Develop Reference

How to access a shared file from multiple threads most effectively?

I'm developing a small web-app whose servlets periodically get access to a shared resource which is a simple text-file on the server side holding some lines of mutable data. Most of the time, servelts just read file for the data, but some servelts may also update it, adding new lines to the file or removing and replacing existing lines. Although file contents is not updated very often, there is still little chance for the data inconsistency and file corruption if two or more servlets decide to read and write to file at the same time.
The first goal is to make the file reading/writing safe. For this purpose, I've created a helper FileReaderWriter class providing some static methods for thread-safe file access. The read and write methods are coordinated by ReentrantReadWiteLock. The rule is quite simple: multiple threads may read from file at any time as far as no other thread is writing to it at the same time.
public class FileReaderWriter {
private static final ReentrantReadWriteLock rwLock = new ReentrantReadWriteLock();
public static List<String> read(Path path) {
List<String> list = new ArrayList<>();
rwLock.readLock().lock();
try {
list = Files.readAllLines(path);
} catch (IOException e) {
e.printStackTrace();
} finally {
rwLock.readLock().unlock();
}
return list;
}
public static void write(Path path, List<String> list) {
rwLock.writeLock().lock();
try {
Files.write(path, list);
} catch (IOException e) {
e.printStackTrace();
} finally {
rwLock.writeLock().unlock();
}
}
}
Then, every servelt may use the above method for file reading like this:
String dataDir = getServletContext().getInitParameter("data-directory");
Path filePath = Paths.get(dataDir, "test.txt");
ArrayList<String> list = FileReaderWriter.read(filePath);
Similarly, writing may be done with FileReaderWriter.write(filePath, list) method. Note: if some data needs to be replaced or removed (which means fetching the data form a file, processing it and writing updated data back to a file), then the whole code paths for this operation should be locked by rwLock.writeLock() for atomicity reasons.
Now, when access to a shared file seems to be safe (at least, I hope so), the next step is to make it fast. From the scalability perspective, reading a file at every user's request to the servlet doesn't sound reasonable. So, what I thought of is to read the contents of file into ArrayList (or other collection) only once during the context initialization time and then share this ArrayList (not the file) as a context-scoped data-holder attribute. Then a context-scoped attribute can be shared by servlets with the same locking mechanism as described above and the contents of the updated ArrayList may be independently stored back to the file on some regular basis.
Another solution (in order to avoid locking) would be to use CopyOnWriteArrayList (or some other collection from java.util.concurrent package) for holding a shared data and designate a single-threaded ExecutorService to dump its contents into a file when needed. I also heard of Java Memory-Mapped Files for mapping the entire file into internal memory, but not sure if such approach is appropriate for this particular situation.
So, could anybody, please, guide me thorough the most effective ways (maybe, suggesting some other alternatives) to solve the problem with a shared file access, provided that the writing to a file is quite infrequent and the contents of it is not expected to exceed a dozens of lines.

You don't explain your real problem, only your current attempt then, is difficult to provide a good solution.
Your approach has two serious problems:
Problem 1: concurrency
a shared resource which is a simple text-file on the server side
holding some lines of mutable data
90% of the solution to a problem is a good data structure. A mutable file it's not. Even popular database engines have important concurrency limitations (eg. SQLite), don't try to reinvent the wheel.
Problem 2: horizontal scalability
Even if he solves his local concurrency problems (eg. synchronous methods), you won't be able to deploy multiple instances (nodes/servers) of your application.
Solution 1: use the right tool for the job
You don't explain exactly the nature of your (data management) problem but probably any NoSQL database will do you good (reading about MongoDB can be a good starting point).
(Bad) solution 2: use FileLock
If for some reason you insist on doing what you indicate, use low level file locks using FileLock. You will only have to deal with partial file locks and even these can be distributed horizontally. You won't have to worry about synchronizing other resources either, as file-level locks will suffice.
(Limited) solution 3: in memory structure
If you don't need horizontal scalability, you can use a shared in memory structure like ConcurrentHashMap but you will lose the horizontal scalability and you could lose transactions if you do not persist the information before an application stop.
Conclusion
Although there are more exotic distributed data models, using a database for even a single table may be the best and simplest solution.

Java In-Application Storage from Xml Data

Hi I am developing a game in Java for learning purposes and have a question regarding in-app handling of data
Say I have a bunch of xml files storing a variety of statistics for, weapons armours etc and I parse these files into my application as lists;
Note, this data will never change and is effectively "final", it is constant data that will be used by the game.
Now my question is, would you store this data (for use while the game is running) in a class (in my example here a singleton) within your application that you can access easily? So for example something like this (where the list has been read elsewhere by the XML parser)
public class WeaponData {
private List<Weapon> weaponData;
public static final WeaponData instance = new WeaponData(parseXml("weapons"));
private WeaponData(List<Weapon> data) {
weaponData = data;
}
public static WeaponData getInstance() {
return instance;
}
public List<Weapon> getWeaponData() {
return weaponData;
}
}
And allows me to use the data with
WeaponData.getInstance().getWeaponData();
If so, is this the way to go about it, or is there a better way?
Or, the only alternative I can think of, is to keep reading the XML whenever its needed (which seems unwise, paticularly given, this data retrieval may (for a commercial application atleast) be a network operation, even if in my case it is just a hdd read). Plus if anything else it would be repeated code.
Thanks for your time!

You should probably read it in once and save it, but don't store it as a singleton. Read the WeaponData during initialization and store an instance of it. If you don't like that you are welcome to use the pattern you suggested, it just might be awkward later.
If you ever get to where WeaponData might be updated while the app is running, you may want to re-read it, but not ever time.

I'm thinking that your approach will work, especially as you have mentioned that the data won't become large enough that it imposes on memory. Another concern would be the frequency in which you must read the data. There's a tradeoff between those two considerations, but it sounds like the small XML size warrants unmarshalling the XML into objects immediately.
Just for completeness, you should be using JAX-B to generate Java classes from your XML schema, rather than rolling your own parser.
If you fear that the XML's data may change, you might consider using the WatchService API to detect changes and re-parse the XML file.

Java saving strings

I have a RuneScape Private Server project coded in Java, and am trying to code a personal "tag" that players can use. I have managed to do this, but everytime there is a restart on the server, their "tag" gets reset to "null".
Their "tag" is initalized by doing a command ";;settag [name]". Their tag is then set to whatever they want. I have done this through a string:
if (command[0].equals("settag")) {
newTag = getCompleteString(command, 1);
newTag = player.yellTag
player.sendMessage("Your tag is now:" +newTag);
}
I am unsure what the most efficient way to fix this would be, I am thinking of just loading and saving through .xml/.txt files. By the way, player.yellTag is where the next command (::mytag) searches it from, which works fine, until there is a restart of the server.

it all depends on the context of your application. If you are planning on having less than a few hundreds players, then a xml file may be ok. You should look at JAXB, which is, afaict, the standard way to store your objects in Java. You can also store them as JSON files, using gson which is way simpler to use and implement than XML stuff.
But if you get to have more than thousands of players, you may want to get some more efficient way to serialize your tags by putting them in a database, and thus an ORM library like hibernate could help you do that.
You may want to make your own stuff, like a tag directory full of files named after unique ids of your players containing the players' tag... It's a lot more "hackish" but still quite efficient.

Using the JNotify library, how can you tell if a deleted file was a file or a directory?

For those unfamiliar with JNotify, this is a library which provides an easy way to monitor events in a directory.
For instance, when a file gets deleted in the selected folder, the method "fileDeleted" gets called, along with a few parameters. Here's an example of the fileDeleted method:
public void fileDeleted(int wd, String rootPath, String name) {
print("deleted " + rootPath + " : " + name);
}
Now, I would like to know if the deleted file was a file or directory. My usual approach is to create a new File object with the given path, and use the methods isFile() and isDirectory()
However, since this file is already deleted, these methods always return false.
So here's my concrete question: I have the path to a deleted file or directory, how can I tell wether it was a file or a directory? Is there a workaround to this? What's the best practice to do here?
Thank you in advance.

I suggest using a better API for this, like Commons IO. It has this distinction in its interface org.apache.commons.io.monitor.FileAlterationListener and its methods onFile...(), onDirectory...(). Alternatively, and this is probably the best approach, use the new standard feature for this that comes with Java 7, WatchService, as discussed here.

How big is the directory structure you're looking at?
My first instinct is to build an internal representation of the directory structure, using some simple graph traversal algorithm, and then do a lookup every time something is removed to figure out what it was.
<edit>
If you know your directory structure is a strict tree you can use a simple recursion to traverse the file system, and create a map of Files or Strings to boolean, so you can do an easy lookup. Then, once you've got the map built it should be easy to maintain using the JNotify events.
<edit/>
even for medium-sized directories I would think this could be made pretty quick. What is this for? Might there be another way of going about achieving the same goal?

I am facing the same problem. Yet as far as I understand it, Java's WatchService does not allow monitoring of subdirectories, so I cannot use it (task is to monitor changes to a structure containing ~40K folders). I will try and go ahead using the simple (and fallible) heuristic
If it contains a dot ('.'), it's a file.
I will post updates if I come across something more sophisticated...

Best approach to create a security environment in Java

I need to create a desktop application that will run third party code, and I need to avoid the third party code from export by any way (web, clipboard, file io) informations from the application.
Somethig like:
public class MyClass {
private String protectedData;
public void doThirdPartyTask() {
String unprotedtedData = unprotect(protectedData);
ThirdPartyClass.doTask(unprotectedData);
}
private String unprotect(String data) {
// ...
}
}
class ThirdPartyClass {
public static void doTask(String unprotectedData) {
// Do task using unprotected data.
// Malicious code may try to externalize the data.
}
}
I'm reading about SecurityManager and AccessControler, but I'm still not sure what's the best approach to handle this.
What should I read about to do this implementation?

First of all, there is pretty much no way you can stop every information leak on a local computer. You can certainly restrict network access, and even a lot of file system access, but there is nothing that would stop the gui from popping up a dialog showing the information to the user on the screen, or any other 100 ways you could "leak" data.
Secondly, you keep talking about the policy file being changeable by the user. yes, it is. it sounds like you are basically trying to recreate DRM. I'd suggest reading up on DRM and the general futility of it. It essentially boils down to giving someone a locked box and the key to the box and telling them not to open it. If someone has physical access to your program, there is almost nothing you can do to stop them from getting data out of it, in java or pretty much any other programming language (at least, not on computers as they are built today).

A general approach would be to run your jvm with a security policy that grants java.security.AllPermission to your codebase (i.e. jar) and no permissions whatsoever to the third-party codebase. Here is some documentation on how to run with a policy file and what to put in said file.