I have one method which write to a file. I need to synchronize file object
class MessageFile{
public static final String fileName="Main.html"
#AutoWired
AppConifg appconfig;
public boolean writeToFile(String fileContent) throws Exception{
String path = appConfig.getNewsPath() + File.separator + fileName; // getNewsPath is non-static method
final File alertFile= new File(path);
FileOutputStream out = null;
synchronized (alertFile) {
if (!alertFile.exists()) {
alertFile.createNewFile();
}
try {
out = new FileOutputStream(alertFile, false);
out.write(fileContent.getBytes());
out.flush();
} finally {
if (out != null) {
out.close();
}
}
}
return true;
}
}
But above code won`t take lock exclusive lock on file object as another instance of this class can have lock on this class and write to file.
So I want to how handle this case ?
I found one workaround creating a temporary file name appending time stamp (so temporary file name will be always unique) and after writing content to it , will first delete original file and then rename temporary file to original file name.
You can try synchronizing on MessageFile.class, if it is the only object accessing the file.
Your program does not get exclusive lock on file because you are using synchronized on a local variable alertFile that is not shared between instances of the class MessageFile (each object has its own alertFile). You have two possibilities to solve this:
1- Create some static object and synchronize on it (you may use fileName as it is already there).
2- Having a references in all the objects that point to the same object (passed in the constructor, for example) and synchronize on it.
You are creating new File object (alertFile) every time method is run, so the lock does nothing as it is different every time method is run - you need to have static File instance shared across all method calls.
If path can be different every time the method is run, you could create static Map<String, File> instance and use it like this:
Get path of the file.
If there is no File associated with this path, create it.
Otherwise, recover existing File instance from map.
Use this File as a lock and do operations on it.
Example based on modified answer:
class MessageFile{
public static final String fileName="Main.html"
#AutoWired
AppConifg appconfig;
private static final Map<String, File> filesMap = new HashMap<>();
public boolean writeToFile(String fileContent) throws Exception{
String path = appConfig.getNewsPath() + File.separator + fileName; // getNewsPath is non-static method
final File alertFile;
synchronized(filesMap) {
if (filesMap.containsKey(path)) {
alertFile = filesMap.get(path);
}
else {
alertFile = new File(path);
filesMap.put(path, alertFile);
}
}
FileOutputStream out = null;
synchronized (alertFile) {
if (!alertFile.exists()) {
alertFile.createNewFile();
}
try {
out = new FileOutputStream(alertFile, false);
out.write(fileContent.getBytes());
out.flush();
} finally {
if (out != null) {
out.close();
}
}
}
return true;
}
}
Synchronize on the class level object ie MessageFile.class or use a static synchronize method wrtietofile() . it will make sure only one thread writes into the file at a time . it also guarantees the lock will be released once the entire data is written to the file by a thread .
Related
How do you mock file reading/writing via JUnit?
Here is my scenario
MyHandler.java
public abstract class MyHandler {
private String path = //..path/to/file/here
public synchronized void writeToFile(String infoText) {
// Some processing
// Writing to File Here
File file = FileUtils.getFile(filepath);
file.createNewFile();
// file can't be written, throw FileWriteException
if (file.canWrite()) {
FileUtils.writeByteArrayToFile(file, infoText.getBytes(Charsets.UTF_8));
} else {
throw new FileWriteException();
}
}
public String readFromFile() {
// Reading from File here
String infoText = "";
File file = new File(path);
// file can't be read, throw FileReadException
if (file.canRead()) {
infoText = FileUtils.readFileToString(file, Charsets.UTF_8);
} else {
throw FileReadException();
}
return infoText
}
}
MyHandlerTest.java
#RunWith(PowerMockRunner.class)
#PrepareForTest({
MyHandler.class
})
public class MyHandlerTest {
private static MyHandler handler = null;
// Some Initialization for JUnit (i.e #Before, #BeforeClass, #After, etc)
#Test(expected = FileWriteException.class)
public void writeFileTest() throws Exception {
handler.writeToFile("Test Write!");
}
#Test(expected = FileReadException.class)
public void readFileTest() throws Exception {
handler.readFromFile();
}
}
Given above source, Scenario when file is not writable (write permission not allowed) is OK, However, when i try to do scenario wherein file is not readable (read permission not allowed). It always read the file, i have already tried to modify the file permission on the test code via below
File f = new File("..path/to/file/here");
f.setReadable(false);
However, I did some reading, setReadable() always returns false (failed) when run on Windows machine.
Is there a way to modify the file permission of the target file programmatically in relation to JUnit?
Note
Target source code to test cannot be modified, meaning
Myhandler.class is a legacy code which is not to be modified.
Instead of relying on the operating system file permissions, use PowerMock to mock FileUtils.getFile(...) and make it return an instance of File (e.g. anonymous sub class) that returns a specific value for canWrite()/canRead().
Mocking static methods with Mockito
Since Mockito cannot mock static methods, use a File factory instead (or refactor your FileUtils to be a factory), then you can mock it and return a mocked File instance as well, where you can also mock any File methods you want.
So instead of FileUtils.getFile(filepath) you will now have something like FileFactory.getInstance().getFile(filepath) for example, where you can mock getFile(String) method easily.
In jUnit there's a handy rule for scenarios like yours.
public class MyHandlerTest {
#Rule
// creates a temp folder that will be removed after each test
public org.junit.rules.TemporaryFolder folder = new org.junit.rules.TemporaryFolder();
private MyHandler handler;
#Before
public void setUp() throws Exception {
File file = folder.newFile("myFile.txt");
// do whatever you need with it - fill with test content and so on.
handler = new MyHandler(file.getAbsolutePath()); // use the real thing
}
// Test whatever behaviour you need with a real file and predefined dataset.
}
I'm trying to rename an existing file using File#renameTo(), but it doesn't seem to work.
The following code represents what I am trying to do:
public class RenameFileDirectory {
public static void main(String[] args) throws IOException {
new RenameFileDirectory();
}
public RenameFileDirectory() throws IOException {
File file = new File("C:\\Users\\User-PC\\Desktop\\Nouveau dossier2\\file.png");
File desFile = new File ("C:\\Users\\User-PC\\Desktop\\Nouveau dossier2\\file2.png");
if (file.renameTo(desFile)) {
System.out.println("successful rename");
} else {
System.out.println("error");
}
}
}
Try using Files.move instead. If you read the javadocs for renameTo, it states that:
Many aspects of the behavior of this method are inherently platform-dependent: The rename operation might not be able to move a file from one filesystem to another, it might not be atomic, and it might not succeed if a file with the destination abstract pathname already exists. The return value should always be checked to make sure that the rename operation was successful.
I've been working on a plugin that requires a fair amount of data being stored.
I have it being stored in a custom config file I found online that works basically the same as the default config.
The problem I'm having is that I am not sure how to actually close the file or if I even need to, as I know little about yaml configurations.
The code for the template I used is below.
I'm also curious as to advice on how I should store larger amounts of data in the future.
public class CustomConfig {
//store name of file to load/edit
private final String fileName;
//store plugin, to get file directory
private final JavaPlugin plugin;
//store actual hard disk file location
private File configFile;
//store ram file copy location
private FileConfiguration fileConfiguration;
//constructor taking a plugin and filename
public CustomConfig(JavaPlugin plugin, String fileName) {
//ensure plugin exists to get folder path
if (plugin == null)
throw new IllegalArgumentException("plugin cannot be null");
//set this classes plugin variable to the one passed to this method
this.plugin = plugin;
//get name of file to load/edit
this.fileName = fileName;
//get directory/folder of file to load/edit
File dataFolder = plugin.getDataFolder();
if (dataFolder == null)
throw new IllegalStateException();
//load config file from hard disk
this.configFile = new File(plugin.getDataFolder(), fileName);
reloadConfig();
}
public void reloadConfig() {
//load memory file from the hard copy
fileConfiguration = YamlConfiguration.loadConfiguration(configFile);
// Look for defaults in the jar
File configFile = new File(plugin.getDataFolder(), fileName);
if (configFile != null) {
YamlConfiguration defConfig = YamlConfiguration.loadConfiguration(configFile);
fileConfiguration.setDefaults(defConfig);
}
}
public FileConfiguration getConfig() {
if (fileConfiguration == null) {
this.reloadConfig();
}
return fileConfiguration;
}
public void saveConfig() {
if (fileConfiguration == null || configFile == null) {
return;
} else {
try {
getConfig().save(configFile);
} catch (IOException ex) {
plugin.getLogger().log(Level.SEVERE, "Could not save config to " + configFile, ex);
}
}
}
public void saveDefaultConfig() {
if (!configFile.exists()) {
this.plugin.saveResource(fileName, false);
}
}
}
No. You do not have to close YamlConfiguration objects.
While the default config (JavaPlugin.getConfig()) is bound to the lifecycle of the plugin, custom ones are disposed when any other Java object is, i.e. when the garbage collector determines that there are no more references pointing to them in the code.
You don't need to close the config. It's not a BufferedWriter. The config keeps all of the data in the memory until the server shuts down. This means that if you change something in the config during the time your plugin is enabled, you will need to use your reloadConfig() method. The only clean up you need to do after using the FileConfiguration#set(String, Object) method is to use FileConfiguration#saveConfig() to tell Bukkit to take the current state of your config and copy it into your config file.
Is it possible to force Properties not to add the date comment in front? I mean something like the first line here:
#Thu May 26 09:43:52 CEST 2011
main=pkg.ClientMain
args=myargs
I would like to get rid of it altogether. I need my config files to be diff-identical unless there is a meaningful change.
Guess not. This timestamp is printed in private method on Properties and there is no property to control that behaviour.
Only idea that comes to my mind: subclass Properties, overwrite store and copy/paste the content of the store0 method so that the date comment will not be printed.
Or - provide a custom BufferedWriter that prints all but the first line (which will fail if you add real comments, because custom comments are printed before the timestamp...)
Given the source code or Properties, no, it's not possible. BTW, since Properties is in fact a hash table and since its keys are thus not sorted, you can't rely on the properties to be always in the same order anyway.
I would use a custom algorithm to store the properties if I had this requirement. Use the source code of Properties as a starter.
Based on https://stackoverflow.com/a/6184414/242042 here is the implementation I have written that strips out the first line and sorts the keys.
public class CleanProperties extends Properties {
private static class StripFirstLineStream extends FilterOutputStream {
private boolean firstlineseen = false;
public StripFirstLineStream(final OutputStream out) {
super(out);
}
#Override
public void write(final int b) throws IOException {
if (firstlineseen) {
super.write(b);
} else if (b == '\n') {
firstlineseen = true;
}
}
}
private static final long serialVersionUID = 7567765340218227372L;
#Override
public synchronized Enumeration<Object> keys() {
return Collections.enumeration(new TreeSet<>(super.keySet()));
}
#Override
public void store(final OutputStream out, final String comments) throws IOException {
super.store(new StripFirstLineStream(out), null);
}
}
Cleaning looks like this
final Properties props = new CleanProperties();
try (final Reader inStream = Files.newBufferedReader(file, Charset.forName("ISO-8859-1"))) {
props.load(inStream);
} catch (final MalformedInputException mie) {
throw new IOException("Malformed on " + file, mie);
}
if (props.isEmpty()) {
Files.delete(file);
return;
}
try (final OutputStream os = Files.newOutputStream(file)) {
props.store(os, "");
}
if you try to modify in the give xxx.conf file it will be useful.
The write method used to skip the First line (#Thu May 26 09:43:52 CEST 2011) in the store method. The write method run till the end of the first line. after it will run normally.
public class CleanProperties extends Properties {
private static class StripFirstLineStream extends FilterOutputStream {
private boolean firstlineseen = false;
public StripFirstLineStream(final OutputStream out) {
super(out);
}
#Override
public void write(final int b) throws IOException {
if (firstlineseen) {
super.write(b);
} else if (b == '\n') {
// Used to go to next line if did use this line
// you will get the continues output from the give file
super.write('\n');
firstlineseen = true;
}
}
}
private static final long serialVersionUID = 7567765340218227372L;
#Override
public synchronized Enumeration<java.lang.Object> keys() {
return Collections.enumeration(new TreeSet<>(super.keySet()));
}
#Override
public void store(final OutputStream out, final String comments)
throws IOException {
super.store(new StripFirstLineStream(out), null);
}
}
Can you not just flag up in your application somewhere when a meaningful configuration change takes place and only write the file if that is set?
You might want to look into Commons Configuration which has a bit more flexibility when it comes to writing and reading things like properties files. In particular, it has methods which attempt to write the exact same properties file (including spacing, comments etc) as the existing properties file.
You can handle this question by following this Stack Overflow post to retain order:
Write in a standard order:
How can I write Java properties in a defined order?
Then write the properties to a string and remove the comments as needed. Finally write to a file.
ByteArrayOutputStream baos = new ByteArrayOutputStream();
properties.store(baos,null);
String propertiesData = baos.toString(StandardCharsets.UTF_8.name());
propertiesData = propertiesData.replaceAll("^#.*(\r|\n)+",""); // remove all comments
FileUtils.writeStringToFile(fileTarget,propertiesData,StandardCharsets.UTF_8);
// you may want to validate the file is readable by reloading and doing tests to validate the expected number of keys matches
InputStream is = new FileInputStream(fileTarget);
Properties testResult = new Properties();
testResult.load(is);
I have multiple threads which are serializing my 'Data' objects to files. The filename is based on 2 fields from the Object
class Data {
org.joda.DateTime time;
String title;
public String getFilename() {
return time.toString() + '_' + title + ".xml";
}
It is possible that 2 Data objects will have the same 'time' and 'title', and so the same filename.
This is acceptable, and I'm happy for either to be saved. (They're probably the same Data object anyway if those are the same)
My problem is that two (or more) threads are writing to a file AT THE SAME TIME, causing malformed XML.
I had a look at java.nio.channels.FileLock, but it's for VM-Wide locking, and specifically NOT suitable for intra-Thread locking.
I could synchronize on DataIO.class (but that will cause a HUGE overhead, since I really only want to synchronize on the individual File).
Synchronizing on the File object will be useless, as multiple File objects can represent the same System-File.
Code Follows:
class DataIO {
public void writeArticleToFile(Article article, String filename, boolean overwrite) throws IOException {
File file = new File(filename);
writeArticleToFile(article, file, overwrite);
}
public void writeDataToFile(Data data, File file, boolean overwrite) throws IOException {
if (file.exists()) {
if (overwrite) {
if (!file.delete()) {
throw new IOException("Failed to delete the file, for overwriting: " + file);
}
} else {
throw new IOException("File " + file + " already exists, and overwrite flag is set to false.");
}
}
File parentFile = file.getParentFile();
if (parentFile != null) {
file.getParentFile().mkdirs();
}
file.createNewFile();
if (!file.canWrite()) {
throw new IOException("You do not have permission to write to the file: " + file);
}
FileOutputStream fos = new FileOutputStream(file, false);
try {
writeDataToStream(data, fos);
logger.debug("Successfully wrote Article to file: " + file.getAbsolutePath());
} finally {
fos.close();
}
}
}
If I am reading this correctly you have a Data object that represents a single file.
You can consider creating a striped set based on the Data object. Possibly having a ConcurrentHashMap of
ConcurrentMap<Data,Lock> lockMap = new ConcurrentHashMap<Data,Lock>();
No when you want to write to this object you can do:
Lock lock = lockMap.get(someMyDataObject);
lock.lock();
try{
//write object here
}finally{
lock.unlock();
}
Keep in mind you would have to write the hashCode and equals method based on the title and DateTime
You could intern() the string that is the filename. Then synchronise on the interned string.
class DataIO {
public void writeArticleToFile(Article article, String filename, boolean overwrite) throws IOException {
synchronized(filename.intern()) {
File file = new File(filename);
writeArticleToFile(article, file, overwrite);
}
}
I agree that using synchronization is the technique you should use. What you need is a distinct object for each file permutation, and more importantly the same object each time. One option might be to create a class called FileLock:
public class FileLock {
DateTime time;
String title;
public FileLock(DateTime time, String title) {
this.time = time;
this.title = title;
}
override equals/hashCode based on those two properties
static Hashtable<FileLock, FileLock> unqiueLocks = new Hashtable<FileLock, FileLock>();
static lockObject = new Object();
public static FileLock getLock(DateTime time, String title) {
synchronized (lockObject) {
FileLock lock = new FileLock(time, title);
if (unqiueLocks.ContainsKey(lock)) {
return unqiueLocks.get(lock);
}
else {
unqiueLocks.put(lock, lock);
return lock;
}
}
}
}
Then callers would use it like:
synchronized (FileLock.getLock(time, title)) {
...
}
Bear in mind this has a memory leak since the Hashtable keeps growing with new file/time permutations. If you need to, you could modify this technique so that callers of getLock also invoke a releaseLock method that you use to keep the Hashtable clean.