i'm developing an application that has to reboot the system after a file has been uploaded and verified. The file system is on an sd card, so it must be synced to be sure the uploaded file has actually been saved on the device.
I was wondering if java.io.file.Files.copy does the sync or not.
My code runs like this:
public int save(MultipartFile multipart) throws IOException {
Files.copy(multipart.getInputStream(), file, standardCopyOption.REPLACE_EXISTING);
if (validate(file)) {
sync(file); <-- is it useless?
reboot();
return 0;
} else {
Files.delete(file);
return -1;
}
}
I tried to find a way to call sync on the fs in the nio package, but the only solution that i've found is:
public void sync(Path file) {
final FileOutputStream fos = new FileOutputStream(file.toFile());
final FileDescriptor fd = fos.getFD();
fd.sync();
}
which relies on old java.io.File .
If you look at the source code for Files.copy(...), you will see that it doesn't perform a sync(). In the end, it will perform a copy of an input stream into an output stream corresponding to the first 2 arguments passed to Files.copy(...).
Furthermore, the FileDescriptor is tied to the stream from which it is obtained. If you don't perform any I/O operation with this stream, other than creating a file with new FileOutputStream(...), there will be nothing to sync() with the fie system, as is the case with the code you shared.
Thus, the only way I see to accomplish your goal is to "revert" to the old-fashioned java.io API and implement a stream-to-stream copy yourself. This will allow you to sync() on the file descriptor obtained from the same FileOutputStream that is used for the copy operation.
I'll say the copy operation is depending on your OS JRE code, so if you want to be sure of the file Copy at OS level, continue to explicitly call the sync() method.
This was because SYNC and DSYNC were annoyingly omitted from StandardCopyOption enum, yet were provided in StandardOpenOption enum for file targets, so you need to use FileChannel and SeekableByteChannel if supported by FileSystemProvider, like :
Set<? extends OpenOption> TARGET_OPEN_OPTIONS = EnumSet.of(StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE);
FileChannel.out = target.getFileSystem().provider().newFileChannel(target, TARGET_OPEN_OPTIONS);
SeekableByteChannel = Files.newByteChannel(source, StandardOpenOption.READ);
out.transferFrom(source, 0, source.size());
out.force(boolean metadata); // false == DSYNC, true == SYNC
Using java.io.FileOutputStream.getFD().sync() is an obsolete "solution", because you lose all support for NIO2 FileSystems, like the often bundled ZipFileSystem, and it can still fail if not supported by the native class implementations or OS!
Using DSYNC or SYNC when opening an OutputStream via a FileSystemProvider is another option, but may cause premature flushing of a FileSystem cache.
Related
I want to check if the file exist on server on multithread environment and if exists return that file content diractly or download from my s3 service server.
My code like this:
final Object lock = new Object();
File file = new File("/file/path");
if (file.exists()) {
return FileUtils.readFileToByteArray(file);
} else {
byte[] bytes = this.downloadFileFromRemoteServer();
if (!file.exists()) {
synchronized (lock) {
if(!file.exists()) {
FileUtils.writeByteArrayToFile(tempFile, bytes);
}
}
}
tempFile.renameTo(file);
return bytes;
}
The above code similar java double checked locking, is method file.exists() behavior like volatile keyword? And pseudo code correctly?
File.exists() checks the file existence with the file-system, and so it should behave like a volatile, so you are covered there
Some issues though -
1) As soon as a thread sees that the file doesn't exist, it starts downloading the file, which is time consuming, so its likely that other threads will also come and start downloading the same file. So the download part should be moved inside the lock
2) You're renaming the temp file outside the lock. A thread may get to that point without creating/writing-to a temp file. Should move the rename inside the lock as-well
Since IO has much more overhead than locking, I think the above 2 steps would be beneficial
You are overly cautious: since you are writing to a temp file, there is no risk of overwriting an existing file, which carries a possibility of reading a half-written file: your reads are going to be consistent.
The only issue that your code is protecting against is writing the same downloaded content into multiple temporary files, which is not much of a performance problem in comparison to multiple downloads, which would happen anyway.
I would simplify your code as follows:
File file = new File("/file/path");
if (!file.exists()) {
byte[] bytes = this.downloadFileFromRemoteServer();
File tempFile = File.createTempFile(...);
FileUtils.writeByteArrayToFile(tempFile, bytes);
tempFile.renameTo(file);
}
return FileUtils.readFileToByteArray(file);
I am trying to open a file for reading or create the file if it was not there.
I use this code:
String location = "/test1/test2/test3/";
new File(location).mkdirs();
location += "fileName.properties";
Path confDir = Paths.get(location);
InputStream in = Files.newInputStream(confDir, StandardOpenOption.CREATE);
in.close();
And I get java.nio.file.NoSuchFileException
Considering that I am using StandardOpenOption.CREATE option, the file should be created if it is not there.
Any idea why I am getting this exception?
It seems that you want one of two quite separate things to happen:
If the file exists, read it; or
If the file does not exist, create it.
The two things are mutually exclusive but you seem to have confusingly merged them. If the file did not exist and you've just created it, there's no point in reading it. So keep the two things separate:
Path confDir = Paths.get("/test1/test2/test3");
Files.createDirectories(confDir);
Path confFile = confDir.resolve("filename.properties");
if (Files.exists(confFile))
try (InputStream in = Files.newInputStream(confFile)) {
// Use the InputStream...
}
else
Files.createFile(confFile);
Notice also that it's better to use "try-with-resources" instead of manually closing the InputStream.
Accordingly to the JavaDocs you should have used newOutputStream() method instead, and then you will create the file:
OutputStream out = Files.newOutputStream(confDir, StandardOpenOption.CREATE);
out.close();
JavaDocs:
// Opens a file, returning an input stream to read from the file.
static InputStream newInputStream(Path path, OpenOption... options)
// Opens or creates a file, returning an output stream that
// may be used to write bytes to the file.
static OutputStream newOutputStream(Path path, OpenOption... options)
The explanation is that OpenOption constants usage relies on wether you are going to use it within a write(output) stream or a read(input) stream. This explains why OpenOption.CREATE only works deliberatery with the OutputStream but not with InputStream.
NOTE: I agree with #EJP, you should take a look to Oracle's tutorials to create files properly.
I think you intended to create an OutputStream (for writing to) instead of an InputStream (which is for reading)
Another handy way of creating an empty file is using apache-commons FileUtils like this
FileUtils.touch(new File("/test1/test2/test3/fileName.properties"));
My Question: How do I open a file (in the system default [external] program for the file) without saving the file to disk?
My Situation: I have files in my resources and I want to display those without saving them to disk first. For example, I have an xml file and I want to open it on the user's machine in the default program for reading xml file without saving it to the disk first.
What I have been doing: So far I have just saved the file to a temporary location, but I have no way of knowing when they no longer need the file so I don't know when/if to delete it. Here's my SSCCE code for that (well, it's mostly sscce, except for the resource... You'll have to create that on your own):
package main;
import java.io.*;
public class SOQuestion {
public static void main(String[] args) throws IOException {
new SOQuestion().showTemplate();
}
/** Opens the temporary file */
private void showTemplate() throws IOException {
String tempDir = System.getProperty("java.io.tmpdir") + "\\BONotifier\\";
File parentFile = new File(tempDir);
if (!parentFile.exists()) {
parentFile.mkdirs();
}
File outputFile = new File(parentFile, "template.xml");
InputStream inputStream = getClass().getResourceAsStream("/resources/template.xml");
int size = 4096;
try (OutputStream out = new FileOutputStream(outputFile)) {
byte[] buffer = new byte[size];
int length;
while ((length = inputStream.read(buffer)) > 0) {
out.write(buffer, 0, length);
}
inputStream.close();
}
java.awt.Desktop.getDesktop().open(outputFile);
}
}
Because of this line:
String tempDir = System.getProperty("java.io.tmpdir") + "\\BONotifier\\";
I deduce that you're working on Windows. You can easily make this code multiplatform, you know.
The answer to your question is: no. The Desktop class needs to know where the file is in order to invoke the correct program with a parameter. Note that there is no method in that class accepting an InputStream, which could be a solution.
Anyway, I don't see where the problem is: you create a temporary file, then open it in an editor or whatever. That's fine. In Linux, when the application is exited (normally) all its temporary files are deleted. In Windows, the user will need to trigger the temporary files deletion. However, provided you don't have security constraints, I can't understand where the problem is. After all, temporary files are the operating system's concern.
Depending on how portable your application needs to be, there might be no "one fits all" solution to your problem. However, you can help yourself a bit:
At least under Linux, you can use a pipe (|) to direct the output of one program to the input of another. A simple example for that (using the gedit text editor) might be:
echo "hello world" | gedit
This will (for gedit) open up a new editor window and show the contents "hello world" in a new, unsaved document.
The problem with the above is, that this might not be a platform-independent solution. It will work for Linux and probably OS X, but I don't have a Windows installation here to test it.
Also, you'd need to find out the default editor by yourself. This older question and it's linked article give some ideas on how this might work.
I don't understand your question very well. I can see only two possibilities to your question.
Open an existing file, and you wish to operate on its stream but do not want to save any modifications.
Create a file, so that you could use file i/o to operate on the file stream, but you don't wish to save the stream to file.
In either case, your main motivation is to exploit file i/o existingly available to your discretion and programming pleasure, am I correct?
I have feeling that the question is not that simple and this my answer is probably not the answer you seek. However, if my understanding of the question does coincide with your question ...
If you wish to use Stream io, instead of using FileOutputStream or FileInputStream which are consequent to your opening a File object, why not use non-File InputStream or OutputStream? Your file i/o utilities will finally boil down to manipulating i/o streams anyway.
http://docs.oracle.com/javase/7/docs/api/java/io/OutputStream.html
http://docs.oracle.com/javase/7/docs/api/java/io/InputStream.html
No need to involve temp files.
Is there a native JDK code to copy files(buffers, streams, or whatever)?
This is the preferred way to copy a file since JDK 1.4 and later
public static void copyFile(final File sourceFile, final File destFile) throws IOException
{
if (!destFile.exists())
{
destFile.createNewFile();
}
FileChannel source = null;
FileChannel destination = null;
try
{
source = new FileInputStream(sourceFile).getChannel();
destination = new FileOutputStream(destFile).getChannel();
destination.transferFrom(source, 0, source.size());
}
finally
{
source.close();
destination.close();
}
}
public abstract long transferFrom(ReadableByteChannel src,
long position,
long count)
throws IOException
... This method is potentially much
more efficient than a simple loop that
reads from this channel and writes to
the target channel. Many operating
systems can transfer bytes directly
from the filesystem cache to the
target channel without actually
copying them. ...
If by "native" you mean "part of the Java standard API" (rather than platform-dependant code, which is usually called "native" in the Java world) and by "copy files" you mean "single method that takes a file and a target path and produces a copy of the file's contents" then no, there is no such method in the standard API. You have to open an InputStream and an OutputStream (optionally get their more efficient FileChannels) and use a buffer to transfer bytes. Convenient single methods to call are found in Apache Commons IO.
Update: Since Java 7, file copy functionality has become part of the Standard API in java.nio.file.Files
Your best option is to use Java NIO:
http://java.sun.com/javase/6/docs/api/java/nio/package-summary.html
For buffers see:
http://java.sun.com/javase/6/docs/api/java/nio/package-summary.html#buffers
For stream however, see the following article:
http://java.sun.com/docs/books/tutorial/essential/io/file.html#readStream
There are frameworks built on top of this, namely Mina and Netty:
Mina - http://mina.apache.org/
Netty - http://www.jboss.org/netty
Just to add that JDK7 defines several copy methods in java.nio.file.Files, including copying files and copy files to/from streams.
I have a cluster of machines, each running a Java app.
These Java apps need to access a unique resource.txt file concurrently.
I need to atomically rename a temp.txt file to resource.txt in Java, even if resource.txt already exist.
Deleting resource.txt and renaming temp.txt doesn't work, as it's not atomic (it creates a small timeframe where resource.txt doesn't exist).
And it should be cross-platform...
For Java 1.7+, use java.nio.file.Files.move(Path source, Path target, CopyOption... options) with CopyOptions "REPLACE_EXISTING" and "ATOMIC_MOVE".
See API documentation for more information.
For example:
Files.move(src, dst, StandardCopyOption.ATOMIC_MOVE);
On Linux (and I believe Solaris and other UNIX operating systems), Java's File.renameTo() method will overwrite the destination file if it exists, but this is not the case under Windows.
To be cross platform, I think you'd have to use file locking on resource.txt and then overwrite the data.
The behavior of the file lock is
platform-dependent. On some platforms,
the file lock is advisory, which means
that unless an application checks for
a file lock, it will not be prevented
from accessing the file. On other
platforms, the file lock is mandatory,
which means that a file lock prevents
any application from accessing the
file.
try {
// Get a file channel for the file
File file = new File("filename");
FileChannel channel = new RandomAccessFile(file, "rw").getChannel();
// Use the file channel to create a lock on the file.
// This method blocks until it can retrieve the lock.
FileLock lock = channel.lock();
// Try acquiring the lock without blocking. This method returns
// null or throws an exception if the file is already locked.
try {
lock = channel.tryLock();
} catch (OverlappingFileLockException e) {
// File is already locked in this thread or virtual machine
}
// Release the lock
lock.release();
// Close the file
channel.close();
} catch (Exception e) {
}
Linux, by default, uses voluntary locking, while Windows enforces it. Maybe you could detect the OS, and use renameTo() under UNIX with some locking code for Windows?
There's also a way to turn on mandatory locking under Linux for specific files, but it's kind of obscure. You have to set the mode bits just right.
Linux, following System V (see System
V Interface Definition (SVID) Version
3), lets the sgid bit for files
without group execute permission mark
the file for mandatory locking
Here is a discussion that relates: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4017593
As stated here, it looks like the Windows OS doesn't even support atomic file rename for older versions. It's very likely you have to use some manual locking mechanisms or some kind of transactions. For that, you might want to take a look into the apache commons transaction package.
If this should be cross-platform I suggest 2 options:
Implement an intermediate service that is responsible for all the file accesses. Here you can use several mechanisms for synchronizing the requests. Each client java app accesses the file only through this service.
Create a control file each time you need to perform synchronized operations. Each java app that accesses the file is responsible checking for the control file and waiting while this control file exists. (almost like a semaphore). The process doing the delete/rename operation is responsible for creating/deleting the control file.
If the purpose of the rename is to replace resource.txt on the fly and you have control over all the programs involved, and the frequency of replacement is not high, you could do the following.
To open/read the file:
Open "resource.txt", if that fails
Open "resource.old.txt", if that fails
Open "resource.txt" again, if that fails
You have an error condition.
To replace the file:
Rename "resource.txt" to "resource.old.txt", then
Rename "resource.new.txt" to "resource.txt", then
Delete "resource.old.txt".
Which will ensure all your readers always find a valid file.
But, easier, would be to simply try your opening in a loop, like:
InputStream inp=null;
StopWatch tmr=new StopWatch(); // made up class, not std Java
IOException err=null;
while(inp==null && tmr.elapsed()<5000) { // or some approp. length of time
try { inp=new FileInputStream("resource.txt"); }
catch(IOException thr) { err=thr; sleep(100); } // or some approp. length of time
}
if(inp==null) {
// handle error here - file did not turn up after required elapsed time
throw new IOException("Could not obtain data from resource.txt file");
}
... carry on
You might get some traction by establishing a filechannel lock on the file before renaming it (and deleting the file you're going to overwrite once you have the lock).
-r
I solve with a simple rename function.
Calling :
File newPath = new File("...");
newPath = checkName(newPath);
Files.copy(file.toPath(), newPath.toPath(), StandardCopyOption.REPLACE_EXISTING);
The checkName function checks if exits.
If exits then concat a number between two bracket (1) to the end of the filename.
Functions:
private static File checkName(File newPath) {
if (Files.exists(newPath.toPath())) {
String extractRegExSubStr = extractRegExSubStr(newPath.getName(), "\\([0-9]+\\)");
if (extractRegExSubStr != null) {
extractRegExSubStr = extractRegExSubStr.replaceAll("\\(|\\)", "");
int parseInt = Integer.parseInt(extractRegExSubStr);
int parseIntPLus = parseInt + 1;
newPath = new File(newPath.getAbsolutePath().replace("(" + parseInt + ")", "(" + parseIntPLus + ")"));
return checkName(newPath);
} else {
newPath = new File(newPath.getAbsolutePath().replace(".pdf", " (" + 1 + ").pdf"));
return checkName(newPath);
}
}
return newPath;
}
private static String extractRegExSubStr(String row, String patternStr) {
Pattern pattern = Pattern.compile(patternStr);
Matcher matcher = pattern.matcher(row);
if (matcher.find()) {
return matcher.group(0);
}
return null;
}
EDIT: Its only works for pdf. If you want other please replace the .pdf or create an extension paramter for it.
NOTE: If the file contains additional numbers between brackets '(' then it may mess up your file names.