Develop Reference

Only download a certain amount of files per second - "decrease" the download speed

It appears to me that my server only allows 60 files to be downloaded per second, but I have 63 of them - all tiny, YAML files. As a result, the last 3 files don't get downloaded and throw error 503. I am using Baeldung's NIO example:
public static void downloadWithJavaNIO(String fileURL, String localFilename) throws MalformedURLException {
String credit = "github.com/eugenp/tutorials/blob/master/core-java-modules/core-java-networking-2/src/main/java/com/baeldung/download/FileDownload.java";
URL url = new URL(fileURL);
try (
ReadableByteChannel readableByteChannel = Channels.newChannel(url.openStream());
FileOutputStream fileOutputStream = new FileOutputStream(localFilename);
FileChannel fileChannel = fileOutputStream.getChannel()
) {
fileChannel.transferFrom(readableByteChannel, 0, Long.MAX_VALUE);
fileOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
I was thinking of saving currentTimeMillis somewhere and checking if a second had passed by the time the 61th file is pending. But are there any other good ideas?

FileChannel.transferFrom: An invocation of this method may or may not transfer all of the requested bytes.
So I am not sure whether it works in all cases. Maybe with those small files.
I would first try a correct (non-optimized) version:
public static void downloadWithJavaNIO(String fileURL, String localFilename)
throws MalformedURLException {
URL url = new URL(fileURL);
Path targetPath = Paths.get(localFilename);
try (InputStream in = url.openStream()) {
Files.copy(in, targetPath);
} catch (IOException e) {
System.getLogger(getClass().getName()).log(Level.ERROR, fileURL, e);
}
}
How fast now? Does it still need throttling? (slowing down, waiting)
Does the capacity error still persist?

Setting permissions for created directory to copy files into it

During the execution of my program it creates a directory which contains two sub-directories/two folders. Into one of these folders I need to copy a Jar-File. My programm resembles an installation routine. The copying of the Jar file is not the problem here, but the permissions of the created directories.
I tried to set the permissions of the directories (before actually creating them with the mkdirs() method) with File.setWritable(true, false) and also with the .setExecutable and .setReadable methods, but the access to the sub-directories is still denied.
Here's an excerpt of my code for the creation of one of the two sub-directories:
folderfile = new File("my/path/to/directory");
folderfile.setExecutable(true, false);
folderfile.setReadable(true, false);
folderfile.setWritable(true, false);
result = folderfile.mkdirs();
if (result) {
System.out.println("Folder created.");
}else {
JOptionPane.showMessageDialog(chooser, "Error");
}
File source = new File("src/config/TheJar.jar");
File destination = folderfile;
copyJar(source, destination);
And my "copyJar" method:
private void copyJar(File source, File dest) throws IOException {
InputStream is = null;
OutputStream os = null;
try {
is = new FileInputStream(source);
os = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer))>0) {
os.write(buffer, 0, length);
}
} catch (Exception e) {
e.printStackTrace();
}
is.close();
os.close();
}
At os = new FileOutputStream(dest); the debugger throws a FileNotFoundException with the description that the access to the directory has been denied.
Does anyone have an idea what I am doing wrong or have a better solution for setting the permissions via Java? Thanks in advance!

A similar question was asked there are several years.
A possible solution for Java 7 and Unix system is available here : How do i programmatically change file permissions?
Or, below the best response, a example with JNA.
I hope that that will help you !

I solved the problem. In the end it was much easier to solve than expected.
The main problem was not the permission issue but the FileNotFoundException. The file that is assigned to the OutputStream is not really a file, but just a directory so that the Stream can't find it. You have to create the file before initializing the OutputStream and after that you copy your source file into the newly created file. The code:
private void copyJar(File source, File dest) throws IOException {
InputStream is = null;
File dest2 = new File(dest+"/TheJar.jar");
dest2.createNewFile();
OutputStream os = null;
try {
is = new FileInputStream(source);
os = new FileOutputStream(dest2);
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer))>0) {
os.write(buffer, 0, length);
}
} catch (Exception e) {
e.printStackTrace();
}
is.close();
os.close();
}

regarding cross checking of files in an directory

I have developed a java program that copies the file from source folder to destination folder
there are 10 serialized files that it copies from source folder to destination folder
but one thing is missing in it is that let say if the files are already exists in the destination folder then in that case it should not copy
so basically a look is done within in one second that will check the destination folder contain those 10 serialized files or not
if not then in that case only it should copy and after copying it should again check within in second whether file exists or not , Please advise how to achieve this
//Create a class extends with TimerTask
class ScheduledTask extends TimerTask {
public void run() {
InputStream inStream = null;
OutputStream outStream = null;
try {
File source = new File("C:\\cache\\");
File target = new File("C:\\Authclient\\cache\\");
// Already exists. do not copy
/*if (target.exists()) {
return;
}*/
File[] files = source.listFiles();
for (File file : files) {
inStream = new FileInputStream(file);
outStream = new FileOutputStream(target + "/" + file.getName());
byte[] buffer = new byte[1024];
int length;
// copy the file content in bytes
while ((length = inStream.read(buffer)) > 0) {
outStream.write(buffer, 0, length);
}
inStream.close();
outStream.close();
}
System.out.println("File is copied successful!");
} catch (IOException e) {
e.printStackTrace();
}
}
}
public class Copycache {
public static void main(String args[]) throws InterruptedException {
Timer time = new Timer();
ScheduledTask task = new ScheduledTask();
time.schedule(task, new Date(), TimeUnit.SECONDS.toMillis(1));
}
}
the above exists implementation which is commented is not working correct rite now please advise

I am curious about your exact requirements. Consider this small example:
File file = new File("test.txt");
if (!file.exists())
{
FileOutputStream fis = new FileOutputStream(file);
fis.write("blabla".getBytes());
fis.close();
}
Now put a breakpoint on the line FileOutputStream fis...
Run it and wait at the breakpoint, then create the test.txt manually and put some data in it.
Then continue running the program.
Your program will overwrite the contents of test.txt without warning.
If timing is so crucial here you will need to figure out a different solution.
Edit: I got curious and did some more testing. It seems it won't even throw an exception if you add the line file.createNewFile();, break there, create the file and then continue the application. I wonder why..

exception during file copy in Java

I have a function that copies binary file
public static void copyFile(String Src, String Dst) throws FileNotFoundException, IOException {
File f1 = new File(Src);
File f2 = new File(Dst);
FileInputStream in = new FileInputStream(f1);
FileOutputStream out = new FileOutputStream(f2);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
in.close();
out.close();
}
and the second function
private String copyDriverToSafeLocation(String driverPath) {
String safeDir = System.getProperty("user.home");
String safeLocation = safeDir + "\\my_pkcs11tmp.dll";
try {
Utils.copyFile(driverPath, safeLocation);
return safeLocation;
} catch (Exception ex) {
System.out.println("Exception occured while copying driver: " + ex);
return null;
}
}
The second function is run for every driver found in the system.
The driver file is copied and I am trying to initialize PKCS11 with that driver.
If initialization failed I go to next driver, I copy it to the tmp location and so on.
The initialization is in try/catch block
After the first failure I am no longer able to copy next driver to the standard location.
I get the exception
Exception occured while copying driver: java.io.FileNotFoundException: C:\Users\Norbert\my_pkcs11tmp.dll (The process cannot access the file because it is being used by another process)
How can I avoid the exception and safely copy the driver file?
For those curious why am I trying to copy the driver ... PKCS11 has nasty BUG, which prevents using drivers stored in the location that has "(" in the path ... and this is a case I am facing.
I will appreciate your help.

I would move the try-catch block into the copyFile method. That way you can properly handle closing the InputStreams (which is probably causing the JVM to hold onto the file handle). Something like this:
public static void copyFile(String Src, String Dst) {
try {
File f1 = new File(Src);
File f2 = new File(Dst);
FileInputStream in = new FileInputStream(f1);
FileOutputStream out = new FileOutputStream(f2);
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
}
catch(Exception e) {
System.out.println("Exception occured while copying driver: " + ex);
}
finally {
in.close();
out.close();
}
}
Then you can remove the try-catch from the copyDriverToSafeLocation method.

Or there's the Java 7 Way:
public static void copyFile(String src, String dst) throws FileNotFoundException, IOException {
try (FileInputStream in = new FileInputStream(new File(src))) {
try (FileOutputStream out = new FileOutputStream(new File(dst))) {
byte[] buf = new byte[1024];
int len;
while ((len = in.read(buf)) > 0) {
out.write(buf, 0, len);
}
}
}
}
Edit: And the Java 7 NIO way.
public static void copyFile(String src, String dst) throws FileNotFoundException, IOException {
copyFile(new File(src), new File(dst));
}
public static void copyFile(File src, File dst) throws FileNotFoundException, IOException {
try (FileInputStream in = new FileInputStream(src)) {
try (FileOutputStream out = new FileOutputStream(dst)) {
copyFile(in, out);
}
}
}
public static void copyFile(FileInputStream in, FileOutputStream out) throws IOException {
FileChannel cin = in.getChannel();
FileChannel cout = out.getChannel();
cin.transferTo(0, cin.size(), cout);
}

If the file is used by an other process and locked, there is no generic solutions to be able to access it. You best chance is to use FileLock but it's plateform-dependant, read the documentation, it's written that the results are "advisory", so be carefull. you can also take a look at the ReentrantReadWriteLock class.

I would choose to go with Apache Commons IO and their FileUtils.copyFile() routine(s).
Standard concise way to copy a file in Java?

I'm not sure why a problem with one file would prevent copying a different file. However, not closing a file when an exception occurs could definitely cause problems. Use try...finally to make sure you call close on every file you open.

java the system can not find the file specified

I used Java to copy file but it appeared a exception (the system can not find the file specified).
The codes are
public static void copyFile(String sourceFile, String destFile){
try {
InputStream in = new FileInputStream(sourceFile);
OutputStream os = new FileOutputStream(destFile);
byte[] buffer = new byte[1024];
int count;
while ((count = in.read(buffer)) > 0) {
os.write(buffer, 0, count);
}
in.close();
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
The test codes
public static void main(String[] args) {
String name = getFileName("D:/z/temp.txt");
String target = "D:/tem.txt";
copyFile(name, target);
}
the exception is java.io.FileNotFoundException: temp.txt(the system can not find the file specified)
The file 'temp.txt' is existence.
The path is right no problem.
I guess that is the problem of Permissions. who can come up with the answer thanks!

We need to see the method getFileName() to be sure, but based on the error message and the method name, I suspect the problem is just that this method returns only the name of the file, removing the path info, so that the file is, indeed, not found.