Develop Reference

Java FTPClient Stuck

I'm trying to download a file from my ftp server, but it gets stuck while retrieving them. I'm using commons-net-3.6.jar
Things I've Noticed
When I use ftpClient.enterRemotePassiveMode(); I can see in the FileZilla server interface that the progress of the download for the connection stuck at 76% (688,128 bytes)
When I use ftpClient.enterLocalPassiveMode(); I can see in the FileZilla server interface that the progress of the download for the connection stuck at 96% (884,736 bytes)
What's interesting is that in these two modes it always gets stuck at that exact same amount of bytes no matter what the file is. If the file size is larger than 884,736 or 688,128 it just gets stuck.
It works perfectly for files less that 884,736 bytes in LocalPassiveMode and 688,128 bytes in RemotePassiveMode.
I've tried downloading from a different server and it didn't work so its definitely not a server related issue.
My Code
public class FTPDownload {
private List<BufferedImage> imageList;
private boolean wasConnected;
public void getFiles(String orderRootDirectory){
wasConnected = true;
imageList = new ArrayList<>();
FTPConnection con = new FTPConnection(); // to get the FTP Credentials
con.readData();
try {
FTPClient ftpClient = new FTPClient();
ftpClient.connect(con.getServerIp());
ftpClient.enterLocalPassiveMode();
ftpClient.login(con.getUsername(), con.getPassword());
ftpClient.setAutodetectUTF8(true);
ftpClient.setBufferSize(1024 * 1024); // tried with and without this no luck there
wasConnected = ftpClient.isConnected();
FTPFile[] files = ftpClient.listFiles(orderRootDirectory);
for (FTPFile file : files) {
String details = file.getName();
if (file.isDirectory()) {
details = "[" + details + "]";
}
String totalFilePath = orderRootDirectory+"/"+file.getName();
InputStream inputStream = ftpClient.retrieveFileStream(totalFilePath); // stuck over here
System.out.println(ftpClient.completePendingCommand());
System.out.println(ftpClient.getReplyString());
System.out.println("Reading File...");
long start=System.currentTimeMillis();
ImageIO.setUseCache(false);
BufferedImage bimg = ImageIO.read(inputStream);
long end=System.currentTimeMillis();
System.out.println("time="+(end-start));
imageList.add(bimg);
details += "\t\t" + file.getSize();
details += "\t\t" + file.getName();
System.out.println(details);
}
System.out.println(imageList.size());
ftpClient.logout();
ftpClient.disconnect();
}catch (Exception e){
e.printStackTrace();
wasConnected = false;
}
}

FTPClient#retrieveFileStream states that:
You must close the InputStream when you finish reading from it. The
InputStream itself will take care of closing the parent data
connection socket upon being closed.
And
To finalize the file transfer you must call completePendingCommand and
check its return value to verify success. If this is not done,
subsequent commands may behave unexpectedly.
Calls should be finalized using completePendingCommand, right now you're executing that method before finalizing (you're reading from the inputStream after you've called completePendingCommand).
Also you're not closing your inputStream which it specifically states.
To fix your issues do something like this, where we first close the inputStream and then call completePendingCommand, all of this should happen after we've read the inputStream.
InputStream inputStream = ftpClient.retrieveFileStream(totalFilePath);
BufferedImage bimg = ImageIO.read(inputStream);
inputStream.close();
if (!ftpClient.completePendingCommand()) {
// Throw some error or do something, file transfer failed
}

Downloaded file with Google Drive API. Can't find it anywhere

I am trying to download a file with google drive api. The code gives no error but I can't find the file anywhere.
This is the code
String fileId = "...";
OutputStream outputStream = new ByteArrayOutputStream();
try {
service.files().export(fileId, "text/csv")
.executeMediaAndDownloadTo(outputStream);
} catch (IOException e) {
System.out.println("Ceva nu a mers bine");
e.printStackTrace();
}
System.out.println(outputStream == null);
Any ideea?

Based from this example in Google documentation, I don't see any error with your code.
String fileId = "1ZdR3L3qP4Bkq8noWLJHSr_iBau0DNT4Kli4SxNc2YEo";
OutputStream outputStream = new ByteArrayOutputStream();
driveService.files().export(fileId, "application/pdf")
.executeMediaAndDownloadTo(outputStream);
Make sure that you are using the correct fileId of the specific file you want to download.
You can check on these links: How to get Google Drive file ID, How to download a file from google drive using drive api java?

Java FTP commons-net error downloading files

I'm using apache commons.net to access an ftp site which is the directory is in unix:
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
I'm looping thru a list with the names of the filenames I want to download on a specific ftp site
String ftpPath = "/home/user1/input/";
FileOutputStream fos = null;
File file;
try {
for (int i = 0; i < fileList.size(); i++) {
file = new File(ftpPath+fileList.get(i).toString());
OutputStream outputStream1 = new BufferedOutputStream(new FileOutputStream(file));
boolean download = ftpClient.retrieveFile("c:/test/downloadedFile.csv", outputStream1);
outputStream1.close();
if (download) {
System.out.println("File downloaded successfully !");
} else {
System.out.println("Error in downloading file ! " + downloadFile);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
But once I try to start to download the files I get this error althougth checking the ftp site the file exists under /home/user1/input/TejasSDH_PM_AU_09_07_2014_09_00.csv -rw-r--r--:
java.io.FileNotFoundException: \home\user1\input\TejasSDH_PM_AU_09_07_2014_09_00.csv (The system cannot find the path specified)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:110)
at com.syntronic.client.FTPDataExtract$1.downloadTejasFiles(FTPDataExtract.java:255)
at com.syntronic.client.FTPDataExtract$1.run(FTPDataExtract.java:133)
I'm thinking as the ftp site I'm connecting, the path dir is in unix home/user1/input and java is converting all the "/" to "\". Anyone has an idea on what the error in eclipse means or is something wrong with my code? thank you

You seem to be switching things around.
You are opening a file outputstream to \home\user1\input\TejasSDH_PM_AU_09_07_2014_09_00.csv and you seem to be on windows so it won't work.
You have the local path where the ftp path should go and the other way around.
Please read your errors more carefully, I'm willing to bet that line 255 in FTPDataExtract.java is:
fos = new FileOutputStream(downloadFile);
Which should alert you to the fact that it's not actually an ftp problem.

for (int i = 0; i < fileList.size(); i++) {
OutputStream output;
output = new FileOutputStream("C:/test/" + fileList.get(i).toString());
ftpClient.retrieveFile(ftpPath + fileList.get(i).toString(), output);
output.close();
}
I wrongfully switch the remote and local path, switching it properly will run the program smoothly.

URLConnection FTP list files

URL url = new URL("ftp://user:pass#ftp.example.com/thefolder/");
URLConnection connection = url.openConnection();
...
// List files in folder...
Using something like the above, I was wondering how I could grab a list of files within folder 'thefolder'?
Following on from this original question, I have put together this simple FTP connection which is all working and looking good. It can see all files in the /live/conf/ location and copies them all to the local /conf/ location.
The only issue is, it is copying the files but there's no content.They are all 0KB and empty.
Can anyone see anything obvious that'd be copying the filename but not file content?
try {
FTPClient ftp = new FTPClient();
ftp.connect("000.000.000.000");
ftp.login("USER", "PASSWORD");
ftp.enterLocalPassiveMode();
ftp.setFileType(FTP.BINARY_FILE_TYPE);
FTPFile[] files = ftp.listFiles("/live/conf/");
for (int i=0; i < files.length; i++) {
if (files[i].getName().contains(".csv")) {
String remoteFile1 = files[i].getName();
File downloadFile1 = new File("/var/local/import/conf/"+files[i].getName());
OutputStream outputStream1 = new BufferedOutputStream(new FileOutputStream(downloadFile1));
ftp.retrieveFile(remoteFile1, outputStream1);
outputStream1.close();
}
}
ftp.disconnect();
} catch (SocketException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}

The Java SE URLConnection is insuitable for the job of retrieving a list of files from a FTP host. As to FTP, it basically only supports the FTP get or put commands (retrieve or upload file). It does not support the FTP ls command (list files) which you're basically looking for, let alone many others.
You need to look for 3rd party libraries supporting the FTP ls command (and many more). A commonly used one is the Apache Commons Net FtpClient. In its javadoc is demonstrated how to issue a ls:
FTPClient f = new FTPClient();
f.connect(server);
f.login(username, password);
FTPFile[] files = f.listFiles(directory);

You could use Apache commons FTPClient
This would allow you to call listFiles with...
public static void main(String[] args) throws IOException {
FTPClient client = new FTPClient();
client.connect("c64.rulez.org");
client.enterLocalPassiveMode();
client.login("anonymous", "");
FTPFile[] files = client.listFiles("/pub");
for (FTPFile file : files) {
System.out.println(file.getName());
}

Check out this class I found. It's does the lifting for you.
Class at nsftools.com
Example would be:
FTPConnection ftpConnect = new FTPConnection();
ftpConnect.connect("ftp.example.com");
ftpConnect.login("user","pass");
System.out.println(ftpConnect.listFiles());

How to copy a file on the FTP server to a directory on the same server in Java?

I'm using Apache Commons FTP to upload a file. Before uploading I want to check if the file already exists on the server and make a backup from it to a backup directory on the same server.
Does anyone know how to copy a file from a FTP server to a backup directory on the same server?
public static void uploadWithCommonsFTP(File fileToBeUpload){
FTPClient f = new FTPClient();
FTPFile backupDirectory;
try {
f.connect(server.getServer());
f.login(server.getUsername(), server.getPassword());
FTPFile[] directories = f.listDirectories();
FTPFile[] files = f.listFiles();
for(FTPFile file:directories){
if (!file.getName().equalsIgnoreCase("backup")) {
backupDirectory=file;
} else {
f.makeDirectory("backup");
}
}
for(FTPFile file: files){
if(file.getName().equals(fileToBeUpload.getName())){
//copy file to backupDirectory
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
Edited code: still there is a problem, when i backup zip file, the backup-ed file is corrupted.
Does any body know the reason for it?
public static void backupUploadWithCommonsFTP(File fileToBeUpload) {
FTPClient f = new FTPClient();
boolean backupDirectoryExist = false;
boolean fileToBeUploadExist = false;
FTPFile backupDirectory = null;
try {
f.connect(server.getServer());
f.login(server.getUsername(), server.getPassword());
FTPFile[] directories = f.listDirectories();
// Check for existence of backup directory
for (FTPFile file : directories) {
String filename = file.getName();
if (file.isDirectory() && filename.equalsIgnoreCase("backup")) {
backupDirectory = file;
backupDirectoryExist = true;
break;
}
}
if (!backupDirectoryExist) {
f.makeDirectory("backup");
}
// Check if file already exist on the server
f.changeWorkingDirectory("files");
FTPFile[] files = f.listFiles();
f.changeWorkingDirectory("backup");
String filePathToBeBackup="/home/user/backup/";
String prefix;
String suffix;
String fileNameToBeBackup;
FTPFile fileReadyForBackup = null;
f.setFileType(FTP.BINARY_FILE_TYPE);
f.setFileTransferMode(FTP.BINARY_FILE_TYPE);
for (FTPFile file : files) {
if (file.isFile() && file.getName().equals(fileToBeUpload.getName())) {
prefix = FilenameUtils.getBaseName(file.getName());
suffix = ".".concat(FilenameUtils.getExtension(file.getName()));
fileNameToBeBackup = prefix.concat(Calendar.getInstance().getTime().toString().concat(suffix));
filePathToBeBackup = filePathToBeBackup.concat(fileNameToBeBackup);
fileReadyForBackup = file;
fileToBeUploadExist = true;
break;
}
}
// If file already exist on the server create a backup from it otherwise just upload the file.
if(fileToBeUploadExist){
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
f.retrieveFile(fileReadyForBackup.getName(), outputStream);
InputStream is = new ByteArrayInputStream(outputStream.toByteArray());
if(f.storeUniqueFile(filePathToBeBackup, is)){
JOptionPane.showMessageDialog(null, "Backup succeeded.");
f.changeWorkingDirectory("files");
boolean reply = f.storeFile(fileToBeUpload.getName(), new FileInputStream(fileToBeUpload));
if(reply){
JOptionPane.showMessageDialog(null,"Upload succeeded.");
}else{
JOptionPane.showMessageDialog(null,"Upload failed after backup.");
}
}else{
JOptionPane.showMessageDialog(null,"Backup failed.");
}
}else{
f.changeWorkingDirectory("files");
f.setFileType(FTP.BINARY_FILE_TYPE);
f.enterLocalPassiveMode();
InputStream inputStream = new FileInputStream(fileToBeUpload);
ByteArrayInputStream in = new ByteArrayInputStream(FileUtils.readFileToByteArray(fileToBeUpload));
boolean reply = f.storeFile(fileToBeUpload.getName(), in);
System.out.println("Reply code for storing file to server: " + reply);
if(!f.completePendingCommand()) {
f.logout();
f.disconnect();
System.err.println("File transfer failed.");
System.exit(1);
}
if(reply){
JOptionPane.showMessageDialog(null,"File uploaded successfully without making backup." +
"\nReason: There wasn't any previous version of this file.");
}else{
JOptionPane.showMessageDialog(null,"Upload failed.");
}
}
//Logout and disconnect from server
in.close();
f.logout();
f.disconnect();
} catch (IOException e) {
e.printStackTrace();
}
}

If you are using apache commons net FTPClient, there is a direct method to move a file from one location to another location (if the user has proper permissions).
ftpClient.rename(from, to);
or, If you are familiar with ftp commands, you can use something like
ftpClient.sendCommand(FTPCommand.yourCommand, args);
if(FTPReply.isPositiveCompletion(ftpClient.getReplyCode())) {
//command successful;
} else {
//check for reply code, and take appropriate action.
}
If you are using any other client, go through the documentation, There wont be much changes between client implementations.
UPDATE:
Above approach moves the file to to directory, i.e, the file won't be there in from directory anymore. Basically ftp protocol meant to be transfer the files from local <-> remote or remote <-> other remote but not to transfer with in the server.
The work around here, would be simpler, get the complete file to a local InputStream and write it back to the server as a new file in the back up directory.
to get the complete file,
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ftpClient.retrieveFile(fileName, outputStream);
InputStream is = new ByteArrayInputStream(outputStream.toByteArray());
now, store this stream to backup directory. First we need to change working directory to backup directory.
// assuming backup directory is with in current working directory
ftpClient.setFileType(FTP.BINARY_FILE_TYPE);//binary files
ftpClient.changeWorkingDirectory("backup");
//this overwrites the existing file
ftpClient.storeFile(fileName, is);
//if you don't want to overwrite it use storeUniqueFile
Hope this helps you..

Try this way,
I am using apache's library .
ftpClient.rename(from, to) will make it easier, i have mentioned in the code below
where to add ftpClient.rename(from,to).
public void goforIt(){
FTPClient con = null;
try
{
con = new FTPClient();
con.connect("www.ujudgeit.net");
if (con.login("ujud3", "Stevejobs27!!!!"))
{
con.enterLocalPassiveMode(); // important!
con.setFileType(FTP.BINARY_FILE_TYPE);
String data = "/sdcard/prerakm4a.m4a";
ByteArrayInputStream(data.getBytes());
FileInputStream in = new FileInputStream(new File(data));
boolean result = con.storeFile("/Ads/prerakm4a.m4a", in);
in.close();
if (result)
{
Log.v("upload result", "succeeded");
//$$$$$$$$$$$$$$$$$$$$$$$$$$$$$Add the backup Here$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$//
// Now here you can store the file into a backup location
// Use ftpClient.rename(from, to) to place it in backup
//$$$$$$$$$$$$$$$$$$$$$$$$$$$$$Add the backup Here$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$//
}
con.logout();
con.disconnect();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}

There's no standard way to duplicate a remote file over FTP protocol. Some FTP servers support proprietary or non-standard extensions for this though.
So if your are lucky that your server is ProFTPD with mod_copy module, you can use FTP.sendCommand to issue these two commands:
f.sendCommand("CPFR sourcepath");
f.sendCommand("CPTO targetpath");
The second possibility is that your server allows you to execute arbitrary shell commands. This is even less common. If your server supports this you can use SITE EXEC command:
SITE EXEC cp -p sourcepath targetpath
Another workaround is to open a second connection to the FTP server and make the server upload the file to itself by piping a passive mode data connection to an active mode data connection. Implementation of this solution (in PHP though) is shown in FTP copy a file to another place in same FTP.
If neither of this works, all you can do is to download the file to a local temporary location and re-upload it back to the target location. This is that the answer by #RP- shows.
See also FTP copy a file to another place in same FTP.

To backup at same Server (move), can you use:
String source="/home/user/some";
String goal ="/home/user/someOther";
FTPFile[] filesFTP = cliente.listFiles(source);
clientFTP.changeWorkingDirectory(goal); // IMPORTANT change to final directory
for (FTPFile f : archivosFTP)
{
if(f.isFile())
{
cliente.rename(source+"/"+f.getName(), f.getName());
}
}