Develop Reference

FTPSClient, What values to give for Remote and Local

I am new to FTPSClient i trying to connect to a FTPS created in my laptop. i don't exactly what some of the methods working and their parameter meaning.
For example,
In my code i have created a FTPSClient as below:
FTPSClient ftps =new FTPSClient();
Then connected to a server use connect() method with ip address.
ftps.connect("172.xx.xx.xxx");
After every step i will check the reply code using.
ftps.getReplyCode();
In the below code i know that
username = system username
password = the password to login
ftps.login(username, password);
In the my system in Internet Information Service(IIS). Created an ftp server with ssl and given the below directory to share.
C:\Users\karan-pt2843\Desktop\FTPS
Want to send the file in below directory to the server.
D:\sam.txt
Now i want to store a file in the server in the given above directory and i tried using
remote="";
local="";
InputStream input;
input = new FileInputStream(local);
ftps.storeFile(remote, input);
input.close();
I don't know what value to give for remote and local. please help me with the values to give on them and the what happens internal.

// Use passive mode as default because most of us are
// behind firewalls these days.
ftps.enterLocalPassiveMode();
...
String remote = "samFromClient.txt"; //Place on FTP
String input = "D:/sam.txt" //Place on your Client
//Your FTP reads from the inputstream and store the file on remote-path
InputStream input = new InputStream(new FileInputStream(input));
ftps.storeFile(remote, input);
input.close();
ftps.logout();
...
Taken from: Apache example

In java is there a way to tell on which physical computer a file resides?

I have an eclipse RCP product that is run by multiple people at our company. All PCs are running some version of Windows. We have access to a shared PC which different people have mapped to different drive letters. That means the same file may be referred to in many different ways depending on the PC on which program is run. E.g.
\communalPC\Shared\foo.txt
Y:\Shared\foo.txt
Z:\Shared\foo.txt
I want to programmatically check if an arbitrary file is on the communnal PC. Is there a robust way to do this in java?
Our current solution below is a bit of a hack It is not robust due to people mapping to different drive letters, changing drive letters, not-portable etc.
private static boolean isOnCommunalPc(File file) {
if(file.getAbsolutePath().toLowerCase().startsWith("\\\\communalPC")) {
return true;
}
if(file.getAbsolutePath().toLowerCase().startsWith("y:")){
return true;
}
if(file.getAbsolutePath().toLowerCase().startsWith("z:")){
return true;
}
return false;
}

Java cannot tell the difference of which machine the file is on, as Windows abstracts that layer away from the JVM. You can, however be explicit with your connection.
Is there a reason why you couldn't have an ftp or http server (or even a custom java server!) on the communal pc, and to access it via a hostname or an ip? That way, it doesn't matter where the user has mapped the network drive, you connected via a static address.
Accessing a remote file in Java is as easy as:
URL remoteUrl = new URL(String.format("%s/%s", hostName, fileName));
InputStream remoteInputStream remoteUrl.openConnection().getInputStream();
//copyStreamToFile(remoteInputStream, new File(destinationPath), false);
If you need the file to be local for a library or code you would prefer not to change, you could:
void copyStreamToFile(InputStream in, File outputFile, boolean doDeleteOnExit) {
//Clean up file after VM exit, if needed.
if(doDeleteOnExit)
outputFile.deleteOnExit();
FileOutputStream outputStream = new FileOutputStream(outputFile);
ReadableByteChannel inputChannel = Channels.newChannel(in);
WritableByteChannel outputChannel = Channels.newChannel(outputStream);
ChannelTools.fastChannelCopy(inputChannel, outputChannel);
inputChannel.close();
outputChannel.close()
}
EDIT Accessing a remote file via Samba with JCIFS is as easy as:
domain = ""; //Your domain, only set if needed.
NtlmPasswordAuthentication npa = new NtlmPasswordAuthentication(domain, userName, password);
SmbFile remoteFile = new SmbFile(String.format("smb://%s/%s", hostName, fileName), npa);
//copyStreamToFile(new SmbFileInputStream(remoteFile), new File(destinationPath), false)
This will probably be the most pragmatic solution, as it requires the least amount of work on the Windows server. This plugs into the existing server framework in Windows, instead of installing more.

Is there a simple method to check if there are changes in a SFTP server?

My objective is to poll the SFTP server for changes. My first thought is to check if the number of files in the dir changed. Then maybe some additional checks for changes in the dir.
Currently I'm using the following:
try {
FileSystemOptions opts = new FileSystemOptions();
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(opts, "no");
SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, true);
SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 60000);
FileSystemManager manager = VFS.getManager();
FileObject remoteFile = manager.resolveFile(SFTP_URL, opts);
FileObject[] fileObjects = remoteFile.getChildren();
System.out.println(DateTime.now() + " --> total number of files: " + Objects.length);
for (FileObject fileObject : fileObjects) {
if (fileObject.getName().getBaseName().startsWith("zzzz")) {
System.out.println("found one: " + Object.getName().getBaseName());
}
}
} catch (Exception e) {
e.printStackTrace();
}
This is using apache commons vfs2 2.2.0. It works "fine", but when the server has too many files, it takes over minutes just to get the count(currently, it takes over 2 minutes to get the count for a server that has ~10k files). Any way to get the count or other changes on the server faster?

Unfortunately there's no simple way in the SFTP protocol to get the changes. If you can have some daemon running on the server OR if the source of the new files can create/update a helper file, creation of such file with the last modification time in its name or contents can be an option.

I know the SFTP protocol fairly well, having developed commercial SFTP clients and an SFTP server (CompleteFTP), and as far as I know there's no way within the protocol to get a count of files in a directory without listing it. Some servers, such as ours, provide ways of adding custom commands to servers that you can invoke from the client, so it would be possible to add a custom command that returns the number of files in a directory. CompleteFTP also allows you to write custom file-systems so you could potentially write one that only shows files that have changed after a given timestamp when you do a listing, which might be another approach. Our server only runs on Windows though, so that might be show-stopper for you.

SFTP: IOException while reading a file with java

I am using com.jcraft.jsch library to read .xls files from an SFTP server. Following is the code to connect to server.
session = jsch.getSession(username, host);
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword(password);
session.connect();
sftpChannel = (ChannelSftp) session.openChannel("sftp");
sftpChannel.connect();
I am using sftpChannel.get(file) to retrieve inputStream to the file. This inputstream is then used to instantiate XSSFWorkbook as shown below:
XSSFWorkbook workbook = new XSSFWorkbook(in);
Problem 1:
When I run the app, it seems to get stuck on the above line for some time (say 5 minutes) and then it throws java.io.IOException: Pipe closed error.
The xls file I am trying to read is 800kb and it works fine when run from local machine.
Problem 2:
The app is designed to process files sequentially. So, if first file fails with IOE, rest of the files also fail as the connection is timed out. To prevent this, I put the below code to check and re-connect:
if(null == session || !session.isConnected()){
log.debug("Session is not connected/timed out. Creating a new session");
openSftpSession();
log.debug("New session is created");
}
//openSftpSession() is the code to create a new session as explained in the beginning of the question.
When this code gets executed, following exception gets thrown:
java.io.IOException: error: 4: RequestQueue: unknown request id 1028332337
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1407)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.PushbackInputStream.read(PushbackInputStream.java:186)
//More lines
edit : code to retrieve input stream
public InputStream getInputStream(String folder, String file) throws Exception{
sftpChannel.cd(root + folder);
log.debug("current directory:" + sftpChannel.pwd());
log.debug("File :" + folder + " " + file);
return sftpChannel.get(file);
}
Can anyone please help me get over this? I believe an alternate approach to prevent timeout is to download the file in some temp directory and process. However, I don't really want to do that.
Thanks in advance.

Have you checked to see whether the approach you describe (download into temp file) works? Just to verify that your inputstream is ok.. How long does it take to download into a local file over that connection?
If you don't want to manage a temp file you could always pull it into a byte[] in memory, so long as you don't have to scale to much more than 800kbs.. Use Apache Commons as such:
InputStream in = sftpChannel.get(file);
byte[] inBytes = org.apache.commons.io.IOUtils.toByteArray(in)
ByteArrayInputStream inByteStream = new ByteArrayInputStream(inBytes)
XSSFWorkbook workbook = new XSSFWorkbook(inByteStream);
As for the Request Id, it looks like the old session/channel is still trying to do a read but is no longer able to. Maybe you aren't closing out that session/channel properly. From the openSftpSession() code it looks like you would only be overwriting the references without properly shutting them down.

getContentLength doesn't work on android for FTP url

Following code prints length -1 for filesize on android, but it works fine on desktop JAVA.
I'm using Android 2.2.
URL url1 = null;
URLConnection uconn = null;
try {
url1 = new URL("ftp://FTPHOST/file.zip");
uconn = url1.openConnection();
uconn.setDoInput(true);
int len= uconn.getContentLength();
int headersize = uconn.getHeaderFields().size();
System.out.println("******************************* "+len);
} catch (Exception e) {
e.printStackTrace();
}
return null;
Let me know if any workaround in android to get filesize..

The Android platform's url connection code uses a different base (Apache HTTP client) under the hood, rather than the Oracle JVM's implementation. Apache HTTP client doesn't natively support FTP download the way the desktop JVM does.
The desktop JVM uses a class that was historically named sun.net.ftp.FtpClient for that FTP functionality. None of the sun classes are available on Android, so that doesn't work. You'll need to get your own FTP client.