I'm attempting to use Apache Commons VFS to SFTP a file onto a server, but I keep getting the following error:
java.lang.RuntimeException: org.apache.commons.vfs2.FileSystemException: Could not connect to SFTP server at "sftp://user:***#xxx.x.xxx.xxx/".
Is it normal for it to not include the remote file path (remoteFilePath) here? It's in my code to include it in the connection string (see below)
I have the following jars included in my pom:
commons-logging-1.1.3.jar
commons-vfs2-2.0.jar
hamcrest-core-1.3.jar
jsch-0.1.50.jar
Code:
public void SftpMethod(String strMsg, String tableName){
String host = "xxx.x.xxx.xxx";
String user = "user";
String pass = "password!";
String localFilePath = "C:\\Users\\exampleDir\\Desktop\\loc.dat";
String remoteFilePath = "/dir/home/user/export/loc.dat";
StandardFileSystemManager manager = new StandardFileSystemManager();
File file = new File(localFilePath);
if (!file.exists())
throw new RuntimeException("Error. Local file not found");
try{
manager.init();
// Create local file object
FileObject localFile = manager.resolveFile(file.getAbsolutePath());
// Create remote file object
FileObject remoteFile = manager.resolveFile(
createConnectionString(host, user, pass, remoteFilePath),
createDefaultOptions());
// Copy local file to SFTP server
remoteFile.copyFrom(localFile, Selectors.SELECT_SELF);
System.out.println("File upload success");
}catch(IOException e){
throw new RuntimeException(e);
}finally{
manager.close();
}
}
public static String createConnectionString(String hostName, String username, String password, String remoteFilePath) {
return "sftp://" + username + ":" + password + "#" + hostName + "/" + remoteFilePath;
}
public static FileSystemOptions createDefaultOptions() throws FileSystemException {
// Create SFTP options
FileSystemOptions opts = new FileSystemOptions();
// SSH Key checking
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(opts, "no");
/*
* Using the following line will cause VFS to choose File System's Root
* as VFS's root. If I wanted to use User's home as VFS's root then set
* 2nd method parameter to "true"
*/
// Root directory set to user home
SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, true);
// Timeout is count by Milliseconds
SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 10000);
return opts;
}
Without the full stack trace its hard to give a conclusive answer, but this is what I saw recently:
Caused by: org.apache.commons.vfs2.FileSystemException: Could not load private key from "/Users/<user>/.ssh/id_rsa".
at org.apache.commons.vfs2.provider.sftp.SftpClientFactory.createConnection(SftpClientFactory.java:131)
Unfortunately, I wasn't trying to use a public/private key. I was only intending to log in with a username/password. I needed a way for it to stop trying to read my private key.
The root cause was that the code was using a default location for my key, and attempting to read it (even though thats not what I wanted).
So the workaround was to override the default location by setting the following property:
System.setProperty("vfs.sftp.sshdir", "/");
This bypassed the attempt to read the ssh key altogether, and successfully connected.
Related
I am trying to download a file in an sftp server via Apache VFS. The VFS finds the server itself, but it does not find the file I want to download:
Could not copy "sftp://host/mnt/myFiles/Register/2022.log" because it does not exist.
The VFS finds my local file to which I want to copy, and the server itself, but not the file in the server.
What am I missing? I understand there are some similar questions on StackOverflow, but I have tried to apply their answers (use SftpFileSystemConfigBuilder.getInstance( ).setUserDirIsRoot(opts,true);, create several managers...) and so far the issue has remained.
Prints:
local found? true! file:///C:/Users/Chaddington/Desktop/MyProject/output/2022.log
base found? true! sftp://host/
SFTP found? false! sftp://host/mnt/myFiles/Register/2022.log
Exception:
org.apache.commons.vfs2.FileSystemException: Could not copy "sftp://host/mnt/myFiles/Register/2022.log" because it does not exist.
My code:
// Relevant variables
String localPath = "C:/Users/Chaddington/Desktop/MyProject/output/2022.log";
String remoteUrl = "sftp://host";
String remotePathBase = "sftp://host"
String remotePathRelative = "mnt/myFiles/Register/2022.log";
String myUsr = "username";
String myPass = "password";
// Makes sure there is a file to copy to in my local directory
File fileCopy = new File(localPath);
if (fileCopy.exists())
fileCopy.delete();
fileCopy.createNewFile();
// Sets up VFS
StandardFileSystemManager manager = (StandardFileSystemManager) VFS.getManager();
StaticUserAuthenticator auth = new StaticUserAuthenticator(remoteUrl, myUsr, myPass);
FileSystemOptions opts = new FileSystemOptions();
DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(opts, auth);
SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, true);
// Tries to find relevant files
FileObject localFile = manager.resolveFile(localPath);
System.out.println("local found? " + localFile.exists() + "! " + localFile.getPublicURIString());
FileObject baseFile = manager.resolveFile(remotePathBase, opts);
System.out.println("base found? " + baseFile.exists() + "! " + baseFile.getPublicURIString());
manager.setBaseFile(baseFile);
FileObject remoteFile = manager.resolveFile(remotePathRelative, opts);
System.out.println("SFTP found? " + remoteFile.exists() + "! " + remoteFile.getPublicURIString());
localFile.copyFrom(remoteFile, Selectors.SELECT_SELF);
I am using:
java 8
apache vfs2 2.6.0
apache commons logging 1.2
jsch 0.1.55
I'm trying to download a file via an ftp(s) server with apache vfs.
Here is the code:
`
String fileToDownload="testdownload.txt";
FileSystem fs = null;
FileSystemOptions opts = new FileSystemOptions();
FtpFileSystemConfigBuilder.getInstance().setPassiveMode(opts, true);
FileSystemManager manager = VFS.getManager();
//
// Create local file object
String filepath = "d:\\butta\\" + "stoca.txt";
File file = new File(filepath);
FileObject localFile = manager.toFileObject(file);
FileObject remote = manager.resolveFile("ftps://user:pwd#ftp-test/DOWNLOAD/"+fileToDownload,opts);
//fs = remote.getFileSystem();
System.out.println("path is:" + remote.getName().getPath());
System.out.println("tipe is: " + remote.getName().getType());
System.out.println("uri is: " + remote.getName().getURI());
localFile.copyFrom(remote, Selectors.SELECT_SELF);
localFile.close();
remote.close();
`
I can connect seamlessy to the server,and I can retrieve path,type (which is file) and uri.
All seems correct but the copyFrom gives:
Could not copy "my filename" because it does not exist.
at org.apache.commons.vfs2.provider.AbstractFileObject.copyFrom(AbstractFileObject.java:271)
I'm running on windows and I can download the file via Filezilla or winscp without problems
I was facing the similar issue. Resolved it by setting user directory as root
FtpFileSystemConfigBuilder.getInstance( ).setUserDirIsRoot(opts,true);
The original answer and explanation is given here - https://stackoverflow.com/a/16358024/2845937
Spring boot application. User selects server of their choice and a file on their local system and the file gets SFTP to that server.
This works by: input stream reading the file into a folder within the root project. The sftp method needs the file path to work. Since I'm unable to get the file path of the file on the users system and just get the file, I bring it locally and then supply the path from there.
This work perfectly when running my project from intellij and NOT so perfectly when packaged as a jar.
After some reading, it looks like it's not great to bring in a file into a jar. Any ideas on a route to take now? Thanks
public void SFTP(MultipartFile file, String friendlyName, String filePath) throws Exception {
String host = "10.3.152.245"; //host:IP
String user = "cloud";
String password = "Password1";
System.out.println("File path: " + filePath);
try {
String fileName = renameLocal(file);
java.util.Properties config = new java.util.Properties();
JSch jsch = new JSch();
Session session = jsch.getSession(user, host, 22);
session.setPassword(password);
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect();
ChannelSftp sftpChannel = (ChannelSftp) session.openChannel("sftp");
sftpChannel.connect();
sftpChannel.cd("/");
sftpChannel.cd(filePath);
sftpChannel.put("\\upload-dir\\" + fileName, ".");
sftpChannel.rename(filePath + "/" + fileName, filePath + "/" + file.getOriginalFilename());
session.disconnect();
sftpChannel.disconnect();
Path path = FileSystems.getDefault().getPath(".\\upload-dir\\" + fileName);
Files.delete(path);
}catch (Exception e){
throw new Exception();
}
}
the method above is the SFTP method that's called when a user uploads a file. Static host user and pass for now. As you see, I'm putting the file locally (method not here) in the upload-dir. Then providing upload-dir\filename to the sftpChannel.put method
Before downloading a file needs check whether the file(file name starts) exist or not:
I had a ftp location, it will generate a file in response to hitting a service(API). I need to check whether file exits or not in ftp location using a starting characters of file name because it will append some data at end of file name.
Can any one help on this using java code with commons.net package
Use FTPFileFilter
try {
String filePattern = "prefix";
FTPClient objFTPClient = new FTPClient();
//objFTPClient - set username, password, host, etc...
FTPFileFilter ftpFileFilter = new FTPFileFilter() {
#Override
public boolean accept(FTPFile ftpFile) {
return ftpFile.getName().toLowerCase().startsWith(filePattern.toLowerCase());
}
};
/* List of file that starts with your given prefix */
FTPFile[] ftpFiles = objFTPClient.listFiles(remoteDirectory, ftpFileFilter);
}catch(Exception ex){
ex.printStackTrace();
}finally{
//close connection, etc....
}
I would like to ask if there is a way how to check if file already exists in the folder using only Apache Commons.
I have method which uploads into the SFTP folder but it overwrites current files anytime the method is running. The method is set to run every 5 minutes. I need a code which will create and if statement which checks if the file is not at the SFTP location already and then, if not executes my copy method, if there is a file, then skips it.
My copy method looks like this
private void copyFileSFTP(File model, String hour) throws IOException {
StandardFileSystemManager manager = new StandardFileSystemManager();
String dest = String.format("%s/%s/model/%s", destinationPath, hour,
model.getName());
remoteDirectory = String.format("%s/%s/model/", destinationPath, hour);
try {
if (!model.exists())
LOG.error("Error. Local file not found");
// Initializes the file manager
manager.init();
// Setup our SFTP configuration
FileSystemOptions opts = new FileSystemOptions();
SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(
opts, "no");
SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts,
false);
SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 10000);
// Create the SFTP URI using the host name, userid, password, remote
// path and file name
String sftpUri = "sftp://" + userId + ":" + password + "#"
+ serverAddress + "/" + remoteDirectory + model.getName();
**HERE I NEED THE CHECK IF THE MODEL EXISTS ALREADY ON SFTP**
// Create local file object
FileObject localFile = manager.resolveFile(model.getAbsolutePath());
// Create remote file object
FileObject remoteFile = manager.resolveFile(sftpUri, opts);
// Copy local file to sftp server
remoteFile.copyFrom(localFile, Selectors.SELECT_SELF);
LOG.info("File upload successful");
LOG.info("New file has been created.");
LOG.info(dest);
} catch (Exception ex) {
LOG.error(ex);
handleBadPath(model, hour);
} finally {
manager.close();
}
}
Thank you for help.
Use FileObject.exists() method.
See https://commons.apache.org/proper/commons-vfs/commons-vfs2/apidocs/org/apache/commons/vfs2/FileObject.html#exists--