I've got a simple task for now: connect to a remote server and get list of files and their info (in particular date of creation).
Tried JSch, but it's like writing unix app 20 years ago. Would like to switch to sshj so if it's possible, please provide some code on how to achieve at least file listing and their info (ideally, I would like to get an array of File objects).
So how can I achieve the goal?
Thanks in advance.
NOTE: AFAIU it's only possible by having ls on server side and parsing it, isn't it?
They have examples bundled with their source distribution. Did you look a them? I found this in 2 minutes: sshj: how to execute remote command example
Edit:
Ok, you could execute for instance (basing on the example I linked):
final Command cmd = session.exec("ls -l /some/interesting/dir");
String lsOutput = cmd.getOutputAsString();
// parse lsOutput and extract required information
...
There is no simplier way if you want to do it over ssh, because it has no notion of files etc. It is just a remote shell. Maybe sftp could provide some better interface here, but I am no expert with sftp.
Here is the code for sftp (JSCH)
ChannelSftp sftp = (ChannelSftp)session.openChannel("sftp");
sftp.connect();
sftp.cd(DIRECTORY);
Vector v = null;
v = sftp.ls("*.txt"); //txt files only
Use with keyfile:
JSch jsch = new JSch();
jsch.setKnownHosts(myKonfig.getKnownHostsFile());
String privKeyFile = myKonfig.getPrivateKeyFile();
jsch.addIdentity(privKeyFile);
Oops, just saw that it doesn't return the creation time, just the modification time.
If you're just looking to get file information from the remote system, I would recommend using the SFTPClient class that's provided within sshj.
use the:
SFTPClient.ls(directory)
command to find all the remote files, then use the:
SFTPClient.stat(file)
to get all the information from the remote files including the date of modification.
Related
i try to connect to sftp server in my local machine i generate the knownHosts file with the command ssh
and i use it like jsch.setKnownHosts(knownHosts);
but i would to run my job in other machine wish i didn't have access to his knownHosts file
so i decided to disable the check of this rsa key and i wont to know if the action is safe
i will use this line to disable it
session.setConfig("StrictHostKeyChecking", "no");
The question itself has a comment about its safety, but I would like to add that if you don't want to have the StrictHostKeyChecking set as "no", and do not want to rely on a knownHosts file, I would recommend you to:
1) Generate the host fingerprint in a way that's compatible with Jsch, please refer to this question. You can output the generated fingerprint to another file or elsewhere as needed.
2) You can then get the generated value and store it as a variable (environment variable, config file, property, etc.) so that your application can use it. You can pass this fingerprint (not filepath) to Jsch with the setKnownHosts method.
I am using JSch in a Java client to connect to a remote server and get some files using SFTP. The following code has been working fine for me: -
JSch ssh = new JSch();
JSch.setConfig(FileTransferConstants.STRICT_HOST_KEY_CHECKING, FileTransferConstants.NO);
Session session = ssh.getSession(userName, host, port);
session.setPassword(password);
session.connect();
Channel channel = session.openChannel(FileTransferConstants.SFTP);
channel.connect();
ChannelSftp sftp = (ChannelSftp) channel;
sftp.cd(remoteDirectoryPath);
sftp.lcd(localDirectoryPath);
sftp.get(remoteDirectoryPath + remoteFileName, remoteFileName);
The problem is that there has now been a change of site policy. I am no longer permitted to log on directly as this user (userName above). I must first log on as my personal user and then su into the user that has access to the files I want to SFTP.
I don't think there is anyway I can refactor the above code to achieve this and so I have started looking at using a shell or exec channel instead. I have had little success so far and cannot seem to find any examples on the web, so I would be very grateful for any advice or pointers in the right direction. Many thanks.
I do not think you can do this directly with JSch. But with some modification of its code, it's probably doable.
Note that my answer assumes that the server is *nix-based (what is backed by your reference to su) and uses OpenSSH SFTP server.
You have to open SSH "exec" channel, to execute something like:
sudo /bin/sftp-server
But on top of that channel, you need to build the ChannelSftp instance, not ChannelExec.
So you will need to implement Session.openChannel-like method, that will open exec channel, but create ChannelSftp for it.
For some background, see how it's possible to do sudo with WinSCP SFTP client.
Note that while the FAQ claims, that you won't be able to use password for the sudo, that's true for WinSCP. But as you have a full control of the session with JSch, you may be able to feed the password to sudo.
For that you might override the ChannelSftp.start() to write the password to the channel input, before starting the actual SFTP session.
You still need the requiretty option be off, as the SFTP cannot work with TTY.
For general considerations when automating operations using a different/root account, see:
Allowing automatic command execution as root on Linux using SSH
I want to transfer file from FTP server to HDFS. I tried this method: FTP to HDFS, the demo code as follows:
Configuration conf = new Configuration();
FTPFileSystem ftpfs = new FTPFileSystem();
ftpfs.setConf(conf);
ftpfs.initialize(new URI(ftpConnectStr), conf);
Path homeDirectory = ftpfs.getHomeDirectory();
System.out.println(homeDirectory.toString());
FileStatus[] fileStatuses = ftpfs.listStatus(new Path("/"));
for(FileStatus fileStatus : fileStatuses){
System.out.println(fileStatuses.length);
System.out.println(fileStatus.toString());
}
boolean test = ftpfs.mkdirs(new Path("test"));
System.out.println(test);
The ftpfs.listStatus(new Path("/")) doesn't work, it shows nothing, but the FTP server has two directories and ftpfs.mkdirs(new Path("test")) work fine, the program running result as follows:
and the FTP server dirctory as follows：
I searched in google，but find a little information. I don't know why. If you can help me, I will be very grateful，thanks
As you have found out, the problem is because Hadoop (or rather the underlying Apache Common Net FtpClient) defaults to the FTP active mode, which hardly works nowadays due to ubiquitous NATs and firewalls.
Since Hadoop 2.9, you can set up FTP passive mode by setting fs.ftp.data.connection.mode configuration option to enable the FTP passive mode:
fs.ftp.data.connection.mode=PASSIVE_LOCAL_DATA_CONNECTION_MODE
See https://issues.apache.org/jira/browse/HADOOP-13953
Finally,I found the problem where it is; In FTP Server,the data tansfer mode is set to passive.
Then I debuged the source code of FTPFileSystem,and i found that it didn't set FTP passive mode;
so,I modify the related code of FTPFileSystem to this:
Rerun the program：
and it works fine:
I want to read the content of a remote directory using java.
The directory is on a machine running Ubuntu. Right clicking on the folder should give the share folder option and its installed samba client for windows sharing, but I don't have any machine running Windows.
I'm looking for a java api library to access the remote directory content?
User will only provide username, password, ip and folder name.
eg [//172.17.0.1/sharefolder/demo/]
Thanks.
For a Samba Share:
Even SAMBA shares in linux use the same protocol as windows shares.
So the post here can help: How can I mount a windows drive in Java?
Basically, you could mount the shared location as a network drive using "net use" command .
You could call this either through windows console, or through a java Process.
For a SFTP location:
If you don't have a problem with calling/using an external command you could use sshfs (either out of java or through Process) to mount the remote directory into a local folder.
See: http://numberformat.wordpress.com/2010/03/01/how-to-mount-a-remote-ssh-filesystem-using-sshfs/
If you want pure java on how to access SFTP,I read that a library called JSch can be used to access SFTP directly from java. See:
http://chrisjordan.ca/post/15052396308/java-sftp
http://jcraft.com/jsch/examples/Sftp.java.html
If it's another type please specify
You might find the The Java CIFS Client Library having the API you need - it is useful for both server and client.
Here is an example taken from their documentation to retrieve a file:
import jcifs.smb.*;
jcifs.Config.setProperty( "jcifs.netbios.wins", "192.168.1.220" );
NtlmPasswordAuthentication auth = new NtlmPasswordAuthentication("domain", "username", "password");
SmbFileInputStream in = new SmbFileInputStream("smb://host/c/My Documents/somefile.txt", auth);
byte[] b = new byte[8192];
int n;
while(( n = in.read( b )) > 0 ) {
System.out.write( b, 0, n );
}
For SFTP consider using JSCAPE's Secure FTP Factory. Documentation with code examples can be found here.
jsch-nio is a fully functional unix/linux java FileSystemProvider over ssh.
I know there is a lot of thread about this problem but I dont found right: I follow this example: Want to invoke a linux shell command from Java to run command. Problem with ssh command is in authentication. When I run it I need to set password
$ ssh root#server 'fgrep Exception *.log*'
Enter passphrase for key '/././.ssh/id_rsa':
How can I pass here password ?
There are libraries are available to invoke ssh. The Java Secure Channel (JSCH) is a very popular library, used by maven, ant and eclipse. It is open source with a BSD style license.
If you need authentication for ssh, you can use through java.
If your still need to by pass password passing, there are two ways to do what you want. One involves a stored password, and one does not.
Both are non-interactive, meaning that they can work when you're not there to enter a password.
The way that does not require a password. You can use public/private
key authentication instead of passwords with SSH. I'm going to
assume that you're using OpenSSH, which comes with practically every
Linux distribution.
Steps :
Configure your SSH server to accept private key logins. In /etc/ssh/sshd_config make sure that there's a line that says PubkeyAuthentication yes (and that there is no # infront of it). If you change this file, you need to restart the sshd service.
On your local machine (not the server), create yourself a pair of keys with ssh-keygen -t rsa (you can use other options than rsa, but I'm keeping it simple). Do not specify a password. Save the keys in the locations prompted.
Open the contents of the id_rsa.pub file that you just created (it's one very long line of text), and copy the contents into the end of the file $HOME/.ssh/authorized_keys on the server machine. Create the file if it doesn't exist.
Further Detail refer here.
The basic idea is to use expect, which is an
administration automation tool, to type your password in to ssh when
prompted. It might not always work, and when it doesn't, it's hard
to figure out why not. I recommend the first method.
Anyway, here's a command that you can poke at until it does what you
want.
The script Code is:
expect -c 'spawn ssh user#remote.host ; expect assword ; send "passphrase\n" ; interact'
Expect might not be installed on your system. Make sure install that
You need to get the InputStream (which has the output) from the execution and wait for it to ask you for the password, then get the OutputStream (into which you give the command its input) and send it the password you want.
Have a read of this article