JSch not uploading complete file to remote SFTP server, only partial - java

I am trying to use the Jsch library to transfer a locally created XML file (marshalled from a Java object using JAXB) to a remote server. However, the file only gets partially uploaded. It is missing the end tag and an arbitrary amount of characters at the end.
My code looks like this (TradeLimits is a JAXB annotated Java class)
TradeLimits limits = getTradeLimits(); //complex object with many fields
JSch jsch = new JSch();
jschSession = jsch.getSession(username, remoteHost);
//to avoid unknown host issues
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
jschSession.setConfig(config);
jschSession.setPassword(password);
jschSession.setPort(22);
jschSession.connect();
ChannelSftp channelSftp = (ChannelSftp) jschSession.openChannel("sftp");
channelSftp.connect();
jaxbContext = JAXBContext.newInstance(TradeLimits.class);
Marshaller marshaller = jaxbContext.createMarshaller();
marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true); //for pretty print
marshaller.marshal(limits, channelSftp.put(limitUploadPathString)); //this uploads only partial xml file to sftp server
marshaller.marshal(limits, System.err)); //THIS WORKS CORRECTLY AND THE FULL XML IS PRINTED!
channelSftp.disconnect();
channelSftp.exit();
Note how this cannot be a JAXB issue because it will print the complete XML elsewhere, but only partial is uploaded to remote server. What can possibly be the issue? Thanks in advance!

Always ensure you flush/close an OutputStream when you are done writing to it.
try(OutputSteam fileStream = channelSftp.put(limitUploadPathString)) {
marshaller.marshal(limits, fileStream);
}

Related

How to get server's SSH version without authenticating in JSch?

I tried to use JSch:
#Test
public void test() throws Exception {
var session = new JSch().getSession("host");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
var version = session.getServerVersion();
System.out.println("version = " + version);
}
but I'm getting
java.lang.NullPointerException: Cannot read the array length because "str" is null
It seems that this implementation requires you to connect and authenticate.
How can I get this information with Java, without the need to authenticate?
This Python code works for me on a server that requires authentication yet I didn't have to authenticate in order to get the remote version.
import paramiko
ssh = paramiko.Transport(("host", 22))
ssh.start_client()
print(ssh.remote_version)
ssh.close()
The SSH identification string is sent in plain text right after opening the connection.
So you do not need an SSH library to obtain the string.
You can do with a simple code like shown here:
Read Data from a Java Socket
Or just try to login using dummy credentials with JSch and then read Session.getServerVersion.
You do not even have to try to login. You can abort the connection right at the host key check:
config.put("StrictHostKeyChecking", "yes");

Why is Jsch asking for Kerberos username? [duplicate]

So I'm writing a little program that needs to connect to a remote server through SFTP, pull down a file, and then processes the file. I came across JSch through some answers here and it looked perfect for the task. So far, easy to use and I've got it working, with one minor thing I'd like to fix. I'm using the following code to connect and pull the file down:
JSch jsch = new JSch();
Session session = null;
try {
session = jsch.getSession("username", "127.0.0.1", 22);
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword("password");
session.connect();
Channel channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftpChannel = (ChannelSftp) channel;
sftpChannel.cd(REMOTE_FTP_DIR);
sftpChannel.lcd(INCOMING_DIR);
sftpChannel.get(TMP_FILE, TMP_FILE);
sftpChannel.exit();
session.disconnect();
} catch (JSchException e) {
e.printStackTrace();
} catch (SftpException e) {
e.printStackTrace();
}
So this works and I get the file. I'm running this code on a linux server and when I run the code JSch asks me for my Kerberos username and password. It looks like:
Kerberos username [george]:
Kerberos password for george:
I just hit enter for both questions and then the program seems to continue on with no problems. However I need this code to be automated through a cron task and so I'd rather not having it pausing the program to ask me these two questions. Is there something I'm not supplying it so that it won't ask this? Something I need to do to stop it asking? Hopefully someone has some ideas. Thanks.
Thought I'd post an answer here since in case anyone else ends up running into a similar issue. Turns out I am missing a piece of code that makes all the difference. I just needed to add
session.setConfig("PreferredAuthentications",
"publickey,keyboard-interactive,password");
before
session.connect();
and everything works perfectly now.
While the solution in the self-accepted answer is correct, it lacks any explanation.
The problem is that the OP have a Kerberos/GSSAPI authentication set as the preferred (the JSch default). Yet OP does not seem to actually use/want it, as OP claims not to specify any username or password for the Kerberos prompts.
This problem can appear spontaneously, when either Kerberos gets installed on the the client PC or the server starts to support Kerberos.
The solution is to remove the Kerberos/GSSAPI (gssapi-with-mic) from the list of preferred authentication methods in JSch:
session.setConfig(
"PreferredAuthentications", "publickey,keyboard-interactive,password");
All answers are correct, I'll just add here the way it can be done for Spring Integration when trying to integrate with an SFTP server.
So, if you are using SFTP Spring Integration and the weird user and password for Kerberos is prompting in the same way the OP is asking.
Then modify your Spring configuration (I'm using Java Spring Integration config, if you are using XML config you can try to translate it yourself - I really don't like XML config :P ):
So in the bean you are using as SessionFactory you need to add this change in config:
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost("hostname");
factory.setPort(22);
factory.setUser("username");
factory.setPassword("superstrongpassword");
factory.setAllowUnknownKeys(true);
factory.setSessionConfig(buildSessionProperties());
return new CachingSessionFactory<>(factory);
}
/**
* Build JSch property PreferredAuthentications without "gssapi-with-mic"
* This way it won't prompt for Kerberos authentication every time it tries to connect
* to the SFTP.
*/
private Properties buildSessionProperties() {
Properties sessionProperties = new Properties();
sessionProperties.setProperty("PreferredAuthentications", "publickey,keyboard-interactive,password");
return sessionProperties;
}

How to edit the existing file on the linux server using jsch

I have a file on the linux server that contains the system date of the application.
The file name is "abc.property." and the file contains the systemdate parameter which application reads.
e.g the content of the file is below:-
1.SYSTEMDATE= 20162201 160
2.ABC
3.XYZ
4.CCC
I want to write a java program that will connect to the server using JSCH and allow me to change the system date through the application.
I already have a code to connect to the server and read the content.
ChannelSftp sftpChannel = (ChannelSftp) session.openChannel("sftp");
session.sendKeepAliveMsg();
sftpChannel.connect();
session.sendKeepAliveMsg();
sftpChannel.cd("path");
InputStream out= null;
out= sftpChannel.get(remoteFile);
BufferedReader br = new BufferedReader(new InputStreamReader(out));
String line;
session.sendKeepAliveMsg();
System.out.println(workingDir);
Can any one help me on this?? i am new to jsch programming.
Thank you in advance.

SFTP: IOException while reading a file with java

I am using com.jcraft.jsch library to read .xls files from an SFTP server. Following is the code to connect to server.
session = jsch.getSession(username, host);
session.setConfig("StrictHostKeyChecking", "no");
session.setPassword(password);
session.connect();
sftpChannel = (ChannelSftp) session.openChannel("sftp");
sftpChannel.connect();
I am using sftpChannel.get(file) to retrieve inputStream to the file. This inputstream is then used to instantiate XSSFWorkbook as shown below:
XSSFWorkbook workbook = new XSSFWorkbook(in);
Problem 1:
When I run the app, it seems to get stuck on the above line for some time (say 5 minutes) and then it throws java.io.IOException: Pipe closed error.
The xls file I am trying to read is 800kb and it works fine when run from local machine.
Problem 2:
The app is designed to process files sequentially. So, if first file fails with IOE, rest of the files also fail as the connection is timed out. To prevent this, I put the below code to check and re-connect:
if(null == session || !session.isConnected()){
log.debug("Session is not connected/timed out. Creating a new session");
openSftpSession();
log.debug("New session is created");
}
//openSftpSession() is the code to create a new session as explained in the beginning of the question.
When this code gets executed, following exception gets thrown:
java.io.IOException: error: 4: RequestQueue: unknown request id 1028332337
at com.jcraft.jsch.ChannelSftp$2.read(ChannelSftp.java:1407)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.PushbackInputStream.read(PushbackInputStream.java:186)
//More lines
edit : code to retrieve input stream
public InputStream getInputStream(String folder, String file) throws Exception{
sftpChannel.cd(root + folder);
log.debug("current directory:" + sftpChannel.pwd());
log.debug("File :" + folder + " " + file);
return sftpChannel.get(file);
}
Can anyone please help me get over this? I believe an alternate approach to prevent timeout is to download the file in some temp directory and process. However, I don't really want to do that.
Thanks in advance.
Have you checked to see whether the approach you describe (download into temp file) works? Just to verify that your inputstream is ok.. How long does it take to download into a local file over that connection?
If you don't want to manage a temp file you could always pull it into a byte[] in memory, so long as you don't have to scale to much more than 800kbs.. Use Apache Commons as such:
InputStream in = sftpChannel.get(file);
byte[] inBytes = org.apache.commons.io.IOUtils.toByteArray(in)
ByteArrayInputStream inByteStream = new ByteArrayInputStream(inBytes)
XSSFWorkbook workbook = new XSSFWorkbook(inByteStream);
As for the Request Id, it looks like the old session/channel is still trying to do a read but is no longer able to. Maybe you aren't closing out that session/channel properly. From the openSftpSession() code it looks like you would only be overwriting the references without properly shutting them down.

Secure FTP using private key authentication in java

import com.jcraft.jsch.*;
import com.jcraft.jsch.JSchException;
import oracle.jdbc.driver.OracleDriver;
import java.io.*;
import java.util.*;
import java.sql.*;
import java.net.*;
public class SecureFTP {
public static void main(String[] args) throws IOException , ClassNotFoundException, JSchException, SftpException{
JSch jsch = new JSch();
File file = new File("/home/xxxxx/.ssh/id_rsa");
Session session = null;
URL keyFileURL = null;
URI keyFileURI = null;
if (file.exists())
{
keyFileURL = file.toURL();
if (keyFileURL == null)
{
System.out.println("what");
throw new RuntimeException("Key file not found in classpath");
}
}
else System.out.println("FIle not found");
try{
keyFileURI = keyFileURL.toURI();
}
catch(Exception URISyntaxException)
{
System.out.println("Wrong URL");
}
String privateKey = ".ssh/id_rsa";
//jsch.addIdentity(privateKey);
jsch.addIdentity(new File(keyFileURI).getAbsolutePath());
System.out.println(new File(keyFileURI).getAbsolutePath() + " LOL");
session = jsch.getSession("username", "servername");
//session.setPassword("password");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
// connect
session.connect();
// get SFTP channel
Channel channel = session.openChannel("sftp");
channel.connect();
ChannelSftp schannel = (ChannelSftp) channel;
FileInputStream fis = new FileInputStream(sourcefile);
schannel.put(fis, destinationfile );
fis.close();
}
schannel.exit();
session.disconnect();
}
}
As you can see from the code I have commented out, I have tried everything possible to get this working and the only thing that works is if I set the password directly. I am trying to use the RSA private key generated, but I keep getting an auth fail.
I have added the public key to the list of authorized keys on the target server. And there is no passphrase.
Is there something else I am supposed to do? Like say, while generating the keys? Is there a step I am missing?
Is there another library I can use to implement the same function?
Make sure the necessary files exist (id_rsa and id_rsa.pub on the client, authorized_keys on the server). Make sure you can use public key authentication with another tool, like ssh, using these files.
If that looks alright, the problem may be with your Java security provider. Read on if you think you have the right files in place.
There are different formats for RSA private key storage, and SSH uses one that is not standard. Most providers expect something called a CRT RSA key, and when JSch doesn't give them a key in that format, they raise an exception which JSch silently eats and goes on to the next authentication method.
What is your provider? The following snippet will help you find out:
import java.security.KeyFactory;
…
KeyFactory f = KeyFactory.getInstance("RSA");
System.out.println(f.getProvider().getName());
Update: I did some checking around, and as of Java 5, the SunPKCS11 provider is installed with the highest precedence on Solaris systems, for performance. Since I don't run Solaris, I can't test it, but I believe this may be causing the problem.
JSch doesn't allow you to specify the provider to use for this operation through its API, so you will have to change the precedence of the installed providers. In fact, I'd suggest trying to remove the SunPKCS11 from this application; run this code once when your application starts up:
Security.removeProvider("SunPKCS11-Solaris");
Have you have copied the key into the file $HOME/.ssh/authorized_keys on the target server? If so, you should probably mention that. If not, that is required for this to work. Also, are you generating the key without a password? If the private key is password protected, you will need to provide that password to addIdentity.
After verifying those things, I'd recommend trying to connect via the command line using OpenSSH, as the Java code you have here looks correct. If the command line does not work, invoke it with -vvv to get verbose output about what it is doing. It is possible that the server is configured with PubkeyAuthentication set to no.

Categories

Resources