Develop Reference

Java - CopyStreamException due to SSLProtocolException when storing file's contents in FTP server

I'm facing an issue when trying to store a file in an FTP server. When trying to upload a file to the FTP server, the file is created but its contents are not copied.
Context
We use the following configuration to access the FTP server using lftp. I cannot change this configuration, and don't know why do we use FTPS with verify-certificates disabled.
# FTPS_OPTIONS:
set ssl:verify-certificate/testhostname no;
set ftp:ssl-protect-data yes;
set ftp:passive-mode on;
I need to store certain files from a Java application. I'm using apache-commons library. The implemented code looks like this:
#Autowired
public FtpService() {
ftpsClient = new FTPSClient();
ftpsClient.addProtocolCommandListener(new PrintCommandListener(new PrintWriter(System.out), true));
}
public void uploadFile(String ftpHost, File tempFile, String destination, String filename)
throws UploadException {
ftpsClient.connect(ftpHost, 990);
ftpsClient.execPBSZ(0);
ftpsClient.execPROT("P");
ftpsClient.enterLocalPassiveMode();
ftpsClient.setKeepAlive(true);
ftpsClient.setControlKeepAliveTimeout(3000);
if(ftpsClient.login("user", "password")) {
try (InputStream fileStream = new FileInputStream(tempFile)) {
if (!ftpsClient.changeWorkingDirectory(destination)) {
throwUploadException("Destination directory not available in FTP server");
}
boolean saved = ftpsClient.storeFile(filename, fileStream);
// Following code is not executed since the exception is thrown in the previous line
if (!saved) {
throwUploadException("Unable to save file in FTP server");
}
log.info("Saved FTP file: {}/{}", destination, filename);
}
catch (UploadException | IOException e)
{
throwUploadException(e.getMessage());
}
finally
{
ftpsClient.disconnect();
if (!tempFile.delete()) {
log.warn("Unable to delete '{}' file", tempFile.getAbsolutePath());
}
}
}
}
Problem
I started with a FTPClient (non FTPSClient) but this way I wasn't able to login.
Currently (FTPSClient), I can:
change the working directory
create directories in the FTP server
I cannot:
storeFile: this method throws the following exception, and creates the file in the FTP server, but this is empty
org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
Cause: javax.net.ssl.SSLProtocolException: Software caused connection abort: socket write error
listFiles()/listDirectories(): when executing this command, the obtained list is always empty. The logged user has all the required permissions in the whole FTP server
Following is the FTP's log (note that I have translated the commands to English between parenthesis), corresponding to the code shown before, that raises the exception mentioned before:
er: testhostname:990
USER *******
331 Usuario testuser OK. Clave requerida ( = User testuser OK. Password required)
PASS *******
230 OK. El directorio restringido actual es / ( = OK. The current restricted directory is /)
CWD /test/upload
250 OK. El directorio actual es /test/upload ( = Ok. The current directory is /test/upload)
PASV
227 Entering Passive Mode (<numbers...>)
[Replacing PASV mode reply address <ip_address> with testhostname]
STOR dummyfile.txt
150 Conexi├│n de datos aceptada ( = Data connection accepted)
If there is anything else I can include to improve the description, please let me know. Thanks for your help!

I had a similar problem from python connecting to an FTPS server. The error was that the server required the data channel session to be the same as the control channel session(reuse the session). The solution was to override one of the methods to do that.
You can test extending FTPClient.java and overriding the next method:
#Override
protected void _prepareDataSocket_(final Socket socket) {
if(preferences.getBoolean("ftp.tls.session.requirereuse")) {
if(socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket) _socket_).getSession();
if(session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
context.setSessionCacheSize(preferences.getInteger("ftp.ssl.session.cache.size"));
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
Method getHostMethod;
try {
getHostMethod = socket.getClass().getMethod("getPeerHost");
}
catch(NoSuchMethodException e) {
// Running in IKVM
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object peerHost = getHostMethod.invoke(socket);
putMethod.invoke(cache, String.format("%s:%s", peerHost, socket.getPort()).toLowerCase(Locale.ROOT), session);
}
catch(NoSuchFieldException e) {
// Not running in expected JRE
log.warn("No field sessionHostPortCache in SSLSessionContext", e);
}
catch(Exception e) {
// Not running in expected JRE
log.warn(e.getMessage());
}
}
else {
log.warn(String.format("SSL session %s for socket %s is not rejoinable", session, socket));
}
}
}
}
I found this Java solution here: https://stackoverflow.com/a/32404418/19599290

Java, SSLSocket, receiving no answer

I'm trying to use SSL with IMAP in java. I do not want to use the IMAP class.
For some reason, when I send the n th message, I receive the answer to message n-2, and not to message n-1. Which means that I don't receive any answer to the first message sent until I send the second message. Can anyone spot what's wrong in the following minimal code ? (It is indeed minimal, apart from the println, which, I guess, help debugging)
import java.io.*;
import javax.net.ssl.*;
public class Mail{
static String server = "imap.gmail.com";
static String user = "straightouttascript#gmail.com";
static String pass = "azerty75";
public static void print (PrintWriter to, String text){
System.out.println("sent : "+text);
to.println(text+ "\r");
to.flush();
}
public static void read (BufferedReader from) throws InterruptedException, IOException {
do {
String line = from.readLine();
System.out.println("received: "+line);
} while (from.ready());
}
public static void main(String[] args){
try{
SSLSocket sslsocket = (SSLSocket) SSLSocketFactory.getDefault().createSocket(server, 993);
System.out.println("Start connexion");
BufferedReader from = new BufferedReader(new InputStreamReader(sslsocket.getInputStream()));
// read(from);
PrintWriter to = new PrintWriter(new BufferedWriter(new OutputStreamWriter(sslsocket.getOutputStream())), true);
print(to,"a1 login "+user+" "+pass);
read(from);/*exepcted:
OK gimap ready
a1 OK login#host authenticated (Success)*/
sslsocket.close();
System.out.println("End connexion");
}
catch (Exception e){
e.printStackTrace();
}
}
}

IMAP is not a pingpong protocol. The server doesn't send one line in response to one of yours.
Rather, you send commands and the server sends information. The server is permitted to send you more information than you asked for, so you can get seven responses to one command, and you can even get a response without sending a command at all, which is then called an unsolicited response. Strange phrase. Unsolicited responses are used by some servers to notify you about new mail, by more to notify you about flag changes on messages, and by (almost?) all to notify you that they're about to close your connection.

Restlet streaming with Thread.sleep()

This example is based on an example from the book Restlet in Action.
If I try
public class StreamResource extends ServerResource
{
#Get
public Representation getStream() throws ResourceException, IOException
{
Representation representation = new WriterRepresentation(MediaType.TEXT_PLAIN)
{
#Override
public void write(Writer writer) throws IOException
{
String json = "{\"foo\" : \"bar\"}";
while (true)
{
writer.write(json);
}
}
};
return representation;
}
}
it works and it continuously sends the json string to the client.
If I introduce a delay in the while loop like this
String json = "{\"foo\" : \"bar\"}\r\n";
while (true)
{
writer.write(json);
try
{
Thread.sleep(250);
}
catch (InterruptedException e)
{}
}
I was hoping that the client would get data 4 times in a second BUT nothing seems to get to the client.
Can anyone explain why the introduction of Thread.sleep() does that? What is a good way to introduce delay in streaming data to the client?

You should try with the Jetty connector instead of the internal Restlet connector. This connector isn't ready for production even though we are working on fixing it.
You can also try the Simple extension which has less dependent JARs than the Jetty extension.

You can try to flush the buffer, like this:
String json = "{\"foo\" : \"bar\"}\r\n";
while (true)
{
writer.write(json);
writer.flush(); // flush the buffer.
try
{
Thread.sleep(250);
}
catch (InterruptedException e)
{}
}
Without writer.flush(), the writer waits to fill the internal buffer before writing the socket. Thread.sleep(250) reduces the output produced at each second, so that far more time is required to fill the buffer.

nearest equivalent of a webpage reload in java

I am taking some data from a database via a servlet and a db handler java class and hosting it at a url. Since the database is changing I'm taking care only to host the changes rather than the entire db data.
I'm getting the required functionality by a browser i.e after every (manual) reload, I'm getting the data as required by me,
1. at the first page load, entire data gets displayed.
2. at subsequent reloads, I get either null data if there is no change in the database, or the appended rows if the database extends. (the database can only extend).
But then in a java program, I'm not getting the same functionality. The java program using HttpUrlConnection.
This is the code for the java client for servlet...
public class HTTPClient implements Runnable {
private CallbackInterface callbackinterface;
private URL url;
private HttpURLConnection http;
private InputStream response;
private String previousMessage = "";
public HTTPClient() {
try {
url = new URL("http://localhost:8080/RESTful-Server/index.jsp");
http = (HttpURLConnection) url.openConnection();
http.connect();
} catch (IOException e) {
}
}
#Override
public void run() {
while (true) {
try {
String currentmessage = "";
response = http.getInputStream();
if (http.getResponseCode() == HttpURLConnection.HTTP_OK) {
BufferedReader buffread = new BufferedReader(new InputStreamReader(response));
String line;
for (; (line = buffread.readLine()) != null;) {
currentmessage += line;
}
if ((!currentmessage.equals(previousMessage)
|| !previousMessage.equals(""))
&& !currentmessage.equals("")) {
//this.callbackinterface.event(currentmessage);\
System.out.println(currentmessage + "\t" + previousMessage);
}
previousMessage = currentmessage;
Thread.sleep(2500);
} else {
throw new IOException();
}
} catch (IOException | InterruptedException e) {
System.err.println("Exception" + e);
}
}
}
The shown class is a thread which read the connections every 2.5 s. If it gets something significant in the getline(), it will issue a callback to a worker method, which takes care of remaining things.
I am thinking the issues is because of the class variable conn, and that reload as in the browser is not getting replicated..
Any idea how to do this?

You're basically connecting (requesting) only once and trying to read the response multiple times, while it can be read only once. You basically need to create a new connection (request) everytime. You need to move the creation of the connection by url.openConnection() to inside the loop. The line http.connect() is by the way superfluous. You can safely omit it. The http.getInputStream() will already implicitly do it.
See also:
Using java.net.URLConnection to fire and handle HTTP requests

How to read response from remote server using Java Sockets

Part of a Java program I'm creating needs to talk to a service on a remote machine. That remote machine is running a service (written in Delphi I believe) on a Windows platform.
I need to connect to that machine, send command strings and receive (String) responses.
If I connect using Linux CLI telnet session I get responses as expected:
[dafoot#bigfoot ~]$ telnet [host IP] [host port]
Trying [host IP]...
Connected to [host IP].
Escape character is '^]'.
Welcome to MidWare server
ping
200 OK
ProcessDownload 4
200 OK
In the above the lines 'ping' and 'ProcessDownload 4' are me typing in the terminal, other lines are responses from remote system.
I created a Main in my Java class that will do the work to call the appropriate methods to try and test this (I've left out irrelevant stuff):
public class DownloadService {
Socket _socket = null; // socket representing connecton to remote machine
PrintWriter _send = null; // write to this to send data to remote server
BufferedReader _receive = null; // response from remote server will end up here
public DownloadServiceImpl() {
this.init();
}
public void init() {
int remoteSocketNumber = 1234;
try {
_socket = new Socket("1.2.3.4", remoteSocketNumber);
} catch (IOException e) {
e.printStackTrace();
}
if(_socket !=null) {
try {
_send = new PrintWriter(_socket.getOutputStream(), true);
_receive = new BufferedReader(new InputStreamReader(_socket.getInputStream()));
} catch (IOException e) {
e.printStackTrace();
}
}
}
public boolean reprocessDownload(int downloadId) {
String response = null;
this.sendCommandToProcessingEngine("Logon", null);
this.sendCommandToProcessingEngine("ping", null);
this.sendCommandToProcessingEngine("ProcessDownload", Integer.toString(downloadId));
try {
_socket.close();
} catch (IOException e) {
e.printStackTrace();
}
return false;
}
private String sendCommandToProcessingEngine(String command, String param) {
String response = null;
if(!_socket.isConnected()) {
this.init();
}
System.out.println("send '"+command+"("+param+")'");
_send.write(command+" "+param);
try {
response = _receive.readLine();
System.out.println(command+"("+param+"):"+response);
return response;
} catch (IOException e2) {
e2.printStackTrace();
}
return response;
}
public static void main(String[] args) {
DownloadServiceImpl service = new DownloadServiceImpl();
service.reprocessDownload(0);
}
}
As you will see in the code, there are a couple of sys.outs to indicate when the program is attempting to send/receive data.
The output generated:
send 'Logon(null)'
Logon(null):Welcome to MidWare server
send 'ping(null)'
So Java is connecting to the server ok to get the "Welcome to Midware" message back, but when I try to send a command ('ping') I don't get a response.
So the questions:
- does the Java look about right?
- could problem be related to character encoding (Java -> windows)?

You need to flush the output stream:
_send.write(command+" "+param+"\n"); // Don't forget new line here!
_send.flush();
or, since you create a auto-flushing PrintWriter:
_send.println(command+" "+param);
The latter has the disadvantage that the line end can be \n or \r\n, depending on the system on which your Java VM runs. So I prefer the first solution.