Develop Reference

How to establish a FTPS data connection to a FileZilla Server 1.2.0

It is a known problem to use the Java FTPSClient of Apache commons-net with session resumption. Session resumption is a security feature which a FTPS server can require for data connections. The Apache FTPSClient does not support session resumption, and the JDK APIs make it hard to build a custom implementation. There are a couple of workarounds using reflection, see e.g. this answer and this commons-net bug entry.
I use such a workaround (see snipped below) in JDK 11 and tested it against a local FileZilla Server. It works with FileZilla Server 0.9.6, but it doesn't with FileZilla Server 1.2.0, which is the latest version at the time of writing. With that version, when trying to establish a data connection, the server responds with:
425 Unable to build data connection: TLS session of data connection not resumed.
As I said, FileZilla Server 0.9.6 is fine with how I do session resumption, and I made sure that the setting for requiring session resumption is activated.
In FileZilla Server 1.2.0, such settings are now set implicitly and cannot be changed via the GUI, maybe not at all. Are there some server settings that I can tweak for this to work? Or is it an issue with how I implemented the workaround? Does anyone experience similar issues?
This is the workaround I am using:
public class FTPSClientWithSessionResumption extends FTPSClient {
static {
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", "false");
}
#Override
protected void _connectAction_() throws IOException {
super._connectAction_();
execPBSZ(0);
execPROT("P");
}
#Override
protected void _prepareDataSocket_(Socket socket) throws IOException {
if (useSessionResumption && socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket)_socket_).getSession();
if (session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
Method getHostMethod;
try {
getHostMethod = socket.getClass().getMethod("getPeerHost");
}
catch (NoSuchMethodException e) {
// Running in IKVM
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object peerHost = getHostMethod.invoke(socket);
InetAddress iAddr = socket.getInetAddress();
int port = socket.getPort();
putMethod.invoke(cache, String.format("%s:%s", peerHost, port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostName(), port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostAddress(), port).toLowerCase(Locale.ROOT), session);
}
catch (Exception e) {
throw new IOException(e);
}
}
else {
throw new IOException("Invalid SSL Session");
}
}
}
}
The address under which the socket is cached is determined using getPeerHost, getInetAddress().getHostName(), and getInetAddress().getHostAddress(). I tried several combinations of doing or not doing these three, but I always get the same result.
Edit:
Here is a screenshot of the server logs of the full session:

As stated in this StackOverflow post it is possible to tell the JVM that only TLS 1.2 should be used.
Here is the link to the original answer which worked for me: command for java to use TLS1.2 only
You have to add a command line parameter at the start of the JVM in this case this is: java -Djdk.tls.client.protocols=TLSv1.2 -jar ... <rest of command line here>
This simple parameter worked for me, now I can connect and transfer data from a FTP-Server wich runs FileZilla FTP-Server 1.3.0

SSLSocket duplex close failed

I am getting following exception:
javax.net.ssl|DEBUG|79|Keep-Alive-Timer|2021-03-29 23:34:12.355 PDT|SSLSocketImpl.java:479|duplex close of SSLSocket<br>
javax.net.ssl|DEBUG|79|Keep-Alive-Timer|2021-03-29 23:34:12.356 PDT|SSLSocketImpl.java:1569|close the underlying socket<br>
javax.net.ssl|DEBUG|79|Keep-Alive-Timer|2021-03-29 23:34:12.356 PDT|SSLSocketImpl.java:1588|close the SSL connection (initiative)<br>
javax.net.ssl|DEBUG|79|Keep-Alive-Timer|2021-03-29 23:34:12.356 PDT|SSLSocketImpl.java:727|close inbound of SSLSocket<br>
javax.net.ssl|WARNING|79|Keep-Alive-Timer|2021-03-29 23:34:12.356 PDT|SSLSocketImpl.java:500|SSLSocket duplex close failed (
"throwable" : {
java.net.SocketException: Socket is closed
at java.base/java.net.Socket.shutdownInput(Socket.java:1538)
at java.base/sun.security.ssl.BaseSSLSocketImpl.shutdownInput(BaseSSLSocketImpl.java:216)
at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:742)
at java.base/sun.security.ssl.SSLSocketImpl.bruteForceCloseInput(SSLSocketImpl.java:692)
at java.base/sun.security.ssl.SSLSocketImpl.duplexCloseOutput(SSLSocketImpl.java:553)
at java.base/sun.security.ssl.SSLSocketImpl.close(SSLSocketImpl.java:485)
at java.base/sun.net.www.http.HttpClient.closeServer(HttpClient.java:1058)
at java.base/sun.net.www.http.KeepAliveCache.run(KeepAliveCache.java:183)
at java.base/java.lang.Thread.run(Thread.java:834)
at java.base/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)
}
I am really puzzled about where is this error came out.

I experienced this behavior with openJDK11 and above, it seems due to the new implementation of SSL/TLS channel in Java 11 in order to support TLSv1.3. The error it is noticed when it is called a server that supports also TLSv1.3, the JVM tries to open a channel using this protocol with no luck, evidently it is not fully supported yet.
I found a workaround (useful in OpenJDK11) by forcing TLSv1.2, just add the following options to JVM at application startup: -Djdk.tls.client.protocols=TLSv1.2 -Dhttps.protocols=TLSv1.2
Notice that this solution does not always work, it depends also by your java code and used libs.
A second way is to downgrade to Jdk1.8, that in default builds does not support TLSv1.3, but it is not always possible.
If you want to investigate more in detail the TLS handshake add also the following JVM parameter at startup: -Djavax.net.debug=ssl (for more details set it to all)
Edit
I initially experienced this error in Jenkins on openJDK11 during updates in "add-on manager" from 02/2021, when the repository added the support to TLSv1.3 (https://updates.jenkins.io). During the investigation of the issue I found and tried with jdk11 the following code from jenkins issue report:
public class DownloadWebpageExample {
public static void main(String[] args) {
try {
URL url = new URL("https://updates.jenkins.io/download/plugins/plugin-util-api/1.2.5/plugin-util-api.hpi");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
readStream(con.getInputStream());
// Give output for the command line
} catch (Exception e) {
e.printStackTrace();
}
}
private static void readStream(InputStream in)
{
char[] buf = new char[1024];
try (BufferedReader reader = new BufferedReader(new InputStreamReader(in));) {
while (reader.read(buf, 0, 1024) != -1) { }
} catch (IOException e) {
e.printStackTrace();
}
}
}
by running it with java -Djavax.net.debug=ssl DownloadWebpageExample it gives the error as above:
javax.net.ssl|DEBUG|01|main|2021-06-11 12:40:59.499 CEST|SSLSocketImpl.java:636|close inbound of SSLSocket
javax.net.ssl|WARNING|01|main|2021-06-11 12:40:59.501 CEST|SSLSocketImpl.java:494|SSLSocket duplex close failed (
"throwable" : {
java.net.SocketException: Socket is closed
at java.base/java.net.Socket.shutdownInput(Socket.java:1521)
By executing it with java -Djdk.tls.client.protocols=TLSv1.2 -Dhttps.protocols=TLSv1.2 -Djavax.net.debug=ssl DownloadWebpageExample does not solve the problem.I tried this code with many jdk versions and I found that jdk8 and openJDK9 are not affected by this behaviour, but it is in openJDK11-16.
In Jenkins I noticed a different behaviour, by adding the parameters in /etc/sysconfig/jenkins file like the following solved the problem with openJDK11. For me it is the confirmation that the error it is also dependent by source code implementation:
JENKINS_JAVA_OPTIONS="-Djava.awt.headless=true -Djdk.tls.client.protocols=TLSv1.2 -Dhttps.protocols=TLSv1.2"

Was looking for the same thing, found https://bugs.openjdk.java.net/browse/JDK-8255148 which indicates that this logging is misleading.

How to start a Mysql server in Java programatically?

My Java application needs to connect to a Mysql database file which is present on disk.
For that, it needs to start a Mysql server and use the server to read the file. The problem is that I am not sure how to start the server from within my Java code, read the Mysql file, make modifications to it, and then stop the server.
Currently, I am using something like below, but I don't see a way to start the server from code.
MysqlDataSource dataSource = new MysqlDataSource();
dataSource.setUser("root");
dataSource.setPassword("root");
dataSource.setServerName("MysqlServer");
Connection conn = dataSource.getConnection();
Using this, I run into
Exception in thread "main" java.lang.RuntimeException: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
which I think is because the server is not up. How do I start the server from code?

Option 1 : Find out how to start mysql from command line. Then use ProcessBuilder to do that
Option 2: Use embeddable version of mysql
, if that suits you : See details

you need to run 'mysqld.exe' to start the mysql server. This file can be found in the 'bin' folder in the mysql main folder (Assuming that you are working in windows). In my case 'mysql-5.6.10-win32\bin\mysqld.exe' .
From Java if you want to do the same - I used the following code and it worked. Hope it helps.
======================================================
String command = "~path~\\mysql-5.6.10-win32\\bin\\mysqld.exe";
try
{
Process process = Runtime.getRuntime().exec(command);
}
catch (IOException e)
{
e.printStackTrace();
}

Look into Runtime.exec(), which will allow you to start mySQL via console.
http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Runtime.html

Runtime runTime = Runtime.getRuntime();
Process p = runTime.exec(String[] cmdarray, String[] envp, File dir);
p.waitFor();

Easiest way is to use JDBC all you need to do then is send queries to the server and interpret the results. A simple way to connect is below.
import java.sql.*;
Connection conn;
private void connect()
{
try
{
Class.forName("com.mysql.jdbc.Driver").newInstance();
conn = DriverManager.getConnection("jdbc:mysql://serverURL","user","pass);
System.out.println("Connection Established");
}
catch (Exception ex)
{
System.err.println("Unable to load the MySQL driver.");
ex.printStackTrace();
}

FTP error while downloading/uploading with ftp4j library in java application

I want to download a file in java application and when I try it, it creates the file on my hard drive but then fails to download it completely. I am using the ftp4j library to do it.
import it.sauronsoftware.ftp4j.*;
public class Main {
public static void main (String args[]){
FTPClient client = new FTPClient();
try{
client.connect("ftp.myaddress.comlu.com");
client.login("username", "password");
System.out.println("Connection created");
client.download("public_html/ZScreen.png", new java.io.File("d:/xxx/ZScreen.png"));
System.out.println("Download successful");
client.disconnect(true);
}
catch (Exception FTPException){
System.out.println("Shit hit the fan");
}
}
}
I always get the Connection created and Shit hit the fan. Also, there is a file created on my hard drive but it's size is 0 bytes.
This is the stack race
Connection created
java.net.SocketException: Connection reset
Shit hit the fan
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
at sun.nio.cs.StreamDecoder.read0(StreamDecoder.java:126)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:112)
at java.io.InputStreamReader.read(InputStreamReader.java:168)
at it.sauronsoftware.ftp4j.NVTASCIIReader.readLine(NVTASCIIReader.java:105)
at it.sauronsoftware.ftp4j.FTPCommunicationChannel.read(FTPCommunicationChannel.java:142)
at it.sauronsoftware.ftp4j.FTPCommunicationChannel.readFTPReply(FTPCommunicationChannel.java:187)
at it.sauronsoftware.ftp4j.FTPClient.openPassiveDataTransferChannel(FTPClient.java:3538)
at it.sauronsoftware.ftp4j.FTPClient.openDataTransferChannel(FTPClient.java:3473)
at it.sauronsoftware.ftp4j.FTPClient.download(FTPClient.java:3302)
at it.sauronsoftware.ftp4j.FTPClient.download(FTPClient.java:3213)
at it.sauronsoftware.ftp4j.FTPClient.download(FTPClient.java:3078)
at Main.main(Main.java:9)

Apparently there is a bug on the Windows 7 firewall related to using FTP on IPv6 that would explain your problem. See bug report here.
Any one of the following workarounds should suffice to fix it:
Run the following as an administrator in a Windows console:
netsh advfirewall set global StatefulFtp disable
Run the JVM with the option: -Djava.net.preferIPv4Stack=true

You do not have the rights to write on the folder. Check if the repertory is not under "read-only" state.

A problem connecting to a MySQL DB using JDBC

Here's how I'm trying to connect:
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
} catch (Exception e) {
throw new DbConnectionException();
}
try {
connection = DriverManager.getConnection(url,username,password);
} catch (SQLException e) {
e.printStackTrace();
throw new DbConnectionException();
}
I'm 100% sure that the url, username, password strings are correct. I've already connected successfully using an external tool (MySQL query browser).
This is the error I receive:
com.mysql.jdbc.CommunicationsException:
Communications link failure due to
underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException MESSAGE:
java.net.ConnectException: Connection
refused
...

Possibly a url issue. If your code is pointing to MySQL localhost, try changing localhost to 127.0.0.1 on your url.
E.g.:
jdbc:mysql://localhost:3306/MY_DB
to
jdbc:mysql://127.0.0.1:3306/MY_DB
And see if this works.

did you run the mysql browser from the same machine where the code is running? What I am getting at is the permissions in mysql can be host-specific, and depending on how you set them up you might not be able to connect from the machine where the code is running.
Also, you might want to double check the url, name, pword again, perhaps with log statements or a debugger to make sure there are no typos, trailing whitespaces, etc...

Double check the format of your url. It should start with "jdbc:mysql:". Make sure you are using a current version for the driver as well.

Check that you can connect to the database from the mysql admin tool, that will drive out whether your mysql is running and that the port is open.

In my case the problem was that I was using a connection from emulator to localhost.
If you use emulator to localhost don't use localhost value in connection String but use 10.0.2.2 instead:
jdbc:mysql://10.0.2.2:3306/MY_DB
Hope this helps.