Develop Reference

How to establish a FTPS data connection to a FileZilla Server 1.2.0

It is a known problem to use the Java FTPSClient of Apache commons-net with session resumption. Session resumption is a security feature which a FTPS server can require for data connections. The Apache FTPSClient does not support session resumption, and the JDK APIs make it hard to build a custom implementation. There are a couple of workarounds using reflection, see e.g. this answer and this commons-net bug entry.
I use such a workaround (see snipped below) in JDK 11 and tested it against a local FileZilla Server. It works with FileZilla Server 0.9.6, but it doesn't with FileZilla Server 1.2.0, which is the latest version at the time of writing. With that version, when trying to establish a data connection, the server responds with:
425 Unable to build data connection: TLS session of data connection not resumed.
As I said, FileZilla Server 0.9.6 is fine with how I do session resumption, and I made sure that the setting for requiring session resumption is activated.
In FileZilla Server 1.2.0, such settings are now set implicitly and cannot be changed via the GUI, maybe not at all. Are there some server settings that I can tweak for this to work? Or is it an issue with how I implemented the workaround? Does anyone experience similar issues?
This is the workaround I am using:
public class FTPSClientWithSessionResumption extends FTPSClient {
static {
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", "false");
}
#Override
protected void _connectAction_() throws IOException {
super._connectAction_();
execPBSZ(0);
execPROT("P");
}
#Override
protected void _prepareDataSocket_(Socket socket) throws IOException {
if (useSessionResumption && socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket)_socket_).getSession();
if (session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
Method getHostMethod;
try {
getHostMethod = socket.getClass().getMethod("getPeerHost");
}
catch (NoSuchMethodException e) {
// Running in IKVM
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object peerHost = getHostMethod.invoke(socket);
InetAddress iAddr = socket.getInetAddress();
int port = socket.getPort();
putMethod.invoke(cache, String.format("%s:%s", peerHost, port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostName(), port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostAddress(), port).toLowerCase(Locale.ROOT), session);
}
catch (Exception e) {
throw new IOException(e);
}
}
else {
throw new IOException("Invalid SSL Session");
}
}
}
}
The address under which the socket is cached is determined using getPeerHost, getInetAddress().getHostName(), and getInetAddress().getHostAddress(). I tried several combinations of doing or not doing these three, but I always get the same result.
Edit:
Here is a screenshot of the server logs of the full session:

As stated in this StackOverflow post it is possible to tell the JVM that only TLS 1.2 should be used.
Here is the link to the original answer which worked for me: command for java to use TLS1.2 only
You have to add a command line parameter at the start of the JVM in this case this is: java -Djdk.tls.client.protocols=TLSv1.2 -jar ... <rest of command line here>
This simple parameter worked for me, now I can connect and transfer data from a FTP-Server wich runs FileZilla FTP-Server 1.3.0

Check for mail server SMTPUTF8 support

I have a system where (registered) users have the possibility to add their own SMTP server for mail sending. Thus I don't know in advance what server a user might add and whether this one supports SMTPUTF8. Now starting from JavaMail 1.6 support for SMTPUTF8 has been added.
After enabling this for some servers the following line is printed to the log:
mail.mime.allowutf8 set but server doesn't advertise SMTPUTF8 support
This is correct for that server, however how can I detect whether a server supports this or not? I see two possibilities:
Somehow detect whether a mail server supports SMTPUTF8 - if that is possible?
Add another parameter to the stmp configuration and then the users decide (which in most cases they don't really know I would guess)
So is there a way to detect whether a server supports SMTPUTF8?
I checked the implementation and the com.sun.mail.SMTPTransport seems to provide such a query possibility.
public boolean supportsExtension(String ext)
So calling
SMTPTransport transport = new SMTPTransport(...);
boolean smtputf8 = transport.supportsExtension("SMTPUTF8");
should work. It is using an EHLO command (defined in rfc1869) when connecting to the server. However in my implementation I just call
...
Transport.send(Message msg);
So I do not directly access SMTPTransport. One should not directly access com.sun packages, right?

Using ESMTP with the EHLO command one can query the smtp server for supported extensions. mail.smtp.ehlo=true; must be enabled.
boolean detectUTF8(String smtpServerHost, int smtpServerPort, String smtpUserName, String smtpUserAccessToken) {
Transport transport = null;
try {
Properties props = new Properties();
Session session = Session.getDefaultInstance(props);
//session.setDebug(true);
transport = session.getTransport();
transport.connect(smtpServerHost, smtpServerPort, smtpUserName, smtpUserAccessToken);
if(transport instanceof SMTPTransport || transport instanceof SMTPSSLTransport) {
return ((SMTPTransport)transport).supportsExtension("SMTPUTF8");
}
return false;
}
catch (Exception ex) {
//Proper loggin & exception handling!!!
//return false;
}
finally {
if(transport!=null) {
try {
transport.close();
}
catch (MessagingException e) {
}
}
}
}
It would be nice if the supportsExtension(...) method would make it into the Transport interface so that javax.mail could be used.

UnboundID LDAP SDK startTLS "No extended operation handler" error

Created an in-memory UnboundID LDAP server. The server listens on an unencrypted port and a TLS port. I can bind to it with LDAP on the first port, and LDAPS on the second port. However, I cannot connect to it using startTLS. The error is:
LDAPException(resultCode=53 (unwilling to perform), errorMessage='No
extended operation handler is defined for extended request OID
'1.3.6.1.4.1.1466.20037'.', diagnosticMessage='No extended operation
handler is defined for extended request OID
'1.3.6.1.4.1.1466.20037'.')
Here is how the server is configured:
try {
InMemoryListenerConfig ldapsConfig = InMemoryListenerConfig.createLDAPSConfig("ldaps",
LDAPS_PORT,
getServerSSLContext().getServerSocketFactory());
InMemoryListenerConfig ldapConfig = InMemoryListenerConfig.createLDAPConfig("ldap",
LDAP_PORT);
InMemoryDirectoryServerConfig serverConfig serverConfig = new InMemoryDirectoryServerConfig(BASE_DN);
serverConfig.addAdditionalBindCredentials(BIND_RDN, BIND_PASSWORD);
serverConfig.setListenerConfigs(ldapConfig, ldapsConfig);
server = new InMemoryDirectoryServer(serverConfig);
server.startListening();
} catch (LDAPException e) {
fail(e.getMessage());
}
The message tells me I need to add some kind of handler to the server, but I cannot find an example in the docs I've skimmed.

Embedded Jetty HTTP/2 not working with Firefox/Chrome but seems ok with Safari

I'm implementing a servlet in an embedded Jetty (9.3.0.v20150612) and I would like to use HTTP/2.
I'm enabling ALPN for protocol negotiation to select either HTTP1.1 or HTTP2.
When I'm sending an HTTPs request to my servlet from Safari 8 (supports only HTTP1.1) or Safari 9 (support both HTTP1.1 & HTTP2) I get an answer from my servlet.
When I execute the same request from Firefox 39, it doesn't work and I just get NS_ERROR_ABORT. I have the same issue with Chrome.
I have two questions:
Why I don't get answer from Chrome & Firefox
How can I know if with Safari 9, HTTP/2 has been used instead of HTTP1.1? (My IOS9 App is also connecting without issue)
Below is the code to perform the initialization of Jetty
private void startHTTP2Server() {
WebServerProperties webProperties = WebServerProperties.getInstance();
HttpConfiguration config = getHttpConfiguration();
HttpConnectionFactory http1 = new HttpConnectionFactory(config);
HTTP2ServerConnectionFactory http2 = new HTTP2ServerConnectionFactory(config);
NegotiatingServerConnectionFactory.checkProtocolNegotiationAvailable();
ALPNServerConnectionFactory alpn = new ALPNServerConnectionFactory();
alpn.setDefaultProtocol(http1.getProtocol()); // sets default protocol to HTTP 1.1
// SSL Connection Factory
SslContextFactory sslContextFactory = new SslContextFactory();
sslContextFactory.setKeyStorePath(webProperties.getKeystore());
sslContextFactory.setKeyStorePassword(webProperties.getKeystorePassword());
//sslContextFactory.setKeyManagerPassword(KEYSTORE_PW);
//sslContextFactory.addExcludeCipherSuites(".*RC4.*");
//sslContextFactory.addExcludeCipherSuites("TLS_DHE_RSA.*");
sslContextFactory.setProtocol(webProperties.getTLSVersion()); // SEB
SslConnectionFactory ssl = new SslConnectionFactory(sslContextFactory, alpn.getProtocol());
Server server = new Server();
//ServerConnector connector = new ServerConnector(server, ssl, alpn, http2, http1);
ServerConnector connector = new ServerConnector(server, ssl, alpn, http2, http1);
connector.setPort(webProperties.getPort());
server.addConnector(connector);
// --- SEB
ServletContextHandler context = new ServletContextHandler(ServletContextHandler.SESSIONS);
context.setContextPath(webProperties.getServletContext());
context.setResourceBase(System.getProperty(webProperties.getServletTmpDir()));
server.setHandler(context);
// Add dump servlet
context.addServlet(IMonServer.class, webProperties.getServletPath());
try {
server.start();
server.join();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private static HttpConfiguration getHttpConfiguration() {
WebServerProperties webProperties = WebServerProperties.getInstance();
HttpConfiguration config = new HttpConfiguration();
config.setSecureScheme("https");
config.setSecurePort(webProperties.getPort());
config.setSendXPoweredBy(true);
config.setSendServerVersion(true);
config.addCustomizer(new SecureRequestCustomizer());
return config;
}
When I'm starting the server, I give also the Java option -Xbootclasspath/p:$installDirectory/lib/alpn-boot-8.1.3.v20150130.jar
Is there anything wrong or missing?
Thanks for your help,
Regards,

Your code is correct, it just lacks two more lines that are important to get HTTP/2 to work correctly.
When you configure the SslContextFactory, add the lines:
sslContextFactory.setCipherComparator(HTTP2Cipher.COMPARATOR);
sslContextFactory.setUseCipherSuitesOrder(true);
What these two lines do is to sort the TLS ciphers to prefer the HTTP/2 ones before the others, and to ask to respect that order.
Without sorting, the server was falling back to an older draft version of HTTP/2 (h2-14) that did not enforce cipher strength, but this is unfortunately rejected by Chrome and Firefox.
I don't know exactly why it is working with Safari: either a Safari bug, or a more relaxed interpretation of the HTTP/2 specification with regard to cipher strength with respect to other browsers.

Java client listening to WebSphere MQ Server?

I need to write a Java client listening to WebSphere MQ Server. Message is put into a queue in the server.
I developed this code, but am not sure it is correct or not. If correct, then how can I test it?
This is a standalone Java project, no application server support. Which jars I should put into classpath?
I have the MQ settings, where I should put into my codes? Standard JMS can skip these settings? confusing ....
import javax.jms.Destination;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Queue;
import javax.jms.QueueConnection;
import javax.jms.QueueConnectionFactory;
import javax.jms.QueueReceiver;
import javax.jms.QueueSession;
import javax.jms.Session;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
public class Main {
Context jndiContext = null;
QueueConnectionFactory queueConnectionFactory = null;
QueueConnection queueConnection = null;
QueueSession queueSession = null;
Queue controlQueue = null;
QueueReceiver queueReceiver = null;
private String queueSubject = "";
private void start() {
try {
queueConnection.start();
queueSession = queueConnection.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = queueSession.createQueue(queueSubject);
MessageConsumer consumer = queueSession.createConsumer(destination);
consumer.setMessageListener(new MyListener());
} catch (Exception e) {
e.printStackTrace();
}
}
private void close() {
try {
queueSession.close();
queueConnection.close();
} catch (Exception e) {
e.printStackTrace();
}
}
private void init() {
try {
jndiContext = new InitialContext();
queueConnectionFactory = (QueueConnectionFactory) this.jndiLookup("QueueConnectionFactory");
queueConnection = queueConnectionFactory.createQueueConnection();
queueConnection.start();
} catch (Exception e) {
System.err.println("Could not create JNDI API " + "context: " + e.toString());
System.exit(1);
}
}
private class MyListener implements MessageListener {
#Override
public void onMessage(Message message) {
System.out.println("get message:" + message);
}
}
private Object jndiLookup(String name) throws NamingException {
Object obj = null;
if (jndiContext == null) {
try {
jndiContext = new InitialContext();
} catch (NamingException e) {
System.err.println("Could not create JNDI API " + "context: " + e.toString());
throw e;
}
}
try {
obj = jndiContext.lookup(name);
} catch (NamingException e) {
System.err.println("JNDI API lookup failed: " + e.toString());
throw e;
}
return obj;
}
public Main() {
}
public static void main(String[] args) {
new Main();
}
}
MQ Queue setting
<queue-manager>
<name>AAA</name>
<port>1423</port>
<hostname>ddd</hostname>
<clientChannel>EEE.CLIENTS.00</clientChannel>
<securityClass>PKIJCExit</securityClass>
<transportType>1</transportType>
<targetClientMatching>1</targetClientMatching>
</queue-manager>
<queues>
<queue-details id="queue-1">
<name>GGGG.NY.00</name>
<transacted>false</transacted>
<acknowledgeMode>1</acknowledgeMode>
<targetClient>1</targetClient>
</queue-details>
</queues>

There is an article with sample code Running a standalone Java application on WebSphere MQ V6.0 which walks you through most of your questions, including how you might test with a free WMQ trial install. The main difference (as discussed in the comments) to v7 or v7.1 is that there is no broker component to start if you want to use topics. Other than that, the article should work fine with the current WMQ client and/or server.
In addition, please refer to the WebSphere MQ Using Java manual (v7.0 client) or the Using WebSphere MQ Classes for Java manual (v7.1 client) for the appropriate CLASSPATH and other settings for your client. Remember to use the Infocenter appropriate to your client version and not to the server version. You can mix and match client and server version but you get only the features supported by the server. For example, using the WMQ v7.1 client with the WMQ v7.0 server is perfectly valid.
Finally, there are a number of sample programs supplied with the free client download that do exactly what you are describing. Some use JNDI to lookup the WMQ resources, others use Java methods and can be populated with standard Java property files. The ones with a -nojndi option show how to initialize your WMQ objects in the code at run time. These are under
[WMQ Install path]\tools\wmqjava\samples
...in the latest Windows client install (SupportPac MQC71). You can also use the v7.0 client (SupportPac MQC7). I would recommend using the samples to get started rather than starting from scratch. Why reinvent the wheel, right?
In addition to the many sample programs, the vendor install includes all of the requisite jar files. Note that what goes in the CLASSPATH changes by WMQ client version so refer to the Infocenter. The later versions are much simpler and require only a couple of jar files in the CLASSPATH.
If you want to download the WMQ trial for testing and do not have Administrator rights on your Windows workstation, you can install it easily on a RedHat or SUSE virtual machine. With a bit of massaging you can also easily install on Ubuntu as described in an Andy Piper blog post.

If you have the option, I would recommend that you introduce the Spring Framework to handle the JMS communication. That way you only need to write your business logic and can leave the error handling up to Spring.
Spring JMS
Download the latest Spring JARS and look at configuring a DefaultMessageListenerContainer for your application. You will then write your own POJO (plain old java object) with an onMessage() event that gets called everytime a new message arrives.
I found this tutorial that you may find useful to start with

There is a small client application that I developed in JavaFx2 for windows and Mac osx. It's available on source forge (https://sourceforge.net/projects/mqconsole).
You can see an example of listening to a queue for new messages.
The program lists the queues, the messages in each queue, view the details of the message and send a message to a queue and listen to the response.
You can check out the code, use it and improve it.