Develop Reference

How to establish a FTPS data connection to a FileZilla Server 1.2.0

It is a known problem to use the Java FTPSClient of Apache commons-net with session resumption. Session resumption is a security feature which a FTPS server can require for data connections. The Apache FTPSClient does not support session resumption, and the JDK APIs make it hard to build a custom implementation. There are a couple of workarounds using reflection, see e.g. this answer and this commons-net bug entry.
I use such a workaround (see snipped below) in JDK 11 and tested it against a local FileZilla Server. It works with FileZilla Server 0.9.6, but it doesn't with FileZilla Server 1.2.0, which is the latest version at the time of writing. With that version, when trying to establish a data connection, the server responds with:
425 Unable to build data connection: TLS session of data connection not resumed.
As I said, FileZilla Server 0.9.6 is fine with how I do session resumption, and I made sure that the setting for requiring session resumption is activated.
In FileZilla Server 1.2.0, such settings are now set implicitly and cannot be changed via the GUI, maybe not at all. Are there some server settings that I can tweak for this to work? Or is it an issue with how I implemented the workaround? Does anyone experience similar issues?
This is the workaround I am using:
public class FTPSClientWithSessionResumption extends FTPSClient {
static {
System.setProperty("jdk.tls.useExtendedMasterSecret", "false");
System.setProperty("jdk.tls.client.enableSessionTicketExtension", "false");
}
#Override
protected void _connectAction_() throws IOException {
super._connectAction_();
execPBSZ(0);
execPROT("P");
}
#Override
protected void _prepareDataSocket_(Socket socket) throws IOException {
if (useSessionResumption && socket instanceof SSLSocket) {
// Control socket is SSL
final SSLSession session = ((SSLSocket)_socket_).getSession();
if (session.isValid()) {
final SSLSessionContext context = session.getSessionContext();
try {
final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
sessionHostPortCache.setAccessible(true);
final Object cache = sessionHostPortCache.get(context);
final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
putMethod.setAccessible(true);
Method getHostMethod;
try {
getHostMethod = socket.getClass().getMethod("getPeerHost");
}
catch (NoSuchMethodException e) {
// Running in IKVM
getHostMethod = socket.getClass().getDeclaredMethod("getHost");
}
getHostMethod.setAccessible(true);
Object peerHost = getHostMethod.invoke(socket);
InetAddress iAddr = socket.getInetAddress();
int port = socket.getPort();
putMethod.invoke(cache, String.format("%s:%s", peerHost, port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostName(), port).toLowerCase(Locale.ROOT), session);
putMethod.invoke(cache, String.format("%s:%s", iAddr.getHostAddress(), port).toLowerCase(Locale.ROOT), session);
}
catch (Exception e) {
throw new IOException(e);
}
}
else {
throw new IOException("Invalid SSL Session");
}
}
}
}
The address under which the socket is cached is determined using getPeerHost, getInetAddress().getHostName(), and getInetAddress().getHostAddress(). I tried several combinations of doing or not doing these three, but I always get the same result.
Edit:
Here is a screenshot of the server logs of the full session:

As stated in this StackOverflow post it is possible to tell the JVM that only TLS 1.2 should be used.
Here is the link to the original answer which worked for me: command for java to use TLS1.2 only
You have to add a command line parameter at the start of the JVM in this case this is: java -Djdk.tls.client.protocols=TLSv1.2 -jar ... <rest of command line here>
This simple parameter worked for me, now I can connect and transfer data from a FTP-Server wich runs FileZilla FTP-Server 1.3.0

Mail service GAE issue - sending mail exception "javax.mail.MessagingException: Could not connect to SMTP host: localhost, port: 25;"

I'd like to send mail from my GAE project. I've followed the documentation example...
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
try {
Message msg = new MimeMessage(session);
msg.setFrom(new InternetAddress("xxx#xxxx.appspotmail.com", "Example.com Admin"));
msg.addRecipient(Message.RecipientType.TO,
new InternetAddress("xxxxx#gmail.com", "Mr. User"));
msg.setSubject("Your Example.com account has been activated");
msg.setText("This is a test");
Transport.send(msg);
} catch (AddressException e) {
e.printStackTrace();
} catch (MessagingException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
}
After deployment, I get this exception message
javax.mail.MessagingException: Could not connect to SMTP host: localhost, port: 25;
But the documentation says that:
When you create a JavaMail Session, if you do not provide any SMTP server configuration, App Engine uses the Mail service for sending messages
But it seems to try connecting to a SMTP server... and obviously there is no SMTP server on localhost...
I've never used this service... my quotas are full available.
Please, help me !

had the same issue today. just got it working. app engine sdk already includes the classes you will need to send email:
https://cloud.google.com/appengine/docs/standard/java/javadoc/com/google/appengine/api/mail/MailService.Message
that and the related classes are the way to invoke the mail service. replace your message classes with those, remove all references to javax.mail. one other thing in case you're referencing this (as I was):
https://cloud.google.com/appengine/docs/standard/java/mail/sending-mail-with-mail-api
I couldn't get it to work, doesn't looks like it would without an smtp host at least. Nice of google to provide nonsensical documentation for a non-working example in their example code base
also, if you follow the "who can send mail" link it tells you that any address of the form anything#[APP_NAME].appspotmail.com or anything#[APP_ALIAS].appspotmail.com should work. using my apps name resulted in "unauthorized sender", but using the app id from the dashboard worked. what should have been a ten minute solution turned into hours of drudgery, but I have a working emailer. thanks, google.

The Mail service API supports the JavaMail (javax.mail) interface which is included with the App Engine SDK. Using any other jars may create the issue. You may follow the code sample in Java 7 and Java 8 which demonstrate how to send mail.
I should note that outbound connections on ports 25, 465, and 587 are not allowed due to spam concerns, so the sender address of a message must be one of the optioned in this link.
You can take your application ID/name (which is the same as the project ID/name) through the dashboard.
Kindly note that Issue Tracker is reserved for reporting bugs and feature requests. If you encounter any issue related to APP_NAME or APP_ALIAS, it is recommended to report the issue there so that we would be able to dig into the problem.

SNI client-side mystery using Java8

I have an Apache web server that runs several TLS virtualhosts with different certs and SNI.
I can access the various virtual hosts just fine using curl (presumably SNI makes it work). I can also access them fine with a little command-line Java program that basically just openConnection()s on a URL.
In my Tomcat application, the basic same client-side code accesses the same Apache server as a client, but always ends up with the default cert (defaulthost.defaultdomain) instead of the cert of the virtual host that was specified in the URL that it attempts to access. (This produces a SunCertPathBuilderException -- basically it can't verify the certificate path to the cert, which of course is true as it is a non-official cert. But then the default cert should not be used anyway.)
It's just as if SNI had been deactivated client-side in my application / Tomcat. I am at a loss why it should behave differently between my app and the command-line; same JDK, same host etc.
I found property jsse.enableSNIExtension, but I verified that it is set to true for both cases. Questions:
Any ideas, even wild ones, why these two programs behave differently?
Any ideas how I would debug this?
This is Arch Linux on 86_64, JDK 8u77, Tomcat 8.0.32.

This answer comes late, but we just have hit the problem (I can't believe it, it seems a very big bug).
All what it said seems true, but it's not default HostnameVerifier the culprit but the troubleshooter. When HttpsClient do afterConnect first try to establish setHost (only when socket is SSLSocketImpl):
SSLSocketFactory factory = sslSocketFactory;
try {
if (!(serverSocket instanceof SSLSocket)) {
s = (SSLSocket)factory.createSocket(serverSocket,
host, port, true);
} else {
s = (SSLSocket)serverSocket;
if (s instanceof SSLSocketImpl) {
((SSLSocketImpl)s).setHost(host);
}
}
} catch (IOException ex) {
// If we fail to connect through the tunnel, try it
// locally, as a last resort. If this doesn't work,
// throw the original exception.
try {
s = (SSLSocket)factory.createSocket(host, port);
} catch (IOException ignored) {
throw ex;
}
}
If you use a custom SSLSocketFactory without override createSocket() (the method without parameters), the createSocket well parametrized is used and all works as expected (with client sni extension). But when second way it's used (try to setHost en SSLSocketImpl) the code executed is:
// ONLY used by HttpsClient to setup the URI specified hostname
//
// Please NOTE that this method MUST be called before calling to
// SSLSocket.setSSLParameters(). Otherwise, the {#code host} parameter
// may override SNIHostName in the customized server name indication.
synchronized public void setHost(String host) {
this.host = host;
this.serverNames =
Utilities.addToSNIServerNameList(this.serverNames, this.host);
}
The comments say all. You need to call setSSLParameters before client handshake. If you use default HostnameVerifier, HttpsClient will call setSSLParameters. But there is no setSSLParameters execution in the opposite way. The fix should be very easy for Oracle:
SSLParameters paramaters = s.getSSLParameters();
if (isDefaultHostnameVerifier) {
// If the HNV is the default from HttpsURLConnection, we
// will do the spoof checks in SSLSocket.
paramaters.setEndpointIdentificationAlgorithm("HTTPS");
needToCheckSpoofing = false;
}
s.setSSLParameters(paramaters);
Java 9 is working as expected in SNI. But they (Oracle) seem not to want fix this:
https://bugs.openjdk.java.net/browse/JDK-8072464
https://bugs.openjdk.java.net/browse/JDK-8144566
and many more

After some hours of debugging the JDK, here is the unfortunate result. This works:
URLConnection c = new URL("https://example.com/").openConnection();
InputStream i = c.getInputStream();
...
This fails:
URLConnection c = new URL("https://example.com/").openConnection();
((HttpsURLConnection)c).setHostnameVerifier( new HostnameVerifier() {
public boolean verify( String s, SSLSession sess ) {
return false; // or true, won't matter for this
}
});
InputStream i = c.getInputStream(); // Exception thrown here
...
Adding the setHostnameVerifier call has the consequence of disabling SNI, although the custom HostnameVerifier is never invoked.
The culprit seems to be this code in sun.net.www.protocol.https.HttpsClient:
if (hv != null) {
String canonicalName = hv.getClass().getCanonicalName();
if (canonicalName != null &&
canonicalName.equalsIgnoreCase(defaultHVCanonicalName)) {
isDefaultHostnameVerifier = true;
}
} else {
// Unlikely to happen! As the behavior is the same as the
// default hostname verifier, so we prefer to let the
// SSLSocket do the spoof checks.
isDefaultHostnameVerifier = true;
}
if (isDefaultHostnameVerifier) {
// If the HNV is the default from HttpsURLConnection, we
// will do the spoof checks in SSLSocket.
SSLParameters paramaters = s.getSSLParameters();
paramaters.setEndpointIdentificationAlgorithm("HTTPS");
s.setSSLParameters(paramaters);
needToCheckSpoofing = false;
}
where some bright mind checks whether the configured HostnameVerifier's class is the default JDK class (which, when invoked, just returns false, like my code above) and based on that, changes the parameters for the SSL connection -- which, as a side effect, turns off SNI.
How checking the name of a class and making some logic depend on it is ever a good idea escapes me. ("Mom! We don't need virtual methods, we can just check the class name and dispatch on that!") But worse, what in the world does SNI have to do with the HostnameVerifier in the first place?
Perhaps the workaround is to use a custom HostnameVerifier with the same name, but different capitalization, because that same bright mind also decided to do case-insensitive name comparison.
'nuff said.

This is a Java 8 bug (JDK-8144566) fixed by 8u141. See Extended server_name (SNI Extension) not sent with jdk1.8.0 but send with jdk1.7.0 for more.

How to share the same email session between all instances of the application?

Maybe this question is already answered, but I couldn't find the proper answer.
I have a web application based in JSF, and I want to share the same email session between all the instances of the application, yet I haven't found how to do that.
My questions are:
a) What I am thinking is stupid? Should I just create a new session every time that I want to send a new mail?
b) If a is false, is there a proper way to do that?
Additional info: I'm working with PrimeFaces 4.0, Apache Tomcat 7.0.41, and JDK 7.
EDIT: I'm establishing an email connection like this (using sun's java mail)
Properties datos = new Properties();
datos.put("mail.smtp.host", "smtp.gmail.com");
datos.setProperty("mail.smtp.starttls.enable", "true");
datos.setProperty("mail.smtp.port", "587");
datos.setProperty("mail.smtp.user", usuarioAutenticacion);
datos.put("mail.smtp.timeout", 5000);
System.out.println(usuarioAutenticacion + " - " + contrasenaAutenticacion);
sesionCorreo = Session.getDefaultInstance(datos, null);
sesionCorreo.setDebug(true);
try {
conexionCorreo = sesionCorreo.getTransport("smtp");
} catch (NoSuchProviderException ex) {
Logger.getLogger(NotificacionesManager.class.getName()).log(Level.SEVERE, null, ex);
}
try {
conexionCorreo.connect(usuarioAutenticacion, contrasenaAutenticacion);
Then I proceed to send the messages in the Queue, but I'm looking for a way for just set that connection once then start sending the mails in the queue when necessary.

The way that the Java EE designers intended you to do this is that you configure your javax.mail.Session object in your server. This is described in the Tomcat 7 JavaMail Sessions documentation.
Your managed beans should then be able to access the session via #Resource:
class MyManagedBean {
#Resource(name="mail/Session") // this name is defined by your configuration
private Session mailSession;
public void someBusinessMethod() {
...
Message message = new MimeMessage(mailSession);
// compose message
...
Transport.send(message);
}
}
If you need to do this from a non-managed bean then you grab your Session instance using JNDI. This is described in the documentation linked above.

Getting Invalid Address with javax.mail when the addresses are fine

I'm using the javax.mail system, and having problems with "Invalid Address" exceptions. Here's the basics of the code:
// Get system properties
Properties props = System.getProperties();
// Setup mail server
props.put("mail.smtp.host", m_sending_host);
// Get session
Session session = Session.getDefaultInstance(props, new Authenticator(){
#Override
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(m_sending_user, m_sending_pass);
}
});
// Define message
MimeMessage message = new MimeMessage(session);
message.setFrom(new InternetAddress(m_sending_from));
message.addRecipient(Message.RecipientType.TO,
new InternetAddress(vcea.get(i).emailaddr));
message.setSubject( replaceEnvVars(subject) );
message.setText(replaceEnvVars(body));
// Send message
try {
Transport.send(message);
} catch (Exception e){
Log.Error("Error sending e-mail to addr (%s): %s",
vcea.get(i).emailaddr, e.getLocalizedMessage() );
}
The issue is that the above code does work, sometimes. But for some e-mail addresses that I know to be valid (because I can send to them via a standard e-mail client), the above code will throw an "Invalid Address" exception when trying to send.
Any clues or hints would be greatly appreciated.
--Update: problem with authentication.
Ok, here's what I've discovered was going on. When receiving e-mail, the code above correctly sets up authentication and the Authenticator.getPasswordAuthentication() callback is actually invoked.
Not so when sending e-mail. You have to do a bit more. Add this:
// Setup mail server
props.put("mail.smtp.host", m_sending_host);
props.put("mail.smtp.auth", "true");
which will force the javax.mail API to do the login authentication. And then use an actual Transport instance instead of the static .send() method:
Transport t = session.getTransport(m_sending_protocol);
t.connect(m_sending_user, m_sending_pass);
...
// Send message
try {
t.sendMessage(message, message.getAllRecipients());
} catch (Exception e){
Without forcing the authentication, the mail server saw me as an unauthorized relay, and just shut me down. The difference between the addresses that "worked" and the addresses that didn't was that the ones that "worked" were all local to the mail server. Therefore, it simply accepted them. But for any non-local "relay" addresses, it would reject the message because my authentication information hadn't been presented by the javax.mail API when I thought it would have.
Thanks for the clues to prompt me to look at the mail server side of things as well.

--Update: problem with authentication.
Ok, here's what I've discovered was going on. When receiving e-mail, the code above correctly sets up authentication and the Authenticator.getPasswordAuthentication() callback is actually invoked.
Not so when sending e-mail. You have to do a bit more. Add this:
// Setup mail server
props.put("mail.smtp.host", m_sending_host);
props.put("mail.smtp.auth", "true");
which will force the javax.mail API to do the login authentication. And then use an actual Transport instance instead of the static .send() method:
Transport t = session.getTransport(m_sending_protocol);
t.connect(m_sending_user, m_sending_pass);
...
// Send message
try {
t.sendMessage(message, message.getAllRecipients());
} catch (Exception e){
Without forcing the authentication, the mail server saw me as an unauthorized relay, and just shut me down. The difference between the addresses that "worked" and the addresses that didn't was that the ones that "worked" were all local to the mail server. Therefore, it simply accepted them. But for any non-local "relay" addresses, it would reject the message because my authentication information hadn't been presented by the javax.mail API when I thought it would have.
Thanks for the clues to prompt me to look at the mail server side of things as well.

I would change the call to InternetAddress to use the "strict" interpretation and see if you get further errors on the addresses you are having trouble with.
message.addRecipient(Message.RecipientType.TO,
new InternetAddress(vcea.get(i).emailaddr, true ));
// ^^^^ turns on strict interpretation
Javadoc for InternetAddress constructor
If this fails, it will throw an AddressException which has a method called getPos() which returns the position of the failure (Javadoc)

A good hint for those using the ssl encryption in the smtp configuration, you should enable it by specifying the property mail.smtp.ssl.enable, as shown below:
props.put("mail.smtp.ssl.enable", "true");
Otherwise it can lead to similar problems as described above.

Try this:
String to="stackoverflow#so.com";
String cc="one#mail.com,two#mail.com"; //The separator ',' works good
message.setRecipients(Message.RecipientType.TO,new InternetAddress[] {
new InternetAddress(to) }); // This is only one mail
InternetAddress[] addr = parseAddressList(cc); //Here add all the rest of the mails
message.setRecipients(Message.RecipientType.CC,addr);
Sorry for my english. It's not good.

This seems to me like a problem that happened at my work.
If the code you are showing is being concurrent, then using directly System.getProperties
could be a problem, because the host value you put on them can be overwritten by next request, while still keeping the user and password from the overwritten host.
In our case, we solved that using a clone of the System.getProperties() hashtable.
Hope that helps (this problem was really hard to track).