Develop Reference

UnknownHostException behind Proxy

I have a java program, that connects to a website to retrieve some XML from it. This works fine on my computer, as well as others outside our company. One of our customers is now not able to connect to the website. I figured out, that they are behind a proxy. I have now found which settings I need to use, and in my test program it works (partially).
In the code below, the downloadFile() call works as expected, and the file can be downloaded without problems. The contactHost() fails on our client machines with an UnknownHostException:
java.net.UnknownHostException: No such host is known (api.myserver.de)
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:925)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1505)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:844)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1495)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1354)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1288)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:111)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
Background: Windows 10 machines, our program is shipped with an internal OpenJDK, version "10.0.2" 2018-07-17. The program is started with the following defines -Djdk.http.auth.tunneling.disabledSchemes="" -Djava.net.preferIPv4Stack=true in order to use IP4 only, and to enable BasicAuthentification for the Proxy. With these settings, the file can be downloaded, however the UnknownHostException is still there.
We have also tried to open the used URL in an browser, and this works as excepted, i.e. in the browser the website is opened.
Here is my code for testing:
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.Authenticator;
import java.net.URL;
import java.net.URLConnection;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
public class LFTProxyTest {
private static String uname = null;
private static String pass = null;
public static void main(String[] args) {
System.setProperty("java.net.useSystemProxies", "true");
// uname = "test"; // whatever that user provides
// pass = "sectret"; // whatever that user provides
Authenticator.setDefault(new ProxyAuth(uname, pass));
contactHost();
downloadFile();
}
private static boolean downloadFile() {
System.out.println("CHECK connection");
int cp = contactHost();
if (cp == 200)
return true;
if (cp == 407)
return false;
else {
try {
System.out.println("Try loading file: ");
URL url = new URL("https://www.google.de");
URLConnection urlConnection = url.openConnection();
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
dBuilder.parse(in);
System.out.println(" FILE DOWNLOAD successfull!");
} catch (Exception e) {
System.out.println(" FILE DOWNLOAD failed:");
System.out.println("***EXCEPTION: " + e.getMessage());
return false;
}
}
System.out.println("CHECK done");
return true;
}
private static int contactHost() {
HttpClient client = HttpClientBuilder.create().build();// new DefaultHttpClient();
String catalogURI = "https://api.myserver.de/query";
HttpGet request = new HttpGet(catalogURI);
try {
int ret = 0;
HttpResponse response = client.execute(request);
ret = response.getStatusLine().getStatusCode();
System.out.println("PROXY test: " + ret);
((CloseableHttpClient) client).close();
return ret;
} catch (IOException e) {
e.printStackTrace();
return -1;
}
}
}
I don't know what do do know, I'm not even sure where the error could be. Any ideas are highly appreciated!

Ok, so after some further digging, I found out that org.apache.http.client.HttpClient is not respecting java.net.useSystemProxies at all, be it set via System or via -D. And it is also ignoring http.proxyHost etc. Solution is to use a ProxySelector like this:
ProxySelector.setDefault(new ProxySelector() {
#Override
public List<Proxy> select(URI uri) {
ArrayList<Proxy> list = new ArrayList<Proxy>();
list.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy1.de", 8000)));
list.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy2.de", 8080)));
return list;
}
#Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
logger.error("Error in ProxySelector, connection Failed: ", ioe);
}
});
I'm getting another exception now, but I might open another thread for this.

UnknownHostException designates a pretty straight forward problem. That the IP address of the remote host you are trying to reach cannot be resolved. So the solution to this is very simple. You should check the input of Socket (or any other method that throws an UnknownHostException), and validate that it is the intended one. If you are not whether you have the correct host name, you can launch a UNIX terminal and use the nslookup command (among others) to see if your DNS server can resolve the host name to an IP address successfully.
If you are on Windows you can use the host command. If that doesn’t work as expected then, you should check if the host name you have is correct and then try to refresh your DNS cache. If that doesn’t work either, try to use a different DNS server, eg Google Public DNS is a very good alternative.

Java IMAP endless loop while writing to EML

We got a serious problem in our system which is processing mails from IMAP and saving it as EML. Thing is, everything works in production but not on dev machine anymore, therefore we cannot make any fixes or further development. This is kind of strange, because also this example does not work:
package com.test;
import java.io.FileOutputStream;
import java.io.FileWriter;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Properties;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.Session;
import javax.mail.Store;
public class ReadMail {
public static void main(String args[]) throws Exception {
String host = "mail";
String user = "xxx";
String password = "zzz";
Properties properties = System.getProperties();
Session session = Session.getDefaultInstance(properties);
Store store = session.getStore("imap");
store.connect(host, user, password);
Folder folder = store.getFolder("a");
folder.open(Folder.READ_ONLY);
Message[] message = folder.getMessages();
for (int i = 0; i < message.length; i++) {
System.out.println("------------ Message " + (i + 1) + " ------------");
System.out.println(message[i].getSentDate());
System.out.println(message[i].getFrom()[0]);
System.out.println(message[i].getSubject());
try (OutputStream out = new FileOutputStream("something"+i+".eml")) { // fix the name of course
message[i].writeTo(out);
}
}
folder.close(true);
store.close();
}
}
In writeTo it makes endless loop and creates file which grows until the disk is full. This is from com.sun.mail.imap.IMAPMessage
/**
* Write out the bytes into the given OutputStream.
*/
public void writeTo(OutputStream os)
throws IOException, MessagingException {
if (bodyLoaded) {
super.writeTo(os);
return;
}
InputStream is = getMimeStream();
try {
// write out the bytes
byte[] bytes = new byte[16*1024];
int count;
while ((count = is.read(bytes)) != -1)
os.write(bytes, 0, count);
} finally {
is.close();
}
}
It will never exit from the while clause. Should it be some error on the Postfix side? The mail is multipart MIME message and the EML created are just multiple copies of EML. Only difference between prod and dev machine is HW. JDK8 is the same. Any hints?
Java mail API used:
<dependency>
<groupId>com.sun.mail</groupId>
<artifactId>javax.mail</artifactId>
<version>1.5.6</version>
</dependency>
UPDATE SOLVED: ESET Nod32 Antivirus software intercept the IMAP traffic in that way, that the buffer never returns -1! After disabling ESET Mail, everything works.

Java Current Directory Returns `null`

I'm trying to use the printWorkingDirectory() from Apache Commons FTP but it's only returning null. I can't navigate directories, list files, etc.
Log in pass all is success but how ever I try I can not change current directory.
I use this following code:
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.net.ftp.FTP;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPFile;
public class FTPDownloadFileDemo {
public static void main(String[] args) {
String server = "FTP server Address";
int port = portNo;
String user = "User Name";
String pass = "Pasword";
FTPClient ftpClient = new FTPClient();
String dir = "stocks/";
try {
ftpClient.connect(server, port);
ftpClient.login(user, pass);
System.out.println( ftpClient.printWorkingDirectory());//Always null
//change current directory
ftpClient.changeWorkingDirectory(dir);
boolean success = ftpClient.changeWorkingDirectory(dir);
// showServerReply(ftpClient);
if (success)// never success
System.out.println("Successfully changed working directory.");
System.out.println(ftpClient.printWorkingDirectory());// Always null
} catch (IOException ex) {
System.out.println("Error: " + ex.getMessage());
ex.printStackTrace();
} finally {
try {
if (ftpClient.isConnected()) {
ftpClient.logout();
ftpClient.disconnect();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}

This is rather old question that deserves an answer. This issue is likely a result of using FTPClient when secure connection is required. You may have to switch to FTPSClient if that is, indeed, the case. Further, output the response from the server with the following code snippet to troubleshoot the issue if secure client doesn't solve the it:
ftpClient.addProtocolCommandListener(
new PrintCommandListener(
new PrintWriter(new OutputStreamWriter(System.out, "UTF-8")), true));
Also, a server can reject your login attempt if your IP address is not white listed. So, being able to see the logs is imperative. The reason you see null when printing current working directory is because you are not logged in. Login method will not throw an exception but rather return a boolean value indicating if the operation succeeded. You are checking for success when changing a directory but not doing so when logging in.
boolean success = ftpClient.login(user, pass);

I faced the same, but I came across with a simple step.
Just added this.
boolean success = ftpClient.changeWorkingDirectory(dir);
ftpClient.printWorkingDirectory(); //add this line after changing the working directory
System.out.println(ftpClient.printWorkingDirectory()); //wont be getting null
Here I have the code and the console output
FTPClient.changeWorkingDirectory - Unknown parser type: "/Path" is current directory
I know I replied too soon ;-P, but I saw this post recently. Hope this helps to future searchers ;-)

java ftp - download from guest account on server without login name or password

I am trying to download a file from a server. The server has a guest account without a login or password.
The code was adapted from http://www.dreamincode.net/forums/topic/32031-ftp-in-java-using-apache-commons-net. The reply code is 220, which means "Service ready for new user", but the size of the downloaded file is 0 Bytes. The size of the file on the server is 845 Bytes.
Thank you for your time.
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.SocketException;
import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPReply;
public class FtpTest {
public static FTPClient ftp = new FTPClient();
public static void main(String []args) throws IOException{
String ftpStr = "ftp.ncbi.nih.gov";
String path = "ftp.ncbi.nih.gov/genomes/MapView/Mus_musculus/non_sequence/README";
try {
ftp.connect(ftpStr);
} catch (SocketException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
int reply = ftp.getReplyCode();
System.out.println(reply); //Output: 220
System.out.println("Connected");
File file = new File("README");
FileOutputStream dfile = new FileOutputStream(file);
ftp.retrieveFile(path,dfile);
ftp.disconnect();
System.out.println("Finished");
}
}

To access an FTP server with a guest account (without username or password), you should use the username anonymous with an empty password.

I know this is two years late, but I had the same issue. The variable remote in: public boolean retrieveFile(String remote, OutputStream local) throws IOException
expects a file name, not the the full address to the file. So you should pass "README" instead of path. Before doing that, you should change the working directory of your ftp client to "genomes/MapView/Mus_musculus/non_sequence"

Can't read property file on Server side with GWT

I want to read property file on Server side. I have DBConfig.java, useDBConfig.java and DBConfig.properties all placed in server package. I can't read the values from property file on Server Side. Your help is highly appreciated.
public interface DBConfig extends Constants {
#DefaultStringValue("host")
String host(String host);
#DefaultStringValue("port")
String port(String port);
#DefaultStringValue("username")
String username(String username);
#DefaultStringValue("password")
String password(String password);
}
public void useDBConfig() {
DBConfig constants = GWT.create(DBConfig.class);
Window.alert(constants.host());
host = constants.host(host);
port = constants.port(port);
username = constants.username(username);
password = constants.password(password);
}
property file...
host=127.0.0.1
port=3306
username=root
password=root
Thanks in advance.

GWT.Create can be used only in client mode.
Are you sure that code execute in server side?
If i write in my application GWT.Create in server side i get this error:
java.lang.UnsupportedOperationException: ERROR: GWT.create() is only usable in client code! It cannot be called, for example, from server code. If you are running a unit test, check that your test case extends GWTTestCase and that GWT.create() is not called from within an initializer or constructor.
You can read a properties files in java. The file is similar that Constants files in GWT.
Example of Properties file:
key = value
host = 127.0.0.1
port = 80
username = guest
password = guest
EOF
You can read this file, see the next code:
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.Properties;
String fileToRead = "MY_PATH"+File.separator+"MY_FILE.properties";
Properties prop = new Properties();
try {
File propertiesFile = new File(fileToRead);
prop.load(new FileInputStream(propertiesFile));
String host = prop.getProperty("host");
String port = prop.getProperty("port");
String username = prop.getProperty("username");
String password = prop.getProperty("password");
System.out.println(host);
System.out.println(port);
System.out.println(username);
System.out.println(password);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
If the key doesnt exists getProperty(String key) return null.
You can use prop.containsKey(String key); to see if the key exists. This function return a boolean (True if exists False in other case).
Greetings