Develop Reference

UnknownHostException behind Proxy

I have a java program, that connects to a website to retrieve some XML from it. This works fine on my computer, as well as others outside our company. One of our customers is now not able to connect to the website. I figured out, that they are behind a proxy. I have now found which settings I need to use, and in my test program it works (partially).
In the code below, the downloadFile() call works as expected, and the file can be downloaded without problems. The contactHost() fails on our client machines with an UnknownHostException:
java.net.UnknownHostException: No such host is known (api.myserver.de)
at java.base/java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.base/java.net.InetAddress$PlatformNameService.lookupAllHostAddr(InetAddress.java:925)
at java.base/java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1505)
at java.base/java.net.InetAddress$NameServiceAddresses.get(InetAddress.java:844)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1495)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1354)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1288)
at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:111)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
Background: Windows 10 machines, our program is shipped with an internal OpenJDK, version "10.0.2" 2018-07-17. The program is started with the following defines -Djdk.http.auth.tunneling.disabledSchemes="" -Djava.net.preferIPv4Stack=true in order to use IP4 only, and to enable BasicAuthentification for the Proxy. With these settings, the file can be downloaded, however the UnknownHostException is still there.
We have also tried to open the used URL in an browser, and this works as excepted, i.e. in the browser the website is opened.
Here is my code for testing:
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.Authenticator;
import java.net.URL;
import java.net.URLConnection;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
public class LFTProxyTest {
private static String uname = null;
private static String pass = null;
public static void main(String[] args) {
System.setProperty("java.net.useSystemProxies", "true");
// uname = "test"; // whatever that user provides
// pass = "sectret"; // whatever that user provides
Authenticator.setDefault(new ProxyAuth(uname, pass));
contactHost();
downloadFile();
}
private static boolean downloadFile() {
System.out.println("CHECK connection");
int cp = contactHost();
if (cp == 200)
return true;
if (cp == 407)
return false;
else {
try {
System.out.println("Try loading file: ");
URL url = new URL("https://www.google.de");
URLConnection urlConnection = url.openConnection();
InputStream in = new BufferedInputStream(urlConnection.getInputStream());
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
dBuilder.parse(in);
System.out.println(" FILE DOWNLOAD successfull!");
} catch (Exception e) {
System.out.println(" FILE DOWNLOAD failed:");
System.out.println("***EXCEPTION: " + e.getMessage());
return false;
}
}
System.out.println("CHECK done");
return true;
}
private static int contactHost() {
HttpClient client = HttpClientBuilder.create().build();// new DefaultHttpClient();
String catalogURI = "https://api.myserver.de/query";
HttpGet request = new HttpGet(catalogURI);
try {
int ret = 0;
HttpResponse response = client.execute(request);
ret = response.getStatusLine().getStatusCode();
System.out.println("PROXY test: " + ret);
((CloseableHttpClient) client).close();
return ret;
} catch (IOException e) {
e.printStackTrace();
return -1;
}
}
}
I don't know what do do know, I'm not even sure where the error could be. Any ideas are highly appreciated!

Ok, so after some further digging, I found out that org.apache.http.client.HttpClient is not respecting java.net.useSystemProxies at all, be it set via System or via -D. And it is also ignoring http.proxyHost etc. Solution is to use a ProxySelector like this:
ProxySelector.setDefault(new ProxySelector() {
#Override
public List<Proxy> select(URI uri) {
ArrayList<Proxy> list = new ArrayList<Proxy>();
list.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy1.de", 8000)));
list.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy2.de", 8080)));
return list;
}
#Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
logger.error("Error in ProxySelector, connection Failed: ", ioe);
}
});
I'm getting another exception now, but I might open another thread for this.

UnknownHostException designates a pretty straight forward problem. That the IP address of the remote host you are trying to reach cannot be resolved. So the solution to this is very simple. You should check the input of Socket (or any other method that throws an UnknownHostException), and validate that it is the intended one. If you are not whether you have the correct host name, you can launch a UNIX terminal and use the nslookup command (among others) to see if your DNS server can resolve the host name to an IP address successfully.
If you are on Windows you can use the host command. If that doesn’t work as expected then, you should check if the host name you have is correct and then try to refresh your DNS cache. If that doesn’t work either, try to use a different DNS server, eg Google Public DNS is a very good alternative.

How to access a HTTPS url from a java application

I know this is a very basic question, but some how I have managed to not find a solution to this problem. I have a java class that has a main method. In that method, I try to access an https url as below:
package helloworld;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import javax.net.ssl.HttpsURLConnection;
public class ConnectHttps
{
public static void main(String[] argsd)
{
System.out.println("***************Https testing started **************");
try
{
URL u = new URL("https://localhost:8443/myapp/test");
HttpsURLConnection http = (HttpsURLConnection) u.openConnection();
http.setAllowUserInteraction(true);
http.setRequestMethod("GET");
http.connect();
InputStream is = http.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder stringBuilder = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null)
{
stringBuilder.append(line
+ "\n");
}
System.out.println(stringBuilder.toString());
System.out.println("***************Https testing completed **************");
}
catch (IOException e)
{
System.out.println("***************Https testing failed **************");
e.printStackTrace();
}
}
}
On executing this program, the output I get is:
***************Https testing started **************
***************Https testing failed **************
javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No name matching localhost found
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1886)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
at helloworld.ConnectHttps.main(ConnectHttps.java:59)
I guess I am doing a very basic mistake here.
I am using JDK 1.7.0_25.

java.net.ConnectException: Connection timed out: connect
This is not really related to SSL/TLS. Rather, your client can't connect to the server at all (at least not within a reasonable time).
It's quite possible that there's a firewall preventing you from making such connections.
You might have to go through a proxy, in which case setting the https.proxyHost and https.proxyPort system properties should be taken into account by HttpsURLConnection.

Java tutorial sample code throws: java.net.SocketException: Connection reset - what does cause it?

So this is a beginner's question.
When executing the sample code from the working with urls chapter it throws:
Exception in thread "main" java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189) ...
Origin is the openStream() method.
Here is the code:
import java.net.*;
import java.io.*;
public class URLReader {
public static void main(String[] args) throws Exception {
URL oracle = new URL("http://www.oracle.com/");
BufferedReader in = new BufferedReader(
new InputStreamReader(oracle.openStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
}
in.close();
}
}
I know there are similar threads regarding that topic, but i could not find an answer that suits me.
What I've tried so far:
I have set the proxy host as suggested
here. Command was: java -Dhttp.proxyHost=dslb-088-071-100-199.pools.arcor-ip.net, I also tried it with inserting System.setProperty("http.proxyHost", "dslb-088-071-100-199.pools.arcor-ip.net"); in the first line of the URLReader class.
I tried JSoup html parser and
org.apache.commons.io.FileUtils.copyURLToFile(URL, File) method to have a similar result.
Whatever I try, I always get the same error: There will happen nothing for 30 seconds or so and then it throws the mentioned SocketException.
I simply dont know how to continue in solving this problem. Helpful would be to get information about what happens in background during the 30seconds before connection reset.
So what could actually cause this Exception?
The smallest hint could help! Thank you!

Your code is working fine for JVM's that can connect to the internet.
Based on the original question and discussion: https://chat.stackoverflow.com/rooms/31264/discussion-between-achingfingers-and-meewok it seems that either:
An intermediate firewall is blocking the JVM from making the connection (or another similar network issue).
An operating system firewall, or antivirus that is causing the problems as well.
My suggestion is to try:
Same app on different computer within same network (to see if it is PC specific).
Same app on different network.

Try Apache HTTPClient. I hope all the imports are included as this code is not tested as it is... Also your 30s is the connection timeout of your client.
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.ProxySelector;
import java.net.SocketAddress;
import java.net.URI;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.HttpStatus;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.CoreConnectionPNames;
public class URLReader {
public static void main(String[] args) throws Exception {
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(
CoreConnectionPNames.CONNECTION_TIMEOUT, timeOut);
httpclient.getParams().setParameter(
CoreConnectionPNames.SO_TIMEOUT, 2 * timeOut);
httpclient.getParams().setParameter(
CoreConnectionPNames.STALE_CONNECTION_CHECK, false);
httpclient.getParams().setParameter(
CoreConnectionPNames.TCP_NODELAY, true);
HttpHost proxy = new HttpHost(%proxyhost%, %proxyport%);
HttpGet httpget = new HttpGet("http://www.oracle.com");
HttpResponse resp = httpclient.execute(httpget);
respCode = resp.getStatusLine().getStatusCode();
BufferedReader br = new BufferedReader(new InputStreamReader(resp
.getEntity().getContent()));
String line = null;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
br.close();
}
}

Using NTLM authentication in Java applications

I want to use Windows NTLM authentication in my Java application to authenticate intranet users transparently. The users should not notice any authentication if using their browsers (single sign-on).
I've found a few libs with NTLM support, but don't know which one to use:
http://spnego.sourceforge.net/
http://sourceforge.net/projects/ntlmv2auth/
http://jcifs.samba.org/
http://www.ioplex.com/jespa.html
http://www.luigidragone.com/software/ntlm-authentication-in-java/
Any suggestions where to start?

Out of the above list, only ntlmv2-auth and Jespa support NTLMv2. Jespa is workable but commercial. ntlmv2-auth I haven't tried but it's based on the code from Liferay, which I've seen working before.
'ntlm-authentication-in-java' is only NTLMv1, which is old, insecure, and works in a dwindling number of environments as people upgrade to newer Windows versions. JCIFS used to have an NTLMv1 HTTP auth filter, but it was removed in later versions, as the way it was implemented amounts to a man-in-the-middle attack on the insecure protocol. (The same appears to be true of 'ntlm-authentication-in-java'.)
The 'spnego' project is Kerberos not NTLM. If you want to replicate full IWA as IIS does it, you'd need to support both NTLMv2 and Kerberos ('NTLM' auth, 'Negotiate' auth, NTLMSSP-in-SPNego auth and NTLM-masquerading-as-Negotiate auth).

Luigi Dragone's script is really old and seems to always fail.
HttpURLConnection can work with NTLM if you add library jcifs, this example works with latest jcifs-1.3.18 :
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.UnknownHostException;
import java.util.HashMap;
import java.util.Map;
import org.apache.http.impl.auth.NTLMEngineException;
public class TestNTLMConnection {
public static void main(String[] args) throws UnknownHostException, IOException, NTLMEngineException {
// Method 1 : authentication in URL
jcifs.Config.registerSmbURLHandler();
URL urlRequest = new URL("http://domain%5Cuser:pass#127.0.0.1/");
// or Method 2 : authentication via System.setProperty()
// System.setProperty("http.auth.ntlm.domain", "domain");
// System.setProperty("jcifs.smb.client.domain", "domain");
// System.setProperty("jcifs.smb.client.username", "user");
// System.setProperty("jcifs.smb.client.password", "pass");
// Not verified // System.setProperty("jcifs.netbios.hostname", "host");
// System.setProperty("java.protocol.handler.pkgs", "jcifs");
// URL urlRequest = new URL("http://127.0.0.1:8180/simulate_get.php");
HttpURLConnection conn = (HttpURLConnection) urlRequest.openConnection();
StringBuilder response = new StringBuilder();
try {
InputStream stream = conn.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(stream));
String str = "";
while ((str = in.readLine()) != null) {
response.append(str);
}
in.close();
System.out.println(response);
} catch(IOException err) {
System.out.println(err);
} finally {
Map<String, String> msgResponse = new HashMap<String, String>();
for (int i = 0;; i++) {
String headerName = conn.getHeaderFieldKey(i);
String headerValue = conn.getHeaderField(i);
if (headerName == null && headerValue == null) {
break;
}
msgResponse.put(headerName == null ? "Method" : headerName, headerValue);
}
System.out.println(msgResponse);
}
}
}
And if you are curious about the content of each handshake, you can find another example using jcifs and Socket on this thread.

Had to recently implement this at work hence here is updated solution with Spring's RestTemplate:
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.NTCredentials;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.HttpClients;
import org.springframework.http.HttpEntity;
import org.springframework.http.ResponseEntity;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.web.client.RestTemplate;
import java.io.IOException;
public class Runner {
public static void main(String[] args) {
var credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new NTCredentials("username", "password", "", "someDomain"));
try (var client = HttpClients.custom()
.setDefaultCredentialsProvider(credentialsProvider)
.build();) {
var requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(client);
RestTemplate restTemplate = new RestTemplate(requestFactory);
ResponseEntity<String> stringResponseEntity = restTemplate.postForEntity("url", new HttpEntity<>("yourDtoObject"), String.class);
} catch (IOException e) {
e.printStackTrace();
}
}
}
dependencies needed are: spring-web and org.apache.httpcomponents
ps: it is important to enter username without domain otherwise it doesn't work. As in if your domain is companyName/username often people just enter that whole thing as username and what you should do is enter them separately where domain="companyName" and username="username"

Ref: https://jcifs.samba.org/src/docs/faq.html#ntlmv2
Q: Does jCIFS support NTLMv2?
A: Yes. As of 1.3.0, JCIFS fully supports NTLMv2 and uses it by default.
Note: The NTLM HTTP SSO Filter that used to be included with JCIFS cannot support NTLMv2.

Relatively from the list you gave,I would go with JCIFS.
The library is mature , and their documentation is good.
To top it off they had fairly regular releases , and the last one being Nov 2011.
Personal Experience : it was fairly easy to get started when compared to others i have tried (spnego and ntmv2auth)

Communication between my Java Server and Local Client

I've been writing easy Java Server. I'm gonna deploy this code to my student's server and run it there.
public class Demo {
public static void main(String[] args) {
String port = "50000";
ServerAttributes attr = new ServerAttributes();
attr.setPort(Integer.parseInt(port));
Socket socket = null;
ServerSocket serverSocket= null;
try {
serverSocket = new ServerSocket(attr.getPort());
System.out.println("Waiting for accept...");
while(true) {
socket = serverSocket.accept();
// TODO
socket.close();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
I wanna create easy Client code which will be 'talking' with my server. Communication Client->Server is easy. My server is visible for client. But what should I do to provide communication in another way?
Maybe REST is good idea? So, how can I 'teach' my server to answer on REST queries?
I've got piece of code which send data to my GAE server:
package enceladus.server.trash.rest;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
public class RESTGAEServer {
static String httpAddress = "http://*********.appspot.com/sign";
public static void main(String[] args) {
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(httpAddress);
try {
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
nameValuePairs.add(new BasicNameValuePair("guestbookName", "default"));
nameValuePairs.add(new BasicNameValuePair("content", "TEST"));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
#SuppressWarnings("unused")
HttpResponse response = client.execute(post);
} catch (IOException e) {
e.printStackTrace();
}
}
}
Thanks in advance

If you are trying to provide RESTFul service from the server, its not an easy task. What you might want to do is user something like Restlet for bootstrapping your RESTFul server and client.
For more information refer to http://wiki.restlet.org/docs_2.0/13-restlet/21-restlet/318-restlet.html

REST is a very simple an easy way of communicating between a client and a server. REST basically says, use HTTP the way it was meant to be used, even when communicating between computer programs.
Read up on HTTP in case you do not have enough knowledge. Here is one good document: http://www.jmarshall.com/easy/http/
Once you understand how to send and receive HTTP messages on the client and on the server, you are ready to develop RESTful server API:s.
What you need to know about REST is that it is mostly a way of thinking when you design your API. Make sure to utilize HTTP to its full extent and send/receive data in whatever format (usually JSON, XML or UrlEncoded key/value pairs).
I would say you are MUCH better off doing this yourself than to try to learn Restlet or some other huge library at the same time you learn REST. REST and HTTP are both easy stuff - once you get down to the "it's just some text going back and fourth". When you understand these things fully, then you could look at some frameworks.
Here is some information about REST:
http://rest.elkstein.org/