I am developing an android application and want to connect to a REST service using URLConnection. The resource is protected using Digest Authentication. I can successfully access the REST service via the browser. I do not want to use HttpClient because URLConnection is the more current way to connect for the future.
But when I try to access the resource via the code below, I get a 401 Error with an java.io.FileNotFoundException. I have researched this thoroughly but no success, solutions appreciated.
Note: My REST service has been developed using Grails, I am running this code in the android emulator with code developed in Eclipse on the windows 64 bit OS.
CODE
URL myurl = new URL("http://10.0.2.2:8080/MyREST/customers");
HttpURLConnection myurlConnection = (HttpURLConnection) myurl.openConnection();
String basicAuth = "Basic " + (Base64.encode(userpass.getBytes(),android.util.Base64.DEFAULT)).toString();
myurlConnection.setRequestProperty ("Authorization", basicAuth);
try {
int responseCode1 = ((HttpURLConnection) myurlConnection).getResponseCode();
Log.i("MyLongOperation", "Check connection" +Integer.toString(responseCode1) );
InputStream in = new BufferedInputStream(myurlConnection.getInputStream());
readStream(in);
}
finally {
myurlConnection.disconnect();
}
I have also tried setting authentication at a global level with no effect
Authenticator.setDefault(new Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(username, password.toCharArray());
}
}
);
I have referred to this article - with no success. Connecting to remote URL which requires authentication using Java
If the resource is protected by "Digest" then sending a "Basic" authorization scheme in your code will not work because the server would not recognize it.
Secondly, by using a "preemptive" authentication, setting the Authorization header w/o it being requested is kind of a security hole. You will be sending information to the server that it has not requested.
Thirdly, the "Authenticator.setDefault" most likely will not be requested as there was some significant back-and-forth caused by MicroSoft's implementation of HTTP digest authentication (YMMV may vary on my recollection of this). As such, Sun/Oracle decided to leave this behavior disabled by default as per this document.
That said, you may be better off looking into utilizing the Apache HTTP client bundled with Android to do this. There is a bundled implementation Digest Authentication included. There is an example of "preemptive" digest authentication located here.
Couple of caveats to be aware of:
Pay CLOSE attention to the "HttpHost" stored in "target" - this MUST MATCH EXACTLY the host name, protocol port, and protocol scheme used in the URL being retrieved.
The example provided is for HTTP Client 4.2.x. I am not 100% sure of the version included in Android but you should be able to locate working examples.
Update Submitter has provided additional comments with regard to the statement that it is recommended by Google to use the HttpURLConnection with articles here and here.
While I trust the statements made by Tim Bray with regard to the reasoning as to why you should be using the provided HttpURLConnection object for performing these calls, I do not agree that they should be immediately accepted on face value.
There is no indication as to the level of support of digest authentication provided by the implementation in Android. As I mentioned earlier, the HttpURLConnection does not support immediately as it has been known to be buggy.
If you are decided that you are going to use HTTP Digest Authentication, regardless of the fact that it has been deemed unstable by the majority of the community, I would attempt to set the following system properties in your application as EARLY as possible during the Android lifecycle:
http.auth.digest.validateServer="true"
http.auth.digest.validateProxy="true"
By doing so, this should enable the digest authentication scheme.
I am, again, going to re-iterate that the Apache HTTP Client bundled with Android was developed and designed specifically to address short-comings of the basic Java HttpURLConnection, providing much a much broader and robust client for dealing with HTTP(s) based data streams.
I would recommend trying a couple of things as well, see if you can configure your container to provide "Basic" authentication protection. The other, more complex option, would be to possibly provide X.509 Certificate Based authentication.
I hope that this clarification helps you get to your goal.
change
'Base64.encode ...'
to
'Base64.encodeToString ...'
Related
CloseableHttpClient httpclient = HttpClients.createDefault();
HttpGet httpget = new HttpGet(url);
System.out.println(httpclient.execute(httpget));
I am using this programming snippet and using this with couple of urls. "https://abc.def.com/file.txt" and "https://ab_c.def.com/file.txt" (actual names i used exist - this is just an example). When i run the code with -Djavax.net.debug=ssl:handshake:data options set, I see _server_name_ extension is added only in case of abc.def.com and not in case of ab_c.def.com.
I am aware that java.net.URL package doesn't allow '_' in hostname. Is this a similar issue? Is there any workaround to get the SSL handshake include the hostname with '_'?
The answer is no. This wont work in java using the standard JDK libraries with HTTPS.
There virtually no "normal" Java way to work around this problem.
Other ideas:
ask the service owner to provide a domain name without the _.
dont use SSL/HTTPS. Most Java HTTP clients that are not doing strict checks will work with a hostname with an underscore in it. This is not a great solution if you are on the open internet
do use HTTPS but go direct to the IP address and disable HTTPS hostname validation. This is also a bad idea if you are on the open internet
introduce a MITM or forward proxy that can provide a HTTP endpoint to your service, but will do HTTPS over the untrusted parts of the network, (SSL onloading?)
just shell out to curl and read the resulting string, as described here on Baeldung. Be very careful here, as it's very easy to unwittingly open yourself up security issues like remote code execution
use jni to get to a network library that will allow the underscore - libcurl is an obvious candidate, but this will be significantly more work than the System.execute method, also possibly unsafe for other reasons (memory management, instability of badly written jni interop, not great for maintainability)
You could possibly try to fork the related URI / URL parsing classes in java.net to remove the check for the underscore. The problem with this is it's a massive hack, possibly of unbounded difficulty, could easily break all manner of things unintentionally and you might need to build your own version of the JDK as I have some recollection of java.net code being special for some reason in that you couldn't just chuck a patched class in ext/lib dir of the JVM (but I could be mixing that up with something else).
I will provide a hotfix.
In your /etc/hosts file, include a new entry without the underscore to the IP.
For example,
abc.com xx.xx.xx.xx
Then, use abc.com in your program.
So I have this situation: I try to download an image from somedomain.com using HTTPS. The domain is probably misconfigured, but unfortunately I can't change that. What exactly is happening:
When I browse to https://somedomain.com/animage.jpg I get a valid certificate issued for somedomain.com, which is perfect. But when I call the same site using it's IP address, say https://123.123.123.123 - I get a (also valid) certificate for *.hostingcompany.com - the certificate of the hosting company.
Now, I try to download the contents of the file using Java's HttpsUrlConnection, nothing special:
var urlConnection = new URL(imageUrl).openConnection();
((HttpURLConnection) urlConnection).getResponseCode();
(I want to first check the response code, but it's not important here.)
This code runs inside a Spring Boot App and is run on request. It works fine for the first request since booting the app. Each subsequent request fails with java.security.cert.CertificateException: No subject alternative DNS name matching somedomain.com found. It's because on each subsequent request the SSL Handshake is sent to the IP, not hostname, and get's the hosting company's certificate.
I was trying to find different settings for the SSL classes, but to no avail. I know there is a workaround where I could supply my own HostnameVerifier which could just return true, but that won't be secure, so I don't want to do that.
Did anyone encounter such problem? Maybe I'm searching in the wrong places? Maybe it's something with DNSes? I will appreciate any help.
Turns out it is a bug in Java 11.01. It is fixed since 11.02. After switching to 11.03. the behaviour I described above is gone. Each request gets a proper certificate.
Here are the details of the bug: https://bugs.openjdk.java.net/browse/JDK-8211806
From SCAVA security scan tool it is reported that some of my code lines are subject to vulnerability.
The vulnerability classification is :
Insufficient_Sensitive_Transport_Layer
The vulnerable code contains
httpclient.addHeader() method sending password to login a server/api.
Application is in java langauge.
From google, I got to know that we can add some encryption method to avoid this vulnerability.
HttpGet getMethod = null;
getMethod = new HttpGet("http/https url to connect");
getMethod.addHeader(USERNAME,PASSWORD); // Vulnerability reported in
//this line as it is sending the password without any protection.
How can i prevent this vulnerability and pass the password in a secure way through some encryption method.
Thanks in advance.
Enable Transport Layer Security (TLS) each time you have to send credentials as plain text. If you have correctly configured https:// with latest secure version of TLS the communication will be secure.
Moreover, there are situations where sending plain text credentials can't be avoided e.g. user login process. Automatic scanning tools might not be smart enough to distinguish valid use cases and raise a false-positive.
Currently at a loss for authenticating with a Microsoft Project Server 2007 instance running on IIS with Integrated Windows Authentication enabled from a Java 1.6(u19) client running on linux, RHEL 5.5.
Note: The client works on my Windows workstation.
I initially was trying to implement a JAX-WS call and found that I could not retrieve the WSDL due to authentication errors, specifically a 401.2, followed by a 500. So I simplified it to a Java class that:
Creates an Authenticator and sets it as the default with a user name/password in AD that has permissions to the project server site
Create a java.net.URL object
Create a java.net.HttpURLConnection and invoke getInputStream
It is at this point where a failure occurs.
With HttpURLConnection debugging turned on I can see:
the initial authentication failure (401.2) returned from the server with "negotiate" and "NTLM" included in the response.
the client creating an NTLM token and sending it back to the server
the server returning with a 500 status code
On the Windows server in the logs, I can see that there is no user name included in the log file only for my requestion and only a "-" which I believe means "anonymous".
My thought is that Project Server isn't liking the NTLM token that is being passed and choking. Based on the many postings on this, NTLM (v1 & v2) are suppose to be supported within Java 1.6.
Any help would be greatly appreciated...
UPDATE 6/20/12: narrowed the issue down to a local security policy setting for Network security: Minimum session security for NTLM SSP based (including RPC) servers. The setting that causes the Java client to fail is Require NTLMv2 security. The goes against what is claimed for NTLM support with the 1.6 JDK..
Some references:
Java HTTP Authentication
Blog showing Java Authenticator Impl
A while back when i had this problem, i ended up using a scheme created by somebody else.
http://devsac.blogspot.com/2010/10/supoprt-for-ntlmv2-with-apache.html
Worked for me when i had to get image files from and iis server with ntlm.
Snippet using the code above..
AuthPolicy.registerAuthScheme(AuthPolicy.NTLM, org.xyz.JCIFS_NTLMScheme.class);
HttpClient client = new HttpClient();
client.getState().setCredentials(AuthScope.ANY, new NTCredentials(userName, password, "", strDomain));
GetMethod get = new GetMethod(strImageFile);
get.setDoAuthentication(true);
client.executeMethod(get);
I can easily get authenticated in this site
via browser. But there's NO POSSIBLE way to connect to it using Java. I always get 401 error. I tried to connect via Jsoup, and HTMLUnit and still nothing.. Anyone has at least some explanation as to why this is happening?
Oh, by the way... In just trying to connect to the page WITHOUT authenticating, I get the 401 error. Which in my opinion is pretty damn weird.
Basic URLConnection = new URLConnection(URL); combo (Native Java API) didn't work.
Jsoup with Document = Response.parse(); didn't work
HTMLUnit using WebClient didn't work either
Any of the above trying to emulate a browser didn't work as well
The 401 error is an authentication error. This is likely because the agent you are specifying (or failing to specify) when you connect isn't considered valid. (There could be more than that, depending on how that site is set up.)
You can specify a user agent in JSoup, see this answer to that question on SO.
If you use HttpURLConnection, maybe it helps...
java.net.Authenticator.setDefault( new Authenticator() {
protected java.net.PasswordAuthentication getPasswordAuthentication() {
return new java.net.PasswordAuthentication ("user", "pass".toCharArray());
}
});