Develop Reference

Why doesn't InputStream work with HTTPS in Android?

While I was changing from open weather api to dark sky api, I found a problem with the connection.
I simply want to get the JSON-response from that api and with the open weather map api, everything worked just well. Now I decided to use the dark sky api instead. I just adapted everything as always but it doesn't work.
Maybe it has a problem with https? (the dark sky api uses an https-URL where the open weather map api used an http-URL).
Anyway, I catch an IOException in
inputStream = connection.getInputStream();
Here is my class:
import com.nymvno.hiob.prototyp_v30.Utils.Utils;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class WeatherHttpClient {
public String getWeatherData(String place) {
HttpURLConnection connection;
InputStream inputStream;
try {
connection = (HttpURLConnection) (new URL(Utils.BASE_URL + place)).openConnection();
connection.setRequestMethod("GET");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.connect();
//Read the response
StringBuffer stringBuffer = new StringBuffer();
inputStream = connection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String line;
while ((line = bufferedReader.readLine()) != null) {
stringBuffer.append(line + "\r\n");
}
inputStream.close();
connection.disconnect();
return stringBuffer.toString();
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
}

You must have to varify your host name from your Application class
check my code
public class MyApp extends Application{
#Override
public void onCreate(){
super.onCreate();
handleSSLHandshake();
}
#SuppressLint("TrulyRandom")
public static void handleSSLHandshake() {
try {
TrustManager[] trustAllCerts = new TrustManager[]{new X509TrustManager() {
#Override
public void checkClientTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException {
}
#Override
public void checkServerTrusted(java.security.cert.X509Certificate[] chain, String authType) throws CertificateException {
}
#Override
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return new java.security.cert.X509Certificate[0];
}
}};
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier() {
#Override
public boolean verify(String hostname, SSLSession arg1) {
if(hostname.equalsIgnoreCase("your host name")){
return true;
}else {
return false;
}
}
});
} catch (Exception ignored) {
}
}
}

As you have mentioned, there is just a difference of http/https protocol. For URLS, having https protocols, you must use HttpsURLConnection API rather than HttpURLConnection. If you use HttpURLConnection API for https url exception will be thrown. For further details you may refer below link.
https://developer.android.com/reference/javax/net/ssl/HttpsURLConnection.html

To download a response of the below wget command in windows

I want to download a response of the below wget command in windows:
wget --no-check-certificate --post-data 'login=xxx&password=yyy' https://AAA.BBB.CCC.DDD:1234/login -O -
This command works fine in linux, but it's not working in windows. I know if I install wget on windows, then it will work. I want to run it without installing it. I want the option for above wget command for above specified url to achieve the target.
So far i have tried the below snippet of code But it only returns html info (source code) of the page without going to the homepage using username and password.
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.X509Certificate;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import org.apache.commons.codec.binary.Base64;
public class ConnectToUrlUsingBasicAuthentication
{
static
{
disableSslVerification();
}
private static void disableSslVerification()
{
try
{
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] {new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(X509Certificate[] certs, String authType) {
}
}
};
// Install the all-trusting trust manager
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
// Create all-trusting host name verifier
HostnameVerifier allHostsValid = new HostnameVerifier()
{
public boolean verify(String hostname, SSLSession session)
{
return true;
}
};
// Install the all-trusting host verifier
HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);
}
catch (NoSuchAlgorithmException e)
{
e.printStackTrace();
}
catch (KeyManagementException e)
{
e.printStackTrace();
}
}
public static void main(String[] args)
{
try
{
String webPage = "https://AAA.BBB.CCC.DDD:1234/login";
String name = "xxx";
String password = "yyy";
String authString = name + ":" + password;
System.out.println("auth string: " + authString);
byte[] authEncBytes = Base64.encodeBase64(authString.getBytes());
String authStringEnc = new String(authEncBytes);
System.out.println("Base64 encoded auth string: " + authStringEnc);
URL url = new URL(webPage);
URLConnection urlConnection = url.openConnection();
urlConnection.setRequestProperty("Authorization", "Basic " + authStringEnc);
javax.net.ssl.HostnameVerifier myHv = new javax.net.ssl.HostnameVerifier(){
public boolean verify(String
hostName,javax.net.ssl.SSLSession session) { return true; }
};
((HttpsURLConnection) urlConnection).setHostnameVerifier(myHv); urlConnection.setDoOutput(true);
urlConnection.setDoInput(true);
InputStream is = urlConnection.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
int numCharsRead;
char[] charArray = new char[1024];
StringBuffer sb = new StringBuffer();
while ((numCharsRead = isr.read(charArray)) > 0)
{
sb.append(charArray, 0, numCharsRead);
}
String result = sb.toString();
System.out.println("*** BEGIN ***");
System.out.println(result);
System.out.println("*** END ***");
}
catch (MalformedURLException e)
{
e.printStackTrace();
}
catch (IOException e)
{
e.printStackTrace();
}
}
}

javax.net.ssl.HttpsURLConnection returning martian poetry

I am writing a simple https client that will pull down the html of a webpage over https. I can connect to the webpage fine however the html I pull down is gibberish.
public String GetWebPageHTTPS(String URI){
BufferedReader read;
URL inputURI;
String line;
String renderedPage = "";
try{
inputURI = new URL(URI);
HttpsURLConnection connect;
connect = (HttpsURLConnection)inputURI.openConnection();
connect.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401");
read = new BufferedReader (new InputStreamReader(connect.getInputStream()));
while ((line = read.readLine()) != null)
renderedPage += line;
read.close();
}
catch (MalformedURLException e){
e.printStackTrace();
}
catch (IOException e){
e.printStackTrace();
}
return renderedPage;
}
When I pass it a string like https://kat.ph/ around 10,000 characters of gibberish is returned
EDIT
Here is my modified code for self-signing certs however I'm still getting the encrypted stream:
public String GetWebPageHTTPS(String URI){
TrustManager[] trustAllCerts = new TrustManager[] {
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
}
};
try {
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (GeneralSecurityException e) {
}
try {
System.out.println("URI: " + URI);
URL url = new URL(URI);
} catch (MalformedURLException e) {
}
BufferedReader read;
URL inputURI;
String line;
String renderedPage = "";
try{
inputURI = new URL(URI);
HttpsURLConnection connect;
connect = (HttpsURLConnection)inputURI.openConnection();
read = new BufferedReader (new InputStreamReader(connect.getInputStream()));
while ((line = read.readLine()) != null)
renderedPage += line;
read.close();
}
catch (MalformedURLException e){
e.printStackTrace();
}
catch (IOException e){
e.printStackTrace();
}
return renderedPage;
}

"is it compressed by any chance? stackoverflow.com/questions/8249522/…" – Mahesh Guruswamy
yes, turns out it was just gzip compressed here is my work around for this
public String GetWebPageGzipHTTP(String URI){
String html = "";
try {
URLConnection connect = new URL(URI).openConnection();
BufferedReader in = null;
connect.setReadTimeout(10000);
connect.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.3) Gecko/20100401");
if (connect.getHeaderField("Content-Encoding")!=null && connect.getHeaderField("Content-Encoding").equals("gzip")){
in = new BufferedReader(new InputStreamReader(new GZIPInputStream(connect.getInputStream())));
} else {
in = new BufferedReader(new InputStreamReader(connect.getInputStream()));
}
String inputLine;
while ((inputLine = in.readLine()) != null){
html+=inputLine;
}
in.close();
return html;
} catch (Exception e) {
return html;
}
}
}

HTTPS always presents a Certificate and the further communication happens on a secure encrypted channel. That is why what you are receiving looks like gibberish.
For any signed certificates, HttpsURLConnection will do the work for you and everything works. Things become muddy when the Certificate is not signed by a certificate authority. In such instances if you open that URL from a browser, it will present the Certificate for you to examine and accept before continuing.
Looks like you have the similar issue here. What you need to do is to tell Java to accept self-signed certificates without complaining. You have two options here, either download the certificate (just open the URL in any browser and it will show you how to) and add it to the keystore inn your JVM or create your own TrustManager and disable the Certificate Validate.
See this SO answer for details of both these options. https://stackoverflow.com/a/2893932/2385178

Unable to post the data to server, getting error java.io.IOException: Server returned HTTP response code: 415

Iam unable to post the data to the server, getting error as . But it is working fine in curl script.
Error reading URL
java.io.IOException: Server returned HTTP response code: 415 for URL: https://8.7.177.4/api/domains/amj.nms.mixnetworks.net/subscribers/9001?do_not_disturb=no
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at CurlAuthentication.authenticatePostUrl(CurlAuthentication.java:109)
at CurlAuthentication.main(CurlAuthentication.java:134)
Error reading URL
Below is the code.
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLSession;
public class CurlAuthentication {
public void authenticatePostUrl() {
HostnameVerifier hv = new HostnameVerifier() {
#Override
public boolean verify(String urlHostName, SSLSession session) {
System.out.println("Warning: URL Host: " + urlHostName
+ " vs. " + session.getPeerHost());
return true;
}
};
// Now you are telling the JRE to trust any https server.
// If you know the URL that you are connecting to then this should
// not be a problem
try {
trustAllHttpsCertificates();
} catch (Exception e) {
System.out.println("Trustall" + e.getStackTrace());
}
HttpsURLConnection.setDefaultHostnameVerifier(hv);
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
try {
URL url = new URL("www.stackoverflow.com");
String credentials = "user" + ":" + "password";
String encoding = Base64Converter.encode(credentials.getBytes("UTF-8"));
HttpsURLConnection uc = (HttpsURLConnection) url.openConnection();
uc.setDoInput(true);
uc.setDoOutput(true);
uc.setRequestProperty("Authorization", String.format("Basic %s", encoding));
uc.setRequestMethod("POST");
uc.setRequestProperty("Content-Type", "application/xml");
uc.setRequestProperty("Accept", "application/xml");
uc.getInputStream();
System.out.println(uc.getContentType());
InputStream content = (InputStream) uc.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(
content));
String line;
while ((line = in.readLine()) != null) {
pw.println(line);
}
} catch (MalformedURLException e) {
e.printStackTrace();
pw.println("Invalid URL");
} catch (IOException e) {
e.printStackTrace();
pw.println("Error reading URL");
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(sw.toString());
}
public static void main(String[] args) {
// TODO Auto-generated method stub
CurlAuthentication au = new CurlAuthentication();
au.authenticatePostUrl();
}
// Just add these two functions in your program
public static class TempTrustedManager implements
javax.net.ssl.TrustManager, javax.net.ssl.X509TrustManager {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public boolean isServerTrusted(
java.security.cert.X509Certificate[] certs) {
return true;
}
public boolean isClientTrusted(
java.security.cert.X509Certificate[] certs) {
return true;
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType)
throws java.security.cert.CertificateException {
return;
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType)
throws java.security.cert.CertificateException {
return;
}
}
private static void trustAllHttpsCertificates() throws Exception {
// Create a trust manager that does not validate certificate chains:
javax.net.ssl.TrustManager[] trustAllCerts =
new javax.net.ssl.TrustManager[1];
javax.net.ssl.TrustManager tm = new TempTrustedManager();
trustAllCerts[0] = tm;
javax.net.ssl.SSLContext sc =
javax.net.ssl.SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, null);
javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(
sc.getSocketFactory());
}
}
Do iam doing anything wrong from the above code?
DO i need to use curl script to post the data?

Well, as Error 415 is already stating:
415 Unsupported Media Type
The request entity has a media type which the server or resource does not support. For example, the client uploads an image as image/svg+xml, but the server requires that images use a different format.
Without knowing the specifications of what the server is expecting of you to send him, it's hard to tell what you are missing. It though seems as if you only want to receive data from the server and not send him anything.
By setting uc.setRequestProperty("Content-Type", "application/xml"); you tell the server that you hand him over some XML data (which you don't) and he probably also wouldn't expect this, so he's giving you that error.
As your sending some urlencoded data in your POST data, try setting this to:
uc.setRequestProperty("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");

Android Session cookies without using CookieManager

My application makes multiple web calls in order to get authentication. I need to store this session in a cookie. I wanted to use Cookie Manager but after doing some research, I found out it is only available to API 9 and above and my application needs to be backward compatible.
I make my web connections using HTTPURLConnection to a secure HTTPS. Quick example of my code
public String iStream_to_String(InputStream is)
{
BufferedReader rd = new BufferedReader(new InputStreamReader(is), 4096);
String line;
StringBuilder sb = new StringBuilder();
try
{
while ((line = rd.readLine()) != null)
{
sb.append(line);
}
rd.close();
} catch (IOException e)
{
e.printStackTrace();
}
String contentOfMyInputStream = sb.toString();
return contentOfMyInputStream;
}
final static HostnameVerifier DO_NOT_VERIFY = new HostnameVerifier()
{
public boolean verify(String hostname, SSLSession session)
{
return true;
}
};
/**
* Trust every server - dont check for any certificate
*/
private static void trustAllHosts()
{
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[]
{ new X509TrustManager()
{
public java.security.cert.X509Certificate[] getAcceptedIssuers()
{
return new java.security.cert.X509Certificate[]
{};
}
public void checkClientTrusted(X509Certificate[] chain,
String authType) throws CertificateException
{
}
public void checkServerTrusted(X509Certificate[] chain,
String authType) throws CertificateException
{
}
} };
// Install the all-trusting trust manager
try
{
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection
.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (Exception e)
{
e.printStackTrace();
}
}
Then I make a request like so
try
{
url = new URL(url1);
trustAllHosts();
HttpsURLConnection https = (HttpsURLConnection) url.openConnection();
https.setHostnameVerifier(DO_NOT_VERIFY);
http = https;
InputStream in = new BufferedInputStream(http.getInputStream());
sAuthenticateP1 = iStream_to_String(in);
in.close();
} catch (Exception e)
{
e.printStackTrace();
}
The full authentication is done in 4 steps. I need to have it so the session is remembered throughout the 4 steps. Seeing I can't use CookieManager, I have been looking around for other ways of doing this, but can't seem to find any. Would anyone be able to point me in the right direction.
Thanks in advance!!

Figured it out. Incase anyone else is having similar problem, will give quick outline of code. As I said before mine is a several step authentication process. So after the first request, after you have received a response, take the cookie like so
String cookie = http.getRequestProperty("Cookie");
if (cookie != null && cookie.length() > 0)
{
sCookie = cookie;
Log.v("cookie2", sCookie);
}
sCookie is a static string variable I have set up. Then in the next request, after this line
https = (HttpsURLConnection) url.openConnection();
Just put
https.setRequestProperty("Cookie", sCookie);
https.setRequestMethod("POST");
https.setDoInput(true);
https.setDoOutput(true);
And do the same thing for each request after that requires the session, and it should work fine