Reading https web page data behind proxy java - java

I want to read a secure webpage data say https://www.paypal.com, i am behind proxy. I tried with
System.setProperty("java.net.useSystemProxies","true");
System.setProperty("htttps.proxyHost","myproxyhost");
System.setProperty("https.proxyPort","443");
URL u = new URL("https://www.paypal.com");
URLConnection uc = u.openConnection();
uc.setDoOutput(true);
StringBuffer sbuf=new StringBuffer();
BufferedReader in = new BufferedReader(
new InputStreamReader(uc.getInputStream()));
String res = in.readLine();
System.out.println(" Response from paypal "+res);
while ((res = in.readLine()) != null){
sbuf.append(res).append(",");
}
in.close();
System.out.println(" Total Data received "+sbuf);
i am getting UnknownHostException all the time, I am successfully fetching data with http websites. Am i missing something?
Thanks,
Rohit

You've got 3 T's in your proxyHost settings, i.e. you are using htttps rather than https.

Related

Android HttpURLConnection always unauthorized 401 response (Spring Boot REST API)

I am attempting to consume a REST API with an Android app I am developing in Android Studio. I have developed the API using Spring Boot. The issue:
Whenever I make a call to my API I am always returned a 401 Unauthorized response. "Pre-authenticated entry point called. Rejecting access."
I have configured my CORS as follows:
cors:
allowed-origins: "*"
allowed-methods: GET, PUT, POST, DELETE, OPTIONS
allowed-headers: "*"
exposed-headers:
allow-credentials: true
max-age: 1800
My api is running on localhost:8080, and I am making the requests from my device with my WLAN IPv4 address.
Such as: curl -v -X GET http://192.168.1.xx:8080/api/users
This returns a 200 OK response.
I am also able to call this in postman/DHC etc.. and receive a 200 OK response.
However, when I call this same address with HttpURLConnection through my android device, I receive a 401 response.
Noob developer here - any ideas as to what might be causing this would be greatly appreciated!
Edit to include my GET Request:
#Override
protected String doInBackground(String... params){
String stringUrl = params[0];
String result;
String inputLine;
try {
//Create a URL object holding our url
URL myUrl = new URL(stringUrl);
//Create a connection
HttpURLConnection connect =(HttpURLConnection)
myUrl.openConnection();
connect.setRequestMethod(REQUEST_METHOD); // GET
connect.setRequestProperty("Host", HOST);
connect.setRequestProperty("Connection", "keep-alive");
connect.setRequestProperty("Origin", ORIGIN);
connect.setRequestProperty("User-Agent", System.getProperty("http.agent"));
connect.setRequestProperty("Content-Type", CONTENT_TYPE);
connect.setRequestProperty("Accept", "*/*");
connect.setRequestProperty("Accept-Encoding", ACCEPT_ENCODING);
connect.setRequestProperty("Accept-Language", ACCEPT_LANGUAGE);
connect.setDoOutput(true);
connect.setDoInput(true);
//Connect to our url
connect.connect();
String responseMessage = connect.getResponseMessage(); // Unathorized
int responseCode = connect.getResponseCode(); //401
//Create a new InputStreamReader
InputStreamReader streamReader = new
InputStreamReader(connect.getInputStream());
//Create a new buffered reader and String Builder
BufferedReader reader = new BufferedReader(streamReader);
StringBuilder stringBuilder = new StringBuilder();
//Check if the line we are reading is not null
while((inputLine = reader.readLine()) != null){
stringBuilder.append(inputLine);
}
//Close our InputStream and Buffered reader
reader.close();
streamReader.close();
//Set our result equal to our stringBuilder
result = stringBuilder.toString();
}
catch(IOException e){
e.printStackTrace();
result = null;
}
return result;
}
Replace:-
InputStreamReader streamReader = new
InputStreamReader(connect.getInputStream());
With:-
InputStreamReader streamReader = new
InputStreamReader(connect.getErrorStram());
This will help you track why 401 then you can proceed solving that issue.
401-- is server side problem, Server is unable to process you request, check if you have to pass data in header or url as you are using get request

How do I send a cookie while trying to grab a sites source?

I am trying to grab a site's source code using this code
private static String getUrlSource(String url) throws IOException {
URL url = new URL(url);
URLConnection urlConn = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
urlConn.getInputStream(), "UTF-8"));
String inputLine;
StringBuilder a = new StringBuilder();
while ((inputLine = in.readLine()) != null)
a.append(inputLine);
in.close();
return a.toString();
}
When I do grab the site code this way I get an error about needing to allow cookies. Is there anyway to allow cookies in a java application just so I can grab some source code? I do have the cookie my browser uses to log me in if that helps.
Thanks
John
This way you would have to deal with raw request data, Go with apache http client that gives you abstraction and some methods to allow to set headers in request

Accessing a URL using a loop

I have created a web application, and hosted in on the server. Now I want to create a java program which will access (or "hit") the URL of my application in a loop, so that I can check how much load can my web application can handle. Also, the program should be able to tell me when the URL was successfully accessed, and when it wasn't.
I tried executing it without using a loop:
try {
URL url = new URL("https://example.com/");
BufferedReader in = new BufferedReader(
new InputStreamReader(url.openStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
}
in.close();
} catch (Exception e) {
System.out.println("e: " + e.toString());
}
But, I got this error:
e: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching example.com found.
Use,
javax.net.ssl.HttpsURLConnection
to connect to https://
something like (note, handle resource closing, exceptions etc left on you)
final URL url = new URL("https://example.com");
final HttpsURLConnection con = (HttpsURLConnection)url.openConnection();
final BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream()));
String input;
while ((input = br.readLine()) != null){
System.out.println(input);
}
However there are lots of tool available to load test it
you are just getting a stream from url, but obviously this url is using HTTPS, so you need a "Public Key" imported to your client application. otherwise the client and server won't be able to communicate with each other.

POST Requests from a Servlet

I am writing a servlet using eclipse that receives POST request from a client that should do some splitting on the received text, access google geolocation api to get some data and display to the user.
On a localhost, this works perfectly fine. On an actual server (tried with Openshift and CloudBees), this doesn't work. I can see the splitting reply but not the reply from google geolocation service. There is always an error logged into the console from google service. However, the same code works perfectly fine on localhost.
After I receive the POST request in the doPost method of the servlet, I am doing the following to access the Google GeoLocation service:
//Attempting to send data to Google Geolocation Service
URL url;
HttpURLConnection connection = null;
try {
//Create connection
url = new URL("https://www.googleapis.com/geolocation/v1/geolocate?key=MyAPI");
connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/json");
connection.setUseCaches (false);
connection.setDoInput(true);
connection.setDoOutput(true);
//Send request with data (output variable has the JSON data)
DataOutputStream wr = new DataOutputStream (
connection.getOutputStream ());
wr.writeBytes (output);
wr.flush ();
wr.close ();
//Get Response
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
StringBuffer response2 = new StringBuffer();
while((line = rd.readLine()) != null) {
response2.append(line);
response2.append('\r');
}
rd.close();
//Write to Screen using out=response.getWriter();
out.println("Access Point's Location = " + response2.toString());
} catch (Exception e) {
e.printStackTrace();
} finally {
if(connection != null) {
connection.disconnect();
}
Could you tell me why this is happening and how can I make this work? Should I resort to something like AJAX or is there someother work around? I am relatively new to coding and hence, trying to refrain from learning AJAX at this stage. Please let me if there's any other way of getting this to work
Your localhost has your localhost IP as a sending IP. Openshift et al has the Openshift et al IP as a sending IP. So the Google API says "I have only seen that localhost IP twice before, that's fine!", whereas it says "I have seen this Openshift IP millions of times before! NO REPLY FOR YOU!"

How to get the Response Back from the Remote Server using an HttpURLConnection Object?

I am trying to send an HTTP POST Request to a remote server using an instance of the HttpURLConnection class. Although, I am able to get a response code and a response message, when I try to write the input stream into a StringBuffer, I am not able to actually read any lines.
When I analyzed the packets sent from WireShark, I noticed that a full response was being sent from the remote server. My only guess as to why I am not able to see it in the Java program is because the time in which I try to read from the InputStream is too late.
So, how do I read the immediate, full response from the remote server using my HttpURLConnection object? Below is the code that I am using:
HttpURLConnection conn = null;
String urlStr = "...";
URL url = null;
try
{
url = new URL(urlStr);
conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("POST");
conn.setDoInput(true);
conn.setDoOutput(true);
...
BufferedReader rd = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
StringBuilder sb = new StringBuilder();
String line;
while ((line = rd.readLine()) != null)
{
sb.append(line);
}
rd.close();
...
}...
Okay, never mind. It turns out that what I was looking for was in the HTTP Respone's header. So, I got what I needed by looking through its headers. ::Face Palm::

Categories

Resources