URL queryUrl = new URL(url);
InputStream inputStream = null;
HttpsURLConnection connection = (HttpsURLConnection) queryUrl.openConnection();
connection.setRequestProperty("User-Agent", "My Client");
connection.setRequestMethod("GET");
connection.setDoInput(true);
inputStream = connection.getInputStream();
Hi,
I'm using the above code to perform an HttpGet query.
I'm getting once in a few tries an exception that server returned error code 502 or 504 (both scenarios occur).
The exception is thrown in the line :
inputStream = connection.getInputStream()
Any ideas?
Your help would be appreciated.
Thanks in advance
The error code in 5xx indicates some issue with Server or proxy. Rfc Statement
Response status codes beginning with the digit "5" indicate cases in which the server is
aware that it has erred or is incapable of performing the request. Except when responding
to a HEAD request, the server SHOULD include an entity containing an explanation of the
error situation, and whether it is a temporary or permanent condition. User agents SHOULD
display any included entity to the user. These response codes are applicable to any request method.
Please check what is the actual error by reading the error steam of url connection as below:
If the HTTP response code is 4nn (Client Error) or 5nn (Server Error), then you may want to read the HttpURLConnection#getErrorStream() to see if the server has sent any useful error information.
InputStream error = ((HttpURLConnection) connection).getErrorStream();
Also I think for working with http requests, you can use Apache HttpClient instead of directly working with HttpUrlConnection. Using HttpClient is lot more easier.
Related
I need to send data to another system in a Java aplication via HTTP POST method. Using the Apache HttpClient library is not an option.
I create a URL, httpconection without problems. But when sending special character like Spanish Ñ, the system complains it is receiving
Ñ instead of Ñ.
I've read many post, but I don't understand some things:
When doing a POST connection, and writing to the connection object, is it mandatory to do the URLEncode.encode(data,encoding) to the data being sent?
When sending the data, in some examples I have seen they use the
conn.writeBytes(strData), and in other I have seen conn.write(strData.getBytes(encoding)). Which one is it better? Is it related of using the encode?
Update:
The current code:
URL url = new URL(URLstr);
conn1 = (HttpsURLConnection) url.openConnection();
conn1.setRequestMethod("POST");
conn1.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(conn1.getOutputStream());
wr.writeBytes(strToSend);//data sent
wr.flush();
wr.close();
(later I get the response)
strToSend has been previously URLENCODE.encode(,"UTF-8")
I still don't know if I must use urlencode in my code and/or setRequestProperty("Contentype","application/x-www-formurlencode");
Or if I must use .write(strToSend.getByte(??)
Any ideas are welcome. I am testing also the real server (I dont know very much about it)
I want to read the second line of the text at this URL: "http://vuln2014.picoctf.com:51818/" (this is a capture-the-flag competition but only asking for flags or direction to flags breaks the competition rules). I am attempting to open an input stream from the URL but I get an Invalid HTTP Response exception. Any help is appreciated, and I recognize that my error is likely quite foolish.
Code:
URL url = new URL("http://vuln2014.picoctf.com:51818");
URLConnection con = url.openConnection();
InputStream is = con.getInputStream()
The error occurs at the third line.
java.io.IOException: Invalid Http response at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1342) at name.main(name.java:41)
curl happily gets the text from the page, and it is perfectly accessible from a web browser.
When you do this:
URL url = new URL("http://vuln2014.picoctf.com:51818");
URLConnection con = url.openConnection();
You are entering into a contract that says that this URL uses the http protocol. When you call openConnection it expects to get http responses because you used http:// in the URL as the protocol. The Java Documentation says:
If for the URL's protocol (such as HTTP or JAR), there exists a public, specialized URLConnection subclass belonging to one of the following packages or one of their subpackages: java.lang, java.io, java.util, java.net, the connection returned will be of that subclass. For example, for HTTP an HttpURLConnection will be returned, and for JAR a JarURLConnection will be returned.
The server you are connecting to just returns a couple lines of data. I retrieved them with the command nc vuln2014.picoctf.com 51818. There is no http response code like HTTP/1.1 200 OK:
Welcome to the Daedalus Corp Spies RSA Key Generation Service. The public modulus you should use to send your updates is below. Remember to use exponent 65537.
b4ab920c4772c5247e7d89ec7570af7295f92e3b584fc1a1a5624d19ca07cd72ab4ab9c8ec58a63c09f382aa319fa5a714a46ffafcb6529026bbc058fc49fb1c29ae9f414db4aa609a5cab6ff5c7b4c4cfc7c18844f048e3899934999510b2fe25fcf8c572514dd2e14c6e19c4668d9ad82fe647cf9e700dcf6dc23496be30bb
In this case I would use java.net.Socket to establish a connection and then read the lines. This is a simplistic approach that assumes there are 2 lines of data:
Socket theSocket;
try {
theSocket = new Socket("vuln2014.picoctf.com", 51818);
BufferedReader inFile = new BufferedReader(new InputStreamReader(theSocket.getInputStream()));
String strGreet = inFile.readLine();
String strData = inFile.readLine();
} catch (IOException e) {
e.printStackTrace();
}
As for why curl and browsers may render it properly? They are likely more lenient about the data they read and will just dump what is read from the port even if it doesn't conform to the specified protocol (like http)
I'm trying to make multiple calls to a REST API using HttpURLConnection with GET method and retrieving the response through InputStream.
It worked fine previously for 3 months but now it's throwing below exception:
SAXException Occurred during getArtifactsUrl method:: org.xml.sax.SAXParseException; Premature end of file.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) [xercesImpl.jar:6.1.0.Final]
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) [xercesImpl.jar:6.1.0.Final]
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) [:1.7.0_03]
Below is the line of code where I'm making the second call to parse the response:
request = (HttpURLConnection) endpointUrl.openConnection();
inputstream = request.getInputStream();
doc = dBuilder.parse(inputstream);
First call is working fine using request and inputstream objects but second call is failing. I tried all possible answers I found in google but no luck:
after every call:
inputstream.close();
request.disconnect();
Remember that request is an HttpURLConnection object.
I greatly appreciate if you can be able to solve this as I this is a high prioirity production issue now!
First you should check for error cases and not assume it's always working.
Try this:
request = (HttpURLConnection) endpointUrl.openConnection();
request.connect(); // not really necessary (done automatically)
int statusCode = request.getResponseCode();
if (statusCode == 200) { // or maybe other 2xx codes?
// Success - should work if server gives good response
inputstream = request.getInputStream();
// if you get status code 200 and still have the same error, you should
// consider logging the stream to see what document you get from server.
// (see below *)
doc = dBuilder.parse(inputstream);
} else {
// Something happened
// handle error, try again if it makes sense, ...
if (statusCode == 404) ... // resource not found
if (statusCode == 500) ... // internal server error
// maybe there is something interesting in the body
inputstream = request.getErrorStream();
// read and parse errorStream (but probably this is not the body
// you expected)
}
Have a look at the List of HTTP status codes.
And in some nasty cases, there are other problems which are not easy to detect if you just sit behind HttpURLConnection. Then you could enable logging or snoop the TCP/IP traffic with an apropriate tool (depends on your infrastructure, rights, OS, ...). This SO post might help you.
*) In your case I suppose that you're getting a non-error status code from the server but unparseable XML. If logging the traffic is not your thing, you could read the InputStream, write it to a file and then process the stream like before. Of course you can write the stream to a ByteArrayOutputStream, get the byte[] and write that Bytes to a file and then convert them to a ByteArrayInputStream and give this to your XML-parser. Or you could use Commons IO TeeInputStream to handle that for you.
There are cases where connection.getResponseCode() throws an exception. Then it was not possible to parse the HTTP header. This should only happen if there are strange errors in server software, hardware or perhaps a firewall, proxy or load balancer not behaving well.
One more thing: You might consider choosing an HTTP Client library and not directly use HttpURLConnection.
I made an app. for Android which uses the C2DM service from Google. I
made a server simulator from some tutorials and it works fine. My
problem is, I tried to build a Java Servlet. From the Android device
it receives fine the message and saves the Registration ID, but when I
try to send a https POST request to the Google C2DM Server it always
gets a SocketTimeoutException : Timeout while fetching:
https://android.clients.google.com/c2dm/send.
I don't get why this is happening when the same works on the Android
device. Here is the code:
//The AuthToken from Google Client Login
String auth_key = TOKEN;
StringBuilder postDataBuilder = new StringBuilder();
//some parameters to pass, I've checked and it's correct, it's working
//with Fiddler
postDataBuilder.append(PARAM_REGISTRATION_ID).append("=").append(REGISTRATION_ID);
postDataBuilder.append("&").append(PARAM_COLLAPSE_KEY).append("=").append("0");
postDataBuilder.append("&").append("data.payload").append("=").append(URLEncoder.encode(message, UTF8));
byte[] postData = postDataBuilder.toString().getBytes(UTF8);
URL url = new URL("https://android.clients.google.com/c2dm/send");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type","application/x-www-form-urlencoded;charset=UTF-8");
conn.setRequestProperty("Content-Length",Integer.toString(postData.length));
conn.setRequestProperty("Authorization", "GoogleLogin auth="+auth_key);
OutputStream out = conn.getOutputStream();
out.write(postData);
out.close();
int responseCode = conn.getResponseCode();
//here comes the error processing, but I can't reach it, because of
//the exception.
if (responseCode == 401 || responseCode == 403) {
//....
}
Thanks for your help :).
The first obvious thing to check is - if you have thought of this I apologise - are you behind a proxy server e.g. a company firewall? If so a timeout is exactly the symptom I'd expect with the above code. (This catches me out all the time!)
With the latter half of your code (from the HttpURLConnection declaration on), unmodified, I see a timeout; on my system (behind a company firewall), with two changes I get a 200 OK back:
addition of a proxy object passed to the HttpUrlConnection factory as follows:
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("...", 8080));
HttpURLConnection conn = (HttpURLConnection) url.openConnection(proxy);
accepting the C2DM server's certificate that wasn't trusted by my JVM. For test purposes I overrode the default hostname verifier and TrustManager as described in Trusting all certificates using HttpClient over HTTPS . For production you should look at a more secure solution.
Another thing I spotted; it doesn't seem to matter but http://code.google.com/android/c2dm/index.html#push says to post to https://android.apis.google.com/c2dm/send, not android.clients.google.com - just something to be aware of that might break in future.
I faced same problem and
I had tried :
URL url = new URL("http://android.apis.google.com/c2dm/send");
instead of :
URL url = new URL("https://android.apis.google.com/c2dm/send");
it worked for me.
I am having an issue with this error:
**Server returned HTTP response code: 501 for URL: http://dev1:8080/data/xml/01423_01.xml**
See this code:
private static Map sendRequest(String hostName, String serviceName) throws Exception {
Map assets = null;
HttpURLConnection connection = null;
Authenticator.setDefault(new Authenticator());
URL serviceURL = new URL(hostName + "/" + serviceName);
connection = (HttpURLConnection)serviceURL.openConnection();
connection.setRequestMethod("GET");
ClientHttpRequest postRequest = new ClientHttpRequest(connection);
InputStream input = null;
/*
At line input = postRequest.post(); I get the following error
Server returned HTTP response code: 501 for URL: http://dev1:8080/data/xml/01423_01.xml
Yet if I enter that url in my browser it opens up fine.
Is this a common problem? Is there some type of content type I need to set?
*/
input = postRequest.post();
connection.disconnect();
return assets;
}
A 501 response means "not implemented", and is usually taken to mean that the server didn't understand the HTTP method that you used (e.g. get, post, etc).
I don't recognise ClientHttpRequest , but you have a line that says
connection.setRequestMethod("GET");
and then a line that says
input = postRequest.post();
I'm not sure what post() actually does, but does that mean send a POST request? If so, then that contradicts the GET specified in the first line.
Either way, the server is saying that it doesn't under the GET or the POST method, whichever one your code is actually sending. You need to find out what method the server does support for that URL, and use that.
Perhaps you should check your port settings:
new URL(hostName + "/" + serviceName);
Looks like the port number ":8080" is missing.
Some server expect additional information from the client in the request like a user agent or some form data. Even cookies could be expected by the application running on the server. You should also check the complete response and not only the response code.
I would recommend you to use a library like httpclient that is more convenient:
https://hc.apache.org/httpcomponents-client-ga/index.html
Here is simple usage example:
https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/test/java/org/apache/hc/client5/http/examples/ClientWithResponseHandler.java