Develop Reference

Http response code 429 while reading HTML

In java I want to read and save all the HTML from an URL(instagram), but getting Error 429 (Too many request). I think it is because I am trying to read more lines than request limits.
StringBuilder contentBuilder = new StringBuilder();
try {
URL url = new URL("https://www.instagram.com/username");
URLConnection con = url.openConnection();
InputStream is =con.getInputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(is));
String str;
while ((str = in.readLine()) != null) {
contentBuilder.append(str);
}
in.close();
} catch (IOException e) {
log.warn("Could not connect", e);
}
String html = contentBuilder.toString();
And the Error is so;
Could not connect
java.io.IOException: Server returned HTTP response code: 429 for URL: https://www.instagram.com/username/
And it shows also that error occurs because of this line
InputStream is =con.getInputStream();
Does anybody have an idea why I get this error and/or what to do to solve it?

The problem might have been caused by the connection not being closed/disconnected.
For the input try-with-resources for automatic closing, even on exception or return is usefull too. Also you constructed an InputStreamReader that would use the default encoding of the machine where the application would run, but you need the charset of the URL's content.
readLine returns the line without line-endings (which in general is very useful). So add one.
StringBuilder contentBuilder = new StringBuilder();
try {
URL url = new URL("https://www.instagram.com/username");
URLConnection con = url.openConnection();
try (BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream(), "UTF-8"))) {
String line;
while ((line = in.readLine()) != null) {
contentBuilder.append(line).append("\r\n");
}
} finally {
con.disconnect();
} // Closes in.
} catch (IOException e) {
log.warn("Could not connect", e);
}
String html = contentBuilder.toString();

How send GET request with value of parameter containing a space?

I'm testing this code below to send GET request with parameters and this code fails when the value of parameter is a string containing a space, Ex: http://company.com/example.php?value=Jhon 123. Already if i send Jhon123 (withou any space) works fine.
Why this happens?
private static void sendGet(String site, String params) throws Exception {
site += params;
URL obj = new URL(site);
try {
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + site);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(
new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
//print result
System.out.println(response.toString());
} catch (Exception ex) {
}
}

You should URL Encode your request.

You can use URLEncoder to encode your parameter:
String url = "http://company.com/example.php?value=" + URLEncoder.encode("Jhon 123", "utf-8");

How to get correct data from HTTP Request

I'm trying to get my user information from stackoverflow api using a simple HTTP request with GET method in Java.
This code I had used before to get another HTTP data using GET method without problems:
URL obj;
StringBuffer response = new StringBuffer();
String url = "http://api.stackexchange.com/2.2/users?inname=HCarrasko&site=stackoverflow";
try {
obj = new URL(url);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
con.setRequestMethod("GET");
int responseCode = con.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
System.out.println(response.toString());
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
But in this case I'm getting just stranger symbols when I print the response var, like this:
�mRM��0�+�N!���FZq�\�pD�z�:V���JX���M��̛yO^���뾽�g�5J&� �9�YW�%c`do���Y'��nKC38<A�&It�3��6a�,�,]���`/{�D����>6�Ɠ��{��7tF ��E��/����K���#_&�yI�a�v��uw}/�g�5����TkBTķ���U݊c���Q�y$���$�=ۈ��ñ���8f�<*�Amw�W�ـŻ��X$�>'*QN�?�<v�ݠ FH*��Ҏ5����ؔA�z��R��vK���"���#�1��ƭ5��0��R���z�ϗ/�������^?r��&�f��-�OO7���������Gy�B���Rxu�#:0�xͺ}�\�����
thanks in advance.

The content is likely GZIP encoded/compressed. The following is a general snippet that I use in all of my Java-based client applications that utilize HTTP, which is intended to deal with this exact problem:
// Read in the response
// Set up an initial input stream:
InputStream inputStream = fetchAddr.getInputStream(); // fetchAddr is the HttpURLConnection
// Check if inputStream is GZipped
if("gzip".equalsIgnoreCase(fetchAddr.getContentEncoding())){
// Format is GZIP
// Replace inputSteam with a GZIP wrapped stream
inputStream = new GZIPInputStream(inputStream);
}else if("deflate".equalsIgnoreCase(fetchAddr.getContentEncoding())){
inputStream = new InflaterInputStream(inputStream, new Inflater(true));
} // Else, we assume it to just be plain text
BufferedReader sr = new BufferedReader(new InputStreamReader(inputStream));
String inputLine;
StringBuilder response = new StringBuilder();
// ... and from here forward just read the response...
This relies on the following imports: java.util.zip.GZIPInputStream; java.util.zip.Inflater; and java.util.zip.InflaterInputStream.

Sending request to GCM results in a JSON_PARSIN_ERROR

I need to write a server which sends push notifications to iOS and Android devices.
I can successfully send notifications to iOS. Luckily it was painless to implement that part.
I'm facing a problem with the easiest part of the task: Android.
I configured GCM and the Android device successfully registers against the server.
When I try to send a push to the device, I get the following error:
STATUS: 400
JSON_PARSING_ERROR: Unexpected token END OF FILE at position 0.
Here the code that sends the request.
AndroidPush push = new AndroidPush();
push.getRegids().add(token);
push.getData().setMessage(message);
push.getData().setMsgcnt(String.format("%d", (badge + 1)));
URL u = new URL(androidEndpoint);
HttpsURLConnection conn = null;
OutputStream os = null;
OutputStreamWriter osw = null;
InputStream is = null;
InputStreamReader isr = null;
BufferedReader br = null;
try{
conn = (HttpsURLConnection)u.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Content-Type", androidContentType);
conn.setRequestProperty("Authorization", androidAuthorization);
conn.connect();
os = conn.getOutputStream();
osw = new OutputStreamWriter(os);
System.out.println(push.toJSON());
osw.write(push.toJSON());
int status = conn.getResponseCode();
System.err.println("STATUS: "+status);
if(status == 200){
is = conn.getInputStream();
} else {
is = conn.getErrorStream();
}
isr = new InputStreamReader(is);
br = new BufferedReader(isr);
StringBuilder sb = new StringBuilder();
String read = null;
do {
read = br.readLine();
if(read != null)
sb.append(read);
} while (read != null);
if(status != 200){
System.err.println(sb.toString());
} else {
System.out.println(sb.toString());
}
} catch(IOException ex) {
ex.printStackTrace();
throw ex;
} finally {
if(isr != null)
isr.close();
if(is != null)
is.close();
if(osw != null)
osw.close();
if(os != null)
os.close();
if(conn != null)
conn.disconnect();
}
Where:
androidEndpoint = "https://android.googleapis.com/gcm/send";
androidContentType = "application/json";
androidAuthorization = "key=<mykey>";
and the output of the push.toJSON() method is the following String:
{
"registration_ids" : ["APA91bEmD8T9NxQj07uhbTAsD1GTWvT7L_no1SXP70YWaQGPX6VO73pdAOa53PN-hyAyy-3erItWxLDb8W1aQ2nh3np0NttJ5g66w2-142d4bXTCsmrF34-J7rWw4IUObutQznaml59XdfweiEGKzv1Otp3quffUEA"],
"data" : {
"message":"Push di prova",
"msgcnt":"13"
}
}
I cannot see anything wrong... What am I missing?
Thank you for your help,

Developing a solution with php, I recieved always the same error WHEN the message had Umlauts (e.g. äöü) in it.
Without the message went through and arrived the reciever.
The solution was finally to utf8_encode( 'your text here äöä' ) your outgoing strings (message, title, etc. whatever).
Cost my at least several hours to detect what was 'wrong'.

Is it the case that you are missing ',' as shown below in bold
{
"registration_ids" : ["APA91bEmD8T9NxQj07uhbTAsD1GTWvT7L_no1SXP70YWaQGPX6VO73pdAOa53PN-hyAyy-3erItWxLDb8W1aQ2nh3np0NttJ5g66w2-142d4bXTCsmrF34-J7rWw4IUObutQznaml59XdfweiEGKzv1Otp3quffUEA"],
"data" : {
"message":"Push di prova",
"msgcnt":"13"**,**
}**,**
}
Refer message given in this link: http://developer.android.com/google/gcm/adv.html#payload

How do I read a HTTP response if the server returned a 502 error?

I used URLConnection to send a POST request to a server and I am using the following code to read the response:
String line;
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
while ((line = reader.readLine()) != null) {
response += line;
}
reader.close();
However, the server will sometimes return a 502 error with a meaningful error message in the response, which I would like to obtain. My problem is that attempting to create the BufferedReader in that case will result in a java.io.IOException:
Server returned HTTP response code: 502 for URL: <url>
How can I bypass this?

Modify your code along these lines so that you can get the errorstream rather than throw the IOException
HttpURLConnection httpConn = (HttpURLConnection)connection;
InputStream is;
if (httpConn.getResponseCode() >= 400) {
is = httpConn.getErrorStream();
} else {
is = httpConn.getInputStream();
}
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
while ((line = reader.readLine()) != null) {
response += line;
}
reader.close();