I am trying to access this json file : http://www.cloudpricingcalculator.appspot.com/static/data/pricelist.json with Java.
But when I read it, sometimes it gives me a JSON string (that's ok) and sometimes it gives me something else and json.simple.parser throw an Unexpected character(<) at position 0.
Based on what I read on stackOverflow, it may be that it returns XML instead of JSON. As my url is "json", how is it possible ?
Here is the code i'm using :
String baseUrl = "http://www.cloudpricingcalculator.appspot.com/static/data/pricelist.json";
...
URL url = new URL(this.baseUrl);
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String l;
String json = "";
System.out.println(url);
while((l=in.readLine()) != null){
System.out.println(l);
json+=l;
}
JSONParser parser = new JSONParser();
JSONObject jsonObject = (JSONObject) parser.parse(json);
and the log is <followed by an lot of squares and unknown characters like ÿÕ[s›È and an error Unexpected character () at position 0.
You are not taking into account the compression and encoding of the resource returned from the server. The response of a HEAD request is the following:
rpax#machine:~$ HEAD http://www.cloudpricingcalculator.appspot.com/static/data/pricelist.json
200 OK
Cache-Control: public, max-age=600
Connection: close
Date: Mon, 21 Aug 2017 12:02:14 GMT
Age: 112
ETag: "n_s_jQ"
Server: Google Frontend
Content-Encoding: gzip <---- *HERE*
Content-Length: 7902
Content-Type: application/json
Expires: Mon, 21 Aug 2017 12:12:14 GMT
...
For avoiding this issue, you can wrap the url stream into a GZIPInputStream :
GZIPInputStream gis = new GZIPInputStream(url.openStream());
BufferedReader in = new BufferedReader(new InputStreamReader(gis));
// ...
And the data returned when executing readline() will be decompressed.
Related
I have a java program that is trying to make an HTTP request over a socket. For some reason slashes in the string are messing it up.
I have a try/catch and it gets caught as soon as the socket is created with a string that has a slash.
Socket socket = new Socket("www.google.ca", port);
Response
HTTP/1.1 400 Bad Request
Content-Length: 54
Content-Type: text/html; charset=UTF-8
Date: Fri, 14 Oct 2016 06:05:43 GMT
Connection: close
<html><title>Error 400 (Bad Request)!!1</title></html>
Now with a slash
Socket socket = new Socket("www.google.ca/", port);
Gets caught.
My request.
outputStream.println("GET / HTTP/1.1");
outputStream.println("");
outputStream.flush();
I'm trying to access a specific site with the hostname and path which has slashes. What is happening?
The first error HTTP/1.1 400 Bad Request happens because of wrong request path. It is hard to find the reason without knowing your code.
The second error happens like Andy Turner already said, because the host name is wrong. InetAddress can not resolve the host names with slashes.
This example works for me:
public static void main(String[] args) throws Exception {
Socket s = new Socket(InetAddress.getByName("google.com"), 80);
PrintWriter pw = new PrintWriter(s.getOutputStream());
pw.println("GET /about/ HTTP/1.1"); // here comes the path
pw.println("f-Modified-Since: Wed, 1 Oct 2017 07:00:00 GMT");
pw.println("");
pw.flush();
BufferedReader br = new BufferedReader(new InputStreamReader(s.getInputStream()));
String line;
while((line = br.readLine()) != null){
System.out.println(line);
}
br.close();
}
You just have to set the path in this line:
pw.println("GET /about HTTP/1.1");
More specifically than IOException, you are getting an UnknownHostException (a subclass of IOException), because host names can't contain a slash.
You should print/log the exception's stack trace in the catch block; this problem would then be a lot more apparent.
i´m trying to solve encoding issue for response (json) from simple rest service in my app for about one day now, no more ideas..
when i open response in restclient, getting correct result:
Status Code: 200 OK
Connection: Keep-Alive
Content-Encoding: gzip
Content-Length: 270
Content-Type: application/json; charset=utf-8
Date: Wed, 13 Apr 2016 09:26:39 GMT
Keep-Alive: timeout=15, max=100
Server: Apache
Vary: Accept-Encoding
response:
{"reqStatus": 1,
"articles": [
{
"articleId": "1",
"articleName": "Schüssel 18cm",
"articlePrice": "34.50",
"articleDate": "2016-03-15 17:34:00",
"userZip": "76879",
"userCity": "Ottersheim",
"articleImages": [ ...
for inputstream getting that:
{"reqStatus":1,"articles":[{"articleId":"1","articleName":"Sch\u00fcssel 18cm","articlePrice":"34.50","articleDate":"2016-03-15 17:34:00","userZip":"76879","userCity":"Ottersheim","articleImages":[ ...
code for request:
URL urlToRequest = new URL(serviceUrl);
urlConnection = (HttpURLConnection) urlToRequest.openConnection();
urlConnection.setConnectTimeout(10000);
urlConnection.setReadTimeout(5000);
urlConnection.setRequestMethod("POST");
if (headers!=null&&headers.length>0){
for (int i=0;i < headers.length;i++){
urlConnection.setRequestProperty(headers[i][0].toString(), headers[i][1].toString());
}
}
urlConnection.connect()
int statusCode = urlConnection.getResponseCode();
if (statusCode != HttpURLConnection.HTTP_OK) {
BufferedReader streamReader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream(), "UTF-8"));
StringBuilder responseStrBuilder = new StringBuilder();
String inputStr;
while ((inputStr = streamReader.readLine()) != null)
responseStrBuilder.append(inputStr);
return new JSONObject(inputStr);
}
any ideas?
I'm trying to make a POST request using JSON with foreign characters, such as the Spanish n with the '~' over it, but I keep getting this request and response error:
POST ...
Accept: application/json
Content-Type: application/json; charset=UTF-8
Content-Length: 151
Content-Encoding: UTF-8
Host: ...
Connection: Keep-Alive
User-Agent: ..
{"numbers":"2","date":"2014-07-15T00:00:00+0000","description":" // this never gets closed
X-Powered-By: ...
Set-Cookie: ...
Cache-Control: ...
Date: Tue, 15 Jul 2014 15:19:12 GMT
Content-Type: application/json
Allow: GET, POST
{"status":"error",
"status_code":400,
"status_text":"Bad Request",
"current_content":"",
"message":"Could not decode JSON, malformed UTF-8 characters (incorrectly encoded?)"}
I can already make a successful POST request with normal ASCII characters, but now that I'm supporting foreign languages, I need to convert the foreign characters to UTF-8 (or whatever the correct encoding ends up being), unless there's a better way to do this.
Here's my code:
JSONObject jsonObject = new JSONObject();
HttpResponse resp = null;
String urlrest = // some url;
HttpPost p = new HttpPost(urlrest);
HttpClient hc = new DefaultHttpClient();
hc = sslClient(hc);
try
{
p.setHeader("Accept", "application/json");
p.setHeader("Content-Type", "application/json");
// setting TimeZone stuff
jsonObject.put("date", date);
jsonObject.put("description", description);
jsonObject.put("numbers", numbers);
String seStr = jsonObject.toString();
StringEntity se = new StringEntity(seStr);
// Answer: The above line becomes new StringEntity(seStr, "UTF-8");
Header encoding = se.getContentType();
se.setContentEncoding("UTF-8");
se.setContentType("application/json");
p.setEntity(se);
resp = hc.execute(p);
When I put a breakpoint and look at se before it's submitted, the characters look right.
UPDATE: code updated with answer a few lines above with a comment identifying it.
The new StringEntity constructor takes a "UTF-8" parameter.
This question already has answers here:
How to decompress a gzipped data in a byte array?
(5 answers)
Closed 9 years ago.
In HTTP request and response, content-encoding is 'gzip' and content is gzipped. Is there a way to decompress the gzipped content so we can see the contents ??
Example for GZipped HTTP request
HTTP/1.1 200 OK
Date: mon, 15 Jul 2014 22:38:34 GMT
Server: Apache/1.3.3.7 (Unix) (Red-Hat/Linux)
Last-Modified: Wed, 08 Jan 2003 23:11:55 GMT
Accept-Ranges: bytes
Content-Length: 438
Connection: close
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
//Response body with non type characters
That shouldn't be necessary. Your application server should handle such requests for you and decompress the payload for you automatically.
If that doesn't happen, you need to wrap the InputStream in a GZipInputStream. But this sounds more like a misconfiguration of your server.
Found the answer.
//reply - Response from Http byte[] reply = getResponseBytes();
int i;
for(i=0;i<reply.length;i++){
//Finding Raw Response by two new line bytes
if(reply[i]==13){
if(reply[i+1]==10){
if(reply[i+2]==13){
if(reply[i+3]==10){
break;
}
}
}
}
}
//Creating new Bytes to parse it in GZIPInputStream
byte[] newb = new byte[4096];
int y=0;
for(int st=i+4;st<reply.length;st++){
newb[y++]=reply[st];
}
GZIPInputStream gzip = new GZIPInputStream (new ByteArrayInputStream (newb));
InputStreamReader reader = new InputStreamReader(gzip);
BufferedReader in = new BufferedReader(reader);
String readed;
while ((readed = in.readLine()) != null) {
//Bingo...
System.out.println(readed);
}
This is what I have so far,
Socket clientSocket = new Socket(HOST, PORT);
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
InputStream is = socket.getInputStream();
byte[] byteChunk = new byte[1024];
int c = is.read(byteChunk);
while (c != -1){
buffer.write(byteChunk, 0, c);
c = is.read(byteChunk);
}
BufferedImage bufferedImage = ImageIO.read(new ByteArrayInputStream(buffer.toByteArray()));
My problem with my code is ImageIO.read() returns null.
When I print the content of ByteArrayOutputStream object what i get is header part
HTTP/1.1 200 OK
Date: Fri, 30 Dec 2011 11:34:19 GMT
Server: Apache/2.2.3 (Debian) ...........
Last-Modified: Tue, 20 Dec 2011 19:12:23 GMT
ETag: "502812-490e-4b48ad8d273c0"
Accept-Ranges: bytes
Content-Length: 18702
Connection: close
Content-Type: image/jpeg
followed with a empty line plus many lines with different characters such as Àã$sU,e6‡Í~áŸP;Öã….
Again my problem is ImageIO.read() function returns null.
Thanks in advance.
Why you don't want to use simple http URL to get image from host?
I mean:
URL imageURL = new URL("http://host:port/address");
BufferedImage bufferedImage = ImageIO.read(imageURL);
If you want to use plain socket you have to parse http response and extract data from the http reply manually: read/skip headers, read binary data and pass it to ImageIO.read (or seek stream to correct position and pass stream to ImageIO.read).