I'm trying to login to a webpage, but even before that, I'm loading the page using HttpGet, and this is one the lines that's being returned,
ÓA;
That's all I could put, won't let me paste any other characters. But they are all like that, like I'm somehow getting the wrong encoding? Here is the code I am using to GET
HttpGet httpget = new HttpGet(url);
if(headers == null) {
headers = getDefaultHeaders();
}
for(String s : headers.keySet()) {
httpget.addHeader(s, headers.get(s));
}
HttpResponse response = getClient().execute(httpget);
HttpEntity entity = response.getEntity();
System.out.println("Status Line: " + response.getStatusLine());
if (entity != null) {
InputStream input = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String ln = "";
while((ln = reader.readLine()) != null) {
System.out.println("During Get - " + ln);
}
}
What am I doing wrong?
Thanks for any help.
If you need any more information like headers, just ask.
The following line is possibly the cause of your problems:
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
You are creating a reader using the default characterset of your platform, and completely ignoring any character set that may be specified in the HTTP response headers.
If you are getting the same problem when reading the content the correct way, then it is possible that the server is at fault for not setting the response header correctly.
DO the entity reading like this:
String content = org.apache.http.util.EntityUtils.toString( entity );
System.out.println(content);
This is going to read it all for you so you can check what's being really returned.
Make sure that you didn't accidentally go to port 443 with a simple HTTP connection. Because in that case you will get back the SSL handshake instead of an HTTP response.
Related
I have following code where I am calling an API which is a PHP built. The code returns json stated as below which I am collecting in a stringBuilder object. Problem is its working on some carriers and on few devices with other carriers / wifi connection its throwing JSONException end of input at character 0 exception, i know this comes when input string is empty, it means stringBuilder object is empty. Problem is i don't have access to the devices on which its throwing these errors.
I am not getting on some device why following code returns empty string and on some its working fine, user has tested on 3G as well as wifi these devices are in other country on different carriers.
HttpClient httpClient = HttpClientBuilder.create().build();
HttpPost postRequest = new HttpPost(ServiceUrls.base_url + ServiceUrls.get_profile_url);
JSONObject object = new JSONObject();
object.put("username", params[0]);
StringEntity input = new StringEntity(object.toString());
input.setContentType("application/json");
postRequest.setEntity(input);
HttpResponse response = httpClient.execute(postRequest);
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ response.getStatusLine().getStatusCode());
}
BufferedReader br = new BufferedReader(
new InputStreamReader((response.getEntity().getContent())));
String output;
StringBuilder stringBuilder = new StringBuilder();
while ((output = br.readLine()) != null) {
stringBuilder.append(output);
}
If it was for all API call then it was logical but doest happen for other API call, this API call returns bigger size JSON string as follows in stringbuilder
{
"status":1,
"parking":{
"name":"ghgjjghghg",
"cost":3,
"ownerId":29,
"address":"xyz pqr",
"slots":4,
"image":"d4bc95c1dd031685746f2c3570788acf.jpg",
"details":"gjhjghjgg",
"amenities":"gjhg",
"id":70,
"lon":73.7898023,
"lat":19.9974533,
"type":0,
"available":1
},
"rating":0,
"ratingCount":0,
"owner":{
"id":29,
"username":"vd#gmail.com",
"password":"",
"fullname":"vi hdjh",
"phone":"23434fddf",
"ccNum":null,
"ccType":null,
"type":1,
"authType":1,
"image":"582e3a77d76ae3203cfd6d6a346da429.jpg",
"dni":"abc123",
"account":"ABCBANK"
}
}
I have no clue whats happening , please help. Any input will be appreciated.
There is nothing unusual about the code you posted. No clues there.
Let me summarize what I think you said about the symptoms.
For some devices / carriers, a specific API call fails. But not all devices / carriers.
For the same devices / carriers as above, other API calls work, all if the time.
The client-side code is identical in all cases, apart from the URLs.
To me, this is pointing at a problem on the server side that is triggered by what the requests look like to it. But either the way, I would try to investigate this by looking at the requests and responses on the server side, and checking the server-side logs. See if there are significant differences in the requests coming from different devices / carriers. Especially the ones that work versus the ones that don't work. And see if the responses are empty when the server sends them.
I found the theory of Leonidos usefull:
https://stackoverflow.com/a/19540249/6076711
And here is my end of solution you can try using the following code.
string output = "";
while(br.ready() && (br.readLine() != null))
{
output += (String.valueOf(br.readLine()))+"\n";
}
The code can be improved by checking (before putting it in the string builder) whether the length of the content is bigger than 0.
HttpClient httpClient = HttpClientBuilder.create().build();
HttpPost postRequest = new HttpPost(ServiceUrls.base_url +
ServiceUrls.get_profile_url);
JSONObject object = new JSONObject();
object.put("username", params[0]);
StringEntity input = new StringEntity(object.toString());
input.setContentType("application/json");
postRequest.setEntity(input);
HttpResponse response = httpClient.execute(postRequest);
String output;
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : "
+ response.getStatusLine().getStatusCode());
} else {
HttpEntity entity = response.getEntity();
if ((entity != null) && (entity.getContentLength() != 0)) {
// Use writing to output stream to prevent problems with chunked responses
ByteArrayOutputStream os = new ByteArrayOutputStream();
entity.writeTo(os);
output = new String(os.toByteArray(),"UTF-8");
} else {
// Handle case there is not content returned
System.out.println("Received no content (HTTP response code " + response.getStatusLine().getStatusCode() + " , reason: " + getReasonPhrase() +")");
}
}
The code above however doesn't solve the issue why you get an empty response. I its only handling the fact it is happening is a more elegant way.
I noted however that you require a username in the request. Are you sure the user exist on the device and in case of non existing user, should there be returned something else?
I have written a REST client for this endpoint:
textmap.com/ethnicity_api/api
However, while passing it a name string like jennífer garcía in the POST params, and setting encoding to UTF-8, the response that I get is not the same string. How to get the same name in the response object?
Below is how I set the request and the response thatI get:
httpClient = HttpClients.createDefault();
httpPost = new HttpPost(baseurl);
StringEntity input = new StringEntity(inputJSON, StandardCharsets.UTF_8);
input.setContentType("application/json");
//input.setContentType("application/json; charset=UTF-8");
httpPost.setEntity(input);
response = httpClient.execute(httpPost);
if (response.getStatusLine().getStatusCode() != 200) {
throw new RuntimeException("Failed : HTTP error code : " + response.getStatusLine().getStatusCode());
}
BufferedReader br = new BufferedReader(new InputStreamReader((response.getEntity().getContent())));
output = org.apache.commons.io.IOUtils.toString(br);
System.out.println(output);
Value of the name in output is : jenn�fer garc�a
This is a completely different charset from what I had sent in the request. How can I get the same charset as I had sent in request?
Secondly, I want the same code to work in both Java-6 and Java-7. The above code is using Java-7 only. How can I make the code work for both these versions?
I think the BufferedReader is breaking the UTF8 encoding, so this is actually pretty unrelated to HTTP. On a side note, the br may not be needed at all.
I have a question about Android(Java) HttpRequest and HttpResponse.
I use Apache library.
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.HttpResponse;
HttpGet httpGet = new HttpGet(uri);
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpResponse response = httpClient.execute(httpGet);
if(response.getStatusLine().getStatusCode() == 200) {
BufferedReader br = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuilder sb = new StringBuilder();
String str = "";
while ((str = br.readLine()) != null) {
sb.append(str);
}
}
And the response content response.getEntity().getContent() size is about 500KB
Can I say that if I get status code 200 and I can read the content until EOF (End Of File) then there's no data lost via network transportation?
Or I cannot make sure if there is data lost so I need a CheckSum mechanism to check?
Thanks.
Eric
Well ... that should be the case, right? :) If you have read all the entity content without exception, you've got the complete, correct response body. No need to add your own checksum.
The only problem I can see, is HTTP/1.0 server or intermediaries. If they use TCP FIN to end a response, the response body might be terminated prematurely without client's knowledge.
I don't think we need to worry about HTTP/1.0 at this point of time. If you do, you can design your data format so that premature termination can be detected on the client side.
im having a strange problem when receiving json results from the server. I have no idea what the problem is. The thing is that my String json result is corrupted, with strange symbols.
The result is like this (taken from eclipse debug)
Image :
Another strange thing that happens is that when I change the URL of the service to an alternative one, it works and the data is not corrupted. The URLs are the same but once redirects everything to the other.
The URL is use always is (example) http://www.hello.com
The URL that works is http://www.hello.com.uy
(cant post the exact link for security reasons)
The second one redirects everything to the first one, its the only thing it does.
I have tried changing the encoding to UTF-8 and it is still not working, here is the code (with one of the URLs commented)
I have also tried using Dev HTTP Client extension from chrome to check the service and it works fine, no corrupted data. Also, it works perfectly on iOS so i think its just and android/java issue.
DevClient:
try {
JSONObject json = new JSONObject();
HttpParams httpParams = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParams, 10000);
HttpConnectionParams.setSoTimeout(httpParams, 10000);
HttpClient client = new DefaultHttpClient(httpParams);
//String url = TAG_BASEURL_REST +"Sucursal";
String url = "http://www.-------.com/rest/Sucursal";
//String url = "http://www.--------.com.uy/rest/Sucursal";
HttpGet request = new HttpGet(url);
request.setHeader("Accept", "application/json");
request.setHeader("Content-type", "application/json");
HttpResponse response = client.execute(request);
HttpEntity entity = response.getEntity();
if (entity != null) {
InputStream is = entity.getContent();
BufferedReader reader = new BufferedReader(
new InputStreamReader(is, "iso-8859-1"), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
String jsonRes = sb.toString();
JSONArray jObj = new JSONArray(jsonRes);
return jObj;
}
} catch (Throwable t) {
Log.i("Error", "Request failed: " + t.toString(), t);
}
return null;
InputStream is = entity.getContent();
// check if the response is gzipped
Header encoding = response.getFirstHeader("Content-Encoding");
if (encoding != null && encoding.getValue().equals("gzip")) {
is = new GZIPInputStream(is);
}
I'm trying out the twitter streaming api. I could succesfully filter tweets by using curl, as stated here:
curl -d #tracking http://stream.twitter.com/1/statuses/filter.json -u <user>:<pass>
where tracking is a plain file with the content:
track=Berlin
Now I tried to do the same thing in JavaSE, using Apache's HTTPComponents:
UsernamePasswordCredentials creds = new UsernamePasswordCredentials(<user>, <pass>);
DefaultHttpClient httpClient = new DefaultHttpClient();
httpClient.getCredentialsProvider().setCredentials(AuthScope.ANY, creds);
HttpPost httpPost = new HttpPost("http://stream.twitter.com/1/statuses/filter.json");
HttpParams params = new BasicHttpParams();
params = params.setParameter("track", "Berlin");
httpPost.setParams(params);
try {
HttpResponse httpResponse = httpClient.execute(httpPost);
HttpEntity entity = httpResponse.getEntity();
if (entity != null) {
InputStream instream = entity.getContent();
String t;
BufferedReader br = new BufferedReader(new InputStreamReader(instream));
while(true) {
t = br.readLine();
if(t != null) {
linkedQueue.offer(t);
}
}
}
} catch (IOException ioe) {
System.out.println(ioe.getMessage());
}
finally{
httpClient.getConnectionManager().shutdown();
}
When I run that, I get:
No filter parameters found. Expect at least one parameter: follow track
as a single entry in my linkedQueue. Seems the api wants the parameter in a different form, but cannot find any hint in the documentation. Can somebody share some experiences with the api or see any other problem with the code? Thanks!
EDIT
Putting the filter parameter into the params was a bad idea. As it's post data, it needs to be defined as an Entity before the request is being made:
StringEntity postEntity = new StringEntity("track=Berlin", "UTF-8");
postEntity.setContentType("application/x-www-form-urlencoded");
httpPost.setEntity(postEntity);
That's what I was doing wrong. Thanks Brian!
I suspect you need to post the data as the contents of your HTTP post. The man page for curl -d says:
(HTTP) Sends the specified data in a
POST request to the HTTP server, in
the same way that a browser does when
a user has filled in an HTML form and
presses the submit button. This will
cause curl to pass the data to the
server using the content-type
application/x-www-form-urlencoded.
so I believe you have to set that content type and put the contents of the tracking file in the body of your post.