Develop Reference

client side freezes when requesting JSON from server

I have a Java program that will freeze during the rush hour currently. I am looking for the reason. Right now, my idea is that it is because some of the codes are not AJAX, so it will hangs in there when requesting JSON object from server during rush hour (Internet speed is slow during rush hour). But I am not very sure that it is not AJAX though. I cannot paste all the codes here because it is related to company's property, so I am going to write some piece of codes/pseudocode. The program uses the following to do the request.
URL url = new URL("http://www.xxxxxxxx.com/");
URLConnection conection = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(conection .getInputStream()));
String inputLine;
String content = "";
while ((inputLine = in.readLine()) != null) {
content += inputLine;
}
in.close();
json = new JSONObject(content);
The above codes are some snippet of the program where I think it causes the freezing. My question is that, This piece of code does not seem to use any AJAX and it is not multithreading, right? So it hangs in there (freezing) when the Internet connection is very slow? I am not very experienced about this. Please help.
If you do not think this is the main reason that causes the freezing, what could be the most possible reason for this kind of problem in general speaking.
Thank you in advance.
Edits/Update:
My program is a swing program that runs in client side. There is no
real server side codes for this application. The URL address is a
3rd party server that provides some kind of data that the program
needs. I did not see the program uses any threads. It is basically core Java without using any threads as I see, and I am not very familiar with multi-threading, so I am not sure. (The program is not written by me, it is written by some former programmers) That is why I am considering that maybe it did not use other threads to run this server request when it should be.
What I mean by freezing is that the UI will freeze for a few seconds during the rush hour when it try to run some specific functions. The above code I wrote is some snippet from those functions. After the UI freezes for awhile, it then works as expected in the rest. However, in non-rush hour, it works like a charm.
Specifically, BufferedReader in = new BufferedReader(new
InputStreamReader(conection .getInputStream())); is what I think
that causes the problem. Could anyone confirm that?

All asynchronous methods like HTTP POST/GET, implement in separate Thread. User don't interest to see what your application does in background.
In addition, its not good practice to use String for appends, like jlordo says, use StringBuilder or StringBuffer.
StringBuilder buff= new StringBuilder();
while ((inputLine = in.readLine()) != null) {
if(!"".equals(line).trim){
buff.append(line).append("\n");
}
}
in.close();
You can add timeout to your session to be sure that application doesn't stuck at all.
[EDIT]
Regard to question changes, I don't think that
BufferedReader in = new BufferedReader(new InputStreamReader(conection .getInputStream()));
it's your problem.
You can try to do followed flow but I still thinks about separate Thread with GUI Thread
InputStream is = conection .getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
while ( (line = br.readLine()) != null) {

You probably want to use a SwingWorker thread to do the call to the server. That way, when it is busy, the UI won't freeze.
However, remember, you won't see the resulting changes on the screen that the new data would provide until the new data arrives.
See Oracle SwingWorker JavaDocs
Tutorial with example similar to yours

Java socket read blocking infinitely

I have a really strange issue while working with Java sockets. This problem is only happening for a VERY small subset of the urls that I am processing. Let's call an example url abc.com.
Edit: url is lists.wikimedia.org/robots.txt that gives me problems.
I can curl/netcat/telnet lists.wikimedia.org with path /robots.txt perfectly fine. Telnet even tells me the IP address for lists.wikimedia.org (see below). However, when I try to do the same using Java socket like the following:
Socket s = new Socket("208.80.154.4", 80); // IP is same as the IP printed by telnet
BufferedWriter writer = new BufferedWriter(s.getOutputStream());
writer.println("HEAD /robots.txt HTTP/1.1");
writer.println("Host: lists.wikimedia.org");
writer.println("Connection: Keep-Alive");
writer.flush();
InputStreamReader r = new InputStreamReader(s.getInputStream());
BufferedReader reader = new BufferedReader(r);
String line;
while ((line = reader.readLine()) != null) {
...
}
The readLine blocks infinitely until the socket times out...
Does anyone have ANY idea why this might be happening? The same code works fine with most of the other URLs, and interestingly enough this bug only happens for some of the ROBOTS.TXT requests... I'm so confused why this might be happening.
Edit:
Interestingly enough, using apache HttpClient library gives me the correct result for lists.wikimedia.org/robots.txt. Is there something else I need to do if I want to manually do it via Socket?

Probably you are missing the additional CRLF to end the HTTP request header. I also would write them explicitly, to avoid platform confusions, like so (untested):
writer.print("HEAD /robots.txt HTTP/1.1\r\n");
writer.print("Host: lists.wikimedia.org\r\n");
writer.print("Connection: Keep-Alive\r\n");
writer.print("\r\n");
writer.flush();
also consider using a HTTPURLConnection instead of plain sockets, takes away all this burdons:
HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
connection.setRequestMethod("HEAD");
...

Slow reading of HTTP response using InputStreamReader and BufferedReader?

I have the following code in Java that sends an HTTP request to a web server and read the response:
StringBuilder response = new StringBuilder(50000);
URL url2 = new URL(ServiceURL);
connection = (HttpURLConnection)url2.openConnection();
connection.setRequestMethod("POST");
//... (some more connection settings) ...
BufferedWriter wr = new BufferedWriter(new OutputStreamWriter(connection.getOutputStream(), "UTF-8"));
wr.write(Request);
wr.flush ();
wr.close ();
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
int i = 0;
while ((i = rd.read()) > 0) {
response.append((char)i);
}
It works for most cases, but I have a problem with one server that returns a rather large XML (something like 500KB; I guess this is pretty large for just a bunch of text..), where I keep getting a read timeout exception.
I believe it's not a network problem, because I've tried making the same request using curl and the response just arrived all right and pretty quick, something like two seconds.
When I look what's going on in the network (using wireshark to capture the packets), I noticed that the TCP receive window in my computer gets full at some point. The TCP stack sometimes survives this; I can see the server sending TCP keep-alive to keep the connection up, but in the end the TCP connection just breaks down.
Could it be that the reading part of the code (appending the received response character-by-character) is slowing my code down? Is there a more efficient way to read an HTTP response?

Reading character by character is quite slow, yes. Try reading chunks at a time into a buffer:
char[] buf = new char[2048];
int charsRead;
while((charsRead = rd.read(buf, 0, 2048)) > 0) {
response.append(buf, 0, charsRead);
}

As Phil already said reading the stream byte by byte is kinda slow. I prefer using the readLine() method of BufferedReader :
StringBuilder response = new StringBuilder();
String line = "";
while((line = rd.readLine()) != null) {
response.append(line + System.getProperty("line.separator");
}
If possible, I would consider using the Apache HTTP Client library. It is easy to use and very powerful in handling HTTP stuff.
http://hc.apache.org/httpcomponents-client-ga/
You should also keep in mind to set the socket and connection timeouts. This way you can control how long a connection is kept open (alt least on you side of the connection).
And last but not least always close your HTTP connections in a finally block after you received the response, otherwise you may run into a too many open files problem.
Hope this heps ;)

How do I recognize EOF in Java Sockets?

I want to recognize end of data stream in Java Sockets. When I run the code below, it just stuck and keeps running (it stucks at value 10).
I also want the program to download binary files, but the last byte is always distinct, so I don't know how to stop the while (pragmatically).
String host = "example.com";
String path = "/";
Socket connection = new Socket(host, 80);
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.write("GET "+ path +" HTTP/1.1\r\nHost: "+ host +"\r\n\r\n");
out.flush();
int dataBuffer;
while ((dataBuffer = connection.getInputStream().read()) != -1)
System.out.println(dataBuffer);
out.close();
Thanks for any hints.

Actually your code is not correct.
In HTTP 1.0 each connection is closed and as a result the client could detect when an input has ended.
In HTTP 1.1 with persistent connections, the underlying TCP connection remains open, so a client can detect when an input ends with 1 of the following 2 ways:
1) The HTTP Server puts a Content-Length header indicating the size of the response. This can be used by the client to understand when the reponse has been fully read.
2)The response is send in Chunked-Encoding meaning that it comes in chunks prefixed with the size of each chunk. The client using this information can construct the response from the chunks received by the server.
You should be using an HTTP Client library since implementing a generic HTTP client is not trivial (at all I may say).
To be specific in your code posted you should have followed one of the above approaches.
Additionally you should read in lines, since HTTP is a line terminated protocol.
I.e. something like:
BufferedReader in =new BufferedReader(new InputStreamReader( Connection.getInputStream() ) );
String s=null;
while ( (s=in.readLine()) != null) {
//Read HTTP header
if (s.isEmpty()) break;//No more headers
}
}
By sending a Connection: close as suggested by khachik, gets the job done (since the closing of the connection helps detect the end of input) but the performance gets worse because for each request you start a new connection.
It depends of course on what you are trying to do (if you care or not)

You should use existing libraries for HTTP. See here.
Your code works as expected. The server doesn't close the connection, and dataBuffer never becomes -1. This happens because connections are kept alive in HTTP 1.1 by default. Use HTTP 1.0, or put Connection: close header in your request.
For example:
out.write("GET "+ path +" HTTP/1.1\r\nHost: "+ host +"\r\nConnection: close\r\n\r\n");
out.flush();
int dataBuffer;
while ((dataBuffer = connection.getInputStream().read()) != -1)
System.out.print((char)dataBuffer);
out.close();

Java HTTP Request Occasionally Hangs

For the majority of the time, my HTTP Requests work with no problem. However, occasionally they will hang.
The code that I am using is set up so that if the request succeeds (with a response code of 200 or 201), then call screen.requestSucceeded(). If the request fails, then call screen.requestFailed().
When the request hangs, however, it does so before one of the above methods are called. Is there something wrong with my code? Should I be using some sort of best practice to prevent any hanging?
The following is my code. I would appreciate any help. Thanks!
HttpConnection connection = (HttpConnection) Connector.open(url
+ connectionParameters);
connection.setRequestMethod(method);
connection.setRequestProperty("WWW-Authenticate",
"OAuth realm=api.netflix.com");
if (method.equals("POST") && postData != null) {
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
connection.setRequestProperty("Content-Length", Integer
.toString(postData.length));
OutputStream requestOutput = connection.openOutputStream();
requestOutput.write(postData);
requestOutput.close();
}
int responseCode = connection.getResponseCode();
System.out.println("RESPONSE CODE: " + responseCode);
if (connection instanceof HttpsConnection) {
HttpsConnection secureConnection = (HttpsConnection) connection;
String issuer = secureConnection.getSecurityInfo()
.getServerCertificate().getIssuer();
UiApplication.getUiApplication().invokeLater(
new DialogRunner(
"Secure Connection! Certificate issued by: "
+ issuer));
}
if (responseCode != 200 && responseCode != 201) {
screen.requestFailed("Unexpected response code: "
+ responseCode);
connection.close();
return;
}
String contentType = connection.getHeaderField("Content-type");
ByteArrayOutputStream baos = new ByteArrayOutputStream();
InputStream responseData = connection.openInputStream();
byte[] buffer = new byte[20000];
int bytesRead = 0;
while ((bytesRead = responseData.read(buffer)) > 0) {
baos.write(buffer, 0, bytesRead);
}
baos.close();
connection.close();
screen.requestSucceeded(baos.toByteArray(), contentType);
} catch (IOException ex) {
screen.requestFailed(ex.toString());
}

Without any trace, I am just shooting in the dark.
Try to add this 2 calls,
System.setProperty("http.keepAlive", "false");
connection.setRequestProperty("Connection", "close");
Keep-alive is a common cause for stale connections. These calls will disable it.

I don't see any issues with the code. It could be that your platform has an intermittent bug, or that the website is causing the connection to hang. Changing connection parameters, such as keep alive, may help.
But, even with a timeout set, Sockets can hang indefinitely - a friend aptly demonstrated this to me some years ago by pulling out the network cable - my program just hung there forever, even with a SO_TIMEOUT set to 30 seconds.
As a "best practice", you can avoid hanging your application by moving all network communication to a separate thread. If you wrap up each request as a Runnable and queue these for exeuction, you maintain control over timeouts (synchronization is still in java, rather than a blocking native I/O call). You can interrupt your waiting thread after (say) 30s to avoid stalling your app. You could then either inform the user, or retry the request. Because the request is a Runnable, you can remove it from the stalled thread's queue and schedule it to execute on another thread.

I see you have code to handle sending a "POST" type request, however there is nothing that writes the POST data in the request. If the connection type is "POST", then you should be doing the following BEFORE the connection.getResponseCode():
set the "Content-Length" header
set the "Content-Type" header (which you're doing)
get an OutputStream from the connection using connection.openOutputStream()
write the POST (form) data to the OutputStream
close the OutputStream

I noticed this problem too on the blackberry OS 5.0. There is no way to reproduce this reliably. We ended up using additional thread using wait/notify along with Timertask.