Note: expressed in Scala. Using a BufferedReader to process a gzipped HTTP stream and iterating through each line to read the incoming data. Problem is that if there is ever a reset connection due to a network I/O issue (provider does weird things sometimes) then I can see the connection staying open for up to 15 seconds before it times out, something I'd like to get down to 1 second. For some reason our office provider resets connections every 11 hours.
Here's how I'm handling the connection:
val connection = getConnection(URL, USER, PASSWORD)
val inputStream = connection.getInputStream()
val reader = new BufferedReader(new InputStreamReader(new StreamingGZIPInputStream(inputStream), GNIP_CHARSET))
var line = reader.readLine()
while(line != null){
parseSupervisor ! ParseThis(line)
line = reader.readLine()
}
throw new ParseStreamCollapseException
and here is getConnection defined:
private def getConnection(urlString: String, user: String, password: String): HttpURLConnection = {
val url = new URL(urlString)
val connection = url.openConnection().asInstanceOf[HttpURLConnection]
connection.setReadTimeout(1000 * KEEPALIVE_TIMEOUT)
connection.setConnectTimeout(1000 * 1)
connection.setRequestProperty("Authorization", createAuthHeader(user, password));
connection.setRequestProperty("Accept-Encoding", "gzip")
connection
}
To summarize: reading HTTP stream line-by-line via java.io.BufferedReader. Keep-alive on stream is 16 seconds, but to prevent further data loss I'd like to narrow it down to hopefully 1-2 seconds (basically check if stream is currently blank or if is network I/O). Some device in the middle is terminating the connection every 11 hours, and it would be nice to have a meaningful workaround to minimize data loss. The HttpURLConnection does not receive a "termination signal" on the connection.
Thanks!
Unfortunately, unless the network device that's killing the connection is closing it cleanly, you're not going to get any sort of notification that the connection is dead. The reason for this is that there is no way to tell the difference between a remote host that is just taking a long time to respond and a broken connection. Either way the socket is silent.
Again, assuming that the connection is just being severed, your only option to detect the broken connection more quickly is to decrease your timeout.
Related
private fun downloadAPKStream() : InputStream? {
val url = URL(this.url)
val connection = url.openConnection() as HttpURLConnection
connection.requestMethod = "GET"
connection.connect() connection.connectTimeout = 5000
fileSize = connection.contentLength
val inputStream = connection.inputStream
return inputStream
}
I'm using this method to download apk file. But here if internet is slow then due to timeout of 5000 ms, download gets stuck in between without get completed. And if I comment this line or I don't provide any **connection.connectTimeout then it runs fine but sometimes get stuck in infinite time loop. What should I do to make it download files of any size and with slow internet as well.
You got timeout meaning wrong. It is not a max. allowed time of given (network in this case) operation, but max. allowed time of inactivity after which operation is considered stalled and fail. So you should set the timeout to some sane value, that would make sense in real life. As value is in milliseconds, the 5000 is not the one because it's just 5 seconds - any small network hiccup and your connection will get axed. Set it to something higher, like 30 secs or 1 minute or more.
Also note that this is connection timeout only. This means you should be able to establish protocol connection to remote server during that time, but this got nothing to data transfer itself. Data transfer oa process that comes next, once connection is established. For data transfer timeout (which definitely should be set higher) you need to use setReadTimeout().
Finally, you must set connection timeout prior calling connect() otherwise it makes no sense as it is already too late - this is what you got in your code now.
PS: use Download Manager instead.
I am new in http connections. The thing I want to realize is that the server should send some data (notifications) to the client periodically by persistent connection.
I wrote a code in server side by php like:
<?php
set_time_limit(0);
header('Connection: keep-alive');
$i = 0;
while($i < 10){
echo "Hello$i<br/>";
sleep(5);
$i++;
}
?>
and tried to connect to the server by java:
public static void main(String[] args) throws Exception {
URL oracle = new URL("http://localhost/connection.php");
URLConnection yc = oracle.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
I expected to get content from the server every five seconds like following:
Hello0<br/>
Hello1<br/>
...
but instead of this the java client is waiting 50 seconds. and printing:
Hello0<br/>Hello1<br/>Hello2<br/>Hello3<br/>Hello4<br/>Hello5<br/>Hello6<br/>Hello7<br/>Hello8<br/>Hello9<br/>
I want the server send notifications itself. instead of the client connect to the server every five seconds.
It's really unnecessary to add Connection: keep-alive header in response for HTTP/1.1 server, UNLESS for backward compatibility.
No matter how long or many times you sleep in that loop, it's seen as ONE request by the client nevertheless.
with that being said, your client snippet, in fact, only make ONE request to http://localhost/connection.php, and it's impossible to reuse URLConnection in order to dispatch another request(achieving persistent).
to sum up:
Persistent Connection behaviour is handled at transport layer (TCP), more specifically, you are required to reuse a client socket for multiple request to the same host plus some other requirements specified in HTTP/1.1.
Go and find some projects that are suitable for your needs, don't reinvent the wheel.
Flushing the connections seemed really good idea. But I think I found better solution. Instead of keeping connection with unlimited timeout, I think it is better to make persistent connection with 5 (N minutes) minutes timeout. It is better because when the user will be offline unexpectedly, the server will keep the connection alive anyway. and it is not good. That's why I am going to make 5 (this number is optional) connections for notification. That is the server will use first one for notification and closes the connection after sending request, and the rest 4 connections will be on duty. When the client (or java client) will receive the notification, it will make new connection to fill missing part or the connection times out.
and the client will be notified immediately every time (of course if connected to the internet).
If someone has better solution I will be happy to see that.
I have an interesting issue.
I have an application where inside of it, I'm trying to account for the condition where the phone is connected to a router, but that router is not connected to the internet.
I've tried multiple methods of establishing the connection, but NONE of the timeouts account for this condition.
I've tried:
HttpParams httpParameters = new BasicHttpParams();
int timeoutSocket = 1000;
HttpConnectionParams.setSoTimeout(httpParameters, timeoutSocket);
HttpConnectionParams.setConnectionTimeout(httpParameters, timeoutSocket);
I've also tried:
HttpURLConnection huc = (HttpURLConnection)serverAddress.openConnection();
huc.setDoOutput(true);
huc.setRequestMethod("PUT"); // For amazon
//huc.setRequestMethod("POST"); // For regular server.
huc.setRequestProperty("Content-Type", "text/plain");
huc.setRequestProperty("Content-Length", String.valueOf(bytes));
huc.setFixedLengthStreamingMode(bytes);
huc.setConnectTimeout(1000); // Establishing connection timeout
huc.setReadTimeout(1000);
But in BOTH cases, when I execute/ get the output stream, it takes about 20 seconds to receive an UnknownHostException error.
I would like that reduced to a maximum of 5 seconds before reaching that conclusion.
Is there any way to do this?
Cheers
Through lots of searching and through the help of this link I've found a solid solution that seams to be working so far.
My understanding of the conclusion is that when I use methods like:
DataOutputStream wr = new DataOutputStream(huc.getOutputStream());
or
InputStream is = ucon.getInputStream();
BufferedInputStream bis = new BufferedInputStream(is);
(uploading or downloading)
There is a lot of things happening under the hood. Including a DNS lookup call. With no-connectivity, but while connected to a router, this is taking about 20 seconds to finally reach a UnknownHostException.
However, if I add this line of code first before the above code is executed:
InetAddress iAddr = InetAddress.getByName("myserverName.com");
Then it will give me the proper SocketTimeOutException and responds exactly how I would hope/expect it to. The above line of code apparently caches the DNS Lookup, and the timeouts work as expected.
Also, something to note: that once the failure is cached, executing the code above will take as long to fail as the other previous code. (Can't tell you exactly what will trigger this) But if you connect to the internet again, and then enter the connected but no connectivity state again, the earlier success will be cached and the timeouts will again work properly.
This wasn't particularly easy to find or figure out, so I hope this helps somebody.
Cheers,
You could implement a CountDownTimer that has a limit of 5000ms see this http://dewful.com/?p=3
I have long-lasting TCP connection between two computers (second not under my control). Second computer can send FIN flag at every moment, and first must close connection corrent (send FIN flag back to second computer).
How can I know that the second computer sending a FIN flag and when I must cause socket.close() method of my Java application?
Normally, you have to read the connection and when this returns -1 for EOF or an appropriate IOException, you can close the connection. Note: SocketTimeoutException doesn't mean the connection is closed.
an example.
boolean ok = false;
try {
int b = in.read();
ok = b >= 0;
if (!ok)
throw new EOFException();
} finally {
if (!ok)
in.close();
}
Detecting a soft connection close by the other side (when they manage to send FIN/RST flags) is only partially possible with the old Java I/O library. You will learn of a broken connection only via a timeout, so it may be far from immediate. Your threads may hang for a long time before they realize that the party at the other end is long gone.
In order to handle it better, you need to use nio. There, such a situation will be recognized by the Selector saying there is data ready for reading but then read on the channel returning less than zero. This will allow you to learn about soft connection resets almost immediately.
On the other hand, a hard connection termination (e.g. someone cutting the wire or network being down) can only be detected via timeouts regardless of which libraries you use as it's a property of the TCP protocol itself.
As explained above, the Socket's properties (isClosed, isConnected, etc) are not helpful. A proper solution would be to set a reasonable SO_TIMEOUT and read from the socket:
In case of connection closed by the peer, the read operation would return with '-1'
In case of read timeout, the read operation would throw SocketTimeoutException.
(Scala code)
val socket = new Socket("localhost", 8888)
socket.setSoTimeout(10000) /* Set reasonable read timeout */
try {
val res = socket.getInputStream().read()
if (res < 0)
... /* Socket closed */
else
... /* Socket read succeeded */
} catch {
case _: SocketTimeoutException => ... /* Socket not closed */
case _ => ... /* Merde */
}
I want to recognize end of data stream in Java Sockets. When I run the code below, it just stuck and keeps running (it stucks at value 10).
I also want the program to download binary files, but the last byte is always distinct, so I don't know how to stop the while (pragmatically).
String host = "example.com";
String path = "/";
Socket connection = new Socket(host, 80);
PrintWriter out = new PrintWriter(connection.getOutputStream());
out.write("GET "+ path +" HTTP/1.1\r\nHost: "+ host +"\r\n\r\n");
out.flush();
int dataBuffer;
while ((dataBuffer = connection.getInputStream().read()) != -1)
System.out.println(dataBuffer);
out.close();
Thanks for any hints.
Actually your code is not correct.
In HTTP 1.0 each connection is closed and as a result the client could detect when an input has ended.
In HTTP 1.1 with persistent connections, the underlying TCP connection remains open, so a client can detect when an input ends with 1 of the following 2 ways:
1) The HTTP Server puts a Content-Length header indicating the size of the response. This can be used by the client to understand when the reponse has been fully read.
2)The response is send in Chunked-Encoding meaning that it comes in chunks prefixed with the size of each chunk. The client using this information can construct the response from the chunks received by the server.
You should be using an HTTP Client library since implementing a generic HTTP client is not trivial (at all I may say).
To be specific in your code posted you should have followed one of the above approaches.
Additionally you should read in lines, since HTTP is a line terminated protocol.
I.e. something like:
BufferedReader in =new BufferedReader(new InputStreamReader( Connection.getInputStream() ) );
String s=null;
while ( (s=in.readLine()) != null) {
//Read HTTP header
if (s.isEmpty()) break;//No more headers
}
}
By sending a Connection: close as suggested by khachik, gets the job done (since the closing of the connection helps detect the end of input) but the performance gets worse because for each request you start a new connection.
It depends of course on what you are trying to do (if you care or not)
You should use existing libraries for HTTP. See here.
Your code works as expected. The server doesn't close the connection, and dataBuffer never becomes -1. This happens because connections are kept alive in HTTP 1.1 by default. Use HTTP 1.0, or put Connection: close header in your request.
For example:
out.write("GET "+ path +" HTTP/1.1\r\nHost: "+ host +"\r\nConnection: close\r\n\r\n");
out.flush();
int dataBuffer;
while ((dataBuffer = connection.getInputStream().read()) != -1)
System.out.print((char)dataBuffer);
out.close();