I am running into an issue with some code in my Android application which downloads a file from a URL, here is a code snippet:
int bytesRead = 0;
final byte[] buffer = new byte[32 * 1024];
InputStream stream = httpUrlConnection.getInputStream();
try {
while ((bytesRead = stream.read(buffer)) > 0) {
Log.i("TAG", "Read from steam, Bytes Read: " + bytesRead);
}
} catch (IOException ex) {
//Recover from lost WIFI connection.
} finally {
stream.close();
}
My application relies on InputStream.read() to throw an IOException if WiFi connectivity is lost. As stated in the Java 8 documentation this method should throw an IOException "if the input stream has been closed, or if some other I/O error occurs". In Android M, this occurs immediately and my code can process and recover from the exception. In Android N, this exception is not thrown which causes my app to simply hang in the read() method, it never breaks out of it. Has anyone else run into this problem and worked around it in such a way that doesn't break backwards compatibility? Is this a new Android N bug?
Reading from a socket can block forever if the connection goes down. You need to use a read timeout.
To avoid reinventing the wheel and find yourself figuring out this and some other common scenarios, I would use a high library such as Volley
see: Transmitting Network Data Using Volley
Volley offers you in a very straight forward way interesting things as:
timeout control
Ease of customization, for example, for retry and backoff
control of several kind of errors
cancellation request API. You can cancel a single request, or you can set blocks or scopes of requests to cancel.
Debugging and tracing tools.
etc
Sending a request is as easy as something like this (from the docs )
final TextView mTextView = (TextView) findViewById(R.id.text);
...
// Instantiate the RequestQueue.
RequestQueue queue = Volley.newRequestQueue(this);
String url ="http://www.google.com";
// Request a string response from the provided URL.
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
#Override
public void onResponse(String response) {
// Display the first 500 characters of the response string.
mTextView.setText("Response is: "+ response.substring(0,500));
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
mTextView.setText("That didn't work!");
}
});
// Add the request to the RequestQueue.
queue.add(stringRequest);
Setting a timeout/retry policy is as easy as:
stringRequest.setRetryPolicy(new DefaultRetryPolicy(20 * 1000, 1, 1.0f));
where the parameters are:
Timeout: Specifies Socket Timeout in millis per every retry attempt.
Number Of Retries: Number of times retry is attempted.
Back Off Multiplier: A multiplier which is used to determine exponential time set to socket for every retry attempt.
Regarding error handling
As you see, you are passing an error listener new Response.ErrorListener() to the request. When there is an error, Volley invokes the onErrorResponse callback public void onErrorResponse method passing an instance of the VolleyError object when there is an error while performing the request.
The following is the list of exceptions in Volley, taken from this post here
AuthFailureError — If you are trying to do Http Basic authentication then this error is most likely to come.
NetworkError — Socket disconnection, server down, DNS issues might result in this error.
NoConnectionError — Similar to NetworkError, but fires when device does not have internet connection, your error handling logic can club
NetworkError and NoConnectionError together and treat them similarly.
ParseError — While using JsonObjectRequest or JsonArrayRequest if the received JSON is malformed then this exception will be generated. If you get this error then it is a problem that should be fixed instead of being handled.
ServerError — The server responded with an error, most likely with 4xx or 5xx HTTP status codes.
TimeoutError — Socket timeout, either server is too busy to handle the request or there is some network latency issue. By default Volley times out the request after 2.5 seconds, use a RetryPolicy if you are consistently getting this error.
So you can do things like
if ((error instanceof NetworkError) || (error instanceof NoConnectionError)) {
//then it was a network error
}
As #EJP says "Reading from a socket can block forever if the connection goes down", just add this line to your code , and also catch java.net.SocketTimeoutException :
int bytesRead = 0;
final byte[] buffer = new byte[32 * 1024];
InputStream stream = httpUrlConnection.getInputStream();
httpUrlConnection.setConnectTimeout(5000);
try {
while ((bytesRead = stream.read(buffer)) > 0) {
Log.i("TAG", "Read from steam, Bytes Read: " + bytesRead);
}
} catch (java.net.SocketTimeoutException ex) {
//Recover from lost WIFI connection.
} finally {
stream.close();
}
Related
I am currently trying to get a little network communication going between a qt server and a java client.
In my example, the client wants to send an image to the Server. My problem is now, that the server never sees the data, so bytesAvailable() returns 0.
I already tried QDataStream, QTextStream and readAll(), still no data.
Server:
QTcpServer* tcpServer;
QTcpSocket* client;
tcpServer = new QTcpServer();
if(!tcpServer->listen(QHostAddress::Any, 7005)){
tcpServer->close();
return;
}
...
tcpServer->waitforNewConnection();
client = tcpServer->nextPendingConnection();
client->waitForConencted();
while(client->state()==connected){
// Syntax here might be iffy, did it from my phone
if(client->bytesAvailable()>0){
//do stuff here, but the program doesnt get here, since bytesAvailable returns 0;
}
}
CLient:
public SendPackage() {
try {
socket = new Socket(ServerIP, Port);
socket.setSoTimeout(60000);
output = new BufferedOutputStream(socket.getOutputStream());
outwriter = new OutputStreamWriter(output);
} catch (ConnectException e) {
System.out.println("Server error, no connection established.");
} catch (Exception e) {
e.printStackTrace();
}
}
public void Send(BufferedImage img) {
try {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write(img, GUI.imageType, baos);
baos.flush();
byte[] imgbyte = baos.toByteArray();
System.out.println(imgbyte.length);
System.out.println("sending");
outwriter.write(imgbyte.length);
outwriter.flush();
// here i'd send the image, if i had a connection ...
output.flush();
} catch (Exception e) {
e.printStackTrace();
}
}
The connection and everything builds up fine, the code even tells me when the socket was disconnected when trying to send, so I guess connection isn't a problem.
I just started using Qt, so if you guys have any idea to why this wouldn't work, I'd be pleased to try it.
client->waitForConencted();
// At this point the client is connected, but it is likely that no data were received yet
client->waitForReadyRead(-1); // <- Add this
// Now there should be at least 1 byte available, unless waitForConencted or waitForReadyRead failed (you should check that)
if(client->bytesAvailable() > 0) {
// ...
}
Note that you can not expect all the data to arrive at once. The TCP stream can get fragmented in any way and the data will be received in randomly sized pieces. You must repeat waiting and reading until you receive everything. This also means that you must somehow know when you did receive everything. So you need to know how much data is coming, or somehow recognize the end of it. You can for example disconnect right after the data transfer, or send the data length first. Depends on your application.
Also have a look at QIDevice::readyRead signal which would allow you to handle reading asynchronously.
I'm trying to do a HTTP GET using the HttpURLConnection object in Android.
UPDATE
I tried connection to a different server. This is also hosted within Cloud 9 (c9.io) and also returns a json response. This time I'm not getting a 301 redirect, but I am getting the actual response the server is supposed to send.
Since this means the problem is localised within the server, I've reorganized the following sections in order to focus reading onto the server-related information. Android related information has been moved to the end of the question.
Where I am connecting:
Development server on Cloud9
Using the Laravel Framework 5.2 (we cannot upgrade to 5.3 at this time, due to unsupported project dependencies)
The server should return a JSON answer
If I connect to the same URL through the browser I get the correct response (JSON string. Required HTTP Headers and a '200' HTTP Response Code)
Where I am connecting FROM
Android phone (Oneplus 3, on Android 6.0)
Compile SDK version: 23
Using Build Tools: "23.0.3"
Using Min SDK verion: 19
Using Target SDK version: 22
I'm connectiong using a HttpURLConnection object, using HTTP Method 'GET'
HTTP Response on Android
When I run my code I get the folling result from the server:
The HTTP response code is 301 but the message is null.
The new URL is exactly the same, but using HTTPS. It seems server is somehow forcing SSL/TSL encryption. Which does not happen when accessing HTTP from the browser.
HTTP Header (on Android):
date => Tue, 04 Oct 2016 05:56:26 GMT
location => https://domain.com/route/ (I modified this bit)
content-length => 382
content-type => text/html; charset=iso-8859-1
X-BACKEND => apps-proxy
X-Android-Selected-Protocol => http/1.1
X-Android-Sent-Millis => 1475560583894
X-Android-Received-Millis => 1475560585637
X-Android-Response-Source => NETWORK 301
null => HTTP/1.1 301
Other data
Since it seems the server wants Android to use HTTPS, I tried modifying the code to use HTTPS (HttpsURLConnection). This may or may not solve this problem, but I am unable to check it since I get an annoying SSL handshake failed error. Plus I have no need for encryption on this application, and therefore I'm reluctant to solve the problems coming with it.
This is all running within an AsyncTask object (since Android get moody when you try to use a network connection on the main thread).
Setting up a new server (outside of Cloud 9 and without any SSL/TSL) could be an option, but I'm reluctant to do this since it would be quite time consuming.
I tried connecting to another Cloud 9 server (which also returns a json response), using the exact same code, and everything works correctly. This suggests that the problem arises from the HTPP 301 error.
I will try to share with you any other information you may require to answer my question!
Native Android stuff (moved on UPDATE, see above)
The response content seems to be an incomplete JSON:
{ 'status':'ERROR'
Note I did NOT forget the closing } character, that's what the response actually containts. This is injected somewhere unknown (to me) during the workflow. When I capture the HTTP response (using Charles on my PC, which is set as a Proxy for my phone's Wi-Fi connection) it's content is (as expected) a simple HTML telling you to redirect (HTPP code 301) to a new route.
The invalid JSON code (above) isn't there, but a valid HTML is.
This would suggest that the invalid JSON appears somewhere internally to my code (not on the server, or transport). But there is no code on my app that generates a JSON string, let alone inject it into the response I'm processing.
Code for the HttpURLConnection
this.setURL(ruta); //gets correct url
HttpURLConnection cxn = (HttpURLConnection) this.getURL().openConnection(); //init
cxn.setRequestMethod("GET"); //use HTTP GET verb
cxn.setUseCaches(false); //no cache
cxn.setRequestProperty("Cache-Control", "no-cache"); //even less cache
cxn.setDoOutput(false); //only true in POST/PUT requests
cxn.setRequestProperty("Connection","keep-alive");
cxn.setRequestProperty("DNT", "1"); //TEMP
cxn.setInstanceFollowRedirects(true); //should follow redirects
cxn.setRequestProperty( "charset", "utf-8");
Code for the reading the result
int status_code = cxn.getResponseCode();
InputStream responseStream = new BufferedInputStream(cxn.getInputStream());
BufferedReader responseStreamReader = new BufferedReader(new InputStreamReader(responseStream));
String line = "";
StringBuilder stringBuilder = new StringBuilder();
while ((line = responseStreamReader.readLine()) != null) {
stringBuilder.append(line).append("\n");
}
responseStreamReader.close();
String response = stringBuilder.toString();
cxn.disconnect();
Remove the code you've used to create the HttpURLConnection and try with this one:
URL url;
HttpURLConnection urlConnection = null;
try {
url = new URL("http://www.domain.com/index.aspx?parameter1=X¶meter2=X"); //Use your url and add the GET parameters
urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setInstanceFollowRedirects(false); /* added line */
InputStream in = urlConnection.getInputStream();
InputStreamReader isw = new InputStreamReader(in);
int data = isw.read();
while (data != -1) {
char current = (char) data;
data = isw.read();
System.out.print(current);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (urlConnection != null) {
urlConnection.disconnect();
}
}
This should be all you need to set for your GET request.
EDIT:
I've tested the webservice using Volley, here's the code I've used in order to retrieve the webservice response:
public class MainActivity extends AppCompatActivity {
public String response;
TextView textView;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
textView = (TextView) findViewById(R.id.rTextView);
RequestQueue queue = Volley.newRequestQueue(this);
String url = "yourWebserviceUrl";
// Request a string response from the provided URL.
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener < String > () {
#Override
public void onResponse(String response) {
textView.setText("Response is: " + response);
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
textView.setText("That didn't work!");
}
});
// Add the request to the RequestQueue.
queue.add(stringRequest);
}
}
And this is the response I got:
{"status":"ok","found":false,"extra":"App\\Scanners"}
Changing the protocol to https worked for me.
I faced the same problem, and I fixed it after reading this source.
All we need to do is handling 3** errors like shown below
if(responseCode > 300 && responseCode < 400) {
String redirectHeader = conn.getHeaderField("Location");
if(TextUtils.isEmpty(redirectHeader)) {
return new JsonResponse(responseCode, "Failed to redirect");
}
JsonRequest newRequest = request;
newRequest.url = redirectHeader;
return getJsonFromUrl(newRequest);
}
Each 3** response should have a header with name Location which contains a redirect link which we should use.
Change the line :
HttpURLConnection cxn = (HttpURLConnection) this.getURL().openConnection();
with :
HttpsURLConnection cxn = (HttpsURLConnection) this.getURL().openConnection();
So you will able to handle https
I'm writing a simple TCP client/server program pair in Java, and the server must disconnect if the client hasn't sent anything in 10 seconds. socket.setSoTimeout() gets me that, and the server disconnects just fine. The problem is - how can I get the client to determine if the server is closed? Currently I'm using DataOutputStream for writing to the server, and some answers here on SO suggest that writing to a closed socket will throw an IOException, but that doesn't happen.
What kind of writer object should I use to send arbitrary byte blocks to the server, that would throw an exception or otherwise indicate that the connection has been closed remotely?
Edit: here's the client code. This is a test function that reads one file from the file system and sends it to the server. It sends it in chunks, and pauses for some time between each chunk.
public static void sendFileWithTimeout(String file, String address, int dataPacketSize, int timeout) {
Socket connectionToServer = null;
DataOutputStream outStream = null;
FileInputStream inStream = null;
try {
connectionToServer = new Socket(address, 2233);
outStream = new DataOutputStream(connectionToServer.getOutputStream());
Path fileObject = Paths.get(file);
outStream.writeUTF(fileObject.getFileName().toString());
byte[] data = new byte[dataPacketSize];
inStream = new FileInputStream(fileObject.toFile());
boolean fileFinished = false;
while (!fileFinished) {
int bytesRead = inStream.read(data);
if (bytesRead == -1) {
fileFinished = true;
} else {
outStream.write(data, 0, bytesRead);
System.out.println("Thread " + Thread.currentThread().getName() + " wrote " + bytesRead + " bytes.");
Thread.sleep(timeout);
}
}
} catch (IOException | InterruptedException e) {
System.out.println("Something something.");
throw new RuntimeException("Problem sending data to server.", e);
} finally {
TCPUtil.silentCloseObject(inStream);
TCPUtil.silentCloseObject(outStream);
TCPUtil.silentCloseObject(connectionToServer);
}
}
I'd expect the outStream.write to throw an IOException when it tries to write to a closed server, but nothing.
I'd expect the outStream.write to throw an IOException when it tries to write to a closed server, but nothing.
It won't do that the first time, because of the socket send buffer. If you keep writing, it will eventually throw an IOException: 'connection reset'. If you don't have data to get to that point, you will never find out that the peer has closed.
I think you need to flush and close your stream after written like outStream.flush(); outStream.close(); inStream.close();
Remember ServerSocket.setSoTimeout() is different from client's function with same name.
For server, this function only throws SocketTimeoutException for you to catch it if timeout is expired, but the server socket still remains.
For client, setSoTimeout() relates to 'read timeout' for stream reading.
In your case, you must show your server code of closing the connected socket after catching SocketTimeoutException => ensure server closed the associated socket with a specified client. If done, at client side, your code line:
throw new RuntimeException("Problem sending data to server.", e);
will be called.
[Update]
I noticed that you stated to set timeout for the accepted socket at server side to 10 secs (=10,000 milliseconds); for that period, did your client complete all the file sending? if it did, never the exception occurs.
[Suggest]
for probing, just comment out your code of reading file content to send to server, and try replacing with several lines of writing to output stream:
outStream.writeUTF("ONE");
outStream.writeUTF("TWO");
outStream.writeUTF("TREE");
Then you can come to the conclusion.
Whenever I try to to send rapid HTTP Post in succession, the server sometimes crashes (java.net.ConnectException: Connection refused: connect), sometimes freezes (no error, but cannot use it anymore), and sometimes works....
If I send HTTP Post to the server slowly, there seems to be no problems at all.
I have the simple code for an http server in node.js - My guess is that sometimes the NodeJS server will receive a request, then receive another request before it sends out the respond, thus causing all sorts of problem. How do I make my server able to accept multiple requests all at once?
var server = http.createServer(function (req, res) {
if (req.method != 'POST') {
res.end();
}
else {
req.on('data', function(chunk) {
//Do some stuff here
file1=JSON.parse(chunk.toString());
console.log("Hello World") ;
}
req.on('end', function() {
res.writeHead(200, "OK", {'Content-Type': 'text/html'});
res.end();
});
}
} server.listen(9000);
EDIT
Here is the java program sending the HTTP POSTS
public static String httpUrlRequest(String requestURL, String json) {
URL url;
String response = "";
HttpURLConnection connection = null;
InputStream is = null;
try {
url = new URL(requestURL);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.getOutputStream().write(json.getBytes());
connection.getOutputStream().flush();
connection.getOutputStream().close();
int code = connection.getResponseCode();
System.out.println("code" + code);
} catch (Exception e) {
e.printStackTrace();
} finally {
connection.disconnect();
}
return response;
}
public static void main(String[] args) {
Date date = new Date();
Gson gson = new Gson();
Map<String, Object> tempMap = gson.fromJson(json, new TypeToken<Map<String, Object>>(){}.getType());
for(int i = 0; i < 10; i++){
date = new Date();
tempMap.put("GetOn", getDateString(date));
httpUrlRequest("http://111.111.11.111:9000" ,gson.toJson(tempMap));
}
}
UPDATE :
If I parse the JSON in the nodejs server, then sometimes I get the error where the connection is refused.
So when I parse the request data, for some reason, nodejs fails to receive the entire json file that was sent (5KB). Instead, it only receives half of it, and my Java console says connect error. And this problem occurs AFTER nodejs correctly parses about 3 to 5 requests. Then on the next request, everything goes wrong. How can I tell if the java breaks connection causing only half of the JSON to be sent, or if only half of the JSON is sent causing nodejs to crash, ultimately resulting in connect error.
If I comment out all the parsing, then I never get the errors anymore. I don't even understand why JSON.Parse in nodejs would induce a java connect error....
Your problem lies here:
req.on('data', function(chunk) {
//Do some stuff here
file1=JSON.parse(chunk.toString());
console.log("Hello World") ;
}
Since the body can be sent in multiple packets, you might receive more than 1 'data' event, thus attempting to parse an incomplete JSON. Try this instead:
var chunks = [];
req.on('data', function(chunk) {
chunks.push(chunk);
}
req.on('end', function () {
// assemble all chunks
var body = Buffer.concat(chunks).toString(),
file1 = JSON.parse(body);
res.writeHead(200, "OK", {'Content-Type': 'text/html'});
res.end();
});
during this kind of tests, it would be easy to hit max throughput via the connection port.
http://www.slideshare.net/sh1mmer/a-million-connections-and-beyond-nodejs-at-scale
https://groups.google.com/forum/#!topic/nodejs/cRRS7ZJkyzc
The simply solution is ulimit -n 100000 before your run your node.js. But never do this in the production, better consider cluster for really large connection if you need to handle it.
I have another device & application transmitting data in real-time (every few ms) and on my receiving device, I want to:
1) read/receive this data, and
2) use it to update a UI element (a dynamic graph in this case)
The data sender uses a socket in a background service, using AsyncTasks every few ms to send data. To initialize, it does the following:
echoSocket = new Socket(HOST, PORT);
out = new PrintWriter(echoSocket.getOutputStream(), true);
And to periodically send data it does:
static class sendDataTask extends AsyncTask<Float, Void, Void> {
#Override
protected Void doInBackground(Float... params) {
try {
JSONObject j = new JSONObject();
j.put("x", params[0]);
j.put("y", params[1]);
j.put("z", params[2]);
String jString = j.toString();
out.println(jString);
} catch (Exception e) {
Log.e("sendDataTask", e.toString());
}
return null;
}
}
How should I go about receiving this data in my application? Should I also use a background service with AsyncTasks that try to read from the socket every few ms? How about communicating with the UI thread?
There are many ways to do this. The simplest would be to use blocking reads in the doInBackground method of an AsyncTask and call publishProgress() to forward the new data to the UI thread.
Then implement onProgressUpdate with code (running in the UI thread) that updates the screen.
You should be aware that your read may not receive the entire message that you sent -- you may need to read more data and append it to the input received so far until you have an entire JSON message.
By blocking reads, I mean something like this (in pseudo code):
open a socket connected to the sender
is = socket.getInputStream()
initialize buffer, offset, and length
while the socket is good
bytesRead = is.read(buffer, offset, length)
if(bytesRead <= 0)
bail out you have an error
offset += bytesRead;
length -= bytesRead
if(you have a complete message)
copy the message out of the buffer (or parse the message here into
some other data structure)
publishProgress(the message)
reset buffer offset and length for the next message.
(remember you may have received part of the following message)
end-if
end-while
The copy-out-of-the-buffer is necessary because the onProgressUpdate does not happen immediately so you need to be sure the next message does not overwrite the current one before it gets handled.