Develop Reference

HttpUrlConnection Expect-100 Broken Pipe

I use a Expect 100-continue in the http header like you see in the code below. The HttpUrlConnection when i get the outputstream dont wait the answer of the 100 Expectation or any other 400 error. (In the RFC they tell it's not a problem: https://www.rfc-editor.org/rfc/rfc7231#section-5.1.1).
I've looked at this post: How to wait for Expect 100-continue response in Java using HttpURLConnection in case they have more info on the problem but i have tested the setChuncked method just to see if it's make a difference but no.
The problem is when the server answer with an error and close the connection without let me finish the upload i have a Broken Pipe Exception. It's right i write bytes on a close connection but i can't handle that correctly. I can't read the server answer. I can't know that the connection is close before the write. If i try to get the ErrorStream i get a null reference same for the InputStream, ResponseCode and other methods to know what happen.
Any idea how can make it right or what i miss to make it work?
javax.net.ssl.SSLException: Write error: ssl=0x73de8c02c0: I/O error during system call, Broken pipe
at com.android.org.conscrypt.NativeCrypto.SSL_write(Native Method)
at com.android.org.conscrypt.OpenSSLSocketImpl$SSLOutputStream.write(OpenSSLSocketImpl.java:824)
at com.android.okhttp.okio.Okio$1.write(Okio.java:76)
at com.android.okhttp.okio.AsyncTimeout$1.write(AsyncTimeout.java:155)
at com.android.okhttp.okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.java:176)
at com.android.okhttp.okio.RealBufferedSink.write(RealBufferedSink.java:46)
at com.android.okhttp.internal.http.HttpConnection$ChunkedSink.write(HttpConnection.java:339)
at com.android.okhttp.okio.RealBufferedSink.emitCompleteSegments(RealBufferedSink.java:176)
at com.android.okhttp.okio.RealBufferedSink$1.write(RealBufferedSink.java:198)
at out.write(buffer, 0, bytes_read);
In the code below, i've remove some not useful part like a global try on this, or compute for the fileSize.
huc = (HttpURLConnection) new URL(upload_url).openConnection();
huc.setRequestMethod("POST");
huc.setRequestProperty("Content-Length", "" + FILE_SZ);
huc.setRequestProperty("User-Agent", "UA");
huc.setRequestProperty("Expect","100-continue");
huc.setFixedLengthStreamingMode(FILE_SZ);
huc.setUseCaches(false);
huc.setDoInput(true);
huc.setDoOutput(true);
huc.setInstanceFollowRedirects(true);
try {
// read input file chunks + publish progress
OutputStream out = new OutputStream(huc.getOutputStream());
buffer_size = 4 * 1024;
buffer = new byte[buffer_size];
total_bytes = 0;
while ((bytes_read = fis.read(buffer)) != -1) {
out.write(buffer, 0, bytes_read);
total_bytes += bytes_read;
}
out.flush();
out.close();
} catch (IOException e) {
e.printStackTrace();
}

Error while reading inputstream : Software caused connection abort' in java server

I am basically trying to host a server on an Android Device. Client devices connect to the server over TCP and send requests. The server performs the action requested by the client and then writes data back to the socket. The connection is not terminated by the server and requests are to be continuously read over the socket and replied to.
Note: The first 4 bytes of each request message contain the length of the actual message/request.
The parseXmlInputAndExecuteCmd function executes various asynchronous operations depending on the content of the input XML string. This eventually causes a change in the boolean value of 'allowResponse' variable to true and a certain response is generated which is stored in the variable of type String called 'response'. Once the boolean 'allowResponse' becomes true, the thread resumes execution and writes the response back to the socket's OutputStream
Some of these asynchronous operations include connecting and disconnecting from the corporate VPN. Could that be a cause of the error ?
Some Class level variables being used are :
private volatile boolean allowResponse = false;
private String response;
Server Code:
private void startServer() {
try {
ServerSocket serverSocket = new ServerSocket(8080);
while (true) {
Socket connectionSocket = serverSocket.accept();
BufferedInputStream bufInpStream = new BufferedInputStream(connectionSocket.getInputStream());
BufferedOutputStream bufOutStream = new BufferedOutputStream(connectionSocket.getOutputStream());
ByteArrayOutputStream contentLengthBaos = new ByteArrayOutputStream();
int c;
int count = 0;
while ((c = bufInpStream.read()) != -1) {
contentLengthBaos.write(c);
count++;
if (count == 4) {
int contLen = getMessageLength(contentLengthBaos);
String content = getMessageContent(bufInpStream, contLen);
parseXmlInputAndExecuteCmd(content);
count = 0;
while (!allowResponse) {
Thread.sleep(1000);
}
allowResponse = false;
byte[] responseDataBytes = response.getBytes();
int outputContLen = responseDataBytes.length;
byte[] contLengthBytes = ByteBuffer.allocate(4).putInt(outputContLen).array();
ByteArrayOutputStream o = new ByteArrayOutputStream();
o.write(contLengthBytes);
o.write(responseDataBytes);
byte finalOutPutBytes[] = o.toByteArray();
bufOutStream.write(finalOutPutBytes);
bufOutStream.flush();
}
}
}
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
#NonNull
private String getMessageContent(BufferedInputStream inFromClient, int contLen) throws IOException {
ByteArrayOutputStream contentBaos = new ByteArrayOutputStream();
byte[] contentBytes = new byte[contLen];
for (int i = 0; i < contLen; i++) {
contentBaos.write(inFromClient.read());
ByteArrayInputStream bais = new ByteArrayInputStream(contentBaos.toByteArray());
bais.read(contentBytes);
}
String content = new String(contentBytes);
Log.d(TAG, "Content : " + content);
return content;
}
private int getMessageLength(ByteArrayOutputStream contentLengthBaos) {
byte[] firstFourBytesArr = contentLengthBaos.toByteArray();
int contLen = new BigInteger(firstFourBytesArr).intValue();
contentLengthBaos = new ByteArrayOutputStream();
Log.d(TAG, "Content length: " + contLen);
return contLen;
}
The server is started using the following lines of code:
Thread serverThread = new Thread(new Runnable() {
#Override
public void run() {
startServer();
}
});
serverThread.start();
I am getting the following error stack trace:
W/System.err: java.net.SocketException: Software caused connection abort
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:114)
W/System.err: at java.net.SocketInputStream.read(SocketInputStream.java:170)
W/System.err: at java.net.SocketInputStream.read(SocketInputStream.java:139)
W/System.err: at java.io.BufferedInputStream.fill(BufferedInputStream.java:248)
W/System.err: at java.io.BufferedInputStream.read(BufferedInputStream.java:267)
at com.example.umathur.myapplication.MainActivity.startServer(MainActivity.java:192)
at com.example.umathur.myapplication.MainActivity.access$000(MainActivity.java:61)
at com.example.umathur.myapplication.MainActivity$1.run(MainActivity.java:139)
at java.lang.Thread.run(Thread.java:764)
I am getting an error called: java.net.SocketException: Software caused connection abort
I'm not sure where I'm going wrong in the usage of the inputstream/outputstream. Im getting the error at the following line(Line number 192 as mentioned in the stacktrace) :
while ((c = inputStream.read()) != -1)
In some similar questions on StackOverflow, I saw people stating that it might be a corporate firewall configuration issue ? Is that correct ? If so, how do I get it fixed ?
Could this be an issue with the Client code (To which I don't have access) not being written correctly ?

I can't reproduce your issue locally, and something tells me that your trigger conditions are pretty specific, but here are a few things to try to help you diagnose and fix your problem.
Your code: If it's your code, then existing web server code should work. Try the code in the code from this nice minimal Medium article on how to create your own Java web server.
The Firewall: If it's the firewall, contact your network admin person/team etc. If you don't know who it is, just talk to as high profile a person in your company that you feel comfortable asking and they should be able to direct you to the right person.
Socket SO_KEEPALIVE Option: I found an interesting SuperUser question on the subject, and read that this issue may be caused by the fact that you don't have the SO_KEEPALIVE flag set on your socket. The answer talks about PuTTY, but I thought it it might be worthwhile to try on your Java socket. It's certainly a low hanging fruit to try. As such, try adding via calling connectionSocket.setKeepAlive(true); just after the Socket connectionSocket = serverSocket.accept(); line.

I tried to run your code in my pc (not in Android) and it work good with a simply call from browser to `localhost:8080'.
I think you can do the same inside your device just to know if it's a client problem.
I edited this two lines to avoid long waiting:
byte[] contentBytes = new byte[10];
for (int i = 0; i < 10; i++) {

The number of bytes received in TrafficStats class is smaller than the number of bytes receiverd from inputsream using HTTP

I wrote a piece of code to test the available bandwidth of a device using cellular network.
I use TrafficStats class to get how many bytes the device have received now, then let the device to download a small file, and finally get how many bytes the device have transferred and the consumed time after downloading.
I set the downloaded file size is 10245 bytes, but the number of getMobileRxBytes() after downloading minus getMobileRxBytes() before downloading is always lower than 10245 bytes.
I am sure that the cellular network is active and the WiFi is disable, and I got the same result if I use getTotalRxBytes() instead.
Here is my code.
InputStream inStream = null;
byte[] buffer = new byte[10245];
HttpURLConnection connection = null;
try {
connection = (HttpURLConnection) new URL(DOWNLOAD_RATE_TEST_10KB_URL).openConnection();
connection.setDoInput(true);
connection.setUseCaches(false);
connection.connect();
} catch (IOException e) { }
long startingBytes = TrafficStats.getMobileRxBytes();
long startingTime = SystemClock.elapsedRealtime();
try {
inStream = connection.getInputStream();
// Do some busy waiting while the inStream is open.
while (inStream.read(buffer) != -1) { }
} catch (IOException e) { }
this.elapsedTime = SystemClock.elapsedRealtime() - startingTime;
this.downloadBytes = TrafficStats.getMobileRxBytes() - startingBytes;
try {
if (inStream != null) { inStream.close(); }
} catch (IOException e) {}
connection.disconnect();
Is there a cache issue? Or I use APIs in the wrong way?

You're downloading over HTTP: there may be compression occurring (gzip/deflate) that is transparent to the HTTP connection but reduces the bytes over the wire.

TCP detect disconnected server from client

I'm writing a simple TCP client/server program pair in Java, and the server must disconnect if the client hasn't sent anything in 10 seconds. socket.setSoTimeout() gets me that, and the server disconnects just fine. The problem is - how can I get the client to determine if the server is closed? Currently I'm using DataOutputStream for writing to the server, and some answers here on SO suggest that writing to a closed socket will throw an IOException, but that doesn't happen.
What kind of writer object should I use to send arbitrary byte blocks to the server, that would throw an exception or otherwise indicate that the connection has been closed remotely?
Edit: here's the client code. This is a test function that reads one file from the file system and sends it to the server. It sends it in chunks, and pauses for some time between each chunk.
public static void sendFileWithTimeout(String file, String address, int dataPacketSize, int timeout) {
Socket connectionToServer = null;
DataOutputStream outStream = null;
FileInputStream inStream = null;
try {
connectionToServer = new Socket(address, 2233);
outStream = new DataOutputStream(connectionToServer.getOutputStream());
Path fileObject = Paths.get(file);
outStream.writeUTF(fileObject.getFileName().toString());
byte[] data = new byte[dataPacketSize];
inStream = new FileInputStream(fileObject.toFile());
boolean fileFinished = false;
while (!fileFinished) {
int bytesRead = inStream.read(data);
if (bytesRead == -1) {
fileFinished = true;
} else {
outStream.write(data, 0, bytesRead);
System.out.println("Thread " + Thread.currentThread().getName() + " wrote " + bytesRead + " bytes.");
Thread.sleep(timeout);
}
}
} catch (IOException | InterruptedException e) {
System.out.println("Something something.");
throw new RuntimeException("Problem sending data to server.", e);
} finally {
TCPUtil.silentCloseObject(inStream);
TCPUtil.silentCloseObject(outStream);
TCPUtil.silentCloseObject(connectionToServer);
}
}
I'd expect the outStream.write to throw an IOException when it tries to write to a closed server, but nothing.

I'd expect the outStream.write to throw an IOException when it tries to write to a closed server, but nothing.
It won't do that the first time, because of the socket send buffer. If you keep writing, it will eventually throw an IOException: 'connection reset'. If you don't have data to get to that point, you will never find out that the peer has closed.

I think you need to flush and close your stream after written like outStream.flush(); outStream.close(); inStream.close();

Remember ServerSocket.setSoTimeout() is different from client's function with same name.
For server, this function only throws SocketTimeoutException for you to catch it if timeout is expired, but the server socket still remains.
For client, setSoTimeout() relates to 'read timeout' for stream reading.
In your case, you must show your server code of closing the connected socket after catching SocketTimeoutException => ensure server closed the associated socket with a specified client. If done, at client side, your code line:
throw new RuntimeException("Problem sending data to server.", e);
will be called.
[Update]
I noticed that you stated to set timeout for the accepted socket at server side to 10 secs (=10,000 milliseconds); for that period, did your client complete all the file sending? if it did, never the exception occurs.
[Suggest]
for probing, just comment out your code of reading file content to send to server, and try replacing with several lines of writing to output stream:
outStream.writeUTF("ONE");
outStream.writeUTF("TWO");
outStream.writeUTF("TREE");
Then you can come to the conclusion.

Java - Read file from Socket (while loop never ends)

My JAVA application sends a command to server (command=filename.ini). When the server receives this command it sends filename.ini contents through Socket.
The first problem I had was receiving only partial contents of the file. That happened when in the code I used while(in.available()!=0){//write bytes} because in.available() does not know how big/long the content of the file is. If I use while((numBytesRead = dis.read(buffer)) != -1){//write bytes} the loop will never terminate since the Socket connection remains always open. My question is how else can I terminate the loop once every byte has been received? Please help me I have tried everything. I understand where the mistake is but I don't know how to fix it.
The following is the part of the code I have at the moment:
public class TCPClient {
protected Socket s = null;
public DataInputStream in = null;
public TCPClient(InetAddress ipa, int port) {
Socket s1 = null;
try { //Open the socket.
s1 = new Socket(ipa.getHostAddress(), port);
} catch (IOException ex) {
System.out.println("Error opening socket!");
return;
}
s = s1;
try { //Create an input stream.
in = new DataInputStream(new BufferedInputStream(s.getInputStream()));
} catch (Exception ex) {
System.out.println("Error creating input stream!");
}
}
public synchronized byte[] receive() {
byte[] buffer = new byte[0];
ByteArrayOutputStream getBytes = new ByteArrayOutputStream();
try {
while (in.available() == 0) {
} //Wait for data.
} catch (IOException ex) {
}
try {
int numBytesRead;
buffer = new byte[1024];
while ((numBytesRead = dis.read(buffer, 0, 1024)) != -1) { //LOOP NEVER ENDS HERE BECAUSE CONNECTION IS ALWAYS OPEN
getBytes.write(buffer, 0, numBytesRead);
}
} catch (IOException ex) {
}
return (getBytes.toByteArray());
}
}

You need to define a micro protocol to say the receiver how long is the file, or just close the connection on the server after finishing sending the file. First method is preferred, since it is a little bit more robust. On the client you should have a timeout too in order to avoid to wait forever in case of network problems.
Clarification for micro protocol: before sending the file itself send a 32 (or 64 if needed) bit integer containing the file length. The client should read that integer and then start retrieving the file.