I'm using a UNIX socket to facilitate communication on and Android device between a system level daemon I've got running in C and an application I've got running in Java. I'm much more of a C coder than a Java coder, so I'm having some issues when trying to read in data from the socket on the Java side. Currently, my code is as follows:
try{//Prepare to write the command and read the ACK
InputStream is = receiver.getInputStream();
OutputStream os = receiver.getOutputStream();
BufferedReader in = new BufferedReader(new InputStreamReader(is));
os.write(message.getBytes());
//FIX THIS!! The following assumes that there is a newline-
// terminated chunk of data waiting for us in the buffer. If
// that assumption is wrong, the code will hang. Find some way
// to determine if/how much data is waiting for us in the socket!
String str = in.readLine();
is.close();
os.close();
receiver.close();
return str;
}catch(IOException ex){
Log.e(TAG,(ex.toString()+"\n"));
return null;
}
As my comment indicates, this implementation works(ish), but has the very bad design flaw of requiring that the server side socket code responds with a newline terminated string. If that's not the case, the world ends.
What I'm trying to figure out is if there is a better means of accomplishing this task. Is it possible to determine if there is any data waiting to be read in a buffered reader, and if so, what the size of that data is? If not, is there a better way of reading in data over a UNIX socket on the Java side of Android that I'm not familiar with?
Thanks in advance!
if the protocol requires a newline, your impl is appropriate. the code SHOULD be blocked until a newline is read. what else can you do but wait?
of course, usually you don't want to wait forever, you need a timeout. see Socket.setSoTimeout()
if a newline is not necessarily given by server, and you just want to read any data as soon as available, use InputStream.read()
Related
I am using ProcessBuilder to input and receive information from a C++ program, using Java. After starting the process once, I would like to be able to input new strings, and receive their output, without having to restart the entire process. This is the approach I have taken thus far:
public void getData(String sentence) throws InterruptedException, IOException{
InputStream stdout = process.getInputStream();
InputStreamReader isr = new InputStreamReader(stdout);
OutputStream stdin = process.getOutputStream();
OutputStreamWriter osr = new OutputStreamWriter(stdin);
BufferedWriter writer = new BufferedWriter(osr);
BufferedReader reader = new BufferedReader(isr);
writer.write(sentence);
writer.close();
String ch = reader.readLine();
preprocessed="";
while (ch!=null){
preprocessed = preprocessed+"~"+ch;
ch = reader.readLine();
}
reader.close();
}
Each time I want to send an input to the running process, I call this method. However, there is an issue: the first time I send an input, it is fine, and the output is received perfectly. However, the second time I call it, I receive the error
java.io.IOException: Stream closed
which is unexpected, as everything is theoretically recreated when the method is called again. Moreover, removing the line the closes the BufferedWriter results in the code halting at the following line, as if the BufferedReader is waiting for the BufferedWriter to be closed.
One final thing - even when I create a NEW BufferedWriter and instruct the method to use that when called for the second time, I get the same exception, which I do not understand at all.
Is there any way this can be resolved?
Thanks a lot!
Your unexpected IOException happens because when Readers and Writers are closed, they close their underlying streams in turn.
When you call your method the first time, everything appears to work. But you close the writer, which closes the process output stream, which closes stdin from the perspective of the process. Not sure what your C++ binary looks like, but probably it just exits happily when it's done with all its input.
So subsequent calls to your method don't work.
There's a separate but similar issue on the Reader side. You call readLine() until it returns null, meaning the Reader has felt the end of the stream. But this only happens when the process is completely done with its stdout.
You need some way of identifying when you're done processing a unit of work (whatever you mean by "sentence") without waiting for the whole entire stream to end. The stream has no concept of the logical pause between outputs. It's just a continuous stream. Reader and Writer are just a thin veneer to buffer between bytes and characters but basically work the same as streams.
Maybe the outputs could have delimiters. Or you could send the length of each chunk of output before actually sending the output and distinguish outputs that way. Or maybe you know in advance how long each response will be?
You only get one shot through streams. So they will have to outlive this method. You can't be opening and closing streams if you want to avoid restarting your process every time. (There are other ways for processes to communicate, e.g. sockets, but that's probably out of scope.)
On an orthogonal note, appending to a StringBuilder is generally more efficient than a big loop of string concatenations when you're accumulating your output.
You might also have some thread check process.exitValue() or otherwise make sure the process is working as intended.
Don't keep trying to create and close your Streams, because once you close it, it's closed for good. Create them once, then in your getData(...) method use the existing Streams. Only close your Streams or their wrapping classes when you're fully done with them.
Note that you should open and close the Streams in the same method, and thus may need additional methods or classes to help you process the Streams. Consider creating a Runnable class for this and then reading from the Streams in another Thread. Also don't ignore the error stream, as that may be sending key information that you will need to fully understand what's going on here.
I've implemented a C server and a Java client which communicate to each other through TCP. The problem is when the client sends a string, the recv() call in C server just reads a character and returns immediately. If I put a breakpoint at recv() and then do a step-over, the server receives the whole string that was sent.
Here's the recv call I'm using in C
char tempBuffer[256] = {'\0'};
int retVal = recv(hClientSocket, tempBuffer, sizeof(tempBuffer), 0);
And here's the Java client.
clientSocket = new Socket("127.0.0.1", 24886);
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
outToServer.writeBytes("alert(\"hi\")");
My question is, is there a way in which I can make this work without having to put a while loop in my C server code to receive each character separately? What's the best way to do this?
I implemented the C++ client given here and it works fine. So I'm guessing it's some problem in Java+C++ interoperability. Thanks!
Although what Scott M says is true on a Java side ( about using flush ), you will have to rework your whole communication protocol.
You will have to either tell to C-code, how many bytes are expected to be sent (this will be the easiest to track), or use some sort of termination character, to recognize that the string input is over ( e.g. null terminator byte ).
There is absolutely no guarantee that the network layer will not chop the transition arbitrarily, so code defensively and expect recv always to recieve a partial data, which you will have to assemble for your internal consumption.
Try flushing the java stream. They are normally buffered, which will cause some delay in the writing of bytes to a stream. Long enough to mess up code at full speed, short enough to send the whole string while debugging.
With TCP there's no way around "putting a while loop around recv()". TCP is a stream protocol built on top of limited-size packet service (IP). TCP does its own "packetization", and is actually very smart about it. This implies that you never know how many bytes the next read from the socket will return.
The well-known strategy that works is to design your application-level protocol so the message itself tells how big it is, then read in a loop until you have full message to process.
Having said that, there's a kludge, namely MSG_WAITALL flag for the recv(2) call, that you can use to wait for given amount of data, assuming you know how much to expect.
I once tried by having a header indicating length of string .. In that way din have to check for the end of message..
I develop the first part of an Android application that allows to broadcast video stream through the network. Currently, I'm sending the video in a very direct way, like this:
Socket socket = new Socket(InetAddress.getByName(hostname), port);
ParcelFileDescriptor pfd = ParcelFileDescriptor.fromSocket(socket);
recorder.setOutputFile(pfd.getFileDescriptor());
But unfortunately, it is not very fluid. I want to buffered the data stream before sending it through the socket. One of the way I tried is to write the stream in a file using the Android API for recording media, and to use another thread to stream the file to the server on a conputer.
So my problem is: how can I send by a socket a file which is still under writing?
As BufferedInputStream has not a blocking method for reading, I tried to do things like this one, but without any success
while (inputStream.available() >= BUFFER_SIZE) {
inputStream.read(buffer);
outputStream.write(buffer);
}
outputStream.flush();
But when i'm doing that, if the network is faster than the datastream, I get quickly out of the loop.
Is there a 'good' way to do that? I though about doing active waiting but it is not a good solution, especially for mobiles. Another way is to do something like this :
while (true) {
while (inputStream.available() < BUFFER_SIZE) {
wait(TIME);
}
inputStream.read(buffer);
outputStream.write(buffer);
}
outputStream.flush();
But it sound quite dirty for me... Is there sleeker solution?
What I do in these situations if simply fill up a byte array (my buffer) until either I've hit the end of the data I'm about to transmit, or the buffer is full. In which case the buffer is ready to be passed to my Socket transmission logic. Admittedly, I do not do this on video or audio though … only on “regular” data.
Something worth noting is this will give a "janky" user experience to the recipient of that data (it might look like the network is stopping for short periods then running normally again ... the time the buffer is using to fill up). So if you have to use a buffered approach on either video or audio be careful on what buffer size you decide to work with.
For things like video it's been my experence to use streaming based logic versus buffered, but you apparently have some different and interesting requirements.
I can't think of a pretty way of doing this, but one option might be to create a local socket pair, use the 'client' end of the pair as the MediaRecorder output fd, and buffer between the local-server socket and the remote-server. This way, you can block on the local-server until there is data.
Another possibility is to use a file-based pipe/fifo (so the disk doesn't fill up), but I can't remember if the Java layer exposes mkfifo functionality.
In any event, you probably want to look at FileReader, since reads on that should block.
Hope this helps,
Phil Lello
We are integrating with an external product that requires us to communicate with it using Java sockets. We've been able to read small responses from the server with no issues, but larger responses are causing some headaches.
I made some changes to the socket handling logic and now we seem to be able to read large responses 90% of the time. It does still fail occasionally. Failure, in this case, means that the Java client stops reading from the socket before the entire response has been read. The client thinks that the read operation is finished, and stops normally - there are no exceptions or timeouts involved.
Here's what the current logic looks like:
StringWriter response = new StringWriter();
PrintWriter writer = new PrintWriter(response);
char[] buf = new char[4096];
int readChars;
do {
readChars = responseBufferedReader.read(buf);
writer.write(buf, 0, readChars);
} while(readChars != -1 && responseBufferedReader.ready());
responseBufferedReader is a BufferedReader wrapped around an InputStreamReader wrapped around the Socket's InputStream.
This code works most of the time, but it seems like checking for readChars != -1 and ready() are not reliable enough to indicate if we've read all of the content from the server. Comparing the number of read characters to the buffer size is also not reliable, since the server seems to be a little slow at sending the response back causing these numbers to differ.
I've tried changing the size of the character buffer; it helped, but it's still not working 100% of the time.
Is there a better and more reliable way to read entirely from a Socket without knowing the size of the expected response? I've been doing some research on SocketChannels, but I'm not sure if there's any benefit to be had by switching.
In case it helps, we're making a single, blocking Socket connection to the server. The Socket is configured for a 100 second timeout
You shouldn't be checking whether the BufferedReader is ready() to tell if you're done. It's possible that bytes are still being read off of the wire and the BufferedReader has nothing for you, but the Socket is not yet closed.
I'm not sure what value the BufferedReader is getting you (well, it might help your performance). I would expect the following to work better:
StringWriter response = new StringWriter();
PrintWriter writer = new PrintWriter(response);
char[] buf = new char[4096];
int readChars;
do {
readChars = inputStreamReader.read(buf);
writer.write(buf, 0, readChars);
} while(readChars != -1);
I think that you're probably dropping out of the loop when the network is the bottleneck - the loop processes the results fast enough so that the Reader isn't ready yet and you assume that you're done reading (even though you're not).
See the answers to the question at Java blocking socket returning incomplete ByteBuffer
If the network breaks the message into packets, you will drop out of your loop before reading all packets.
See my answer to this question. The same answer applies whenever you are reading from a stream, and the only reliable way to know when you're done if EOF doesn't apply (as in this case) is to encode the end-of-message in the protocol somehow -- how you do it is up to you.
Edit:
If you can't modify the protocol to include message boundaries, you might investigate a timeout-based approach. You have to decide how long to wait before detecting the end of a message... but then you have to handle the case where you get two messages in the same stream. Can your code handle this situation now? If so, then your answer is to buffer the input and use that logic to detect record boundaries.
I have a client connecting to my server. The client sends some messages to the server which I do not care about and do not want to waste time parsing its messages if I'm not going to be using them. All the i/o I'm using is simple java i/o, not nio.
If I create the input stream and just never read from it, can that buffer fill up and cause problems? If so, is there something I can do or a property I can set to have it just throw away data that it sees?
Now what if the server doesn't create the input stream at all? Will that cause any problems on the client/sending side?
Please let me know.
Thanks,
jbu
When you accept a connection from a client, you get an InputStream. If you don't read from that stream, the client's data will buffer up. Eventually, the buffer will fill up and the client will block when it tries to write more data. If the client writes all of its data before reading a response from the server, you will end up with a pretty classic deadlock situation. If you really don't care about the data from the client, just read (or call skip) until EOF and drop the data. Alternatively, if it's not a standard request/response (like HTTP) protocol, fire up a new thread that continually reads the stream to keep it from getting backed up.
If you get no useful data from the client, what's the point of allowing it to connect?
I'm not sure of the implications of never reading from a buffer in Java -- I'd guess that eventually the OS would stop accepting data on that socket, but I'm not sure there.
Why don't you just call the skip method of your InputStream occasionally with a large number, to ensure that you discard the data?
InputStream in = ....
byte[] buffer = new byte[4096] // or whatever
while(true)
in.read(buffer);
if you accept the connection, you should read the data. to tell you the truth i have never seen (or could forsee) a situation where this (a server that ignores all data) could be useful.
I think you get the InputStream once you accept the request, so if you don't acknowledge that request the underlying framework (i.e. tomcat) will drop that request (after some lapsed time).
Regards.