Develop Reference

HTTP Webserver ignoring last line of POST request [duplicate]

Right now, I'm trying to write a GUI based Java tic-tac-toe game that functions over a network connection. It essentially works at this point, however I have an intermittent error in which several chars sent over the network connection are lost during gameplay. One case looked like this, when println statements were added to message sends/reads:
Player 1:
Just sent ROW 14 COLUMN 11 GAMEOVER true
Player 2:
Just received ROW 14 COLUMN 11 GAMEOV
Im pretty sure the error is happening when I read over the network. The read takes place in its own thread, with a BufferedReader wrapped around the socket's InputStream, and looks like this:
try {
int input;
while((input = dataIn.read()) != -1 ){
char msgChar = (char)input;
String message = msgChar + "";
while(dataIn.ready()){
msgChar = (char)dataIn.read();
message+= msgChar;
}
System.out.println("Just received " + message);
this.processMessage(message);
}
this.sock.close();
}
My sendMessage method is pretty simple, (just a write over a DataOutputStream wrapped around the socket's outputstream) so I don't think the problem is happening there:
try {
dataOut.writeBytes(message);
System.out.println("Just sent " + message);
}
Any thoughts would be highly appreciated. Thanks!

As it turns out, the ready() method guaruntees only that the next read WON'T block. Consequently, !ready() does not guaruntee that the next read WILL block. Just that it could.
I believe that the problem here had to do with the TCP stack itself. Being stream-oriented, when bytes were written to the socket, TCP makes no guarantees as to the order or grouping of the bytes it sends. I suspect that the TCP stack was breaking up the sent string in a way that made sense to it, and that in the process, the ready() method must detect some sort of underlying break in the stream, and return false, in spite of the fact that more information is available.
I refactored the code to add a newline character to every message send, then simply performed a readLine() instead. This allowed my network protocol to be dependent on the newline character as a message delimiter, rather than the ready() method. I'm happy to say this fixed the problem.
Thanks for all your input!

Try flushing the OutputStream on the sender side. The last bytes might remain in some intenal buffers.

It is really important what types of streamed objects you use to operate with data. It seems to me that this troubleshooting is created by the fact that you use DataOutputStream for sending info, but something else for receiving. Try to send and receive info by DataOutputStream and DataInputStream respectively.
Matter fact, if you send something by calling dataOut.writeBoolean(b)
but trying to receive this thing by calling dataIn.readString(), you will eventually get nothing. DataInputStream and DataOutputStream are type-sensitive. Try to refactor your code keeping it in mind.
Moreover, some input streams return on invocation of read() a single byte. Here you try to convert this one single byte into char, while in java char by default consists of two bytes.
msgChar = (char)dataIn.read();
Check whether it is a reason of data loss.

How does a BufferedReader interact with a Socket? About how Sockets work

I have been following a tutorial on how to build a simple chat application in Java. I get the basic principle of how sever and client communicate. So far I used a PrintWriter to send, and a BufferedReader to recieve information for both sides. But what I don´t get is how this does not lead to errors?
Socket soc = new Socket(ipAddress, port);
BufferedReader in = new BufferedReader( new InputStreamReader(soc.getInputStream));
while(true) {
String str = in.readLine();
}
My question would be the following. Is the in.readLine() statement "waiting" for input? If it does not, shouldn´t the program throw Errors if using the str since it has no value? Or in general terms, do Socket specific functions like Socket soc = serverSocket.accept(); or as in the example given above the reading of an InputStream "wait" for a signal whatsoever? Did I miss some essential concept about Sockets? It would be great if someone could help me understand this topic further.
Thank you in advance,
Appa

My question would be the following. Is the in.readLine() statement "waiting" for input?
Yes, it is "waiting for input" since .readLine() is a blocking method
If it does not, shouldn´t the program throw Errors if using the str since it has no value?
I cannot parse this -- the statement doesn't make sense. What error / exception would you be expecting?
Or in general terms, do Socket specific functions like Socket soc = serverSocket.accept(); or as in the example given above the reading of an InputStream "wait" for a signal whatsoever?
Again, I'm afraid that this is not too clear (at least to me). ServerSocket#accept() is a blocking method and will wait until a socket connection can be made, if that is what you are asking.
Note that thiscode:
while(true) {
String str = in.readLine();
}
while valid, would not be particularly useful, since you would be reading a String and then discarding it, doing nothing with it. Better to either print the String obtained, or pass it into a method where it can be used in some way. On the other hand, I suppose this could be useful if all you wanted to do was to "gobble" the stream to prevent the OS's buffers from overflowing.

Yes readLine does in fact block. It waits till you read a carriage return/line feed, or the socket closes.

Socket communication between C and java

I have a client and a server both running in C. My task is to introduce java program in which I create a server to the C client and a client to the C server. I am successful in trying to get the connections set up properly. However the problem is in communicating the data between both C programs. Below is what I have written in my java program:
while(true){
while((userInput1=br1.readLine())!=null||(userInput2=br2.readLine())!=null){
if(userInput1=!null){
bw1.write(userInput1);
bw1.flush();
}
if(userInput2=!null){
bw2.write(userInput2);
bw2.flush();
}
}
While debugging the above, it is seen that the execution is stuck at the second while statement meaning that the input stream is waiting for the input for the C client for ever. I am using BufferedReader and BufferedWriter for the streams. The C client and server are using send and recv functions to communicate.
Kindly help with any inputs to make the java program help both the C programs communicate with each other as they do without this.

Have you correctly considered the effect of Java's "short circuit" or operator?
With || if the first clause is true the second is never evaluated.
while(
(userInput1=br1.readLine())!=null ||
(userInput2=br2.readLine())!=null) {
So you successfully read
userInput1=br1.readLine())!=null
and immediately enter your processing, then come back to while and read the next line into userInput1 again. Hence userInput2 never will receive a value.
You need separate logic like
read first line
read second line
But exactly what should you do when reading line2 and a the data is not ready? Try again? Is the line you read next the expected line2 or a new line1? This is quite tricky to get right.
I would prefer not to rely on two separate readlines in my protocol.

while((userInput1=br1.readLine())!=null||(userInput2=br2.readLine())!=null){
This condition means that you are going to read br1 all the way to EOS before you ever read anything from br2. Is that what you really intended?
Conversely, if you are stuck at br2.readLine() it means two things: (a) br1 is at EOS, and (b) the peer associated with br2 hasn't sent anything, or at least hasn't sent a line terminated by a newline.
Are you perhaps suffering from the common delusion that readLine() returns null when there is no data ready to be read?
Also you are reading lines terminated by newlines, which are removed by the readLine() call, and then writing them out without any newlines, which can hardly be correct.
It appears to me that what you are really writing is a proxy, in which case you need two threads per socket, one reading from A and writing to B, and the other reading from B and writing to A. And if it's a proxy you should use InputStreams and OutputStreams rather than Readers and Writers, as you probably have no reason to inspect the data, and you therefore shouldn't put it through the byte->char and char->byte conversion processes implied by using Readers and Writers. There are further subtleties when writing proxies but I'll wait for your confirmation before elucidating them.

the reason I am using the parity character is to interpret the end of the stream. Otherwise using using just the read() is making the program halt for the input forever (even after the actual had sent all its data). Am using the ready() in the following way:
//The proxy client
while(true){
if(br1.ready()){
while((temp1=br1.read())!=(int)par)
userInput1=userInput1+(char)temp1;
System.out.println("Input to Actual Server: " + userInput1);
bw1.write(userInput1);
bw1.flush();
System.out.flush();
userInput1="";
temp1=0;
}
if(br2.ready()){
while((temp2=br2.read())!=(int)par)
userInput2=userInput2+(char)temp2;
System.out.println("Response from Actual Server: " + userInput2);
userInput2=userInput2+par;
bw2.write(userInput2);
bw2.flush();
System.out.flush();
userInput2="";
temp2=0;
}
}
//The proxy server
while(true){
if(br1.ready()){
while((temp1=br1.read())!=(int)par)
userInput1=userInput1+(char)temp1;
System.out.println("Input from Actual Client: " + userInput1);
userInput1=userInput1+par;
bw1.write(userInput1);
bw1.flush();
System.out.flush();
userInput1="";
temp1=0;
}
if(br2.ready()){
while((temp2=br2.read())!=(int)par)
userInput2=userInput2+(char)temp2;
System.out.println("Response to Actual Client: " + userInput2);
bw2.write(userInput2);
bw2.flush();
System.out.flush();
userInput2="";
temp2=0;
}
}
Kindly suggest if there is any problem of using ready().

Extract first valid line of string from byte array

I am writing a utility in Java that reads a stream which may contain both text and binary data. I want to avoid having I/O wait. To do that I create a thread to keep reading the data (and wait for it) putting it into a buffer, so the clients can check avialability and terminate the waiting whenever they want (by closing the input stream which will generate IOException and stop waiting). This works every well as far as reading bytes out of it; as binary is concerned.
Now, I also want to make it easy for the client to read line out of it like '.hasNextLine()' and '.readLine()'. Without using an I/O-wait stream like buffered stream, (Q1) How can I check if a binary (byte[]) contain a valid unicode line (in the form of the length of the first line)? I look around the String/CharSet API but could not find it (or I miss it?). (NOTE: If possible I don't want to use non-build-in library).
Since I could not find one, I try to create one. Without being so complicated, here is my algorithm.
1). I look from the start of the byte array until I find '\n' or '\r' without '\n'.
2). Then, I cut the byte array from the start to that point and using it to create a string (with CharSet if specified) using 'new String(byte[])' or 'new String(byte[], CharSet)'.
3). If that success without exception, we found the first valid line and return it.
4). Otherwise, these bytes may not be a string, so I look further to another '\n' or '\r' w/o '\n'. and this process repeat.
5. If the search ends at the end of available bytes I stop and return null (no valid line found).
My question is (Q2)Is the following algorithm adequate?
Just when I was about to implement it, I searched on Google and found that there are many other codes for new line, for example U+2424, U+0085, U+000C, U+2028 and U+2029.
So my last question is (Q3), Do I really need to detect these code? If I do, Will it increase the chance of false alarm?
I am well aware that recognize something from binary is not absolute. I am just trying to find the best balance.
To sum up, I have an array of byte and I want to extract a first valid string line from it with/without specific CharSet. This must be done in Java and avoid using any non-build-in library.
Thanks you all in advance.

I am afraid your problem is not well-defined. You write that you want to extract the "first valid string line" from your data. But whether somet byte sequence is a "valid string" depends on the encoding. So you must decide which encoding(s) you want to use in testing.
Sensible choices would be:
the platform default encoding (Java property "file.encoding")
UTF-8 (as it is most common)
a list of encodings you know your clients will use (such as several Russian or Chinese encodings)
What makes sense will depend on the data, there's no general answer.
Once you have your encodings, the problem of line termination should follow, as most encodings have rules on what terminates a line. In ASCII or Latin-1, LF,CR-LF and LF-CR would suffice. On Unicode, you need all the ones you listed above.
But again, there's no general answer, as new line codes are not strictly regulated. Again, it would depend on your data.

First of all let me ask you a question, is the data you are trying to process a legacy data? In other words, are you responsible for the input stream format that you are trying to consume here?
If you are indeed controlling the input format, then you probably want to take a decision Binary vs. Text out of the Q1 algorithm. For me this algorithm has one troubling part.
`4). Otherwise, these bytes may not be a string, so I look further to
another '\n' or '\r' w/o '\n'. and this process repeat.`
Are you dismissing input prior to line terminator and take the bytes that start immediately after, or try to reevaluate the string with now 2 line terminators? If former, you may have broken binary data interface, if latter you may still not parse the text correctly.
I think having well defined markers for binary data and text data in your stream will simplify your algorithm a lot.
Couple of words on String constructor. new String(byte[], CharSet) will not generate any exception if the byte array is not in particular CharSet, instead it will create a string full of question marks ( probably not what you want ). If you want to generate an exception you should use CharsetDecoder.
Also note that in Java 6 there are 2 constructors that take charset
String(byte[] bytes, String charsetName) and String(byte[] bytes, Charset charset). I did some simple performance test a while ago, and constructor with String charsetName is magnitudes faster than the one that takes Charset object ( Question to Sun: bug, feature? ).

I would try this:
make the IO reader put strings/lines into a thread safe collection (for example some implementation of BlockingQueue)
the main code has only reference to the synced collection and checks for new data when needed, like queue.peek(). It doesn't need to know about the io thread nor the stream.
Some pseudo java code (missing exception & io handling, generics, imports++) :
class IORunner extends Thread {
IORunner(InputStream in, BlockingQueue outputQueue) {
this.reader = new BufferedReader(new InputStreamReader(in, "utf-8"));
this.outputQueue = outputQueue;
}
public void run() {
String line;
while((line=reader.readLine())!=null)
this.outputQueue.put(line);
}
}
class Main {
public static void main(String args[]) {
...
BlockingQueue dataQueue = new LinkedBlockingQueue();
new IORunner(myStreamFromSomewhere, dataQueue).start();
while(true) {
if(!dataQueue.isEmpty()) { // can also use .peek() != null
System.out.println(dataQueue.take());
}
Thread.sleep(1000);
}
}
}
The collection decouples the input(stream) more from the main code. You can also limit the number of lines stored/mem used by creating the queue with a limited capacity (see blockingqueue doc).
The BufferedReader handles the checking of new lines for you :) The InputStreamReader handles the charset (recommend setting one yourself since the default one changes depending on OS etc.).

The java.text namespace is designed for this sort of natural language operation. The BreakIterator.getLineInstance() static method returns an iterator that detects line breaks. You do need to know the locale and encoding for best results, though.

Q2: The method you use seems reasonable enough to work.
Q1: Can't think of something better than the algorithm that you are using
Q3: I believe it will be enough to test for \r and \n. The others are too exotic for usual text files.

I just solved this to get test stubb working for Datagram - I did byte[] varName= String.getBytes(); then final int len = varName.length; then send the int as DataOutputStream and then the byte array and just do readInt() on the rcv then read bytes(count) using the readInt.
Not a lib, not hard to do either. Just read up on readUTF and do what they did for the bytes.
The string should construct from the byte array recovered that way, if not you have other problems. If the string can be reconstructed, it can be buffered ... no?
May be able to just use read / write UTF() in DataStream - why not?
{ edit: per OP's request }
//Sending end
String data = new String("fdsfjal;sajssaafe8e88e88aa");// fingers pounding keyboard
DataOutputStream dataOutputStream = new DataOutputStream();//
final Integer length = new Integer(data.length());
dataOutputStream.writeInt(length.intValue());//
dataOutputStream.write(data.getBytes());//
dataOutputStream.flush();//
dataOutputStream.close();//
// rcv end
DataInputStream dataInputStream = new DataInputStream(source);
final int sizeToRead = dataInputStream.readInt();
byte[] datasink = new byte[sizeToRead.intValue()];
dataInputStream.read(datasink,sizeToRead);
dataInputStream.close;
try
{
// constructor
// String(byte[] bytes, int offset, int length)
final String result = new String(datasink,0x00000000,sizeToRead);//
// continue coding here
Do me a favor, keep the heat off of me. This is very fast right in the posting tool - code probably contains substantial errors - it's faster for me just to explain it writing Java ~ there will be others who can translate it to other code language ( s ) which you can too if you wish it in another codebase. You will need exception trapping an so on, just do a compile and start fixing errors. When you get a clean compile, start over from the beginnning and look for blunders. ( that's what a blunder is called in engineering - a blunder )

In Java, what's the fastest way to "build" and use a string, character by character?

I have a Java socket connection that is receiving data intermittently. The number of bytes of data received with each burst varies. The data may or may not be terminated by a well-known character (such as CR or LF). The length of each burst of data is variable.
I'm attempting to build a string out of each burst of data. What is the fastest way (speed, not memory), to build a string that would later need to be parsed?
I began by using a byte array to store the incoming bytes, then converting them to a String with each burst, like so:
byte[] message = new byte[1024];
...
message[i] = //byte from socket
i++;
...
String messageStr = new String(message);
...
//parse the string here
The obvious disadvantage of this is that some bursts may be longer than 1024. I don't want to arbitrarily create a larger byte array (what if my burst is larger?).
What is the best way of doing this? Should I create a StringBuilder object and append() to it? That way I don't have to convert from StringBuilder to String (since the former has most of the methods I need).
Again, speed of execution is my biggest concern.
TIA.

I would probably use an InputStreamReader wrapped around a BufferedInputStream, which in turn wraps the socket. And write code that processes a message at a time, potentially blocking for input. If the input is bursty, I might run on a background thread and use a concurrent queue to hold the messages.
Reading a buffer at a time and trying to convert it to characters is exactly what BufferedInputStream/InputStreamReader does. And it does so while paying attention to encoding, something that (as other people have noted) your solution does not.
I don't know why you're focused on speed, but you'll find that the time to process data coming off a socket is far less than the time it takes to transmit over that socket.

Note that as you're transmitting across network layers, your speed of conversion may not be the bottleneck. It would be worth measuring, if you believe this to be important.
Note (also) that you're not specifying a character encoding in your conversion from bytes to String (via characters). I would enforce that somehow, otherwise your client/server communication can become corrupted if/when your client/server run in different environments. You can enforce that via JVM runtime args, but it's not a particularly safe option.
Given the above, you may want to consider StringBuilder(int capacity) to configure it in advance with an appropriate size, such that it doesn't have to reallocate on the fly.

First of all, you are making a lot of assumptions about charachter encoding that you receive from your client. Is it US-ASCII, ISO-8859-1, UTF-8?
Because in Java string is not a sequence of bytes, when it comes to building portable String serialization code you should make explicit decisions about character encoding. For this reason you should NEVER use StringBuilder to convert bytes to String. If you look at StringBuilder interface you will notice that it does not even have an append( byte ) method, and that's not because designers just overlooked it.
In your case you should definetly use a ByteArrayOutputStream. The only drawback of using straight implementation of ByteArrayOutputStream is that its toByteArray() method returns a copy of the array held by the object internaly. For this reason you may create your own subclass of ByteArrayOutputStream and provide direct access to the protected buf member.
Note that if you don't use default implementation, remember to specify byte array bounds in your String constructor. Your code should look something like this:
MyByteArrayOutputStream message = new MyByteArrayOutputStream( 1024 );
...
message.write( //byte from socket );
...
String messageStr = new String(message.buf, 0, message.size(), "ISO-8859-1");
Substitute ISO-8859-1 for the character set that's suitable for your needs.

StringBuilder is your friend. Add as many characters as needed, then call toString() to obtain the String.

I would create a "small" array of characters and append characters to it.
When the array is full (or transmission ends), use the StringBuilder.append(char[] str) method to append the content of the array to your string.
Now for the "small" size of the array - you will need to try various sizes and see which one is fastest for your production environment (performance "may" depend on the JVM, OS, processor type and speed and so on)
EDIT: Other people mentioned ByteArrayOutputStream, I agree it is another option as well.

You may wish to look at ByteArrayOutputStream depending if you are dealing with Bytes instead of Characters.
I generally will use a ByteArrayOutputStream to assemble a message then use toString/toByteArray to retrive it when the message is finished.
Edit: ByteArrayOutputStream can handle various Character set encoding through the toString call.

Personally, independent of language, I would send all characters to an in-memory data stream and once I need the string, I would read all characters from this stream into a string.
As an alternative, you could use a dynamic array, making it bigger whenever you need to add more characters. Even better, keep track of the actual length and increase the array with additional blocks instead of single characters. Thus, you would start with 1 character in an array of 1000 chars. Once you get at 1001, the array needs to be resized to 2000, then 3000, 4000, etc...
Fortunately, several languages including Java have a special build-in class that specializes in this. These are the stringbuilder classes. Whatever technique they use isn't that important but they have been created to boost performance so they should be your fastest option.

Have a look at the Text class. It's faster (for the operations you perform) and more deterministic than StringBuilder.
Note: the project containing the class is aimed at RTSJ VMs. It is perfectly usable in standard SE/EE environments though.