Develop Reference

Input Stream getting filled with null data

So I am creating a server that I am trying to get to handle ASCII data. While I can get the Streams to work and call the methods. However the listening thread to add items to a Queue (ArrayBlockingQueue), and it will loop until the queue is full with null data.
Server code, Client Handler (compressed, let me know if I left something out.):
class ClientThread extends Thread {
// ASCII commands defined here (byte NUL=0x00; byte SOH=0x01; etc.)
private Socket socket;
private InputStream sInput;
private OutputStream sOutput;
BlockingQueue<byte[]> queueIn = new ArrayBlockingQueue<>(30, true);
private boolean goodToGo = false;
ClientThread(Socket socket){
id = ++Server.uniqueId; /* To be replaced with UIDs */
this.socket = socket;
/* Create Data Streams */
try {
sInput = (socket.getInputStream());
sOutput= (socket.getOutputStream());
goodToGo = true;
} catch (IOException ex) {
ServerInit.logger.log(Level.WARNING, "Error Openning Streams!", ex);
}
}
#Override
public void run(){
boolean keepGoing = true;
System.out.println("Client thread started.");
/* Start listening thread */
new Thread() {
#Override
public void run(){
while(goodToGo) {
System.out.println("Listening thread looping.");
try {
byte[] temp = IOUtils.toByteArray(sInput); // read client input using Apache Commons IO.
// Add the result to the queue.
queueIn.put(temp);
} catch (EOFException eof){
ServerInit.logger.log(Level.INFO,"Remote client closed connection.");
close();
}
catch (IOException ex) {
ServerInit.logger.log(Level.WARNING, "Error Reading Stream!", ex);
close();
}
}
}
}.start();
while (keepGoing && goodToGo){
System.out.println("Main thread looping.");
try{
byte[] message = queueIn.take();
if (message.length >= 4){
/* Message picked apart and worked with here */
} else if (message.length == 0 ){
// Do nothing.
} else {
ServerInit.logger.log(Level.WARNING, "Unable to process item from queue.");
}
} catch (Exception e) {
/* Here just for completeness, I don't catch Exceptions this way. :) */
}
}
}
protected void close(){
// try to close the conection
goodToGo = false;
try {
if (sOutput != null) {
sOutput.close();
}
if (sInput != null) {
sInput.close();
}
if (socket != null) {
socket.close();
}
ServerInit.SERVER.remove(id);
} catch (Exception e){
ServerInit.logger.log(Level.FINER, "Error closing client connections.", e);
}
}
}
And client code:
public class TestClient{
public static void main(String args[]){
try{
Socket socket = new Socket("localhost", 5525);
OutputStream outputStream = socket.getOutputStream();
byte[] buffer = { 0x02, 0x05, 0x07, 0x04 };
outputStream.write(buffer);
outputStream.flush();
outputStream.close();
} catch (Exception e) {
/* Again, I don't catch exceptions like normally. */
}
}
}
My Questions: What is causing the "listening" thread to loop and add null data to queue indefinitely?
And While I know this is not the Code Review exchange, if anyone can think of better classes to utilize, If they could just mention it.
EDIT:
Following a suggestion, I changed the queue from an ArrayList<> to an ArrayBlockingQueue.

IOUtils.toByteArray() is not appropriate for this usage. It will read to end of stream and return you one big byte array, not a sequence of messages. So there is certainly no point in calling it twice, or in a loop. After the initial result, all you can get is an infinity of empty byte arrays.

I've not used IOUtils.toByteArray but my suspicion is that if there is no data in the stream when you call it then it either returns null or an empty array.
This makes sense if you think about it since otherwise it has no idea how many bytes to read. It has no way to know if you are sending an array containing 1, 4 or 1000 bytes so it just reads everything that is ready when you call it.
You need to somehow sleep between each call to toByteArray and ignore any empty responses. A better way would be to see if you can sleep until more data arrives on the socket.

Sockets don't work as excpected with LAN or NET, but perfectly locally

I'm programming a network software with Java, but I have a real problem using my application through a "true" network.
Let a software be a host, and listening for client connexions.
Here is my Server loop :
public void run() {
while (mServerSocket != null) {
try {
Socket wClient = mServerSocket.accept();
System.out.println("Client connecté");
wClient.setSoTimeout(50);
wClient.setTcpNoDelay(false);
Client c = new Client(wClient);
synchronized(this) {
mWaitingClients.add(c);
c.start();
}
} catch(Exception ex) {
System.out.println("Server error : " + ex.getMessage());
}
}
}
When a client tried to connect to the server, I use this function :
public Client connect(InetAddress addr, int port) throws Exception {
Socket socket = new Socket(addr, port);
socket.setSoTimeout(50);
socket.setTcpNoDelay(false);
Client c = new Client(socket);
c.start();
return c;
}
And here is the client loop :
public void run() {
try {
ObjectOutputStream out = new ObjectOutputStream(mSocket.getOutputStream());
ObjectInputStream in = new ObjectInputStream(mSocket.getInputStream());
while(mSocket.isConnected() && !mSocket.isClosed()) {
for (int i = 0; i < mOutMessages.size(); i++) {
Message msg = mOutMessages.get(i);
out.writeObject(msg);
}
out.flush();
mOutMessages.clear();
Thread.sleep(50);
out.reset();
while(true) {
try {
Message m = (Message) in.readObject();
mInMessages.add(m);
} catch (Exception e) {
break;
}
}
Thread.sleep(50);
}
} catch(Exception ex) {
try {
mSocket.close();
} catch(Exception exx) {
exx.printStackTrace();
}
ex.printStackTrace();
}
}
Some other parts of the program do Message and put them in the Output list of the Client (mOutMessages).
Some other parts of the program read Message from the mInMessages of the Client.
But something is wrong with this. It works fine locally (server and client on the same computer), but fail or is hazardous (some messages are sent but never received) using two computers (with LAN or through the Internet).
Server ever detect connexions from the clients, send "handshake" messages to the client, but the client never receives them.
I'm more a C programmer than a Java one, and I never had this kind of problem using libc Sockets, so, why my way of doing is wrong ?
Thank you !
Edit :
My Server is created using this function :
public void open(int port) throws Exception {
mServerSocket = new ServerSocket(port);
start(); // Call the run mentionned above.
}
Edit :
Here is my solution, maybe it's not perfect but it works !
public void run() {
try {
BufferedOutputStream buf_out = new BufferedOutputStream(
mSocket.getOutputStream()
);
BufferedInputStream buf_in = new BufferedInputStream(
mSocket.getInputStream()
);
ObjectOutputStream out = new ObjectOutputStream(buf_out);
out.flush();
ObjectInputStream in = new ObjectInputStream(buf_in);
while(mSocket.isConnected() && !mSocket.isClosed()) {
for (int i = 0; i < mOutMessages.size(); i++) {
Message msg = mOutMessages.get(i);
out.writeObject(msg);
out.flush();
}
mOutMessages.clear();
out.reset();
while(true) {
try {
Message m = (Message) in.readObject();
mInMessages.add(m);
} catch (Exception e) {
break;
}
}
}
} catch(Exception ex) {
try {
mSocket.close();
} catch(Exception exx) {
exx.printStackTrace();
}
ex.printStackTrace();
}

If I understand right, both client and server use the run method. If both client and server happen to write sufficiently large messages (not fitting in involved buffers) at the same time then you get a deadlock because neither partner advances to reading (which would drain full buffers). Due to network delays, this might only happen in the non-local scenario, i.e. there may be enough time to pile up enough messages in the mOutMessages buffer.
Note that documentation of Socket.setSoTimeout (which you used) only says that it affects read()s. (For example, in my JDK, ObjectOutputStream seems to use a BlockDataOutputStream with a buffer size of 1024 bytes).
I recommend to either use a separate thread for reading/writing or (if you know the maximum messages size) use a sufficiently large buffer (by wrapping the SocketOutputStream in a BufferedOutputStream). If you opt for larger buffers, you may also want to write one message at a time (and try to read messages after each).

Java Sockets passing between methods

I am just starting out with Java Socket Programming, and I have been reading literature on sockets here. The below code is a sample from a textbook I've taken which asks me to find the bug. Comparing with the literature though I am not seeing any bugs. The creation of the socket, bufferedreader, and printwriter seem correct, and they are surrounded in a try-catch block as well. The are properly "close()"ed in a try-catch block as well. Is there an error when passing these to process()? Any help would be appreciated.
import java.net.*;
import java.io.*;
class main{
public void process(PrintWriter out, BufferedReader in, Socket echoSocket){
//....
}
public void processData() {
Socket echoSocket;
PrintWriter out;
BufferedReader in;
try{
echoSocket = new Socket("server.company.com", 8081);
out = new PrintWriter(echoSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream()));
}
catch (Exception e) {
System.err.println("Exception has occured");
return;
}
process(out, in, echoSocket);
try {
out.close();
in.close();
echoSocket.close();
}
catch(IOException e) {
System.err.println("IOException has occurred.");
}
}
}

Although, typos notwithstanding, one can only guess what the actual "bug" is, this code has an issue with error handling. Specifically, in the disposal of resources.
Discussion about resources
What are resources ?
Basically : any Java Object that relies on underlying OS level resources. Mostly : IO resources (input and output streams, channels), Sockets. But more importantly : if the "thing" you're using has a close, dispsose, shutdown or any of the like, it surely holds on to resources internally.
There are some exceptions (notably ByteArrayInputStream holds no resource but memory), but these are implementation details : if you stick to their interface (and you should, this is a "contract"), every stream should be closed.
Since Java 7, most of these objects in the Java API implement the AutoCloseable interface, but many 3rd parties have not necessarily ported this to their code (and maybe some can't for other reasons).
As one of the code reviewers at my company : I stop reading and I reject any code as soon as I do not see a secure call to the close method of a resource. By secure I mean inside a finally clause, that is guaranteed to be executed.
Rule of thumb about resources
Any resource obtained by your program should be freed in a finally clause (some even add : of its own).
What is the typical lifecycle of a resource
Well:
You obtain it
You use it
You release it
In your code, that is
ResourceObject myObject = null;
try {
myObject = getResource();
processResource(myObject);
} finally {
if(myObject != null) {
try {
myObject.close();
} catch (Exception e) {
// Usually there is nothing one can do but log
}
}
}
Since Java 7, if the resource object implements AutoCloseableyou have a new way of writing that, it's called the "try with resources".
try(ResourceObject myObject = getResource()) {
process(myObject);
}
You do not see the finally, but it's there, the compiler writes the finally clause for you in that case.
What about multiple resources ?
Well : multiple resources, multiple finallys. The idea is to separate the causes of failures in different finally clauses.
Say you want to copy a file...
public void myCopy() throws IOException {
InputStream source = null;
try {
source = new FileInputStream("yourInputFile");
// If anything bad happens, I have a finally clause that protects this now
OutputStream destination = null;
try {
destination = new FileOutputStream("yourOurputFile"); // If fails, my Input will be closed thanks to its own finally
performCopy(source, destination); // If this fail, my destination will also be closed thanks to its own finally
} finally {
if(destination!=null) { try { destination.close(); } catch (Exception e) {/* log*/ }}
}
} finally {
if(source!=null) { try { source.close(); } catch (Exception e) {/* log*/ }}
}
}
Or, with Java 7 syntax, we have the shorter (disclaimer : I have no Java7 right now, so can't really check if this compiles) :
try(
InputStream input = new FileInputStream("in");
OutputStream output = new FileOutputStream("out")) {
performCopy(input, output);
} catch(IOException e) {
// You still have to deal with it of course.
}
This is SO MUCH BOILERPLATE !
Yes it is. That's why we have libraries. One could argue you should not write such code. Use standard, well behaved libraries like commons IO, or use one of their utility methods. Or newer JDK methods like the Files API, and see how this works.
Commons IO has a handy IOUtils.closeQuietly() suite of methods for closing streams.
Try with resources Gotchas
There are ramifications in the "try with resources" call that go a bit deeper than that. These include: What if I want to do something with the exceptions that occur in the finally clause ? How do I differentiate that from an exception that would have occured during performCopy?
Another case is : what happens here :
try(Reader reader = new InputStreamReader(new FileInputStream("in"), "an encoding that is not supported")) {
// Whatever
}
It happens that an UnsupportedEncodingException is thrown but after the FileInputStream is instanciated. But as the FileInputStream is not the subject of the try clause, it will NOT be closed. An you have a File descriptor leak. Try that a thousand times, and your JVM will not be able to open files anymore, you OS will tell you "max number of open files exceeded" (ulimit generally does that in UNIX)
Back to your sockets
So what are your resources ?
Well, first, we can notice that you have only ONE true resource, your Socket instance, because the Socket javadoc says (javadoc):
* <p> Closing this socket will also close the socket's
* {#link java.io.InputStream InputStream} and
* {#link java.io.OutputStream OutputStream}.
So your Input and Output streams are tied to your Socket, and this is enough.
What's wrong with your code
Adding comments one your original code:
try{
echoSocket = new Socket("server.company.com", 8081);
out = new PrintWriter(echoSocket.getOutputStream(), true); // This can throw IOException
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream())); // Ditto
}
catch (Exception e) {
// If an exception was thrown getting any of the streams, we get there
System.err.println("Exception has occured");
// And you return without closing the socket. It's bad !
return;
}
// Let's assume everything worked, no exception.
process(out, in, echoSocket); // This may throw an exception (timeout, socket closed by peer, ...)
// that is uncaught (no catch clause). Your socket will be left unclosed as a result.
try {
out.close(); // This can fail
in.close(); // This too
echoSocket.close(); // And this too - although nothing you can do about it
}
catch(IOException e) {
// if out.close fails, we get here, and in.close and socket.close
// never got a chance to be called. You may be leaking resources
System.err.println("IOException has occurred.");
}
A safe implementation
Socket echoSocket = null;
try {
// open socket,
echoSocket = new Socket("server.company.com", 8081); // protected by finally
out = new PrintWriter(echoSocket.getOutputStream(), true); // protected
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream())); // protected
process(out, in, echoSocket); // Still protected
} catch (Exception e) {
// Your current error handling
} finally {
// Anyway, this close will be called if needs be.
if(echoSocket != null) {
try { echoSocket.close(); } catch (Exception e) { /* log */}
// See javadoc, this has closed the in & out streams too.
}
}

public void process(){PrintWriter out, BufferedReader in, Socket echoSocket){
should be
public void process(PrintWriter out, BufferedReader in, Socket echoSocket){
otherwise everything seems fine to me

Try this I think you missed one semicolon
public void processData() {
Socket echoSocket;
PrintWriter out;
BufferedReader in;
try{
echoSocket = new Socket("localhost", 8080);
out = new PrintWriter(echoSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream()));
}
catch (IOException e) {
System.err.println("Exception has occured");
return;
}
process(out, in, echoSocket);
try {
out.close();
in.close();
echoSocket.close();
}
catch(IOException e) {
System.err.println("IOException has occurred.");
}
}
public void process (PrintWriter out, BufferedReader in, Socket echoSocket)
{
}

Java Socket Problems

So the end result of my program is an updating game client, but what i have so far is a server that is able to accept multiple connections, and a client that connects to the server. this is the code for the client portion:
public void client() {
Socket socket = null;
ObjectInputStream in = null;
ObjectOutputStream out = null;
try {
socket = new Socket(IP, Integer.parseInt(port));
in = new ObjectInputStream(socket.getInputStream());
out = new ObjectOutputStream(socket.getOutputStream());
} catch (Exception e) {
e.printStackTrace();
}
do {
// have a conversation
try {
message = (String) in.readObject();
System.out.println("\n" + message);
} catch (Exception e) {
System.out.println("\n idk wtf that user sent!");
}
} while (!message.equals("CLIENT - END")); // When the user types "END"
System.err.println("CLOSED!!!");
System.exit(0);
}
and this is the code for the server portion:
public void run() {
// where everything happens
System.out.println("server- connected");
try {
in = new ObjectInputStream(socket.getInputStream());
out = new ObjectOutputStream(socket.getOutputStream());
out.writeObject("hi");
out.flush();
Thread.sleep(5000);
out.writeObject("close");
out.flush();
System.out.println("closed");
} catch (Exception e) {
e.printStackTrace();
}
}
now, i am running into this problem where, when my server sends the object "hi" the client appears to not receive it. i'm not totally sure if it does, but if it is getting it, it isnt printing it out like i wanted. i previously have made a chat program that does this same thing, and i pretty much copied it to this, but it isnt communicating. the most i get is the confirmation that they are connected. im not sure what is going on, but any help would be appreciated! thanks in advance!

create the ObjectOutputStreams before the ObjectInputStreams and flush them immediately after creation.
the constructor of an ObjectInputStream reads the stream header. this stream header is written by the constructor of the ObjectOutputStream (kind of an ugly implementation, but it is what it is). if you construct the OIS's first, they hang waiting for the header bytes.

readObject() blocking on particular objects

I'm writing a server/client program. The client sends "Request"s (which are objects designed for this purpose) to the server and server decodes them using an ObjectInputStream. All the "Request" objects are of the same class and just differ in data fields.
Everything usually works; but in some particular states (maybe when the Request object is a bit larger, not however more than 200 kb!) the readObject() on the serverside just blocks with no exception.
Any idea?!
The server code:
public class ConnectionThread extends Thread {
Socket con;
Request request;
public ConnectionThread(Socket s) {
con = s;
try {
ObjectInputStream in = new ObjectInputStream(con.getInputStream());
// Works till here; the object "in" is initialized.
request = (Request) in.readObject();
// This line is not reached, in particular cases.
} catch (ClassNotFoundException ex) {
Logger.getLogger(ConnectionThread.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(ConnectionThread.class.getName()).log(Level.SEVERE, null, ex);
}
}
...
}
The client code:
public static void saveStoreDataForTable(DataTable tb) {
try {
Socket s = new Socket("localhost", portNo);
ObjectOutputStream out = new ObjectOutputStream(s.getOutputStream());
out.writeObject(new Request("saveStoreData",
new Object[]
{tb.getEdited(), tb.getRemoved(), tb.getNewTables(), tb.getAlterations()}));
out.flush();
// These lines work. But I can't get servers respone; again, in some particular cases.
...
}

You should move that I/O from the constructor to the start() method. At the moment you're doing the I/O in the thread that constructs this thread, which is almost certsinly the wrong thread.