I am just starting out with Java Socket Programming, and I have been reading literature on sockets here. The below code is a sample from a textbook I've taken which asks me to find the bug. Comparing with the literature though I am not seeing any bugs. The creation of the socket, bufferedreader, and printwriter seem correct, and they are surrounded in a try-catch block as well. The are properly "close()"ed in a try-catch block as well. Is there an error when passing these to process()? Any help would be appreciated.
import java.net.*;
import java.io.*;
class main{
public void process(PrintWriter out, BufferedReader in, Socket echoSocket){
//....
}
public void processData() {
Socket echoSocket;
PrintWriter out;
BufferedReader in;
try{
echoSocket = new Socket("server.company.com", 8081);
out = new PrintWriter(echoSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream()));
}
catch (Exception e) {
System.err.println("Exception has occured");
return;
}
process(out, in, echoSocket);
try {
out.close();
in.close();
echoSocket.close();
}
catch(IOException e) {
System.err.println("IOException has occurred.");
}
}
}
Although, typos notwithstanding, one can only guess what the actual "bug" is, this code has an issue with error handling. Specifically, in the disposal of resources.
Discussion about resources
What are resources ?
Basically : any Java Object that relies on underlying OS level resources. Mostly : IO resources (input and output streams, channels), Sockets. But more importantly : if the "thing" you're using has a close, dispsose, shutdown or any of the like, it surely holds on to resources internally.
There are some exceptions (notably ByteArrayInputStream holds no resource but memory), but these are implementation details : if you stick to their interface (and you should, this is a "contract"), every stream should be closed.
Since Java 7, most of these objects in the Java API implement the AutoCloseable interface, but many 3rd parties have not necessarily ported this to their code (and maybe some can't for other reasons).
As one of the code reviewers at my company : I stop reading and I reject any code as soon as I do not see a secure call to the close method of a resource. By secure I mean inside a finally clause, that is guaranteed to be executed.
Rule of thumb about resources
Any resource obtained by your program should be freed in a finally clause (some even add : of its own).
What is the typical lifecycle of a resource
Well:
You obtain it
You use it
You release it
In your code, that is
ResourceObject myObject = null;
try {
myObject = getResource();
processResource(myObject);
} finally {
if(myObject != null) {
try {
myObject.close();
} catch (Exception e) {
// Usually there is nothing one can do but log
}
}
}
Since Java 7, if the resource object implements AutoCloseableyou have a new way of writing that, it's called the "try with resources".
try(ResourceObject myObject = getResource()) {
process(myObject);
}
You do not see the finally, but it's there, the compiler writes the finally clause for you in that case.
What about multiple resources ?
Well : multiple resources, multiple finallys. The idea is to separate the causes of failures in different finally clauses.
Say you want to copy a file...
public void myCopy() throws IOException {
InputStream source = null;
try {
source = new FileInputStream("yourInputFile");
// If anything bad happens, I have a finally clause that protects this now
OutputStream destination = null;
try {
destination = new FileOutputStream("yourOurputFile"); // If fails, my Input will be closed thanks to its own finally
performCopy(source, destination); // If this fail, my destination will also be closed thanks to its own finally
} finally {
if(destination!=null) { try { destination.close(); } catch (Exception e) {/* log*/ }}
}
} finally {
if(source!=null) { try { source.close(); } catch (Exception e) {/* log*/ }}
}
}
Or, with Java 7 syntax, we have the shorter (disclaimer : I have no Java7 right now, so can't really check if this compiles) :
try(
InputStream input = new FileInputStream("in");
OutputStream output = new FileOutputStream("out")) {
performCopy(input, output);
} catch(IOException e) {
// You still have to deal with it of course.
}
This is SO MUCH BOILERPLATE !
Yes it is. That's why we have libraries. One could argue you should not write such code. Use standard, well behaved libraries like commons IO, or use one of their utility methods. Or newer JDK methods like the Files API, and see how this works.
Commons IO has a handy IOUtils.closeQuietly() suite of methods for closing streams.
Try with resources Gotchas
There are ramifications in the "try with resources" call that go a bit deeper than that. These include: What if I want to do something with the exceptions that occur in the finally clause ? How do I differentiate that from an exception that would have occured during performCopy?
Another case is : what happens here :
try(Reader reader = new InputStreamReader(new FileInputStream("in"), "an encoding that is not supported")) {
// Whatever
}
It happens that an UnsupportedEncodingException is thrown but after the FileInputStream is instanciated. But as the FileInputStream is not the subject of the try clause, it will NOT be closed. An you have a File descriptor leak. Try that a thousand times, and your JVM will not be able to open files anymore, you OS will tell you "max number of open files exceeded" (ulimit generally does that in UNIX)
Back to your sockets
So what are your resources ?
Well, first, we can notice that you have only ONE true resource, your Socket instance, because the Socket javadoc says (javadoc):
* <p> Closing this socket will also close the socket's
* {#link java.io.InputStream InputStream} and
* {#link java.io.OutputStream OutputStream}.
So your Input and Output streams are tied to your Socket, and this is enough.
What's wrong with your code
Adding comments one your original code:
try{
echoSocket = new Socket("server.company.com", 8081);
out = new PrintWriter(echoSocket.getOutputStream(), true); // This can throw IOException
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream())); // Ditto
}
catch (Exception e) {
// If an exception was thrown getting any of the streams, we get there
System.err.println("Exception has occured");
// And you return without closing the socket. It's bad !
return;
}
// Let's assume everything worked, no exception.
process(out, in, echoSocket); // This may throw an exception (timeout, socket closed by peer, ...)
// that is uncaught (no catch clause). Your socket will be left unclosed as a result.
try {
out.close(); // This can fail
in.close(); // This too
echoSocket.close(); // And this too - although nothing you can do about it
}
catch(IOException e) {
// if out.close fails, we get here, and in.close and socket.close
// never got a chance to be called. You may be leaking resources
System.err.println("IOException has occurred.");
}
A safe implementation
Socket echoSocket = null;
try {
// open socket,
echoSocket = new Socket("server.company.com", 8081); // protected by finally
out = new PrintWriter(echoSocket.getOutputStream(), true); // protected
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream())); // protected
process(out, in, echoSocket); // Still protected
} catch (Exception e) {
// Your current error handling
} finally {
// Anyway, this close will be called if needs be.
if(echoSocket != null) {
try { echoSocket.close(); } catch (Exception e) { /* log */}
// See javadoc, this has closed the in & out streams too.
}
}
public void process(){PrintWriter out, BufferedReader in, Socket echoSocket){
should be
public void process(PrintWriter out, BufferedReader in, Socket echoSocket){
otherwise everything seems fine to me
Try this I think you missed one semicolon
public void processData() {
Socket echoSocket;
PrintWriter out;
BufferedReader in;
try{
echoSocket = new Socket("localhost", 8080);
out = new PrintWriter(echoSocket.getOutputStream(), true);
in = new BufferedReader(new InputStreamReader(echoSocket.getInputStream()));
}
catch (IOException e) {
System.err.println("Exception has occured");
return;
}
process(out, in, echoSocket);
try {
out.close();
in.close();
echoSocket.close();
}
catch(IOException e) {
System.err.println("IOException has occurred.");
}
}
public void process (PrintWriter out, BufferedReader in, Socket echoSocket)
{
}
Related
I'm working on a task and have had a question on the code review- 'do we need to ensure the input stream is consumed here?'
public void processInputStream(final DataInputStream dataInputStream, final OutputStream output) {
try {
// doing something with dataInputStream!!
} catch (IOException e) {
// doing something with IOException
}
}
I have a few questions -
#1 I assume that if the InputStream processing is interrupted, then my catch block will be triggered.
Is that correct? And if so, does that negate the need to ensure that the stream has been consumed?
#2 How would I check the InputStream has been consumed in this case?
Thanks
Update -
Part of processing my InputStream involves using -
copyInputStreamToFile(..)
From Apache commons https://commons.apache.org/proper/commons-io/javadocs/api-2.7/org/apache/commons/io/FileUtils.html#copyInputStreamToFile-java.io.InputStream-java.io.File-
Their documentation says -
Copies bytes from an InputStream source to a file destination. The directories up to destination will be created if they don't already exist. destination will be overwritten if it already exists. The source stream is closed. See copyToFile(InputStream, File) for a method that does not close the input stream.
Does this mean that given the source stream is closed, then this covers checking the InputStream has been consumed?
You could check whether an InputStream is exhausted or not using this method:
package example;
import java.io.IOException;
import java.io.InputStream;
public class SO {
public static boolean isExhausted(InputStream in) throws IOException {
final boolean exhausted;
if (in.markSupported()) {
in.mark(1);
exhausted = in.read() == -1;
in.reset();
} else {
throw new IllegalStateException("mark is not supported on this inputstream");
}
return exhausted;
}
}
Note that this only works if the InputStream supports mark and reset methods (in.markSupported())
This did the trick!
private void consumeQuietly(final InputStream inputStream) {
try (OutputStream out = NullOutputStream.NULL_OUTPUT_STREAM) {
IOUtils.copy(inputStream, out);
} catch (IOException ioException) {
// Log something!!
}
}
public void processInputStream(final DataInputStream dataInputStream, final OutputStream output)
{
try
{
// doing something with dataInputStream!!
}
catch (InterruptedException ie)
{
// doing something with InterruptedException
}
catch (IOException ioe)
{
// doing something with IOException
}
}
You can utilize inputStream.available() method to determine whether input stream is consumed or not.
I have web service written on JAVA (version 1.8) which connects HSM and sends/receives data via socket. My application is deployed on Apache Tomcat/8.5.14 on linux.
Although I'm closing socket connection properly I have
java.net.SocketException: Too many open files
and here is myclass
public class myClass implements AutoCloseable {
Socket socket;
DataInputStream in;
DataOutputStream out;
public myClass(String ip, int port) throws Exception {
try {
socket = new Socket(ip, port);
in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
} catch (IOException e) {
throw new Exception("Connecting to HSM failed" + e);
}
}
public String sendCommandToHsm(String command) throws IOException {
out.writeUTF(command);
out.flush();
return in.readUTF();
}
#Override
public void close() {
if (socket != null && !socket.isClosed()) {
try {
socket.close();
} catch (IOException e) {
lgg.info("Closing of socket failed", e);
}
}
if (in != null) {
try {
in.close();
} catch (IOException e) {
lgg.info("Closing of inputStream failed", e);
}
}
if (out != null) {
try {
out.close();
} catch (IOException e) {
lgg.info("Closing of outputStream failed", e);
}
}
}
}
Here is using of my class
try (MyClass myClass = new MyClass(ip, port);) {
myClass.sendCommandToHsm("my command");
}
I increased maximum open files limit on server from default value(1024) to 8192 and few times later the same Exception occurred again.
I'm thinking about creating Socket Connection Pool, is it good idea?
Can you suggest any other solutions?
Although I'm closing socket connection properly ...
It looks like you are, but I think there are a couple of problems. (I don't know that these are the cause of your leak, but the first one is a plausible explanation.)
Problem 1.
public myClass(String ip, int port) throws Exception {
try {
socket = new Socket(ip, port);
in = new DataInputStream(new BufferedInputStream(socket.getInputStream()));
out = new DataOutputStream(new BufferedOutputStream(socket.getOutputStream()));
} catch (IOException e) {
throw new Exception("Connecting to HSM failed" + e);
}
}
If an exception is thrown during setup of the streams, then the socket will be leaked.
Problem 2.
public void close() {
if (socket != null && !socket.isClosed()) {
try {
socket.close();
} catch (IOException e) {
lgg.info("Closing of socket failed", e);
}
}
if (in != null) {
try {
in.close();
} catch (IOException e) {
lgg.info("Closing of inputStream failed", e);
}
}
if (out != null) {
try {
out.close();
} catch (IOException e) {
lgg.info("Closing of outputStream failed", e);
}
}
}
You are closing in the wrong order. You should1 close in and out before closing socket. In particular, if out has buffered data, then closing out will attempt to flush ... which will fail if you have closed socket already.
Also if socket.close() or in.close() fails for some other reason than an IOException, then the subsequent closes will be skipped. So you should use a finally here.
Also the isClosed() call is redundant. Calling close() on a resource that is already closed should do nothing. This is part of the contract of close().
Finally, calling close() on a socket should2 automatically close the low-level file descriptors beneath in and out. So it is arguably best to just do this:
public void close() {
if (socket != null) {
try {
socket.close();
} catch (IOException e) {
lgg.info("Closing of socket failed", e);
}
}
}
If this doesn't fix your leaks, I suggest that you use netstat and lsof to try to find out if the leakage is open files or open sockets.
I'm thinking about creating Socket Connection Pool, is it good idea?
Yes ... if you can find an existing (well designed and tested) library that meets your requirements. Implementing a reliable pool from scratch is not trivial.
But note:
An incorrectly implemented (or used) pool can leak file descriptors.
The server-side needs to be able to cope with a series of requests / replied on the same connection.
If you have too many simultaneous open connections to different places, then the pool needs a way to close some of them ...
1 - It is debatable whether you should close out before in. On the one hand, closing out flushes outstanding data to the server. On the other hand, by the time that myClass.close() has been called there won't be anything reading the server's response. And besides, the sendCommandToHsm method flushes ... so there shouldn't be any outstanding data.
2 - The javadoc for Socket.close() says: "Closing this socket will also close the socket's InputStream and OutputStream."
I am currently learning networking, specifically client-server classes.
I have done much research and implemented various test programs but I can't figure out why/when one would need to use the flush() method.
How can there be data mistakenly left in the output stream if it is always read in by the input stream? As dictated by the client-server code.
I tried to test my basic echo client server program by omitting the flush() but I could not break it.
When testing the flush() by writing twice from the client side and only reading once for the server's reply all that happened was a backlog (I assume the stream acts like a queue?) in the server's replies.
Then I took the same code and added flush() before and after the second write and it made no difference. It's as if the flush() doesn't actually clear the stream.
So can someone please explain in what scenario with regards to client/server stream interactions would flush() be required?
Server:
public class ServerApp
{
private ServerSocket listener;
private Socket clientCon;
public ServerApp()
{
try
{
listener = new ServerSocket(1234, 10);
} catch (IOException e)
{
e.printStackTrace();
}
}
public void listen()
{
try
{
System.out.println("Server is listening!");
clientCon = listener.accept();
System.out.println("Server: Connection made with Client");
processClient();
} catch (IOException e)
{
e.printStackTrace();
}
}
public void processClient()
{
try(ObjectOutputStream out = new ObjectOutputStream(clientCon.getOutputStream()); ObjectInputStream in = new ObjectInputStream(clientCon.getInputStream()))
{
String msg;
while(!(msg = (String)in.readObject()).equalsIgnoreCase("Shutdown"))
{
out.writeObject("Server: " + msg);
out.flush();
}
out.writeObject("Server is powering down...");
out.close();
in.close();
} catch (IOException | ClassNotFoundException e)
{
e.printStackTrace();
}
}
public static void main (String args[])
{
ServerApp sa = new ServerApp();
sa.listen();
}
}
Client:
public class ClientApp
{
private Socket serverCon;
public ClientApp()
{
try
{
serverCon = new Socket("127.0.0.1", 1234);
} catch (IOException e)
{
e.printStackTrace();
}
}
public void communicate()
{
try (ObjectOutputStream out = new ObjectOutputStream(serverCon.getOutputStream()); ObjectInputStream in = new ObjectInputStream(serverCon.getInputStream());
BufferedReader br = new BufferedReader(new InputStreamReader(System.in)))
{
String response = null;
do
{
System.out.println("Enter your message for server: ");
out.writeObject(br.readLine());
out.flush();
out.writeObject("Flush not working");
out.flush();
response = (String) in.readObject();
System.out.println(response);
response = (String) in.readObject();
System.out.println(response);
} while (!response.equalsIgnoreCase("Server is powering down..."));
} catch (IOException | ClassNotFoundException e)
{
e.printStackTrace();
}
}
public static void main(String args[])
{
ClientApp ca = new ClientApp();
ca.communicate();
}
}
The method flush() is used to flush out any internal buffers that may be in use. For example using a BufferedOutputStream the contents are written in chunks to improve performance (it's slower to write each byte as they come).
Depending on usage, you might never have to call flush(). However let's say you send a small String (converted to byte[]) and it fits nicely in the internal buffer. The contents of the buffer won't be sent until the buffer is full or flush() is called.
Now let's say you're writing over the network, and you expect the other side to answer something to your small String. Since it's still in the buffer, the other side won't receive it and it can result in both sides waiting forever.
Object streams are another beast, and I'm a little disappointed that so many beginners are using them. There should be a warning in the class saying "Objects may be more difficult to send/receive than they appear".
ObjectOutputStream delegates the flush() call to its internal BlockDataOutputStream which has 3 buffers sized 1024, 5 and 256 for "blockdata", header data and characters respectively.
Try it with new ObjectOutputStream(new BufferedOutputStream(clientCon.getOutputStream())) and you'll see a difference with and without flush(). It causes flushing of the underlying buffered output stream. Without a buffered stream there is no buffer to flush so it does nothing.
I was just looking back over some code we wrote in a java class that I'm taking. I noticed that in the finally block there is a try/catch for closing the reader, but not the writer. I'll copy the code below. Can anyone explain why that would be? I'd like to understand better.
public class UsingFiles {
public static void main(String[] args) {
// open the input stream (from the file of this program)
BufferedReader reader = null;
PrintWriter writer = null;
try {
reader = new BufferedReader(new FileReader("./src/UsingFiles.java"));
writer = new PrintWriter("reverseFile.txt");
// String line;
// while ((line = reader.readLine()) != null) {
// System.out.println(line);
// }
// print the file in reverse order
// use recursion
reverseFile(reader, writer);
} catch (FileNotFoundException e) {
System.out.println("Couldn't open the file!");
} catch (IOException e) {
System.out.println("Problem reading the file");
} finally {
if (reader != null) {
try {
reader.close();
} catch (IOException e) {
System.out.println("Couldn't close the reader");
}
}
if (writer != null) {
writer.close();
}
}
}
private static void reverseFile(BufferedReader reader, PrintWriter writer)
throws IOException {
String line = reader.readLine();
if (line != null) {
reverseFile(reader, writer);
writer.println(line);
}
}
There are two possibilities that I can think of:
It's an oversight
Both calls to close() can throw an exception. If the first one throws an exception, the second one would be skipped - unless of course the first one was wrapped in its own try/catch block. The second one doesn't need a try/catch block since if it fails, there is no subsequent code that will be skipped
In the "real world", I would say that the answer is #1. The reason I would think #2 to be unlikely is that there is usually some other code that you will want to execute, even if you can't close some stream. This would be especially true had the catch blocks not caught an exception (or re-threw a different exception) since a fresh exception in the finally block would replace the original exception and you would never know that it had happened.
Update
As another answer has pointed out, PrintWriter.close() does not in fact throw an IOException, even though the parent interface Writer does declare that close() can throw an IOException. So that may be a better explanation.
I believe the intent was to attempt to close the writer even if the reader failed to close. If closing the reader throws an IOException, you will never execute the rest of the finally block.
This is because PrintWriter never throws exception during close(). See API. This
try {
writer.close();
} catch(IOException e) {
System.out.println("Couldn't close the writer");
}
will result in compiler error: Unreachable catch block for IOException. This exception is never thrown from the try statement body
It actually SHOULD be closed.
http://docs.oracle.com/javase/7/docs/api/java/io/PrintWriter.html
Anytime you use a resource on the system, it's great practice to close a the objects that have access to it.
There is no need to close reader in try block of finally block if you are using try with resource
try(reader = new BufferedReader(new FileReader("./src/UsingFiles.java"))
{
}
catch(Exception e)
{
}
I'm trying to read an object from client program over tcp. As you can see in this line I created objectInput:
ObjectInputStream objectInput = new ObjectInputStream(incoming.getInputStream());
And then read my input from the other program. It used to work fine until i made minor changes to clean up the program. Personally Assume I added
objectInput.clsoe();
My question is, After reading the object should I close the objectInputStream or Keep remain without close? Should I close it straight away after using it or at the end of if block or at the end of program? What are the effect of the close? By the way I have read the close documentation.
Here is the error:
java.io.EOFException
at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2280)
at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2749)
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:779)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:279)
at Server.ClientWorker.run(MyCollectionServer.java:116)
at java.lang.Thread.run(Thread.java:680)
Here is my code:
public static void main(String[] args) {
ServerSocket serverSocket = null;
try
{
serverSocket = new ServerSocket(port);
}
catch (IOException e)
{
e.printStackTrace();
}
while(true)
{
ClientWorker w;
try
{
w = new ClientWorker(serverSocket.accept());
Thread t = new Thread(w);
t.start();
}
catch(IOException e)
{
e.printStackTrace();
break;
}
}
}
}
class ClientWorker implements Runnable
{
.....
private Socket incoming;
public ClientWorker(Socket incoming)
{
myList = new ArrayList<PureAlbum>();
loadList();
this.incoming = incoming;
}
.....
public synchronized void run()
{
else if(request.compareTo("AddAlbum")==0)
{
try
{
ObjectInputStream objectInput = new ObjectInputStream(incoming.getInputStream()); //This is the line mentioned in the error
PureAlbum object = (PureAlbum) objectInput.readObject();
if(object instanceof CDAlbum)
{
CDAlbum a = (CDAlbum) object;
myList.add(a);
System.out.println("Artist = " + a.getArtist());
}
else if(object instanceof Client.au.edu.uow.Collection.DVDAlbum)
{
myList.add((DVDAlbum) object);
}
else
{
System.err.println("Warning : The object to add to database is unknown! "+ object.getClass() + "*");
System.exit(0);
}
}
catch (UnknownHostException e)
{
System.err.println("Can not read the host name");
e.printStackTrace();
}
catch (IOException e)
{
System.err.println("Can not read the FILE name"); //This exception has been called
e.printStackTrace();
}
catch (ClassNotFoundException e)
{
e.printStackTrace();
}
}
Your code fragment is pretty long, so I will try to give you a general answer and it will hopefully help you.
The typical patterns of usage of stream.close() prior to java 7 are:
InputStream in = null;
try {
InputStream in = .....; // create your input stream;
// use input stream
} catch (IOException e) {
// do what you need here
} finally {
if (in != null) {
in.close();
}
}
or simply declare the method as throws IOException and then write:
InputStream in = null;
try {
InputStream in = .....; // create your input stream;
// use input stream
} finally {
if (in != null) {
in.close();
}
}
Pay atention that this example does not contain catch section.
Starting from java 7 we can enjoy the new facilities of the language:
try (
InputStream in = .....; // create your input stream;
) {
// use input stream
}
You even do not have to call close() at all. All resources defined into header of try block that implement interface Closable will be closed automatically.
This line of stack trace shows that the exception is occurring when you are initializing the ObjectInputStream, not when you are closing.
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:279)
The most likely cause is that the remote client did not open an ObjectOutputStream. It might have written some other kind of data, or it might have closed its output stream or simply exited.
You should close the Stream when you have completed your reading or writing.
here in this case , you should close the InputStream when you have read the file completely and you no longer require to read file from stream.
In Short , You should close the Stream when its work is over.
It may be in the end of program or after if loop....depends on your use case.
Hope this will help.
I am doing it this way (different example):
private void readFile() throws Exception {
FileInputStream fis = null;
ObjectInputStream ois = null;
Object aux = null;
try {
fis = new FileInputStream("lib.dat");
ois = new ObjectInputStream(fis);
do {
aux = ois.readObject();
if (aux instanceof MyObject)
this.myObjectInstance.add((MyObject) aux);
} while (true);
} catch (EOFException e) {
ois.close();
}
}
This way I am sending any relevant "Error" Exception upstairs to be handled, and once the EndOfFileException is launched this is specifically captured to close the stream properly.
The object has to be defined outside the Try block to be accessible from the Catch block.
The close() method could as well throw an IOException and this can't be caught by our Try block, this would have to be passed by the generic "throw Exception" of readFile() method.