Develop Reference

Error while reading data through socket communication

Following scenario that explains my problem.
I've a PLC that acts as a server socket program. I've written a Client Java program to communicate through socket communication with the PLC.
Steps that take place in this process are:
1) For each second my Client program happen to communicate with the PLC, read the data in stream, store the data temporarily in a ByteArrayOutputStream and closing both input stream and socket. Following snippet gives the idea
try {
socket = new Socket(host, port);
is = socket.getInputStream();
outputBuffer = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int read;
if((read = is.read(buffer)) != -1) {
outputBuffer.write(buffer, 0, read);
}
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
System.out.println("Before closing the socket");
try {
is.close();
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("After closing the socket");
} catch (Exception e) {
e.printStackTrace();
}
}
2) Processing stored data according to my requirement is what I'm trying to do. So for every 1 second, client program connects to Server, read the data(if data is present), store the data, close socket and process it. And it has to happen for a very long run, probably till the Server program is on. And that may happen till for every few weeks.
3) Problem what I'm facing is, I'm able to run the above show for 1-2 hours, but from then, Client Program unable to fetch the data from the Server Program(PLC in this case), though both are connected through socket. I.e 128 bytes of data present, but Client program isn't able to read that data. And this started happening after program run successfully for almost 2hours
4) Please find the brief code which may help for you to look into.
public class LoggingApplication {
public static void main(String[] args) throws NumberFormatException {
if (args.length > 0 && args.length == 2) {
String ipAddress = mappingService.getIpAddress();
int portNo = (int) mappingService.getPortNo();
ScheduledExecutorService execService = Executors.newScheduledThreadPool(1);
execService.schedule(new MyTask(execService, ipAddress, portNo, mappingService), 1000, TimeUnit.MILLISECONDS);
} else {
throw new IllegalArgumentException("Please pass IPAddress and port no as arguments");
}
}
}
Runnable Code:
public class MyTask implements Runnable {
public ScheduledExecutorService execService;
private String ipAddress;
private int portNo;
private ConfigurationMappingService mappingService;
private MySocketSocketUtil mySocketSocketUtil;
public MyTask(ScheduledExecutorService execService, String ipAddress, int portNo, ConfigurationMappingService mappingService) {
this.execService = execService;
this.ipAddress = ipAddress;
this.portNo = portNo;
this.mappingService = mappingService;
}
public void run() {
MySocketSocketUtil mySocketSocketUtil = new MySocketSocketUtil(ipAddress, portNo);
execService.schedule(new MyTask(execService, ipAddress, portNo, mappingService), 1000, TimeUnit.MILLISECONDS);
mySocketSocketUtil.getData(); //It's able to fetch the data for almost 2 hours but from then, it's just getting empty data and it's keep on giving empty data from then. and so on.
/*
*
*Some code
*/
}
}
Here's where, I'm having the problem
mySocketSocketUtil.getData(); is able to fetch the data for almost 2 hours but from then, it's just getting empty data and it's keep on giving empty data from then. and so on. It's a big question I know, And I want to understand what might have gone wrong.
Edit: I'm ignoring the condition to check end of the stream and closing a socket based on it is because, I knew I'm going to read first 1024 bytes of data only always. And So, I'm closing the socket in finally block

socket = new Socket(host, port);
if(socket != null && socket.isConnected())
It is impossible for socket to be null or socket.isConnected() to be false at this point. Don't write pointless code.
if((read = is.read(buffer)) != -1) {
outputBuffer.write(buffer, 0, read);
};
Here you are ignoring a possible end of stream. If read() returns -1 you must close the socket. It will never not return -1 again. This completely explains your 'empty data':
from then, it's just getting empty data and it's keep on giving empty data from then, and so on
And you should not create a new Socket unless you have received -1 or an exception on the previous socket.
} else {
System.err.println("Socket couldn't be connected");
}
Unreachable: see above. Don't write pointless code.

You should never disconnect from the established connection. Connect once in the LoggingApplication. Once the socket is connected keep it open. Reuse the socket on the next read.

I think there are couple of points you need to fix before getting to the solution to your problem. Please try to follow the following suggestions first:
As #EJP said this code block is not needed.
if(socket != null && socket.isConnected()) {
also you are using a byte array of length 1024 and not using while or for loop to read the data stream. Are you expecting only a block of data which will never exceed 1024 bytes?
byte[] buffer = new byte[1024];
int read;
if((read = is.read(buffer)) != -1) {
This is also not needed as it is unreachable.
} else {
System.err.println("Socket couldn't be connected");
}
Can you explain the data stream behavior you are expecting?
Last but not the least is.read(buffer) is a blocking call so if there is no data to read yet, it will hold the thread execution at that point.
Please try to answer the questions I have asked.
#KishoreKumarKorada from your description in the comment section, it seems like you are monitoring the data change on server side. Socket stream works in a read-once fashion. So,
First thing is, you need to request from server every time and the server needs to RESEND the data on every request.
Second, the way you presented is more like you are operating on byte level, which is not very good way to do that unless you have any legitimate reason to do so. The good way is to wrap the data in JSON or XML format and send it over the stream. But to reduce bandwidth consumption, you may need to operate on byte stream sometimes. You need to decide on that.
Third, for monitoring the data change, the better way is to use some timestamp to compare when the data has changed on the server side and what is the timestamp stored on the client side, if they match, data has not changed. Otherwise fetch the data from the server side and update the client side.
Fourth, when there is data available that you are not able to read, can you debug the ins.read(...) statement to see if its getting executed and the execution goes inside the if block or if statement is evaluated to false? if true then examine the read value and let me know what you have found?
Thanks.

Android socket connects but cant write to it

I was trying to get some networking going in my app, but i encountered some issues. It seems that I cant write to the OutputStream object. Though my server recieves the connection, it does not recieve any data. I've tried using Writer, DataOutputStream among others. none seemed to work.
My app uses asynctasks that call this object with a Socket object and a message. (The socket object has already been used to set Streams after initialisation using the setStreams method.)
can someone please try and find the problem? I will be very thankful.
public class NetworkingUtils {
private OutputStream out = null;
private InputStream in = null;
//set streams
public void setStreams(Socket sock){
if (sock.isConnected()) {
try {
this.out = (OutputStream) sock.getOutputStream();
this.in = (InputStream) sock.getInputStream();
} catch (Throwable e) {
Log.d("SOCKET", "FAILED TO SET STREAMS");
e.printStackTrace();
}
}
}
//send \n terminated messages to pre defined socket
public void sendMessage(Socket sock, String message) throws Throwable {
if (sock.isConnected()) {
try {
this.out.write(message.getBytes());
Log.d("SOCKET","WRITING COMPLETE. " + message);
} catch (Throwable e) {
throw e;
}
}
}
public String recvMessage(Socket sock) throws Throwable {
//receives \n terminated message from pre defined socket
String answer = null;
if (sock.isConnected()){
try{
answer = this.convertStreamToString(this.in);
Log.d("SOCKET","READING COMPLETE");
}
catch (Throwable e){
Log.d("socket",e.getLocalizedMessage());
throw e;
}
}
else{
Log.d("socket","is not connected!!!");
}
if (answer.length() == 0){
//empty string answer from server
throw new IOException();
}
else {
return answer;
}
}
private String convertStreamToString(java.io.InputStream is) {
java.util.Scanner s = null;
try{
s = new java.util.Scanner(is).useDelimiter("\r\n");}
catch (Throwable e){
e.printStackTrace();
}
return s.hasNext() ? s.next() : "";
}
}

I can only see one client-side that might cause this ... and I'm doubtful about it. (That is to say: try this, just in case it makes a differences, but I don't think it will.)
this.out.write(message.getBytes());
Log.d("SOCKET","WRITING COMPLETE. " + message);
The potential problem is that if out is a "buffered" stream, then a write may only result in the bytes being written to the buffer. It may be necessary to call this.out.flush() to "push" to the server.
But I am doubtful it will help, because (to my knowledge) a socket output stream isn't buffered in Java. I think it is more likely that the real problem is on the server side.
If you are stumped with figuring out which side the problem is occuring, I suggest you try using a network monitoring / packet sniffing tool (on the server side) to check if the data is reaching the server host.
While I have your attention, your exception code is really, really bad.
Don't declare methods as throws Throwable (or throws Exception). That basically says "this method may throw ANY exception, and I'm not telling you which one". When you do that, the caller code has to cope with any exception, which is basically impossible to do intelligently.
What you should do is to declare the method as throwing the checked exceptions that the code can throw. For example, in your case, IOException is probably sufficient.
It is not a good idea to catch an exception, log it, and then rethrow it. Why? Because further up the stack there are probably other methods that will see the exception. They can't know if the exception has already been logged or not. So should they log it (possibly resulting in duplicate logs events for the same problem) or not (possibly resulting in the exception going unlogged.)
Don't throw exceptions without a message:
throw new IOException();
It is lazy. You should always include a simple message that can (at least) be grep'd or googled for.
In addition, your testing of Socket.isConnected() all over the place is unnecessary. According to the javadoc:
Returns: true if the socket was successfuly connected to a server
Note: Closing a socket doesn't clear its connection state, which means
this method will return true for a closed socket (see isClosed()) if
it was successfuly connected prior to being closed.
So repeatedly testing isConnected is nugatory. If it returns true once, it will will always return true from then on.
Even the initial isConnected test in setStreams is doubtful. I'd just call getInputStream without testing, and rely on the Socket API throwing an IOException if the socket is in the wrong state.

You're effectively reading lines with that obscure Scanner usage, but you're not writing lines. So the scanner will block until a line terminator or EOS arrives.
You need to append a line terminator when sending.

I'm having troubles with Java sockets in a client/server type application when having to accept many connections

First of all, thanks for reading. This is my first time in stackoverflow as user, although I've always read it and found useful solutions :D. By the way, sorry if I'm not clear enough explaining myself, I know that my English isn't very good.
My socket based program is having a strange behaviour, and some performance issues. The client and server communicate with each other by reading/writing serialized objects into object input and output streams, in a multi-threaded way. Let me show you the code basics. I have simplified it to be more readable and a complete exception handling for example is intentionally ommited. The server works like this:
Server:
// (...)
public void serve() {
if (serverSocket == null) {
try {
serverSocket = (SSLServerSocket) SSLServerSocketFactory
.getDefault().createServerSocket(port);
serving = true;
System.out.println("Waiting for clients...");
while (serving) {
SSLSocket clientSocket = (SSLSocket) serverSocket.accept();
System.out.println("Client accepted.");
//LjServerThread class is below
new LjServerThread(clientSocket).start();
}
} catch (Exception e) {
// Exception handling code (...)
}
}
}
public void stop() {
serving = false;
serverSocket = null;
}
public boolean isServing() {
return serving;
}
LjServerThread class, one instance created per client:
private SSLSocket clientSocket;
private String IP;
private long startTime;
public LjServerThread(SSLSocket clientSocket) {
this.clientSocket = clientSocket;
startTime = System.currentTimeMillis();
this.IP = clientSocket.getInetAddress().getHostAddress();
}
public synchronized String getClientAddress() {
return IP;
}
#Override
public void run() {
ObjectInputStream in = null;
ObjectOutputStream out = null;
//This is my protocol handling object, and as you will see below,
//it works processing the object received and returning another as response.
LjProtocol protocol = new LjProtocol();
try {
try {
in = new ObjectInputStream(new BufferedInputStream(
clientSocket.getInputStream()));
out = new ObjectOutputStream(new BufferedOutputStream(
clientSocket.getOutputStream()));
out.flush();
} catch (Exception ex) {
// Exception handling code (...)
}
LjPacket output;
while (true) {
output = protocol.processMessage((LjPacket) in.readObject());
// When the object received is the finish mark,
// protocol.processMessage()object returns null.
if (output == null) {
break;
}
out.writeObject(output);
out.flush();
out.reset();
}
System.out.println("Client " + IP + " finished successfully.");
} catch (Exception ex) {
// Exception handling code (...)
} finally {
try {
out.close();
in.close();
clientSocket.close();
} catch (Exception ex) {
// Exception handling code (...)
} finally {
long stopTime = System.currentTimeMillis();
long runTime = stopTime - startTime;
System.out.println("Run time: " + runTime);
}
}
}
And, the client, is like this:
private SSLSocket socket;
#Override
public void run() {
LjProtocol protocol = new LjProtocol();
try {
socket = (SSLSocket) SSLSocketFactory.getDefault()
.createSocket(InetAddress.getByName("here-goes-hostIP"),
4444);
} catch (Exception ex) {
}
ObjectOutputStream out = null;
ObjectInputStream in = null;
try {
out = new ObjectOutputStream(new BufferedOutputStream(
socket.getOutputStream()));
out.flush();
in = new ObjectInputStream(new BufferedInputStream(
socket.getInputStream()));
LjPacket output;
// As the client is which starts the connection, it sends the first
//object.
out.writeObject(/* First object */);
out.flush();
while (true) {
output = protocol.processMessage((LjPacket) in.readObject());
out.writeObject(output);
out.flush();
out.reset();
}
} catch (EOFException ex) {
// If all goes OK, when server disconnects EOF should happen.
System.out.println("suceed!");
} catch (Exception ex) {
// (...)
} finally {
try {
// FIRST STRANGE BEHAVIOUR:
// I have to comment the "out.close()" line, else, Exception is
// thrown ALWAYS.
out.close();
in.close();
socket.close();
} catch (Exception ex) {
System.out.println("This shouldn't happen!");
}
}
}
}
Well, as you see, the LjServerThread class which handles accepted clients in the server side, measures the time it takes... Normally, it takes between 75 - 120 ms (where the x is the IP):
Client x finished successfully.
Run time: 82
Client x finished successfully.
Run time: 80
Client x finished successfully.
Run time: 112
Client x finished successfully.
Run time: 88
Client x finished successfully.
Run time: 90
Client x finished successfully.
Run time: 84
But suddenly, and with no predictable pattern (at least for me):
Client x finished successfully.
Run time: 15426
Sometimes reaches 25 seconds!
Ocasionally a small group of threads go a little slower but that doesn't worry me much:
Client x finished successfully.
Run time: 239
Client x finished successfully.
Run time: 243
Why is this happening? Is this perhaps because my server and my client are in the same machine, with the same IP? (To do this tests I execute the server and the client in the same machine, but they connect over internet, with my public IP).
This is how I test this, I make requests to the server like this in main():
for (int i = 0; i < 400; i++) {
try {
new LjClientThread().start();
Thread.sleep(100);
} catch (Exception ex) {
// (...)
}
}
If I do it in loop without "Thread.sleep(100)", I get some connection reset exceptions (7 or 8 connections resetted out of 400, more or less), but I think I understand why it happens: when serverSocket.accept() accepts a connection, a very small amount of time has to be spent to reach serverSocket.accept() again. During that time, the server cannot accept connections. Could it be because of that? If not, why? It would be rare 400 connections arriving to my server exactly at the same time, but it could happen. Without "Thread.sleep(100)", the timing issues are worse also.
Thanks in advance!
UPDATED:
How stupid, I tested it in localhost... and it doesn't give any problem! With and without "Thread.sleep(100)", doesn't matter, it works fine! Why! So, as I can see, my theory about why the connection reset is beeing thrown is not correct. This makes things even more strange! I hope somebody could help me... Thanks again! :)
UPDATED (2):
I have found sightly different behaviours in different operating systems. I usually develop in Linux, and the behaviour I explained was about what was happening in my Ubuntu 10.10. In Windows 7, when I pause 100ms between connections, all its fine, and all threads are lighting fast, no one takes more than 150ms or so (no slow connection issues!). This is not what is happening in Linux. However, when I remove the "Thread.sleep(100)", instead of only some of the connections getting the connection reset exception, all of them fail and throw the exception (in Linux only some of them, 6 or so out of 400 were failing).
Phew! I've just find out that not only the OS, the JVM enviroment has a little impact also! Not a big deal, but noteworthy. I was using OpenJDK in Linux, and now, with the Oracle JDK, I see that as I reduce the sleep time between connections, it starts failing earlier (with 50 ms OpenJDK works fine, no exceptions are thrown, but with Oracle's one quite a lot with 50ms sleep time, while with 100ms works fine).

The server socket has a queue that holds incoming connection attempts. A client will encounter a connection reset error if that queue is full. Without the Thread.sleep(100) statement, all of your clients are trying to connect relatively simultaneously, which results in some of them encountering the connection reset error.

Two points I think you may further consider researching. Sorry for a bit vague here but this is what I think.
1) Under-the-hood, at tcp level there are few platform dependent things control the amount of time it takes to send/receive data across a socket. The inconsistent delay could be because of the settings such as tcp_syn_retries. You may be interested to look at here http://www.frozentux.net/ipsysctl-tutorial/chunkyhtml/tcpvariables.html#AEN370
2)Your calculated execution time is not only the amount of time it took to complete the execution but includes the time until the finalization is done which is not guaranteed to happen immediately when an object is ready for finalization.

Concurrent read/write of named pipe in Java (on windows)

I'm trying to provide communication between a C# app and a Java app on windows using named pipes with the method described by v01ver in this question: How to open a Windows named pipe from Java?
I'm running into a problem on the Java side because I have a reader thread constantly waiting for input on the pipe and when I try to write to the pipe from my main thread it gets stuck forever.
final RandomAccessFile pipe;
try {
pipe = new RandomAccessFile("\\\\.\\pipe\\mypipe", "rw");
}
catch (FileNotFoundException ex) {
ex.printStackTrace();
return;
}
Thread readerThread = new Thread(new Runnable() {
#Override
public void run() {
String line = null;
try {
while (null != (line = pipe.readLine())) {
System.out.println(line);
}
}
catch (IOException ex) {
ex.printStackTrace();
}
}
});
readerThread.start();
try { Thread.sleep(500); } catch (InterruptedException e) {}
try {
System.out.println("Writing a message...");
pipe.write("Hello there.\n".getBytes());
System.out.println("Finished.");
}
catch (IOException ex) {
ex.printStackTrace();
}
The output is: Writing a message...
and then it waits forever.
How can I write to a named pipe while waiting for input in another thread?

This is expected behaviour of pipes. It is supposed to hang untill other process connects to the pipe and reads it.

I have a same problem -- communication between a C#/Python app and a Java app on windows using named pipes:
We have example of Client Code written on Java, but in line String echoResponse = pipe.readLine(); tread waits forever.
try {
// Connect to the pipe
RandomAccessFile pipe = new RandomAccessFile("\\\\.\\pipe\\testpipe", "rw");
String echoText = "Hello word\n";
// write to pipe
pipe.write ( echoText.getBytes() );
// read response
String echoResponse = pipe.readLine();
System.out.println("Response: " + echoResponse );
pipe.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Solution of problem:
I have a ServerPipe code written on Python from here Example Code - Named Pipes:
and run its on Python 2.6.6
from ctypes import *
PIPE_ACCESS_DUPLEX = 0x3
PIPE_TYPE_MESSAGE = 0x4
PIPE_READMODE_MESSAGE = 0x2
PIPE_WAIT = 0
PIPE_UNLIMITED_INSTANCES = 255
BUFSIZE = 4096
NMPWAIT_USE_DEFAULT_WAIT = 0
INVALID_HANDLE_VALUE = -1
ERROR_PIPE_CONNECTED = 535
MESSAGE = "Default answer from server\0"
szPipename = "\\\\.\\pipe\\mynamedpipe"
def ReadWrite_ClientPipe_Thread(hPipe):
chBuf = create_string_buffer(BUFSIZE)
cbRead = c_ulong(0)
while 1:
fSuccess = windll.kernel32.ReadFile(hPipe, chBuf, BUFSIZE,
byref(cbRead), None)
if ((fSuccess ==1) or (cbRead.value != 0)):
print chBuf.value
cbWritten = c_ulong(0)
fSuccess = windll.kernel32.WriteFile(hPipe,
c_char_p(MESSAGE),
len(MESSAGE),
byref(cbWritten),
None
)
else:
break
if ( (not fSuccess) or (len(MESSAGE) != cbWritten.value)):
print "Could not reply to the client's request from the
pipe"
break
else:
print "Number of bytes written:", cbWritten.value
windll.kernel32.FlushFileBuffers(hPipe)
windll.kernel32.DisconnectNamedPipe(hPipe)
windll.kernel32.CloseHandle(hPipe)
return 0
def main():
THREADFUNC = CFUNCTYPE(c_int, c_int)
thread_func = THREADFUNC(ReadWrite_ClientPipe_Thread)
while 1:
hPipe = windll.kernel32.CreateNamedPipeA(szPipename,
PIPE_ACCESS_DUPLEX,
PIPE_TYPE_MESSAGE |
PIPE_READMODE_MESSAGE
|
PIPE_WAIT,
PIPE_UNLIMITED_INSTANCES,
BUFSIZE, BUFSIZE,
NMPWAIT_USE_DEFAULT_WAIT,
None
)
if (hPipe == INVALID_HANDLE_VALUE):
print "Error in creating Named Pipe"
return 0
fConnected = windll.kernel32.ConnectNamedPipe(hPipe, None)
if ((fConnected == 0) and (windll.kernel32.GetLastError() ==
ERROR_PIPE_CONNECTED)):
fConnected = 1
if (fConnected == 1):
dwThreadId = c_ulong(0)
hThread = windll.kernel32.CreateThread(None, 0,
thread_func, hPipe, 0, byref(dwThreadId))
if (hThread == -1):
print "Create Thread failed"
return 0
else:
windll.kernel32.CloseHandle(hThread)
else:
print "Could not connect to the Named Pipe"
windll.kernel32.CloseHandle(hPipe)
return 0
if __name__ == "__main__":
main()
After server have start you can use slightly modified version of the Java Client code:
try {
// Connect to the pipe
RandomAccessFile pipe = new RandomAccessFile("\\\\.\\pipe\\mynamedpipe", "rw");
String echoText = "Hello world\n";
// write to pipe
pipe.write(echoText.getBytes());
//String aChar;
StringBuffer fullString = new StringBuffer();
while(true){
int charCode = pipe.read();
if(charCode == 0) break;
//aChar = new Character((char)charCode).toString();
fullString.append((char)charCode);
}
System.out.println("Response: " + fullString);
pipe.close();
}
catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
It works well in NetBeans 6.9.1.

I suppose that RandomAccessFile is not the right API here. Try a FileInputStream + FileOutputStream on the Java side. But that is only a guess, as I last used the Windows API in times when named pipes didn't yet exist.

Don't worry, using RandomAccessFile to access a named pipe is correct. A named pipe is a file system object. Under Linux/Unix it is also called "fifo". Those objects are readable just like a file. (and not the same as pipes used between processes which are abstracted by Java Pipe class).
However I see two problems with your program. I cannot test it currently as I would need your test server (feel free to publish). Your reader thread waits for answers from the other side (i.e. the server). It uses readLine(), I would use a different method (for debugging reading char by char might be the best).
With Java (without JNI) you cannot actually create a named pipe (server side). Opening a named pipe with the generic method used by RandomAccessFile you will get a byte-type stream which can be one-way or duplex.
BTW: JTDS (the free JDBC driver for SQL Server) can optionally use a named pipe to access SQL server, even over the network. And it is using exactly the RandomAccessFile method.
BTW2: there is a makepipe.exe test server on older MS SQL Server installation media, however I did not find a trusted source to get that file.

I'm not familiar with JAVA, and my C# is pretty elementary too. However I'm had a similar problem with a multithreaded C++ client that I fixed by opening the pipe for overlapped IO. Until I did this, Windows serialized reads and writes, effectively causing an unsatisfied (blocking) ReadFile to prevent completion of a subsequent WriteFile until the read was done.
See CreateFile function
FILE_FLAG_OVERLAPPED

Java Gridgain application starts to fail after 1 day of stress testing

So I have a an application which is running on top of gridgain and does so quite successfully for about 12-24 hours of stress testing before it starts to act funny. After this period of time the application will suddenly start replying to all queries with the exception java.nio.channels.ClosedByInterruptException (full stack trace is at http://pastie.org/664717
The method that is failing from is (edited to use #stephenc feedback)
public static com.vlc.edge.FileChannel createChannel(final File file) {
FileChannel channel = null;
try {
channel = new FileInputStream(file).getChannel();
channel.position(0);
final com.vlc.edge.FileChannel fileChannel = new FileChannelImpl(channel);
channel = null;
return fileChannel;
} catch (FileNotFoundException e) {
throw new VlcRuntimeException("Failed to open file: " + file, e);
} catch (IOException e) {
throw new VlcRuntimeException(e);
} finally {
if (channel != null) {
try {
channel.close();
} catch (IOException e){
// noop
LOGGER.error("There was a problem closing the file: " + file);
}
}
}
}
and the calling function correctly closes the object
private void fillContactBuffer(final File signFile) {
contactBuffer = ByteBuffer.allocate((int) signFile.length());
final FileChannel channel = FileUtils.createChannel(signFile);
try {
channel.read(contactBuffer);
} finally {
channel.close();
}
contactBuffer.rewind();
}
The application basically serves as a distributed file parser so it does a lot of these types of operations (will typically open about 10 such channels per query per node). It seems that after a certain period it stops being able to open files and I'm at a loss to explain why this could be happening and would greatly appreciate any one who can tell me what could be causing this and how I could go about tracking it down and fixing it. If it is possibly related to file handle exhaustion, I'd love to hear any tips for finding out for sure... i.e. querying the JVM while it's running or using linux command line tools to find out more information about what handles are currently open.
update: I've been using command line tools to interrogate the output of lsof and haven't been able to see any evidence that file handles are being held open... each node in the grid has a very stable profile of openned files which I can see changing as the above code is executed... but it always returns to a stable number of open files.
Related to this question: Freeing java file handles

There are a couple of scenarios where file handles might not be being closed:
There might be some other code that opens files.
There might be some other bit of code that calls createChannel(...) and doesn't call fillContactBuffer(...)
If channel.position(0) throws an exception, the channel won't be closed. The fix is to rearrange the code so that the following statements are inside the try block.
channel.position(0);
return new FileChannelImpl(channel);
EDIT: Looking at the stack trace, it seems that the two methods are in different code-bases. I'd point the finger of blame at the createChannel method. It is potentially leaky, even if it is not the source of your problems. It needs an in internal finally clause to make sure that the channel is closed in the event of an exception.
Something like this should do the trick. Note that you need to make sure that the finally block does not closes the channel on success!
public static com.vlc.edge.FileChannel createChannel(final File file) {
final FileChannel channel = null;
try {
channel = new FileInputStream(file).getChannel();
channel.position(0);
FileChannel res = new FileChannelImpl(channel);
channel = null;
return res;
} catch (FileNotFoundException e) {
throw new VlcRuntimeException("Failed to open file: " + file, e);
} catch (IOException e) {
throw new VlcRuntimeException(e);
} finally {
if (channel != null) {
try {
channel.close();
} catch (...) {
...
}
}
}
}
FOLLOWUP much later
Given that file handle leakage has been eliminated as a possible cause, my next theory would be that the server side is actually interrupting its own threads using Thread.interrupt(). Some low-level I/O calls respond to an interrupt by throwing an exception, and the root exception being thrown here looks like one such exception.
This doesn't explain why this is happening, but at a wild guess I'd say that it was the server-side framework trying to resolve an overload or deadlock problem.