Develop Reference

ZeroMQ blocked in a context.term() call. Why? How to prevent?

I have a java program that using ZeroMQ.
But I found the program blocked in context.term(); if receiving a message( recvMsg() ) time out!
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket socket = context.socket(ZMQ.REQ);
socket.connect(mAddress);
ZMsg ZM = new ZMsg();
ZM.add(qString);
ZM.send(socket, true);
socket.setReceiveTimeOut(mTimeout);
ZMsg receivedZM = ZMsg.recvMsg(socket);
if(receivedZM != null) {
System.out.println(receivedZM.getFirst().toString());
}
socket.close();
context.term();
What is the reason cause it to blocked?
And how to solve this problem?

ZeroMQ is a system using many tricks behind the Context()-factory
I always advocate to automatically set .setsockopt( ZMQ_LINGER, 0 ) right upon a Socket-instantiation, right due to these types of behaviour, that otherwise remain outside of your local-code domain of control. A hanging Context-instance IO-thread(s) ( be it after a programmed .term() having been issued in spite of the not yet successful .close() of all socket-instances, instantiated under this Context-instance the .term() is to dismantle and release back all system resources from, or an unhandled exception case, when things just went straight wreck havoc ) is one of such never-more-s.
Feel free to follow schoolbook and online hacks/snippet examples, but a serious distributed system designer ought take all reasonable steps and measures so as to prevent her/his system code to fall into any deadlock-state ( the less into an un-salvageable one ).
What is the reason?
As documentation states - it is a designed-in feature of ZeroMQ:
attempting to terminate the socket's context with zmq_ctx_term() shall block until all pending messages have been sent to a peer.
Any case, where a .send()-dispatched ( just dispatched -- by no means meaning that it has already been sent-to-wire ) message is still inside the local-queue for any of the recognised ( and potentially disconnected or busy or ... ) peer-nodes, the just-default configured .term() cannot proceed and will block.
What is the solution:
Newer API versions started to say, a default LINGER value to stop being -1 == INFINITY, but as you never know, which version will your code interface with, an explicit ( manual ) call to a .setsockopt( ZMQ_LINGER, 0 ) method is a self-disciplining step and increases your team awareness on how to build reliable distributed-systems' code.
Using the try: / except: / finally: syntax-handlers is needless to be raised here. You simply always have to design with failures & collisions in mind, haven't you?

According to the API, http://api.zeromq.org/4-2:zmq-term, it will block when there's still messages to transmit. This suggests that you other machine or process, the one that will open the REP socket; isn't running.

Handling different exceptions of the same type in Java?

When handling errors in Java it's common to see the superclasses being the errors that are caugh, such as
Exception, IOException, SocketException, etc.
However how do you go about finding the nitty-gritty details on the exception? How do you single a certain exception type out from the others. For instance, I'm currently working on a small project using Netty.io which throws an IOException for every type of read/write error you can name. This makes sense, because ultimately this is input/output errors, but how would I handle them individually.
Example exceptions:
java.io.IOException: An existing connection was forcibly closed by the remote host
java.io.IOException: Connection reset by peer
java.io.IOException: Stream closed
The list just continues to go on, but how would you go about handling these seperately, one approach that I've found while looking around and seems really nasty is the following.
try {
// ...
} catch (IOException e) {
if(e.getMessage().contains("An existing connection was forcibly closed by the remote host")) {
// Handle error
} else //...
}
This seems very tedious and there's bound to be a better way to do this, a correct way if you will. I've looked through quite a bit of error handling writeups over the last few hours and they all only talk about the big boys that are used commonly. IOException, Exception, SocketException, NullPointerException, and FileNotFoundException. Where I believe SocketException and FileNotFoundException would be directly related to the IOException, more than likely a subclass, correct me if I'm wrong.
Anyway, what's the proper way to go about handling these exceptions and how do you figure out exactly what kind of exception you need to be handling? All I can really do is handle IOException until something more precise comes up, but when developing applications it's always good to be able to handle each error uniquely.

In most of these cases the message is irrelevant from the point of view of your code. It's just something to be shown to the user, or logged. The only salient fact is that the connection is broken, for whatever reason, and there aren't different code paths you can use depending on which message it was.
The only one that's different is 'socket closed', which indicates a coding bug.
EDIT Regarding your comments below:
Any IOException other than SocketTimeoutException on a socket is fatal to the connection.
Invalid packets don't cause IOException: that's an application-layer problem that throws application-layer exceptions, or subclasses of IOException: e.g., java.io.StreamCorruptedException.
There is no such thing as IOException: connection closed by remote host. If the peer closes the connection, that causes an end-of-stream condition, which manifests itself as either read() returning -1, readLine() returning null, or readXXX() throwing EOFException for any other X.

I would suggest catching the exceptions in order, from most specific to least - such that you will notice a circuit break pattern when the exception you are looking for is reached. This is the best I can come up with:
try {
/// Something that can result in IOException or a SocketException
catch (IOException e){
//Do something specific
}catch (SocketExcpetion e){
}catch (Exception e) { //or some superclass of the above exceptions
///
}
Don't forget that you can also catch multiple exceptions of different types using the | command: catch (IOException|SocketException|

The documentation (http://docs.oracle.com/javase/7/docs/api/java/io/IOException.html) contains a long list of direct subclasses. You might want to look through them and check which ones you want to treat differently.
Once you know that, you can use multiple catch-blocks, first the subclasses, then the most general IOException:
catch(SSLException se) {
// do something
}
catch(HttpRetryException he) {
// do something else
}
catch(IOException ioe) {
// nop
}

'ServerSocket.accept()' Loop Without SocketTimeoutException (Java) (Alternative Solution)

Explanation
I'm revisiting the project I used to teach myself Java.
In this project I want to be able to stop the server from accepting new clients and then perform a few 'cleanup' operations before exiting the JVM.
In that project I used the following style for a client accept/handle loop:
//Exit loop by changing running to false and waiting up to 2 seconds
ServerSocket serverSocket = new ServerSocket(123);
serverSocket.setSoTimeout(2000);
Socket client;
while (running){ // 'running' is a private static boolean
try{
client = serverSocket.accept();
createComms(client); //Handles Connection in New Thread
} catch (IOException ex){
//Do Nothing
}
}
In this approach a SocketTimeoutException will be thrown every 2 seconds, if there are no clients connecting, and I don't like relying on exceptions for normal operation unless it's necessary.
I've been experimenting with the following style to try and minimise relying on Exceptions for normal operation:
//Exit loop by calling serverSocket.close()
ServerSocket serverSocket = new ServerSocket(123);
Socket client;
try{
while ((client = serverSocket.accept()) != null){
createComms(client); //Handles Connection in New Thread
}
} catch (IOException ex){
//Do Nothing
}
In this case my intention is that an Exception will only be thrown when I call serverSocket.close() or if something goes wrong.
Question
Is there any significant difference in the two approaches, or are they both viable solutions?
I'm totally self-taught so I have no idea if I've re-invented the wheel for no reason or if I've come up something good.
I've been lurking on SO for a while, this is the first time I've not been able to find what I need already.
Please feel free to suggest completely different approaches =3

The problem with second approach is that the server will die if an exception occurs in the while loop.
The first approach is better, though you might want to add logging exceptions using Log4j.
while (running){
try{
client = serverSocket.accept();
createComms(client);
} catch (IOException ex){
// Log errors
LOG.warn(ex,ex);
}
}

Non-blocking IO is what you're looking for. Instead of blocking until a SocketChannel (non-blocking alternative to Socket) is returned, it'll return null if there is currently no connection to accept.
This will allow you to remove the timeout, since nothing will be blocking.
You could also register a Selector, which informs you when there is a connection to accept or when there is data to read. I have a small example of that here, as well as a non-blocking ServerSocket that doesnt use a selector
EDIT: In case something goes wrong with my link, here is the example of non-blocking IO, without a selector, accepting a connection:
class Server {
public static void main(String[] args) throws Exception {
ServerSocketChannel ssc = ServerSocketChannel.open();
ssc.configureBlocking(false);
while(true) {
SocketChannel sc = ssc.accept();
if(sc != null) {
//handle channel
}
}
}
}

The second approach is better (for the reasons you mentioned: relying on exceptions in normal program flow is not a good practise) allthough your code suggests that serverSocket.accept() can return null, which it can not. The method can throw all kinds of exceptions though (see the api-docs). You might want to catch those exceptions: a server should not go down without a very good reason.
I have been using the second approach with good success, but added some more code to make it more stable/reliable: see my take on it here (unit tests here). One of the 'cleanup' tasks to consider is to give some time to the threads that are handling the client communications so that these threads can finish or properly inform the client the connection will be closed. This prevents situations where the client is not sure if the server completed an important task before the connection was suddenly lost/closed.

Issues receiving in RXTX

I've been using RXTX for about a year now, without too many problems. I just started a new program to interact with a new piece of hardware, so I reused the connect() method I've used on my other projects, but I have a weird problem I've never seen before.
The Problem
The device works fine, because when I connect with HyperTerminal, I send things and receive what I expect, and Serial Port Monitor(SPM) reflects this.
However, when I run the simple HyperTerminal-clone I wrote to diagnose the problem I'm having with my main app, bytes are sent, according to SPM, but nothing is received, and my SerialPortEventListener never fires. Even when I check for available data in the main loop, reader.ready() returns false. If I ignore this check, then I get an exception, details below.
Relevant section of connect() method
// Configure and open port
port = (SerialPort) CommPortIdentifier.getPortIdentifier(name)
.open(owner,1000)
port.setSerialPortParams(baud, databits, stopbits, parity);
port.setFlowControlMode(fc_mode);
final BufferedReader br = new BufferedReader(
new InputStreamReader(
port.getInputStream(),
"US-ASCII"));
// Add listener to print received characters to screen
port.addEventListener(new SerialPortEventListener(){
public void serialEvent(SerialPortEvent ev) {
try {
System.out.println("Received: "+br.readLine());
} catch (IOException e) { e.printStackTrace(); }
}
});
port.notifyOnDataAvailable();
Exception
java.io.IOException: Underlying input stream returned zero bytes
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:268)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:306)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:158)
at java.io.InputStreamReader.read(InputStreamReader.java:167)
at java.io.BufferedReader.fill(BufferedReader.java:136)
at java.io.BufferedReader.read(BufferedReader.java:157)
at <my code>
The big question (again)
I think I've eliminated all possible hardware problems, so what could be wrong with my code, or the RXTX library?
Edit: something interesting
When I open HyperTerminal after sending a bunch of commands from java that should have gotten responses, all of the responses appear immediately, as if they had been put in the buffer somewhere, but unavailable.
Edit 2: Tried something new, same results
I ran the code example found here, with the same results. No data came in, but when I switched to a new program, it came all at once.
Edit 3
The hardware is fine, and even a different computer has the same problem. I am not using any sort of USB adapter.
I've started using PortMon, too, and it's giving me some interesting results. HyperTerminal and RXTX are not using the same settings, and RXTX always polls the port, unlike HyperTerminal, but I still can't see what settings would affect this. As soon as I can isolate the configuration from the constant polling, I'll post my PortMon logs.
Edit 4
Is it possible that some sort of Windows update in the last 3 months could have caused this? It has screwed up one of my MATLAB mex-based programs once.
Edit 5
I've also noticed some things that are different between HyperTerminal, RXTX, and a separate program I found that communicates with the device (but doesn't do what I want, which is why I'm rolling my own program)
HyperTerminal - set to no flow control, but Serial Port Monitor's RTS and DTR indicators are green
Other program - not sure what settings it thinks it's using, but only SPM's RTS indicator is green
RXTX - no matter what flow control I set, only SPM's CTS and DTR indicators are on.
From Serial Port Monitor's help files (paraphrased):
the indicators display the state of the serial control lines
RTS - Request To Send
CTS - Clear To Send
DTR - Data Terminal Ready

OK, sorry it's taken me so long to come back to this question. Here's how I got things working.
Note: This method will NOT work for everyone, please read below before copy/pasting into your own code
public void connect(CommPortIdentifier portId) throws Failure {
if (portId == null)
throw new Failure("No port set");
try { port = (SerialPort) portId.open(getClass().getName(), 10000); }
catch (PortInUseException e) {
throw new Failure("Port in use by " + e.currentOwner,e); }
try {
port.setSerialPortParams(9600, SerialPort.DATABITS_8,
SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);
port.setFlowControlMode(SerialPort.FLOWCONTROL_RTSCTS_IN
| SerialPort.FLOWCONTROL_RTSCTS_OUT);
} catch (UnsupportedCommOperationException e) { throw new Failure(e); }
port.setRTS(true);
// More setup
}
So, in my case, the problem was that my particular device requires RTS flow control. Other devices may require different things (CTS, XON/XOFF), so check that device's manual. By default, RXTX disables all flow control mechanisms (unlike Hypertrm or other programs). Enabling each one is a two-step process.
Once you have a SerialPort object, call the setFlowControlMode() method, and bitwise-OR ('|') the necessary SerialPort.FLOWCONTROL_ constants
Set the appropriate flow control to true or false (like I did with port.setRTS(true))
For the others with similar problems, if this doesn't work, I suggest
Using a serial port monitoring program like Serial Port Monitor and/or PortMon (both Windows) to see what is actually going on.
Emailing the RXTX developers at rxtx#qbang.org (they are very helpful)

There is a simpler solution to this problem. This is what I did:
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String line;
while (keepRunning) {
try {
while ((br.ready()) && (line = br.readLine()) != null) {
....
}
If you check that the buffer "is ready" before you read it there should be no problem.

Ok, I do realize this thread is extremely old, but none of these solutions worked for me. I had the same problem and I tried everything to fix it, to no avail. Then I did some research on what causes the problem, and, when not dealing with Serial Communication, it happens at the end of a file. So, I figured I needed to add an ending to whatever is being received by the Java Application, specifically, a line return (\n). And sure enough, it fixed the problem for me! Hopefully this helps someone new, as I'm not expecting this to help anyone already on this thread...

(might be too simple, but might as well start somewhere...)
Is the port in use? Rather than:
port = (SerialPort) CommPortIdentifier.getPortIdentifier(name)
.open(owner,1000)
what about:
CommPortIdentifier portIdentifier;
try {
portIdentifier = CommPortIdentifier.getPortIdentifier(name);
} catch (NoSuchPortException nspe) {
// handle?
}
if (portIdentifier.isCurrentlyOwned()) {
// handle?
}
port = portIdentifier.open(owner, 1000);
if (!(port instanceof SerialPort)) {
// handle?
}
Are you swallowing any exceptions?

I tried RXTX a few months ago and ran into similar problems. I suggest two things:
Create a virtual comport using com0com. Enable trace logging. Compare the logs for when you use Hyperterminal versus when you run your own program. The difference will highlight what you are doing wrong.
In my humble opinion, RXTX's design is flawed and its implementation is quite buggy (take a look at its source-code, what a mess!). I've published an alternative library at http://kenai.com/projects/jperipheral with the following caveats: It's Windows-only and there are no pre-built binaries. Both of these will change in the near future. If you are interested in trying it out send me an email using http://desktopbeautifier.com/Main/contactus and I'll send you a pre-built version.

If anyone is still getting java.io.IOException: Underlying input stream returned zero bytes after you've read your characters using br.readline() for RXTX (even when you are checking first to see if br.readline() == null), just do this simple fix with a try/catch:
String line;
while (true){
try{
line = br.readLine();
}catch(IOException e){
System.out.println("No more characters received");
break;
}
//Print the line read
if (line.length() != 0)
System.out.println(line);
}
I've done some searching and it appears that this is the best/easiest way to get around this problem.
EDIT : I take that back. I tried this and still ended up having some problems. I'd recommend working with the raw InputStream directly, and implementing your own read/readLine method using InputStream.read(). That worked for me.

BindException: address already in use on a client socket?

I've got a client-server tiered architecture with the client making RPC-like requests to the server. I'm using Tomcat to host the servlets, and the Apache HttpClient to make requests to it.
My code goes something like this:
private static final HttpConnectionManager CONN_MGR = new MultiThreadedHttpConnectionManager();
final GetMethod get = new GetMethod();
final HttpClient httpClient = new HttpClient(CONN_MGR);
get.getParams().setCookiePolicy(CookiePolicy.IGNORE_COOKIES);
get.getParams().setParameter(HttpMethodParams.USER_AGENT, USER_AGENT);
get.setQueryString(encodedParams);
int responseCode;
try {
responseCode = httpClient.executeMethod(get);
} catch (final IOException e) {
...
}
if (responseCode != 200)
throw new Exception(...);
String responseHTML;
try {
responseHTML = get.getResponseBodyAsString(100*1024*1024);
} catch (final IOException e) {
...
}
return responseHTML;
It works great in a lightly-loaded environment, but when I'm making hundreds of requests per second I start to see this -
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:336)
at java.net.Socket.bind(Socket.java:588)
at java.net.Socket.<init>(Socket.java:387)
at java.net.Socket.<init>(Socket.java:263)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
Any thoughts on how to fix this? I'm guessing it's something to do with the client trying to reuse the ephemeral client ports, but why is this happening / how can I fix it?
Thanks!

A very good discussion of the problem you are running into can be found here. On the Tomcat side, by default it will use the SO_REUSEADDR option, which will allow the server to reuse sockets which are in TIME_WAIT. Additionally, the Apache http client will by default use keep-alives, and attempt to reuse connections.
Your problems seems to be caused by not calling releaseConnection on the HttpClient. This is required in order for the connection to be reused. Otherwise, the connection will remain open until garbage collector comes and closes it, or the server disconnects the keep-alive. In both cases, it won't be returned to the pool.

With hundreds of connections a second, and without knowing how long your connections keep to open, do their thing, close, and get recycled, I suspect that this is just a problem you're going to have. One thing you can do is catch the BindException in your try block, use that to do anything you need to do in the bind-unsuccessful case, and wrap the whole call in a while loop that depends on a flag indicating whether the bind succeeded. Off the top of my head:
boolean hasBound = false;
while (!hasBound) {
try {
hasBound = true;
responseCode = httpClient.executeMethod(get);
} catch (BindException e) {
// do anything you want in the bound-unsuccessful case
} catch (final IOException e) {
...
}
}
Update with question: One curious question: what are the maximum total and per-host number of connections allowed by your MultiThreadedHttpConnectionManager? In your code, that'd be:
CONN_MGR.getParams().getDefaultMaxConnectionsPerHost();
CONN_MGR.getParams().getMaxTotalConnections();

Thus, you've fired more requests than TCP/IP ports are allowed to be opened. I don't do HttpClient, so I can't go in detail about this, but in theory there are three solutions for this particular problem:
Hardware based: add another NIC (network interface card).
Software based: close connections directly after use and/or increase the connection timeout.
Platform based: increase the amount of TCP/IP ports which are allowed to be opened. May be OS-specific and/or NIC driver-specific. The absolute maximum is 65535, of which several may already be reserved/in use (e.g. port 80).

So it turns out the problem was that one of the other HttpClient instances accidentally wasn't using the MultiThreadedHttpConnectionManager I instantiated, so I effectively had no rate limiting at all. Fixing this problem fixed the exception being thrown.
Thanks for all the suggestions, though!

Even though we invoke HttpClientUtils.closeQuietly(client); but in your code in case trying to read the content from HttpResponse entity like InputStream contentStream = HttpResponse.getEntity().getContent(), then you should close the inputstream also then only HttpClient connection get closed properly.