Develop Reference

Problem at creating EMS application supporting Failover/FaultTolerance

I am starting to study how can I implement an application supporting Failover/FaultTolerance on top of JMS, more precisely EMS
I configured two EMS servers working both with FaultTolerance enabled:
For EMS running on server on server1 I have
in tibemsd.conf
ft_active = tcp://server2:7232
in factories.conf
[GenericConnectionFactory]
type = generic
url = tcp://server1:7232
[FTTopicConnectionFactory]
type = topic
url = tcp://server1:7232,tcp://server2:7232
[FTQueueConnectionFactory]
type = queue
url = tcp://server1:7232,tcp://server2:7232
And for EMS running on server on server2 I have
in tibemsd.conf
ft_active = tcp://server1:7232
in factories.conf
[GenericConnectionFactory]
type = generic
url = tcp://server2:7232
[FTTopicConnectionFactory]
type = topic
url = tcp://server2:7232,tcp://server1:7232
[FTQueueConnectionFactory]
type = queue
url = tcp://server2:7232,tcp://server1:7232
I am not a TIBCO EMS expert but my config seems to be good: When I start EMS on server1 I get:
$ tibemsd -config tibemsd.conf
...
2022-07-20 23:04:58.566 Server is active.
2022-07-20 23:05:18.563 Standby server 'SERVERNAME#server1' has connected.
then if I start EMS on server2, I get
$ tibemsd -config tibemsd.conf
...
2022-07-20 23:05:18.564 Accepting connections on tcp://server2:7232.
2022-07-20 23:05:18.564 Server is in standby state for 'tcp://server1:7232'
Moreover, if I kill active EMS on server1, I immediately get the following message on server2:
2022-07-20 23:21:52.891 Connection to active server 'tcp://server1:7232' has been lost.
2022-07-20 23:21:52.891 Server activating on failure of 'tcp://server1:7232'.
...
2022-07-20 23:21:52.924 Server is now active.
Until here, everything looks OK, active/standby EMS servers seems to be correctly configured
Things get more complicated when I write a piece of code how is supposed to connect to these EMS servers and to periodically publish messages. Let's try with the following code sample:
#Test
public void testEmsFailover() throws JMSException, InterruptedException {
int NB = 1000;
TibjmsConnectionFactory factory = new TibjmsConnectionFactory();
factory.setServerUrl("tcp://server1:7232,tcp://server2:7232");
Connection connection = factory.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
connection.start();
for (int i = 0; i < NB; i++) {
LOG.info("sending message");
Queue queue = session.createQueue(QUEUE__CLIENT_TO_FRONTDOOR__CONNECTION_REQUEST);
MessageProducer producer = session.createProducer(queue);
MapMessage mapMessage = session.createMapMessage();
mapMessage.setStringProperty(PROPERTY__CLIENT_KIND, USER.toString());
mapMessage.setStringProperty(PROPERTY__CLIENT_NAME, "name");
producer.send(mapMessage);
LOG.info("done!");
Thread.sleep(1000);
}
}
If I run this code while both active and standby servers are up, everything looks good
23:26:32.431 [main] INFO JmsEndpointTest - sending message
23:26:32.458 [main] INFO JmsEndpointTest - done!
23:26:33.458 [main] INFO JmsEndpointTest - sending message
23:26:33.482 [main] INFO JmsEndpointTest - done!
Now If I kill the active EMS server, I would expect that
the standby server would instantaneously become the active one
my code would continue to publish such as if nothing had happened
However, in my code I get the following error:
javax.jms.JMSException: Connection is closed
at com.tibco.tibjms.TibjmsxLink.sendRequest(TibjmsxLink.java:307)
at com.tibco.tibjms.TibjmsxLink.sendRequestMsg(TibjmsxLink.java:261)
at com.tibco.tibjms.TibjmsxSessionImp._createProducer(TibjmsxSessionImp.java:1004)
at com.tibco.tibjms.TibjmsxSessionImp.createProducer(TibjmsxSessionImp.java:4854)
at JmsEndpointTest.testEmsFailover(JmsEndpointTest.java:103)
...
and in the logs of the server (the previous standby server supposed to be now the active one) I get
2022-07-20 23:32:44.447 [anonymous#cersei]: connect failed: server not in active state
2022-07-20 23:33:02.969 Connection to active server 'tcp://server2:7232' has been lost.
2022-07-20 23:33:02.969 Server activating on failure of 'tcp://server2:7232'.
2022-07-20 23:33:02.969 Server rereading configuration.
2022-07-20 23:33:02.971 Recovering state, please wait.
2022-07-20 23:33:02.980 Recovered 46 messages.
2022-07-20 23:33:02.980 Server is now active.
2022-07-20 23:33:03.545 [anonymous#cersei]: reconnect failed: connection unknown for id=8
2022-07-20 23:33:04.187 [anonymous#cersei]: reconnect failed: connection unknown for id=8
2022-07-20 23:33:04.855 [anonymous#cersei]: reconnect failed: connection unknown for id=8
2022-07-20 23:33:05.531 [anonymous#cersei]: reconnect failed: connection unknown for id=8
I would appreciate any help to enhance my code
Thank you

I think I found the origin of my problem:
according to the page Tibco-Ems Failover Issue, the error message
reconnect failed: connection unknown for id=8
means: "the store (ems db) was'nt share between the active and the standby node, so when the active ems failed, the new active ems was'nt able to recover connections and messages."
I realized that it is painful to configure a shared store. To avoid it, I configured two tibems on the same host, by following the page Step By Step How to Setup TIBCO EMS In Fault Tolerant Mode:
two tibemsd.conf configuration files
configure a different listen port in each file
configure ft_active with url of other server
configure factories.conf
By doing so, I can replay my test and it works as expected

Customize the automatic reconnection settings to IBM MQ

I have written a code to connect to IBM MQ, and i am using ConnectionNameList which automatically reconnects to ibm mq.
I want to customize the reconnection which is happening implicitly. I have referred many articles on the internet but i am not able to figure it out.
This is my Queue Manager Config:
#Configuration
public class QM1Config{
public String queueManager;
public String queue;
public String channel;
public String connName;
public String user;
public String password;
private static final int RECONNECT_TIMEOUT = 10;
#Autowired
MQService config;
#Bean
public MQConnectionFactory mqQueueConnectionFactory() {
this.channel = config.getHosts().get(0).getChannel();
this.user = config.getHosts().get(0).getUser();
this.password = config.getHosts().get(0).getPassword();
this.queueManager = config.getHosts().get(0).getQueueManager();
this.queue = config.getHosts().get(0).getQueue();
this.connName = config.getHosts().get(0).getConnName();
System.out.println(channel+" "+connName+" "+queueManager+" "+user);
MQConnectionFactory mqQueueConnectionFactory = new MQConnectionFactory();
try {
mqQueueConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
mqQueueConnectionFactory.setBooleanProperty(WMQConstants.USER_AUTHENTICATION_MQCSP, false);
mqQueueConnectionFactory.setCCSID(1208);
mqQueueConnectionFactory.setChannel(channel);
mqQueueConnectionFactory.setStringProperty(WMQConstants.USERID, user);
mqQueueConnectionFactory.setStringProperty(WMQConstants.PASSWORD, password);
mqQueueConnectionFactory.setQueueManager(queueManager);
mqQueueConnectionFactory.setConnectionNameList(connName);
} catch (Exception e) {
e.printStackTrace();
}
return mqQueueConnectionFactory;
}
#Bean
public JmsListenerContainerFactory<?> qm1JmsListenerContainerFactory(#Qualifier("mqQueueConnectionFactory") MQConnectionFactory mqQueueConnectionFactory, DefaultJmsListenerContainerFactoryConfigurer configurer) throws InterruptedException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
this.queue = config.getHosts().get(0).getQueue();
configurer.configure(factory, mqQueueConnectionFactory);
return factory;
}
#Bean("jmsTemplate1")
public JmsTemplate jmsTemplate(#Qualifier("mqQueueConnectionFactory") MQConnectionFactory mqQueueConnectionFactory) {
JmsTemplate jmsTemplate1 = new JmsTemplate(mqQueueConnectionFactory);
return jmsTemplate1;
}
}
When i stop the queue manager, i get the following exceptions every 5 seconds:
2022-04-24 01:17:43.194 WARN 6644 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'Q5' - trying to recover. Cause: JMSWMQ2002: Failed to get a message from destination 'Q5'.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
2022-04-24 01:17:43.232 ERROR 6644 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'Q5' - retrying using FixedBackOff{interval=5000, currentAttempts=0, maxAttempts=unlimited}. Cause: JMSWMQ0018: Failed to connect to queue manager 'QM5' with connection mode 'Client' and host name 'Client'.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2059' ('MQRC_Q_MGR_NOT_AVAILABLE').
2022-04-24 01:17:48.243 ERROR 6644 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'Q5' - retrying using FixedBackOff{interval=5000, currentAttempts=1, maxAttempts=unlimited}. Cause: JMSWMQ0018: Failed to connect to queue manager 'QM5' with connection mode 'Client' and host name 'Client'.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
2022-04-24 01:17:53.245 ERROR 6644 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'Q5' - retrying using FixedBackOff{interval=5000, currentAttempts=2, maxAttempts=unlimited}. Cause: JMSWMQ0018: Failed to connect to queue manager 'QM5' with connection mode 'Client' and host name 'Client'.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
2022-04-24 01:17:58.250 ERROR 6644 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'Q5' - retrying using FixedBackOff{interval=5000, currentAttempts=3, maxAttempts=unlimited}. Cause: JMSWMQ0018: Failed to connect to queue manager 'QM5' with connection mode 'Client' and host name 'Client'.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
So i want that first 3 reconnection attempts should be a warn messages instead of error message as shown in above logs and 4th attempt onwards i want it to be an error message. and reconnection attempt to be every 10/15 seconds.
How do i configure these re-connection settings ?
Any help would be greatly appreciated ! Thanks !
EDIT: I have added an exception listener as follows:
public class MQExceptionListener implements ExceptionListener {
private static final Logger LOGGER = LoggerFactory.getLogger(MQExceptionListener.class);
int count = -1;
#Override
public void onException(JMSException ex) {
if(count > 2) {
System.out.println("COUNT - "+ count);
count++;
LOGGER.error("***********************************************");
LOGGER.error(ex.toString()+" THIS IS EX TO STRING");
if (ex.getLinkedException() != null) {
LOGGER.error(ex.getLinkedException().toString()+" THIS IS getLinkedException TO STRING");
}
LOGGER.error("================================================");
}else {
System.out.println("COUNT - "+ count);
count++;
LOGGER.warn("***********************************************");
LOGGER.warn(ex.toString()+" THIS IS EX TO STRING");
if (ex.getLinkedException() != null) {
LOGGER.warn(ex.getLinkedException().toString()+" THIS IS getLinkedException TO STRING");
}
LOGGER.warn("================================================");
}
}
}
Now My logs are as follows:
COUNT - 1
2022-04-24 14:41:04.905 WARN 9268 --- [enerContainer-1] com.mq.sslMQ.MQExceptionListener : ***********************************************
2022-04-24 14:41:04.905 WARN 9268 --- [enerContainer-1] com.mq.sslMQ.MQExceptionListener : com.ibm.msg.client.jms.DetailedIllegalStateException: JMSWMQ0018: Failed to connect to queue manager 'QM5' with connection mode 'Client' and host name 'Client'.
Check the queue manager is started and if running in client mode, check there is a listener running. Please see the linked exception for more information. THIS IS EX TO STRING
2022-04-24 14:41:04.905 WARN 9268 --- [enerContainer-1] com.mq.sslMQ.MQExceptionListener : com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE'). THIS IS getLinkedException TO STRING
2022-04-24 14:41:04.905 WARN 9268 --- [enerContainer-1] com.mq.sslMQ.MQExceptionListener : ================================================
2022-04-24 14:41:04.905 ERROR 9268 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'Q5' - retrying using FixedBackOff{interval=5000, currentAttempts=1, maxAttempts=unlimited}. Cause: JMSWMQ0018: Failed to connect to queue manager 'QM5' with connection mode 'Client' and host name 'Client'.; nested exception is com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
I dont want the default message listener container log to be printed onto console. how do we achieve that ?

It says in IBM Docs:-
By default, the reconnection attempts happen at the following intervals:
The first attempt is made after an initial delay of 1 second, plus a random element up to 250 milliseconds.
The second attempt is made 2 seconds, plus a random interval of up to 500 milliseconds, after the first attempt fails.
The third attempt is made 4 seconds, plus a random interval of up to 1 second, after the second attempt fails.
The fourth attempt is made 8 seconds, plus a random interval of up to 2 seconds, after the third attempt fails.
The fifth attempt is made 16 seconds, plus a random interval of up to 4 seconds, after the fourth attempt fails.
The sixth attempt, and all subsequent attempts are made 25 seconds, plus a random interval of up to 6 seconds and 250 milliseconds after the previous attempt fails.
The reconnection attempts are delayed by intervals that are partly fixed and partly random. This is to prevent all of the IBM MQ classes for JMS applications that were connected to a queue manager that is no longer available from reconnecting simultaneously.
If you need to increase the default values, to more accurately reflect the amount of time that is required for a queue manager to recover, or a standby queue manager to become active, modify the ReconDelay attribute in the Channel stanza of the client configuration file.
You can read more about this attribute here.
Sounds like you need to put the following into your mqclient.ini file.
CHANNELS:
ReconDelay=(10000,5000)
That is requesting a delay of 10 seconds plus a random element up to 5 seconds, which is my interpretation of your request for 10/15 seconds. You haven't asked for any of the reconnection attempts to be different in timing, although you can do this if you need to.
Note, it is not possible to change the WARN/ERROR status of the messages.
Remember that you can always turn off automatic reconnect and implement whatever you need yourself by catching the connection failures in your application. Automatic reconnect was designed for applications that were unable (or unwilling) to catch the connection failures.

I would add that the 5 second interval reconnect attempts are the DefaultMessageListenerContainer trying to reconnect. The default reconnect interval is 5 seconds - DEFAULT_RECOVERY_INTERVAL, so I'm not thinking that this involves the MQ reconnect mechanism.
With the exception handler listed above in place, you could programatically change the DefaultMessageListenerContainer setRecoveryInterval() or use setBackOff() to control the backoff interval.
As to disabling the logging, setting the log level for the DefaultMessageListenerContainer to FATAL should do it.

I/O timeout exception (java.net.ConnectException) When calling API

I'm using HttpClient in order to call an API and get its response, and I have set a 60 secs timeout. In this 60 seconds, Java try and retry to connect to the API by each 20 seconds and shows the exception. In the final of 60 secs, it stops retrying.
My doubt is: this I/O Exception is caused by the API? Since I set a timeout higher than the exception returns (each 20 seconds).
Here's the code and the log:
HttpClient client = new HttpClient();
GetMethod getMethod = new GetMethod(GET_TICKETS_URL);
getMethod.setRequestHeader("Content-Type", "application/json");
getMethod.setRequestHeader("Accept", "application/json");
getMethod.getParams().setSoTimeout(60000);
logger.info("Calling service: " + getMethod.getPath());
client.getHttpConnectionManager().getParams().setConnectionTimeout(60000);
client.getHttpConnectionManager().getParams().setSoTimeout(60000);
client.getParams().setSoTimeout(60000);
client.getParams().setConnectionManagerTimeout(60000);
int getTicketsResponse = client.executeMethod(getMethod);
[2020-08-27 13:41:25,215] pool-3-thread-1 br.com.pfm.tasks.baml.task Task INFO - Calling service: /
[2020-08-27 13:41:46,228] pool-3-thread-1 org.apache.commons.httpclient.HttpMethodDirector INFO - I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
[2020-08-27 13:41:46,228] pool-3-thread-1 org.apache.commons.httpclient.HttpMethodDirector INFO - Retrying request
[2020-08-27 13:42:07,242] pool-3-thread-1 org.apache.commons.httpclient.HttpMethodDirector INFO - I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
[2020-08-27 13:42:07,242] pool-3-thread-1 org.apache.commons.httpclient.HttpMethodDirector INFO - Retrying request
[2020-08-27 13:42:28,258] pool-3-thread-1 org.apache.commons.httpclient.HttpMethodDirector INFO - I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
[2020-08-27 13:42:28,258] pool-3-thread-1 org.apache.commons.httpclient.HttpMethodDirector INFO - Retrying request
[2020-08-27 13:42:49,271] pool-3-thread-1 br.com.pfm.tasks.baml.task ERROR - java.net.ConnectException: Connection timed out: connect

I/O exception is occur when there is an issue in reading from the url, can you able to get the stack stacktrace, will give an insight about the issue, it could be the server unavailable or not found anything from http 400+
so please add this to your catch block and will help you get an idea, what went wrong.
catch(Exception e)
{
e.printStacktrace();
}

Based on the somewhat incomplete information presented here, a stack trace will likely reveal the point of failure is where you try to kick off the HTTP transaction (actually tell it to go a head and connect).
Assuming that you've set everything up correctly in GetMethod (and I cant tell), may i recomend that you check that the target you are trying to reach, is actually reachable from the machine? try a wget or if its on a desk top try to access it with your browser. If that works, its likely you set things up incorrectly in your GetMethod code - posting that would be very helpful to troubleshoot further.

Qpid Java client exception: java.lang.IllegalArgumentException: unknown code: 105

I'm using simple qpid java client which consumes messages from broker and sends it to through SOAP service. On the producer side we put all data to map and then send this map to qpid. Here is a snippet:
QueueSender conBusQueueSender = (QueueSender) Component.getInstance("conBusQueueSender");
QueueSession queueSession = org.jboss.seam.jms.QueueSession.instance();
Map<String,Object> map = new HashMap<String,Object>();
map.put("applicationId", applicationId);
map.put("soapAction", "urn:changeApplicationStatus");
map.put("soapXML", changeApplicationStatusString);
MapMessage message = queueSession.createMapMessage();
message.setObject("map",map);
conBusQueueSender.send(message);
On the client side we receive message and trying to send it to web service through SOAP
while (true) {
MapMessage m = (MapMessage) consumer.receive();
#SuppressWarnings("unchecked")
Map<String, Object> map = (Map<String, Object>) m.getObject("map");
SOAPMessage message = createSoapMessage(map);
while (true) {
try {
SOAPMessage result = makeSoapCall(message, (String)map.get("applicationId"), (String)map.get("soapAction"));
if(result.getSOAPBody().hasFault()){
System.out.println("SOAPFault: "+result.getSOAPBody().getFault().getFaultString());
System.out.println("SOAPMessage tried to send: " + (String)map.get("soapXML"));
}
m.acknowledge();
// if we came here successfully -> break and process next message
break;
} catch (SOAPException e) {
// if we have exception -> sleep and retry
e.printStackTrace();
Thread.sleep(1000L);
continue;
}
}
}
Everything works fine but when consumer tries to send a little bigger message ~66KB it just prints this error:
73 [main] INFO org.apache.qpid.client.security.DynamicSaslRegistrar - Additional SASL providers successfully registered.
87 [main] INFO org.apache.qpid.client.AMQConnection - Connection:amqp://guest:********#localhost/?brokerlist='tcp://localhost:5672'
314 [main] INFO org.apache.qpid.client.protocol.AMQProtocolSession - Using ProtocolVersion for Session:0-10
330 [main] INFO org.apache.qpid.client.handler.ClientMethodDispatcherImpl - New Method Dispatcher:AMQProtocolSession[null]
341 [main] INFO org.apache.qpid.client.AMQConnection - Connecting with ProtocolHandler Version:0-10
414 [IoReceiver - localhost/127.0.0.1:5672] INFO org.apache.qpid.transport.ClientDelegate - The broker does not support the configured connection idle timeout of 120 sec, using the brokers max supported value of 0 sec instead.
420 [main] INFO org.apache.qpid.client.AMQConnection - Connected with ProtocolHandler Version:0-10
Connection established to amqp://guest:guest#localhost/?brokerlist='tcp://localhost:5672'
443 [main] INFO org.apache.qpid.client.AMQSession - Created session:org.apache.qpid.client.AMQSession_0_10#112c3327
Session created...
476 [main] INFO org.apache.qpid.client.AMQSession - Prefetching delayed existing messages will not flow until requested via receive*() or setML().
Consumer initialized...
481 [main] INFO org.apache.qpid.client.AMQSession.Dispatcher - Dispatcher-Channel-0 created
481 [Dispatcher-Channel-0] INFO org.apache.qpid.client.AMQSession.Dispatcher - Dispatcher-Channel-0 started
492 [Dispatcher-Channel-0] ERROR org.apache.qpid.client.BasicMessageConsumer - Caught exception (dump follows) - ignoring...
java.lang.IllegalArgumentException: unknown code: 105
at org.apache.qpid.transport.codec.AbstractDecoder.getType(AbstractDecoder.java:354)
at org.apache.qpid.transport.codec.AbstractDecoder.readMap(AbstractDecoder.java:287)
at org.apache.qpid.transport.codec.BBDecoder.readMap(BBDecoder.java:34)
at org.apache.qpid.transport.codec.AbstractDecoder.read(AbstractDecoder.java:455)
at org.apache.qpid.transport.codec.AbstractDecoder.readMap(AbstractDecoder.java:288)
at org.apache.qpid.transport.codec.BBDecoder.readMap(BBDecoder.java:34)
at org.apache.qpid.client.message.AMQPEncodedMapMessage.populateMapFromData(AMQPEncodedMapMessage.java:96)
at org.apache.qpid.client.message.JMSMapMessage.<init>(JMSMapMessage.java:71)
at org.apache.qpid.client.message.AMQPEncodedMapMessage.<init>(AMQPEncodedMapMessage.java:52)
at org.apache.qpid.client.message.AMQPEncodedMapMessageFactory.createMessage(AMQPEncodedMapMessageFactory.java:36)
at org.apache.qpid.client.message.AbstractJMSMessageFactory.create010MessageWithBody(AbstractJMSMessageFactory.java:135)
at org.apache.qpid.client.message.AbstractJMSMessageFactory.createMessage(AbstractJMSMessageFactory.java:166)
at org.apache.qpid.client.message.MessageFactoryRegistry.createMessage(MessageFactoryRegistry.java:150)
at org.apache.qpid.client.BasicMessageConsumer_0_10.createJMSMessageFromUnprocessedMessage(BasicMessageConsumer_0_10.java:221)
at org.apache.qpid.client.BasicMessageConsumer_0_10.createJMSMessageFromUnprocessedMessage(BasicMessageConsumer_0_10.java:47)
at org.apache.qpid.client.BasicMessageConsumer.notifyMessage(BasicMessageConsumer.java:693)
at org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:205)
at org.apache.qpid.client.BasicMessageConsumer_0_10.notifyMessage(BasicMessageConsumer_0_10.java:47)
at org.apache.qpid.client.AMQSession$Dispatcher.notifyConsumer(AMQSession.java:3392)
at org.apache.qpid.client.AMQSession$Dispatcher.dispatchMessage(AMQSession.java:3336)
at org.apache.qpid.client.AMQSession$Dispatcher.access$900(AMQSession.java:3117)
at org.apache.qpid.client.AMQSession.dispatch(AMQSession.java:3110)
at org.apache.qpid.client.message.UnprocessedMessage.dispatch(UnprocessedMessage.java:55)
at org.apache.qpid.client.AMQSession$Dispatcher.run(AMQSession.java:3264)
at java.lang.Thread.run(Thread.java:680)
And there is no any errors like "Enqueue capacity threshold exceeded on queue "bus-status-queue"". What is wrong?

This is due to a restriction in AMPQ protocol related to String size. str16 is a 64K String size. The code part where this restriction is placed come into action when data-structures such as Maps and List are used to contain the Strings. With simple TextMessage, it is treated as raw data.
The solution to your question is, use byte[] instead, which is defined as vbin32 which can contain 4GB data.

Preventing EOF message on a Java RMI socket connection

I have a Java application which uses RMI for client/server communication.
To secure this communication the traffic is tunneled through an ssh connection.
Everything works well, except that the connection keeps getting closed automatically after a few seconds.
I have set the keep alive property true of:
SSHD connection
SSH client connection
ServerSocket server side
ClientSocket client side
A common connection routine of connecting to the register (port 4000) and invoking a method on an object (port 4005) outputs the following log:
INFO org.apache.sshd.server.session.ServerSession - Authentication succeeded
INFO org.apache.sshd.server.session.ServerSession - Received SSH_MSG_CHANNEL_OPEN direct-tcpip
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Receiving request for direct tcpip: hostToConnect=ThinkPad, portToConnect=4000, originatorIpAddress=127.0.0.1, originatorPort=64539
INFO org.apache.sshd.server.session.ServerSession - Received SSH_MSG_CHANNEL_OPEN direct-tcpip
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Receiving request for direct tcpip: hostToConnect=ThinkPad, portToConnect=4005, originatorIpAddress=127.0.0.1, originatorPort=64540
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Received SSH_MSG_CHANNEL_EOF on channel 1
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Send SSH_MSG_CHANNEL_CLOSE on channel 1
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Received SSH_MSG_CHANNEL_CLOSE on channel 1
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Closing channel 1 immediately
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Closing channel 1 immediately
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Send SSH_MSG_CHANNEL_EOF on channel 1
INFO org.apache.sshd.server.session.ServerSession - Closing session
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Closing channel 0 immediately
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Closing channel 0 immediately
INFO org.apache.sshd.server.channel.ChannelDirectTcpip - Send SSH_MSG_CHANNEL_EOF on channel 0
The line ** Received SSH_MSG_CHANNEL_EOF on channel 1 ** suggests that the method invoked on the object has generated an EOF message. This then causes the session to close...
Possible solutions I can think of:
Intercept or prevent the EOF message (but where and how?)
Try to configure the server side sessionfactory to ignore the EOF messages (feels wrong...)

RMI connections are pooled at the client and closed if not reused within 15 seconds. You can adjust this behaviour via system properties: see the Sun system properties page linked from the RMI home page.