Develop Reference

Not able to receive tcp(mode is server) message on channel with undefined terminator

Currently we are using Spring Integration 2.1.0 Release(Due to legacy application can not switch on latest version ) in our application.
Application flow is as below:
All the configuration details are defined in a configuration file, like host name, port number, terminator etc
Get the message from TCP using tcp-inbound-channel-adapter via channel.
Pass it to splitter for further flow.
Here issue is if message has terminator other than, which is defined in configuration file,message does not come to class defined for splitter, if terminator is same, it is working fine.
Requirement is if terminator value is different it should show a error message on same channel using tcp-outbound-channel-adapter(inbound and outbound is used due asynchronous call).
I have enabled the application and spring logging at Trace level but not able to understand why and where message is stuck.
Code for Configuration file is
<Config>
<host>localhost</host>
<port>8888</port>
<mode>server</mode>
<terminator>10</terminator>
<msgLength>65535</msgLength>
<inChannel>telnetInboundCustomChannel</inChannel>
</Config>
XML for connection details
<beans:bean id="serverCustomSerializer"
class="com.core.serializer.CustomSerializer">
<beans:property name="terminatingChar" value="${server.terminator}"/>
<beans:property name="maxLength" value="${server.msgLength}"/>
</beans:bean>
<beans:bean id="serverFactoryTaskExecutor"
class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<beans:property name="corePoolSize" value="5" />
<beans:property name="queueCapacity" value="0" />
</beans:bean
<int:channel id="telnetLandingChannel" />
<ip:tcp-connection-factory id="serverFactory" type="server"
host="${server.host}" port="${server.port}" single-use="false"
serializer="${server.serializer}" deserializer="${server.serializer}" task-
executor="serverFactoryTaskExecutor"/>
<ip:tcp-inbound-channel-adapter id="serverInboundAdpater"
channel="telnetLandingChannel" connection-factory="serverFactory"
error-channel="errorChannel" auto-startup="false"/>
<ip:tcp-outbound-channel-adapter id="serverOutboundAdapter"
channel="serverReplyChannel"
connection-factory="serverFactory"
auto-startup="true"/>
XML for Channel details and flow are:
<int:channel id="telnetInboundCustomChannel" />
<int:splitter id="messageSplitter"
input-channel="telnetInboundCustomChannel" ref="telnetCustomMessageSplitter"
method="splitCustomMessageStream"
outputchannel="base24CustomSplitterChannel" />
<int:filter id="messageFilter" input-
channel="base24CustomSplitterChannel"
output-channel="base24CustomCoreMessageChannel"
ref="telnetCustomMessageFilter"
method="customMessageFilter" />
<!--Other code to get data from filer and pass it to correct router -->
If somehow message is visible in filter class, I can apply the logic to written error code on TCP connection.
I have applied the break points on run() of TcpNetConnection class as well. I am not able to understand Spring Integration internal flow. How message is coming even till splitter.
I have noticed one more thing if I send message with correct terminator, after sending with wrong terminator, Spring will append new message with old message.
Looks like without correct terminator spring is not able to cut the frame and it is stuck in telnetInboundCustomChannel.
Please guide how to fix this issue and reason of issue for better understanding.

It's not clear how you can detect a bad terminator. By definition the deserializer needs to know a message is complete before returning. You could detect a socket close (bite < 0) and n>0 and return a special message but I don't see how else you can emit a message unless you know what invalid terminator(s) to look for.
EDIT
If you mean check for another "special" (non-printable) character, then you can use something like...
if (n > 0 && (bite == bytes.byteValue())) {
break;
}
else (if bite < 0x20) {
return ("Bad terminator for" + new String(buffer, 0, n)).getBytes();
}

The requirement is strictly meaningless. There is no such thing as a message in TCP, and no such thing as a message with an undefined terminator in any protocol.

mybatis setting for mysql time-out on tomcat server

using java7 tomcat7 and mybatis as ORM
config.xml is like this
<transactionManager type="JDBC" />
<dataSource type="POOLED">
<property name="driver" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/xxxxdb" />
<property name="username" value="xxxxxxx" />
<property name="password" value="xxxxxxx" />
<property name="poolPingEnabled" value="true" />
<property name="poolPingQuery" value="SELECT 1 " />
</dataSource>
</environment>
mysql settings are all default set.
Hence interactive_timeout is 28800.
When I login my service, it fails for the first time, then it succeeds for the second time.
Above error sometimes happens even though re-login within 28800 seconds.
I paste the error message in server
2015 10:03:49 org.apache.ibatis.datasource.pooled.PooledDataSource warn
WARN: Execution of ping query 'SELECT 1' failed: Communications link failure
The last packet successfully received from the server was 30,572,290 milliseconds ago. The last packet sent successfully to the server was 1 milliseconds ago.
org.apache.ibatis.exceptions.PersistenceException:
### Error querying database. Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 36,001,604 milliseconds ago. The last packet sent successfully to the server was 36,001,633 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
### The error may exist in sql.xml
### The error may involve com.xxx.isRegistered-Inline
### The error occurred while setting parameters
### SQL: [query for login];
### Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 36,001,604 milliseconds ago. The last packet sent successfully to the server was 36,001,633 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
I tried to add "autoReconnect=true" to the end of connection url, but it doesn't solve the problem

Possible duplicate of Java app handling for connections getting dropped
Reporting here BalusC's answer:
This exception suggests that you're opening the connection only once during application's startup and keeping forever open during the application's lifetime. This is bad. The DB will reclaim the connection sooner or later because it's been open for too long. You should close connections properly in the finally block of the very same try block as you're opening it and executing the query on it.
E.g.
public Entity find(Long id) throws SQLException {
Connection connection = null;
// ...
try {
connection = database.getConnection();
// ...
} finally {
// ...
if (connection != null) try { connection.close(); } catch (SQLException logOrIgnore) {}
}
return entity;
}
If you have a performance concern regarding this (which is very reasonable as connecting is the most expensive task), then you should be using a connection pool. It also transparently handles this kind of "connection dropped" problems. For example, BoneCP. Please note that also in case of a connection pool, you should still be closing the connections in the finally block as per the above JDBC code idiom. It will namely make them available for reuse.

Spring Integration: Why is client socket waiting for response from server?

I am using spring integration to make a client socket (java code), which must send messages to server socket (flash client). Basically, i want to push messages to flash through socket communication without caring for any response from it. The message which should be sent is coming from GW upon send method is called.
I am able to push a message, but the problem is that my client socket is waiting for a response and if it doesn't get one it times out at some point. Here is my configuration:
<int:gateway id="gw"
service-interface="integration.MessageGateway"
default-request-channel="input"/>
<int-ip:tcp-connection-factory id="client"
type="client"
host="localhost"
port="6767"
serializer="clientSerializer"
single-use="true" so-keep-alive="true"
so-timeout="10000"/>
<bean id="clientSerializer" class="org.springframework.integration.ip.tcp.serializer.ByteArrayCrLfSerializer" />
<int:channel id="input" />
<int-ip:tcp-outbound-gateway id="outGateway"
request-channel="input"
connection-factory="client"
request-timeout="10000"
reply-timeout="10000"/>

Consider to use one-way <int-ip:tcp-outbound-channel-adapter>: http://docs.spring.io/spring-integration/docs/latest-ga/reference/html/ip.html#tcp-adapters

Message to Websphere MQ doesn't receive reply (Fuse ESB 7.1)

I'm sending a "RequestSnapshot" to a Websphere-MQ queue which is define by the vendor as 'S' (synchronous) - so I am expecting to receive a reply via the Apache Camel inOut integration pattern, running in Fuse ESB.
from("direct:" + SEND_SUBSCRIPTION)
.routeId(getFinalRouteName(SEND_SUBSCRIPTION))
.log("Sending Request Snapshot request to webspheremq")
.bean(CreateSnapshotRequest.class)
.marshal(myDataFormat)
.convertBodyTo(String.class)
.inOut("webspheremq:queue:SNAPSHOT_REQUESTS.HT") // **1** works, but no Reply!!
.log("RequestSnapshot response: ${body}") // doesnt reach this line
I know that the snapshot request message is transmitted correctly, because after I send it I receive a data on a different MQ topic.
The problem is that I am not receiving a response, the execution stops on the line marked **1** - and after 20 seconds I get a timeout.
org.apache.camel.ExchangeTimedOutException: The OUT message was not received
within: 20000 millis due reply message with correlationID:
Camel-ID-XYZ012-54037-1385986757895-0-3 not received.
Normally, via java code, this would be done differently, but here we are using Fuse ESB and the inOut mechanism should send the message and get a reply on what is called the DYNAMIC REPLY-TO QUEUE.
My understanding of the "reply-to" queue is somewhat limited, I just understand that Fuse ESB should listen for a response on some sort of temporary mq, but that this should work transparently via inOut.
Here is my configuration bean from blueprint.xml:
<bean id="webspheremq" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory">
<bean class="com.ibm.mq.jms.MQConnectionFactory">
<property name="transportType" value="1"/>
<property name="hostName" value="1.2.3.4"/>
<property name="port" value="1417"/>
<property name="channel" value="SOME.CHANNEL"/>
<property name="temporaryModel" value="SOME_MODEL_QUEUE"/>
<property name="CCSID" value="789"/>
<property name="queueManager" value="SOMEGATE"/>
<property name="brokerSubQueue" value="SYSTEM.JMS.ND.MACHINE.USER.*"/>
<property name="brokerControlQueue" value="SOME_SUBSCRIPTION.HT"/>
<property name="brokerQueueManager" value="SOMEHUB"/>
<property name="tempQPrefix" value="MACHINE.USER.*"/>
</bean>
</property>
</bean>
The other question in my mind is firewall ports, we asked to open 1417 and 1499, but only 1417 seems to be open. Plus I do not know if the firewall ports are open in the opposite direction, from WebsphereMQ back to me.
I would be extremely grateful if anyone can offer any advice!
Edit 1:
I tried the replyTo=queue:XYZ suggestion from Claus, and the result was like this:
2013-12-03 14:38:04,636 | WARN | eplyManager[FOO] | entQueueMessageListenerContainer
| ? ? | 153 - org.springframework.jms -
3.0.7.RELEASE | Setup of JMS message listener invoker failed for destination 'FOO'
- trying to recover. Cause: MQJMS2008: failed to open MQ queue ; nested exception is
com.ibm.mq.MQException: MQJE001: Completion Code 2, Reason 2085

You can use named reply queues also. Maybe that works better with WMQ. So in the Camel endpoint uri, you just set replyTo=queue:foobar to use the queue named foobar as the reply queue.
That would set the JMSReplyTo header as foobar as the queue name, so the "other side" would use that for reply queue, instead of a temporary queue name.
See the Camel JMS documentation for more details about request/reply over JMS as there is many options you can configure and tweak: http://camel.apache.org/jms
Also in the past WMQ didn't link long correlation ids, and the option useMessageIDAsCorrelationID can be set to true, to use the message id that WMQ generated for the request message. Otherwise a long random id is generated by Camel.

How to catch when JMS connection is established?

I have message producers that are sending JMS messages about some events using ActiveMQ.
However, connection to ActiveMQ might not be up all the time. Thus, events are stored and when connection is established they are suppose to be read and sent over. Here is my code:
private void sendAndSave(MyEvent event) {
boolean sent = sendMessage(event);
event.setProcessed(sent);
boolean saved = repository.saveEvent(event);
if (!sent && !saved) {
logger.error("Change event lost for Id = {}", event.getId());
}
}
private boolean sendMessage(MyEvent event) {
try {
messenger.publishEvent(event);
return true;
} catch (JmsException ex) {
return false;
}
}
I'd like to create some kind of ApplicationEventListener that will be invoked when connection is established and process unsent events.
I went through JMS, Spring framework and ActiveMQ documentation but couldn't find any clues how to hook up my listener with ConnectionFactory.
If someone can help me out, I'll appreciate it greatly.
Here is what my app Spring context says about JMS:
<!-- Connection factory to the ActiveMQ broker instance. -->
<!-- The URI and credentials must match the values in activemq.xml -->
<!-- These credentials are shared by ALL producers. -->
<bean id="jmsTransportListener" class="com.rhd.ams.service.common.JmsTransportListener"
init-method="init" destroy-method="cleanup"/>
<bean id="amqJmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${jms.publisher.broker.url}"/>
<property name="userName" value="${jms.publisher.username}"/>
<property name="password" value="${jms.publisher.password}"/>
<property name="transportListener" ref="jmsTransportListener"/>
</bean>
<!-- JmsTemplate, by default, will create a new connection, session, producer for -->
<!-- each message sent, then close them all down again. This is very inefficient! -->
<!-- PooledConnectionFactory will pool the JMS resources. It can't be used with consumers.-->
<bean id="pooledAmqJmsConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" destroy-method="stop">
<property name="connectionFactory" ref="amqJmsConnectionFactory" />
</bean>
<!-- Although JmsTemplate instance is unique for each message, it is -->
<!-- thread-safe and therefore can be injected into referenced obj's. -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<constructor-arg ref="pooledAmqJmsConnectionFactory"/>
</bean>

The way you describe the issue, it sure sounds like an open-and-shut case of JMS Durable Subscriptions. You might want to consider a more traditional implementation before going down this road. Caveats aside, ActiveMQ provides Advisory Messages which you can listen for and which will be sent for various events including new connections.
=========
Shoot, sorry... I did not understand what the issue was. I don't think Advisories are the solution at all.... after all, you need to be connected to the broker to get them, but being connected is what you know about.
So if I understand it correctly (prepare for retry #2....), what you need is a client connection which, when it fails, attempts to reconnect indefinitely. When it does reconnect, you want to trigger an event (or more) that flushes pending messages to the broker.
So detecting the lost connection is easy. You just register a JMS ExceptionListener. As far as detecting a reconnect, the simplest way I can think of is to start a reconnect thread. When it connects, stop the reconnect thread and notify interested parties using Observer/Observable or JMX notifications or the like. You could use the ActiveMQ Failover Transport which will do a connection retry loop for you, even if you only have one broker. At least, it is supposed to, but it's not doing that much for you that would not be done by your own reconnect thread... but if you're willing to delegate some control to it, it will cache your unflushed messages (see the trackMessages option), and then send them when it reconnects, which is sort of all of what you're trying to do.
I guess if your broker is down for a few minutes, that's not a bad way to go, but if you're talking hours, or you might accumulate 10k+ messages in the downtime, I just don't know if that cache mechanism is as reliable as you would need it to be.
==================
Mobile app ... right. Not really appropriate for the failover transport. Then I would implement a timer that periodically connects (might be a good idea to use the http transport, but not relevant). When it does connect, if there's nothing to flush, then see you in x minutes. If there is, send each message, wait for a handshake and purge the message from you mobile store. Then see you again in x minutes.
I assume this is Android ? If not, stop reading here. We actually implemented this some time ago. I only did the server side, but if I remember correctly, the connection timer/poller spun every n minutes (variable frequencies, I think, because getting too aggressive was draining the battery). Once a successful connection was made, I believe they used an intent broadcast to nudge the message pushers to do their thing. The thinking was that even though there was only one message pusher, we might add more.