Develop Reference

How to Make Subscribers Not Consuming The Same Messages - Redis Pubsub Messaging Using Spring Data Redis

So recently I was exploring on how to make a client app utilising Redis PubSub feature and make it resemble Apache Kafka messaging. I know that those two have their own characteristics, therefore I might not be able to make the client app, a Redis Subscriber, to work RESEMBLE Apache Kafka Consumer.
What I'm currently tweaking is The Redis Subscribers behaviour on the client app side, so they will not receiving (or 'consuming') the same message sent their subbed channel. I know basically it's kind of impossible, given Redis Pubsub's behaviour that all subscribers will received the published message equally...
... But, I did found a way.
FYI, I am using :
Java
Maven, as dependecy injection
SpringBoot
Dependecies...
Spring Data Redis
Jedis (3.6.0)
json - org.json (20210307)
So here's my setup...
BeanConfiguration.java
#Configuration
public class BeanConfiguration{
#Bean
public JedisConnectionFactory connectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration("localhost", 6379);
JedisConnectionFactory connFactory = new JedisConnectionFactory(redisStandaloneConfiguration);
return connFactory;
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> connTemplate = new RedisTemplate<String, Object>();
connTemplate.setConnectionFactory(connectionFactory());
connTemplate.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
connTemplate.setKeySerializer(new StringRedisSerializer());
connTemplate.setHashKeySerializer(new GenericJackson2JsonRedisSerializer());
connTemplate.setValueSerializer(new GenericJackson2JsonRedisSerializer());
return connTemplate;
}
#Bean
public MessageListenerAdapter messageListener() {
return new MessageListenerAdapter(new RedisMessageSubscriber(redisTemplate()));
}
#Bean
public ChannelTopic topic() {
return new ChannelTopic("mychannel");
}
#Bean
public RedisMessageListenerContainer redisContainer() {
RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory(connectionFactory());
container.addMessageListener(messageListener(), topic());
container.setTaskExecutor(Executors.newFixedThreadPool(4));
return container;
}}
RedisMessageSubscriber.java
#Service
public class RedisMessageSubscriber implements MessageListener {
private RedisTemplate<String, Object> redisTemplate;
public RedisMessageSubscriber(RedisTemplate<String, Object> redisTemplate) {
this.redisTemplate = redisTemplate;
}
#Override
public void onMessage(Message message, byte[] pattern) {
JSONObject jsonMessage = new JSONObject(message.toString());
lockAndProcessMessage(jsonMessage);
}
private void lockAndProcessMessage(JSONObject jsonMessage) {
String key = jsonMessage.getString("id");
String value = jsonMessage.getString("value");
Boolean isNotExist = redisTemplate.opsForValue().setIfAbsent(key, value, 5, TimeUnit.MINUTES);
if (isNotExist) {
System.out.println("SUCCESSFULLY SET ['" + key + "'] Expired in '1 minute'");
try {
System.out.println("Processing ['" + key + "'] ...");
Thread.sleep(60000);
} catch(InterruptedException ex) {
ex.printStackTrace();
} finally {
redisTemplate.opsForValue().getOperations().delete(key);
System.out.println("UNLOCKED ['" + key + "']");
}
} else {
System.out.println("FAILED TO LOCKED ['" + key + "']");
}
}}
Using codes above, I was able to achieve the "no-double-consuming" condition, but there is some circumstances that made the "double-consuming" happened.
If instance "A" is processing at queue 15, while instance "B" is still processing at queue 6 and by the time "A" is done with 15 and "B" caught up with queue 15, "B" WILL PROCESSED queue 15 again due to the marker left by "A" had already been deleted (Due to the processing of 15 is done by "A").
Is there anyway to counter this "weakness"? I am also open to any solution, suggestion, discussion, and findings of weakness in my sample codes. Thank you
Sorry if my english is bad :D

Connect JMS queue using JNDI with Spring Boot

I had a hard time figuring out how to implement a Spring Boot JMS Listener, listening to an ActiveMQ queue within a JBoss application server.
Therefore I choose to post a question and answer it with my final solution, hoping it could save some of you a few hours.

ActiveMQ is supported by Spring Boot autoconfiguration, but since it was inside the JBoss server Spring Boot was failing to connect ActiveMQ.
In fact you need to define connectionFactory and jmsListenerContainerFactory beans yourself by doing a lookup on the JNDI provider.
#Configuration
#EnableJms
public class ActiveMqConnectionFactoryConfig {
#Value("${broker.url}")
String brokerUrl;
#Value("${borker.username}")
String userName;
#Value("${borker.password}")
String password;
#Value("${queue}")
String queueName;
private static final String INITIAL_CONTEXT_FACTORY = "org.jboss.naming.remote.client.InitialContextFactory";
private static final String CONNECTION_FACTORY = "jms/RemoteConnectionFactory";
#Bean
public ConnectionFactory connectionFactory() {
try {
System.out.println("Retrieving JMS queue with JNDI name: " + CONNECTION_FACTORY);
JndiObjectFactoryBean jndiObjectFactoryBean = new JndiObjectFactoryBean();
jndiObjectFactoryBean.setJndiName(CONNECTION_FACTORY);
jndiObjectFactoryBean.setJndiEnvironment(getEnvProperties());
jndiObjectFactoryBean.afterPropertiesSet();
return (QueueConnectionFactory) jndiObjectFactoryBean.getObject();
} catch (NamingException e) {
System.out.println("Error while retrieving JMS queue with JNDI name: [" + CONNECTION_FACTORY + "]");
} catch (Exception ex) {
System.out.println("Error");
}
return null;
}
Properties getEnvProperties() {
Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, INITIAL_CONTEXT_FACTORY);
env.put(Context.PROVIDER_URL, brokerUrl);
env.put(Context.SECURITY_PRINCIPAL, userName);
env.put(Context.SECURITY_CREDENTIALS, password);
return env;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
JndiDestinationResolver jndiDestinationResolver = new JndiDestinationResolver();
jndiDestinationResolver.setJndiEnvironment(getEnvProperties());
factory.setDestinationResolver(jndiDestinationResolver);
return factory;
}
Then if you want to consume the queue you just define your JMS consumer class with a method annotated with #JmsListener(destination = "${queue}")
#JmsListener(destination = "${queue}")
public void receive(Message message) {
System.out.println("Received Message: " + message);
}
Hope that helps save a few hours of research ;)
Cheers

RabbitMQ fail connection

I need help, my app (client) connect to RabbitMQ server and when server shutdown, my app cannot start....
Listener can't created and app failed start.
Or when virtual host dont have queue my app cant start too
So my question
1) How to process exception in my config (all exception, i need was my app start if RabbitMQ server have problems)
2) What in my config look bad and need refactor
i use
Spring 4.2.9.RELEASE
org.springframework.amqp 2.0.5.RELEASE
Java 8
My 2 classes
1) Config for Beans RabbitMq
2) Listener annotation
#EnableRabbit
#Configuration
public class RabbitMQConfig {
#Bean
public ConnectionFactory connectionFactory() {
com.rabbitmq.client.ConnectionFactory factoryRabbit = new com.rabbitmq.client.ConnectionFactory();
factoryRabbit.setNetworkRecoveryInterval(10000);
factoryRabbit.setAutomaticRecoveryEnabled(true);
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory(factoryRabbit);
connectionFactory.setHost("DRIVER_APP_IP");
connectionFactory.setPort(5672);
connectionFactory.setConnectionTimeout(5000);
connectionFactory.setRequestedHeartBeat(10);
connectionFactory.setUsername("user");
connectionFactory.setPassword("pass");
connectionFactory.setVirtualHost("/vhost");
return connectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
try {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory());
rabbitTemplate.setRoutingKey(this.DRIVER_QUEUE);
rabbitTemplate.setQueue(this.DRIVER_QUEUE);
return rabbitTemplate;
} catch (Exception ex){
return new RabbitTemplate();
}
}
#Bean
public Queue queue() {
return new Queue(this.DRIVER_QUEUE);
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(3);
factory.setMaxConcurrentConsumers(10);
return factory;
}
}
#Component
public class RabbitMqListener {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = DRIVER_QUEUE, durable = "true"),
exchange = #Exchange(value = "exchange", ignoreDeclarationExceptions = "true", autoDelete = "true"))
)
public String balancer(byte[] message) throws InterruptedException {
String json = null;
try {
"something move"
} catch (Exception ex) {
}
}

I found solution my problem
First it's Bean container!
We need this
factory.setMissingQueuesFatal(false);
this property give our when queue lost on server RabbitMQ, our app don't crash and can start
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setMissingQueuesFatal(false);
factory.setConcurrentConsumers(3);
factory.setStartConsumerMinInterval(3000L);
factory.setMaxConcurrentConsumers(10);
factory.setRecoveryInterval(15000L);
factory.setStartConsumerMinInterval(1000L);
factory.setReceiveTimeout(10000L);
factory.setChannelTransacted(true);
return factory;
}
and second
#Component
public class RabbitMqListener {
#RabbitListener(containerFactory = "rabbitListenerContainerFactory", queues = DRIVER_QUEUE)
public String balancer(byte[] message) throws InterruptedException {
String json = null;
try {
"something move"
} catch (Exception ex) {
}
}
I set containerFactory and Queue in #RabbitListener and drop other propertys,
because i don't need it
I hope it's help somebody, thank all for you attention and sorry for my English

Spring Integration: TCP Client/Server opening one client connection per server connection

I'm trying to implement a TCP client/server application with Spring Integration where I need to open one TCP client socket per incoming TCP server connection.
Basically, I have a bunch of IoT devices that communicate with a backend server over raw TCP sockets. I need to implement extra features into the system. But the software on both the devices and the server are closed source so I can't do anything about that. So my thought was to place middleware between the devices and the server that will intercept this client/server communication and provide the added functionality.
I'm using a TcpNioServerConnectionFactory and a TcpNioClientConnectionFactory with inbound/outbound channel adapters to send/receive messages to/from all parties. But there's no information in the message structure that binds a message to a certain device; therefore I have to open a new client socket to the backend every time a new connection from a new device comes on the server socket. This client connection must be bound to that specific server socket's lifecycle. It must never be reused and if this client socket (backend to middleware) dies for any reason, the server socket (middleware to device) must also be closed. How can I go about this?
Edit: My first thought was to subclass AbstractClientConnectionFactory but it appears that it doesn't do anything except provide a client connection when asked. Should I rather look into subclassing inbound/outbound channel adapters or elsewhere? I should also mention that I'm also open to non-Spring integration solutions like Apache Camel, or even a custom solution with raw NIO sockets.
Edit 2: I got halfway there by switching to TcpNetServerConnectionFactory and wrapping the client factory with a ThreadAffinityClientConnectionFactory and the devices can reach the backend fine. But when the backend sends something back, I get the error Unable to find outbound socket for GenericMessage and the client socket dies. I think it's because the backend side doesn't have the necessary header to route the message correctly. How can I capture this info? My configuration class is as follows:
#Configuration
#EnableIntegration
#IntegrationComponentScan
public class ServerConfiguration {
#Bean
public AbstractServerConnectionFactory serverFactory() {
AbstractServerConnectionFactory factory = new TcpNetServerConnectionFactory(8000);
factory.setSerializer(new MapJsonSerializer());
factory.setDeserializer(new MapJsonSerializer());
return factory;
}
#Bean
public AbstractClientConnectionFactory clientFactory() {
AbstractClientConnectionFactory factory = new TcpNioClientConnectionFactory("localhost", 3333);
factory.setSerializer(new MapJsonSerializer());
factory.setDeserializer(new MapJsonSerializer());
factory.setSingleUse(true);
return new ThreadAffinityClientConnectionFactory(factory);
}
#Bean
public TcpReceivingChannelAdapter inboundDeviceAdapter(AbstractServerConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
return inbound;
}
#Bean
public TcpSendingMessageHandler outboundDeviceAdapter(AbstractServerConnectionFactory connectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(connectionFactory);
return outbound;
}
#Bean
public TcpReceivingChannelAdapter inboundBackendAdapter(AbstractClientConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
return inbound;
}
#Bean
public TcpSendingMessageHandler outboundBackendAdapter(AbstractClientConnectionFactory connectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(connectionFactory);
return outbound;
}
#Bean
public IntegrationFlow backendIntegrationFlow() {
return IntegrationFlows.from(inboundBackendAdapter(clientFactory()))
.log(LoggingHandler.Level.INFO)
.handle(outboundDeviceAdapter(serverFactory()))
.get();
}
#Bean
public IntegrationFlow deviceIntegrationFlow() {
return IntegrationFlows.from(inboundDeviceAdapter(serverFactory()))
.log(LoggingHandler.Level.INFO)
.handle(outboundBackendAdapter(clientFactory()))
.get();
}
}

It's not entirely clear what you are asking so I am going to assume that you mean you want a spring integration proxy between your clients and servers. Something like:
iot-device -> spring server -> message-transformation -> spring client -> back-end-server
If that's the case, you can implement a ClientConnectionIdAware client connection factory that wraps a standard factory.
In the integration flow, bind the incoming ip_connectionId header in a message to the thread (in a ThreadLocal).
Then, in the client connection factory, look up the corresponding outgoing connection in a Map using the ThreadLocal value; if not found (or closed), create a new one and store it in the map for future reuse.
Implement an ApplictionListener (or #EventListener) to listen for TcpConnectionCloseEvents from the server connection factory and close() the corresponding outbound connection.
This sounds like a cool enhancement so consider contributing it back to the framework.
EDIT
Version 5.0 added the ThreadAffinityClientConnectionFactory which would work out of the box with a TcpNetServerConnectionFactory since each connection gets its own thread.
With a TcpNioServerConnectionFactory you would need the extra logic to dynamically bind the connection to the thread for each request.
EDIT2
#SpringBootApplication
public class So51200675Application {
public static void main(String[] args) {
SpringApplication.run(So51200675Application.class, args).close();
}
#Bean
public ApplicationRunner runner() {
return args -> {
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("foo\r\n".getBytes());
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
System.out.println(reader.readLine());
socket.close();
};
}
#Bean
public Map<String, String> fromToConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public Map<String, String> toFromConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public IntegrationFlow proxyInboundFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(serverFactory()))
.transform(Transformers.objectToString())
.<String, String>transform(s -> s.toUpperCase())
.handle((p, h) -> {
mapConnectionIds(h);
return p;
})
.handle(Tcp.outboundAdapter(threadConnectionFactory()))
.get();
}
#Bean
public IntegrationFlow proxyOutboundFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(threadConnectionFactory()))
.transform(Transformers.objectToString())
.<String, String>transform(s -> s.toUpperCase())
.enrichHeaders(e -> e
.headerExpression(IpHeaders.CONNECTION_ID, "#toFromConnectionMappings.get(headers['"
+ IpHeaders.CONNECTION_ID + "'])").defaultOverwrite(true))
.handle(Tcp.outboundAdapter(serverFactory()))
.get();
}
private void mapConnectionIds(Map<String, Object> h) {
try {
TcpConnection connection = threadConnectionFactory().getConnection();
String mapping = toFromConnectionMappings().get(connection.getConnectionId());
String incomingCID = (String) h.get(IpHeaders.CONNECTION_ID);
if (mapping == null || !(mapping.equals(incomingCID))) {
System.out.println("Adding new mapping " + incomingCID + " to " + connection.getConnectionId());
toFromConnectionMappings().put(connection.getConnectionId(), incomingCID);
fromToConnectionMappings().put(incomingCID, connection.getConnectionId());
}
}
catch (Exception e) {
e.printStackTrace();
}
}
#Bean
public ThreadAffinityClientConnectionFactory threadConnectionFactory() {
return new ThreadAffinityClientConnectionFactory(clientFactory()) {
#Override
public boolean isSingleUse() {
return false;
}
};
}
#Bean
public AbstractServerConnectionFactory serverFactory() {
return Tcp.netServer(1234).get();
}
#Bean
public AbstractClientConnectionFactory clientFactory() {
AbstractClientConnectionFactory clientFactory = Tcp.netClient("localhost", 1235).get();
clientFactory.setSingleUse(true);
return clientFactory;
}
#Bean
public IntegrationFlow serverFlow() {
return IntegrationFlows.from(Tcp.inboundGateway(Tcp.netServer(1235)))
.transform(Transformers.objectToString())
.<String, String>transform(p -> p + p)
.get();
}
#Bean
public ApplicationListener<TcpConnectionCloseEvent> closer() {
return e -> {
if (fromToConnectionMappings().containsKey(e.getConnectionId())) {
String key = fromToConnectionMappings().remove(e.getConnectionId());
toFromConnectionMappings().remove(key);
System.out.println("Removed mapping " + e.getConnectionId() + " to " + key);
threadConnectionFactory().releaseConnection();
}
};
}
}
EDIT3
Works fine for me with a MapJsonSerializer.
#SpringBootApplication
public class So51200675Application {
public static void main(String[] args) {
SpringApplication.run(So51200675Application.class, args).close();
}
#Bean
public ApplicationRunner runner() {
return args -> {
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("{\"foo\":\"bar\"}\n".getBytes());
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
System.out.println(reader.readLine());
socket.close();
};
}
#Bean
public Map<String, String> fromToConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public Map<String, String> toFromConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public MapJsonSerializer serializer() {
return new MapJsonSerializer();
}
#Bean
public IntegrationFlow proxyRequestFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(serverFactory()))
.<Map<String, String>, Map<String, String>>transform(m -> {
m.put("foo", m.get("foo").toUpperCase());
return m;
})
.handle((p, h) -> {
mapConnectionIds(h);
return p;
})
.handle(Tcp.outboundAdapter(threadConnectionFactory()))
.get();
}
#Bean
public IntegrationFlow proxyReplyFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(threadConnectionFactory()))
.<Map<String, String>, Map<String, String>>transform(m -> {
m.put("foo", m.get("foo").toLowerCase() + m.get("foo"));
return m;
})
.enrichHeaders(e -> e
.headerExpression(IpHeaders.CONNECTION_ID, "#toFromConnectionMappings.get(headers['"
+ IpHeaders.CONNECTION_ID + "'])").defaultOverwrite(true))
.handle(Tcp.outboundAdapter(serverFactory()))
.get();
}
private void mapConnectionIds(Map<String, Object> h) {
try {
TcpConnection connection = threadConnectionFactory().getConnection();
String mapping = toFromConnectionMappings().get(connection.getConnectionId());
String incomingCID = (String) h.get(IpHeaders.CONNECTION_ID);
if (mapping == null || !(mapping.equals(incomingCID))) {
System.out.println("Adding new mapping " + incomingCID + " to " + connection.getConnectionId());
toFromConnectionMappings().put(connection.getConnectionId(), incomingCID);
fromToConnectionMappings().put(incomingCID, connection.getConnectionId());
}
}
catch (Exception e) {
e.printStackTrace();
}
}
#Bean
public ThreadAffinityClientConnectionFactory threadConnectionFactory() {
return new ThreadAffinityClientConnectionFactory(clientFactory()) {
#Override
public boolean isSingleUse() {
return false;
}
};
}
#Bean
public AbstractServerConnectionFactory serverFactory() {
return Tcp.netServer(1234)
.serializer(serializer())
.deserializer(serializer())
.get();
}
#Bean
public AbstractClientConnectionFactory clientFactory() {
AbstractClientConnectionFactory clientFactory = Tcp.netClient("localhost", 1235)
.serializer(serializer())
.deserializer(serializer())
.get();
clientFactory.setSingleUse(true);
return clientFactory;
}
#Bean
public IntegrationFlow backEndEmulatorFlow() {
return IntegrationFlows.from(Tcp.inboundGateway(Tcp.netServer(1235)
.serializer(serializer())
.deserializer(serializer())))
.<Map<String, String>, Map<String, String>>transform(m -> {
m.put("foo", m.get("foo") + m.get("foo"));
return m;
})
.get();
}
#Bean
public ApplicationListener<TcpConnectionCloseEvent> closer() {
return e -> {
if (fromToConnectionMappings().containsKey(e.getConnectionId())) {
String key = fromToConnectionMappings().remove(e.getConnectionId());
toFromConnectionMappings().remove(key);
System.out.println("Removed mapping " + e.getConnectionId() + " to " + key);
threadConnectionFactory().releaseConnection();
}
};
}
}
and
Adding new mapping localhost:56998:1234:55c822a4-4252-45e6-9ef2-79263391f4be to localhost:1235:56999:3d520ca9-2f3a-44c3-b05f-e59695b8c1b0
{"foo":"barbarBARBAR"}
Removed mapping localhost:56998:1234:55c822a4-4252-45e6-9ef2-79263391f4be to localhost:1235:56999:3d520ca9-2f3a-44c3-b05f-e59695b8c1b0

Spring Integration - How to implement an asynchronous TCP Socket requests/responses over the same connection?

I have a Python TCP Socket server service which:
Allows only one client connection at time;
Its inputstream/outputstream operates independently.
On the other side, I have a Java Spring Boot client application using Spring Integration. My actual TCP Socket configurator
implementation uses:
#MessagingGateway(defaultRequestChannel = REQUEST_CHANNEL, errorChannel = ERROR_CHANNEL)
public interface ClientGtw {
Future<Response> send(Request request);
}
#Bean
#ServiceActivator(inputChannel = REQUEST_CHANNEL)
public MessageHandler outboundGateway(TcpNioClientConnectionFactory connectionFactory) {
TcpOutboundGateway gateway = new TcpOutboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestTimeout(TimeUnit.SECONDS.toMillis(timeout));
gateway.setRemoteTimeout(TimeUnit.SECONDS.toMillis(timeout));
return gateway;
}
#Bean
public TcpNioClientConnectionFactory clientConnectionFactory(AppConfig config) {
Host host = getHost(config);
TcpNioClientConnectionFactory factory = new TcpNioClientConnectionFactory(host.name, host.port);
factory.setSingleUse(false);
factory.setSoTimeout((int) TimeUnit.SECONDS.toMillis(timeout));
SerializerDeserializer sd = new SerializerDeserializer();
factory.setDeserializer(sd);
factory.setSerializer(sd);
return factory;
}
This actual approach works fine, however, when a request is sent it hangs the connection until a response is received. This is a problem due the fact that some times a request can get too much time to receive a response and the system has other requests incomming whose response can be achieved faster. I would like to send and receive as much as possible requests and responses independetly (decoupled between them). The object transported (serialized and deserialized) contains a key pair that can do the correct correlation.
TL;DR: How to implement an Asynchronous requests/responses over the same connection?
The Spring TcpOutboundGateway javadoc mentions: Use a pair of outbound/inbound adapters for that use case.
So, in addition to the declaration above:
1st Attempt
#Bean
public TcpInboundGateway inboundGateway(AbstractServerConnectionFactory connectionFactory) {
TcpInboundGateway gateway = new TcpInboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestTimeout(TimeUnit.SECONDS.toMillis(timeout));
return gateway;
}
#Bean
public AbstractServerConnectionFactory serverFactory(AppConfig config) {
Host host = getHost(config);
AbstractServerConnectionFactory connectionFactory = new TcpNetServerConnectionFactory(host.port);
connectionFactory.setSingleUse(true);
connectionFactory.setSoTimeout(timeout);
return connectionFactory;
}
The requests are blocked until a response is delivered as before.
2nd Attempt
#Bean
public TcpInboundGateway inboundGateway(TcpNioClientConnectionFactory connectionFactory) {
TcpInboundGateway gateway = new TcpInboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestTimeout(TimeUnit.SECONDS.toMillis(timeout));
gateway.setClientMode(true);
return gateway;
}
org.springframework.integration.ip.tcp.connection.TcpNioClientConnectionFactory may only be used by one inbound adapter
Any clue?

Use a pair of channel adapters instead of an outbound gateway. Instead of using a MessagingGateway, you can do the correlation yourself in your application, or you can use the same technique as is used in the tcp-client-server-multiplex sample app. It uses an aggregator to aggregate a copy of the outbound message with an inbound message, replying to the gateway.
It's old, and uses XML configuration, but the same techniques apply.
<publish-subscribe-channel id="input" />
<ip:tcp-outbound-channel-adapter id="outAdapter.client"
order="2"
channel="input"
connection-factory="client" /> <!-- Collaborator -->
<!-- Also send a copy to the custom aggregator for correlation and
so this message's replyChannel will be transferred to the
aggregated message.
The order ensures this gets to the aggregator first -->
<bridge input-channel="input" output-channel="toAggregator.client"
order="1"/>
<!-- Asynch receive reply -->
<ip:tcp-inbound-channel-adapter id="inAdapter.client"
channel="toAggregator.client"
connection-factory="client" /> <!-- Collaborator -->
<!-- dataType attribute invokes the conversion service, if necessary -->
<channel id="toAggregator.client" datatype="java.lang.String" />
<aggregator input-channel="toAggregator.client"
output-channel="toTransformer.client"
expire-groups-upon-completion="true"
expire-groups-upon-timeout="true"
discard-channel="noResponseChannel"
group-timeout="1000"
correlation-strategy-expression="payload.substring(0,3)"
release-strategy-expression="size() == 2" />
<channel id="noResponseChannel" />
<service-activator input-channel="noResponseChannel" ref="echoService" method="noResponse" />
<transformer input-channel="toTransformer.client"
expression="payload.get(1)"/> <!-- The response is always second -->
(This simple sample correlates on the first 3 bytes).

Gary, thanks for your guidance.
To solve this issue is important to first understand Messaging Channel types.
So, in the configurer class:
#Bean(name = REQUEST_CHANNEL)
public DirectChannel sender() {
return new DirectChannel();
}
#Bean(name = RESPONSE_CHANNEL)
public PollableChannel receiver() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = REQUEST_CHANNEL)
public TcpSendingMessageHandler outboundClient(TcpNioClientConnectionFactory connectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(connectionFactory);
outbound.setRetryInterval(TimeUnit.SECONDS.toMillis(timeout));
outbound.setClientMode(true);
return outbound;
}
#Bean
public TcpReceivingChannelAdapter inboundClient(TcpNioClientConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
inbound.setRetryInterval(TimeUnit.SECONDS.toMillis(timeout));
inbound.setOutputChannel(receiver());
inbound.setClientMode(true);
return inbound;
}
This scratch #Singleton class illustrates how to operate the requests and responses (considering that requests and responses contains a UID to correlate them):
#Autowired
private DirectChannel sender;
#Autowired
private PollableChannel receiver;
private BlockingQueue<Request> requestPool = new LinkedBlockingQueue<>();
private Map<String, Response> responsePool = Collections.synchronizedMap(new HashMap<>());
#PostConstruct
private void init() {
new Receiver().start();
new Sender().start();
}
/*
* It can be called as many as necessary without hanging for a response
*/
public void send(Request req) {
requestPool.add(req);
}
/*
* Check for a response until a socket timout
*/
public Response receive(String key) {
Response res = responsePool.get(key);
if (res != null) {
responsePool.remove(key);
}
return res;
}
private class Receiver extends Thread {
#Override
public void run() {
while (true) {
try {
tcpReceive();
Thread.sleep(250);
} catch (InterruptedException e) { }
}
}
private void tcpReceive() {
Response res = (Message<Response>) receiver.receive();
if (res != null) {
responsePool.put(res.getUID(), res);
}
}
}
private class Sender extends Thread {
#Override
public void run() {
while (true) {
try {
tcpSend();
Thread.sleep(250);
} catch (InterruptedException e) { }
}
}
private void tcpSend() {
Request req = requestPool.poll(125, TimeUnit.MILLISECONDS);
if (req != null) {
sender.send(MessageBuilder.withPayload(req).build());
}
}
}
UPDATED
I forgot to mention this:
#Bean
public TcpNioClientConnectionFactory clientConnectionFactory(Config config) {
// Get host properties
Host host = getHost(config);
// Create socket factory
TcpNioClientConnectionFactory factory = new TcpNioClientConnectionFactory(host.name, host.port);
factory.setSingleUse(false); // IMPORTANT FOR SINGLE CHANNEL
factory.setSoTimeout((int) TimeUnit.SECONDS.toMillis(timeout));
return factory;
}
Feel free to make any considerations.