I'm trying to implement a TCP client/server application with Spring Integration where I need to open one TCP client socket per incoming TCP server connection.
Basically, I have a bunch of IoT devices that communicate with a backend server over raw TCP sockets. I need to implement extra features into the system. But the software on both the devices and the server are closed source so I can't do anything about that. So my thought was to place middleware between the devices and the server that will intercept this client/server communication and provide the added functionality.
I'm using a TcpNioServerConnectionFactory and a TcpNioClientConnectionFactory with inbound/outbound channel adapters to send/receive messages to/from all parties. But there's no information in the message structure that binds a message to a certain device; therefore I have to open a new client socket to the backend every time a new connection from a new device comes on the server socket. This client connection must be bound to that specific server socket's lifecycle. It must never be reused and if this client socket (backend to middleware) dies for any reason, the server socket (middleware to device) must also be closed. How can I go about this?
Edit: My first thought was to subclass AbstractClientConnectionFactory but it appears that it doesn't do anything except provide a client connection when asked. Should I rather look into subclassing inbound/outbound channel adapters or elsewhere? I should also mention that I'm also open to non-Spring integration solutions like Apache Camel, or even a custom solution with raw NIO sockets.
Edit 2: I got halfway there by switching to TcpNetServerConnectionFactory and wrapping the client factory with a ThreadAffinityClientConnectionFactory and the devices can reach the backend fine. But when the backend sends something back, I get the error Unable to find outbound socket for GenericMessage and the client socket dies. I think it's because the backend side doesn't have the necessary header to route the message correctly. How can I capture this info? My configuration class is as follows:
#Configuration
#EnableIntegration
#IntegrationComponentScan
public class ServerConfiguration {
#Bean
public AbstractServerConnectionFactory serverFactory() {
AbstractServerConnectionFactory factory = new TcpNetServerConnectionFactory(8000);
factory.setSerializer(new MapJsonSerializer());
factory.setDeserializer(new MapJsonSerializer());
return factory;
}
#Bean
public AbstractClientConnectionFactory clientFactory() {
AbstractClientConnectionFactory factory = new TcpNioClientConnectionFactory("localhost", 3333);
factory.setSerializer(new MapJsonSerializer());
factory.setDeserializer(new MapJsonSerializer());
factory.setSingleUse(true);
return new ThreadAffinityClientConnectionFactory(factory);
}
#Bean
public TcpReceivingChannelAdapter inboundDeviceAdapter(AbstractServerConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
return inbound;
}
#Bean
public TcpSendingMessageHandler outboundDeviceAdapter(AbstractServerConnectionFactory connectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(connectionFactory);
return outbound;
}
#Bean
public TcpReceivingChannelAdapter inboundBackendAdapter(AbstractClientConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
return inbound;
}
#Bean
public TcpSendingMessageHandler outboundBackendAdapter(AbstractClientConnectionFactory connectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(connectionFactory);
return outbound;
}
#Bean
public IntegrationFlow backendIntegrationFlow() {
return IntegrationFlows.from(inboundBackendAdapter(clientFactory()))
.log(LoggingHandler.Level.INFO)
.handle(outboundDeviceAdapter(serverFactory()))
.get();
}
#Bean
public IntegrationFlow deviceIntegrationFlow() {
return IntegrationFlows.from(inboundDeviceAdapter(serverFactory()))
.log(LoggingHandler.Level.INFO)
.handle(outboundBackendAdapter(clientFactory()))
.get();
}
}
It's not entirely clear what you are asking so I am going to assume that you mean you want a spring integration proxy between your clients and servers. Something like:
iot-device -> spring server -> message-transformation -> spring client -> back-end-server
If that's the case, you can implement a ClientConnectionIdAware client connection factory that wraps a standard factory.
In the integration flow, bind the incoming ip_connectionId header in a message to the thread (in a ThreadLocal).
Then, in the client connection factory, look up the corresponding outgoing connection in a Map using the ThreadLocal value; if not found (or closed), create a new one and store it in the map for future reuse.
Implement an ApplictionListener (or #EventListener) to listen for TcpConnectionCloseEvents from the server connection factory and close() the corresponding outbound connection.
This sounds like a cool enhancement so consider contributing it back to the framework.
EDIT
Version 5.0 added the ThreadAffinityClientConnectionFactory which would work out of the box with a TcpNetServerConnectionFactory since each connection gets its own thread.
With a TcpNioServerConnectionFactory you would need the extra logic to dynamically bind the connection to the thread for each request.
EDIT2
#SpringBootApplication
public class So51200675Application {
public static void main(String[] args) {
SpringApplication.run(So51200675Application.class, args).close();
}
#Bean
public ApplicationRunner runner() {
return args -> {
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("foo\r\n".getBytes());
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
System.out.println(reader.readLine());
socket.close();
};
}
#Bean
public Map<String, String> fromToConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public Map<String, String> toFromConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public IntegrationFlow proxyInboundFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(serverFactory()))
.transform(Transformers.objectToString())
.<String, String>transform(s -> s.toUpperCase())
.handle((p, h) -> {
mapConnectionIds(h);
return p;
})
.handle(Tcp.outboundAdapter(threadConnectionFactory()))
.get();
}
#Bean
public IntegrationFlow proxyOutboundFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(threadConnectionFactory()))
.transform(Transformers.objectToString())
.<String, String>transform(s -> s.toUpperCase())
.enrichHeaders(e -> e
.headerExpression(IpHeaders.CONNECTION_ID, "#toFromConnectionMappings.get(headers['"
+ IpHeaders.CONNECTION_ID + "'])").defaultOverwrite(true))
.handle(Tcp.outboundAdapter(serverFactory()))
.get();
}
private void mapConnectionIds(Map<String, Object> h) {
try {
TcpConnection connection = threadConnectionFactory().getConnection();
String mapping = toFromConnectionMappings().get(connection.getConnectionId());
String incomingCID = (String) h.get(IpHeaders.CONNECTION_ID);
if (mapping == null || !(mapping.equals(incomingCID))) {
System.out.println("Adding new mapping " + incomingCID + " to " + connection.getConnectionId());
toFromConnectionMappings().put(connection.getConnectionId(), incomingCID);
fromToConnectionMappings().put(incomingCID, connection.getConnectionId());
}
}
catch (Exception e) {
e.printStackTrace();
}
}
#Bean
public ThreadAffinityClientConnectionFactory threadConnectionFactory() {
return new ThreadAffinityClientConnectionFactory(clientFactory()) {
#Override
public boolean isSingleUse() {
return false;
}
};
}
#Bean
public AbstractServerConnectionFactory serverFactory() {
return Tcp.netServer(1234).get();
}
#Bean
public AbstractClientConnectionFactory clientFactory() {
AbstractClientConnectionFactory clientFactory = Tcp.netClient("localhost", 1235).get();
clientFactory.setSingleUse(true);
return clientFactory;
}
#Bean
public IntegrationFlow serverFlow() {
return IntegrationFlows.from(Tcp.inboundGateway(Tcp.netServer(1235)))
.transform(Transformers.objectToString())
.<String, String>transform(p -> p + p)
.get();
}
#Bean
public ApplicationListener<TcpConnectionCloseEvent> closer() {
return e -> {
if (fromToConnectionMappings().containsKey(e.getConnectionId())) {
String key = fromToConnectionMappings().remove(e.getConnectionId());
toFromConnectionMappings().remove(key);
System.out.println("Removed mapping " + e.getConnectionId() + " to " + key);
threadConnectionFactory().releaseConnection();
}
};
}
}
EDIT3
Works fine for me with a MapJsonSerializer.
#SpringBootApplication
public class So51200675Application {
public static void main(String[] args) {
SpringApplication.run(So51200675Application.class, args).close();
}
#Bean
public ApplicationRunner runner() {
return args -> {
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("{\"foo\":\"bar\"}\n".getBytes());
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
System.out.println(reader.readLine());
socket.close();
};
}
#Bean
public Map<String, String> fromToConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public Map<String, String> toFromConnectionMappings() {
return new ConcurrentHashMap<>();
}
#Bean
public MapJsonSerializer serializer() {
return new MapJsonSerializer();
}
#Bean
public IntegrationFlow proxyRequestFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(serverFactory()))
.<Map<String, String>, Map<String, String>>transform(m -> {
m.put("foo", m.get("foo").toUpperCase());
return m;
})
.handle((p, h) -> {
mapConnectionIds(h);
return p;
})
.handle(Tcp.outboundAdapter(threadConnectionFactory()))
.get();
}
#Bean
public IntegrationFlow proxyReplyFlow() {
return IntegrationFlows.from(Tcp.inboundAdapter(threadConnectionFactory()))
.<Map<String, String>, Map<String, String>>transform(m -> {
m.put("foo", m.get("foo").toLowerCase() + m.get("foo"));
return m;
})
.enrichHeaders(e -> e
.headerExpression(IpHeaders.CONNECTION_ID, "#toFromConnectionMappings.get(headers['"
+ IpHeaders.CONNECTION_ID + "'])").defaultOverwrite(true))
.handle(Tcp.outboundAdapter(serverFactory()))
.get();
}
private void mapConnectionIds(Map<String, Object> h) {
try {
TcpConnection connection = threadConnectionFactory().getConnection();
String mapping = toFromConnectionMappings().get(connection.getConnectionId());
String incomingCID = (String) h.get(IpHeaders.CONNECTION_ID);
if (mapping == null || !(mapping.equals(incomingCID))) {
System.out.println("Adding new mapping " + incomingCID + " to " + connection.getConnectionId());
toFromConnectionMappings().put(connection.getConnectionId(), incomingCID);
fromToConnectionMappings().put(incomingCID, connection.getConnectionId());
}
}
catch (Exception e) {
e.printStackTrace();
}
}
#Bean
public ThreadAffinityClientConnectionFactory threadConnectionFactory() {
return new ThreadAffinityClientConnectionFactory(clientFactory()) {
#Override
public boolean isSingleUse() {
return false;
}
};
}
#Bean
public AbstractServerConnectionFactory serverFactory() {
return Tcp.netServer(1234)
.serializer(serializer())
.deserializer(serializer())
.get();
}
#Bean
public AbstractClientConnectionFactory clientFactory() {
AbstractClientConnectionFactory clientFactory = Tcp.netClient("localhost", 1235)
.serializer(serializer())
.deserializer(serializer())
.get();
clientFactory.setSingleUse(true);
return clientFactory;
}
#Bean
public IntegrationFlow backEndEmulatorFlow() {
return IntegrationFlows.from(Tcp.inboundGateway(Tcp.netServer(1235)
.serializer(serializer())
.deserializer(serializer())))
.<Map<String, String>, Map<String, String>>transform(m -> {
m.put("foo", m.get("foo") + m.get("foo"));
return m;
})
.get();
}
#Bean
public ApplicationListener<TcpConnectionCloseEvent> closer() {
return e -> {
if (fromToConnectionMappings().containsKey(e.getConnectionId())) {
String key = fromToConnectionMappings().remove(e.getConnectionId());
toFromConnectionMappings().remove(key);
System.out.println("Removed mapping " + e.getConnectionId() + " to " + key);
threadConnectionFactory().releaseConnection();
}
};
}
}
and
Adding new mapping localhost:56998:1234:55c822a4-4252-45e6-9ef2-79263391f4be to localhost:1235:56999:3d520ca9-2f3a-44c3-b05f-e59695b8c1b0
{"foo":"barbarBARBAR"}
Removed mapping localhost:56998:1234:55c822a4-4252-45e6-9ef2-79263391f4be to localhost:1235:56999:3d520ca9-2f3a-44c3-b05f-e59695b8c1b0
Related
I need help, my app (client) connect to RabbitMQ server and when server shutdown, my app cannot start....
Listener can't created and app failed start.
Or when virtual host dont have queue my app cant start too
So my question
1) How to process exception in my config (all exception, i need was my app start if RabbitMQ server have problems)
2) What in my config look bad and need refactor
i use
Spring 4.2.9.RELEASE
org.springframework.amqp 2.0.5.RELEASE
Java 8
My 2 classes
1) Config for Beans RabbitMq
2) Listener annotation
#EnableRabbit
#Configuration
public class RabbitMQConfig {
#Bean
public ConnectionFactory connectionFactory() {
com.rabbitmq.client.ConnectionFactory factoryRabbit = new com.rabbitmq.client.ConnectionFactory();
factoryRabbit.setNetworkRecoveryInterval(10000);
factoryRabbit.setAutomaticRecoveryEnabled(true);
CachingConnectionFactory connectionFactory =
new CachingConnectionFactory(factoryRabbit);
connectionFactory.setHost("DRIVER_APP_IP");
connectionFactory.setPort(5672);
connectionFactory.setConnectionTimeout(5000);
connectionFactory.setRequestedHeartBeat(10);
connectionFactory.setUsername("user");
connectionFactory.setPassword("pass");
connectionFactory.setVirtualHost("/vhost");
return connectionFactory;
}
#Bean
public RabbitTemplate rabbitTemplate() {
try {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory());
rabbitTemplate.setRoutingKey(this.DRIVER_QUEUE);
rabbitTemplate.setQueue(this.DRIVER_QUEUE);
return rabbitTemplate;
} catch (Exception ex){
return new RabbitTemplate();
}
}
#Bean
public Queue queue() {
return new Queue(this.DRIVER_QUEUE);
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(3);
factory.setMaxConcurrentConsumers(10);
return factory;
}
}
#Component
public class RabbitMqListener {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = DRIVER_QUEUE, durable = "true"),
exchange = #Exchange(value = "exchange", ignoreDeclarationExceptions = "true", autoDelete = "true"))
)
public String balancer(byte[] message) throws InterruptedException {
String json = null;
try {
"something move"
} catch (Exception ex) {
}
}
I found solution my problem
First it's Bean container!
We need this
factory.setMissingQueuesFatal(false);
this property give our when queue lost on server RabbitMQ, our app don't crash and can start
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setMissingQueuesFatal(false);
factory.setConcurrentConsumers(3);
factory.setStartConsumerMinInterval(3000L);
factory.setMaxConcurrentConsumers(10);
factory.setRecoveryInterval(15000L);
factory.setStartConsumerMinInterval(1000L);
factory.setReceiveTimeout(10000L);
factory.setChannelTransacted(true);
return factory;
}
and second
#Component
public class RabbitMqListener {
#RabbitListener(containerFactory = "rabbitListenerContainerFactory", queues = DRIVER_QUEUE)
public String balancer(byte[] message) throws InterruptedException {
String json = null;
try {
"something move"
} catch (Exception ex) {
}
}
I set containerFactory and Queue in #RabbitListener and drop other propertys,
because i don't need it
I hope it's help somebody, thank all for you attention and sorry for my English
I'm working on simple chat module for my application using Spring WebFlux with ReactiveMongoRepository on backend and Angular 4 on front. I'm able to receive data through WebSocketSession but after streaming all messages from db i want to keep the connection so i could update message list. Can anyone give me clues how to achieve that, or maybe i'm following wrong assumptions ?
Java Backend responsible for WebSocket, my subscriber only logs current state, nothing relevant there:
WebFluxConfiguration:
#Configuration
#EnableWebFlux
public class WebSocketConfig {
private final WebSocketHandler webSocketHandler;
#Autowired
public WebSocketConfig(WebSocketHandler webSocketHandler) {
this.webSocketHandler = webSocketHandler;
}
#Bean
#Primary
public HandlerMapping webSocketMapping() {
Map<String, Object> map = new HashMap<>();
map.put("/websocket-messages", webSocketHandler);
SimpleUrlHandlerMapping mapping = new SimpleUrlHandlerMapping();
mapping.setOrder(10);
mapping.setUrlMap(map);
return mapping;
}
#Bean
public WebSocketHandlerAdapter handlerAdapter() {
return new WebSocketHandlerAdapter();
}
}
WebSocketHandler Implementation
#Component
public class MessageWebSocketHandler implements WebSocketHandler {
private final MessageRepository messageRepository;
private ObjectMapper mapper = new ObjectMapper();
private MessageSubscriber subscriber = new MessageSubscriber();
#Autowired
public MessageWebSocketHandler(MessageRepository messageRepository) {
this.messageRepository = messageRepository;
}
#Override
public Mono<Void> handle(WebSocketSession session) {
session.receive()
.map(WebSocketMessage::getPayloadAsText)
.map(this::toMessage)
.subscribe(subscriber::onNext, subscriber:: onError, subscriber::onComplete);
return session.send(
messageRepository.findAll()
.map(this::toJSON)
.map(session::textMessage));
}
private String toJSON(Message message) {
try {
return mapper.writeValueAsString(message);
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}
private Message toMessage(String json) {
try {
return mapper.readValue(json, Message.class);
} catch (IOException e) {
throw new RuntimeException("Invalid JSON:" + json, e);
}
}
}
and MongoRepo
#Repository
public interface MessageRepository extends
ReactiveMongoRepository<Message,String> {
}
FrontEnd Handling:
#Injectable()
export class WebSocketService {
private subject: Rx.Subject<MessageEvent>;
constructor() {
}
public connect(url): Rx.Subject<MessageEvent> {
if (!this.subject) {
this.subject = this.create(url);
console.log('Successfully connected: ' + url);
}
return this.subject;
}
private create(url): Rx.Subject<MessageEvent> {
const ws = new WebSocket(url);
const observable = Rx.Observable.create(
(obs: Rx.Observer<MessageEvent>) => {
ws.onmessage = obs.next.bind(obs);
ws.onerror = obs.error.bind(obs);
ws.onclose = obs.complete.bind(obs);
return ws.close.bind(ws);
});
const observer = {
next: (data: Object) => {
if (ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify(data));
}
}
};
return Rx.Subject.create(observer, observable);
}
}
in other service i'm mapping observable from response to my type
constructor(private wsService: WebSocketService) {
this.messages = <Subject<MessageEntity>>this.wsService
.connect('ws://localhost:8081/websocket-messages')
.map((response: MessageEvent): MessageEntity => {
const data = JSON.parse(response.data);
return new MessageEntity(data.id, data.user_id, data.username, data.message, data.links);
});
}
and finally subscribtion with send function which i can't use because of closed connection:
ngOnInit() {
this.messages = [];
this._ws_subscription = this.chatService.messages.subscribe(
(message: MessageEntity) => {
this.messages.push(message);
},
error2 => {
console.log(error2.json());
},
() => {
console.log('Closed');
}
);
}
sendTestMessage() {
this.chatService.messages.next(new MessageEntity(null, '59ca30ac87e77d0f38237739', 'mickl', 'test message angular', null));
}
Assuming your chat messages are being persisted to the datastore as they're being received, you could use the tailable cursors feature in Spring Data MongoDB Reactive (see reference documentation).
So you could create a new method on your repository like:
public interface MessageRepository extends ReactiveSortingRepository< Message, String> {
#Tailable
Flux<Message> findWithTailableCursor();
}
Note that tailable cursors have some limitations: you mongo collection needs to be capped and entries are streamed in their order of insertion.
Spring WebFlux websocket support does not yet support STOMP nor message brokers, but this might be a better choice for such a use case.
I have a Python TCP Socket server service which:
Allows only one client connection at time;
Its inputstream/outputstream operates independently.
On the other side, I have a Java Spring Boot client application using Spring Integration. My actual TCP Socket configurator
implementation uses:
#MessagingGateway(defaultRequestChannel = REQUEST_CHANNEL, errorChannel = ERROR_CHANNEL)
public interface ClientGtw {
Future<Response> send(Request request);
}
#Bean
#ServiceActivator(inputChannel = REQUEST_CHANNEL)
public MessageHandler outboundGateway(TcpNioClientConnectionFactory connectionFactory) {
TcpOutboundGateway gateway = new TcpOutboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestTimeout(TimeUnit.SECONDS.toMillis(timeout));
gateway.setRemoteTimeout(TimeUnit.SECONDS.toMillis(timeout));
return gateway;
}
#Bean
public TcpNioClientConnectionFactory clientConnectionFactory(AppConfig config) {
Host host = getHost(config);
TcpNioClientConnectionFactory factory = new TcpNioClientConnectionFactory(host.name, host.port);
factory.setSingleUse(false);
factory.setSoTimeout((int) TimeUnit.SECONDS.toMillis(timeout));
SerializerDeserializer sd = new SerializerDeserializer();
factory.setDeserializer(sd);
factory.setSerializer(sd);
return factory;
}
This actual approach works fine, however, when a request is sent it hangs the connection until a response is received. This is a problem due the fact that some times a request can get too much time to receive a response and the system has other requests incomming whose response can be achieved faster. I would like to send and receive as much as possible requests and responses independetly (decoupled between them). The object transported (serialized and deserialized) contains a key pair that can do the correct correlation.
TL;DR: How to implement an Asynchronous requests/responses over the same connection?
The Spring TcpOutboundGateway javadoc mentions: Use a pair of outbound/inbound adapters for that use case.
So, in addition to the declaration above:
1st Attempt
#Bean
public TcpInboundGateway inboundGateway(AbstractServerConnectionFactory connectionFactory) {
TcpInboundGateway gateway = new TcpInboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestTimeout(TimeUnit.SECONDS.toMillis(timeout));
return gateway;
}
#Bean
public AbstractServerConnectionFactory serverFactory(AppConfig config) {
Host host = getHost(config);
AbstractServerConnectionFactory connectionFactory = new TcpNetServerConnectionFactory(host.port);
connectionFactory.setSingleUse(true);
connectionFactory.setSoTimeout(timeout);
return connectionFactory;
}
The requests are blocked until a response is delivered as before.
2nd Attempt
#Bean
public TcpInboundGateway inboundGateway(TcpNioClientConnectionFactory connectionFactory) {
TcpInboundGateway gateway = new TcpInboundGateway();
gateway.setConnectionFactory(connectionFactory);
gateway.setRequestTimeout(TimeUnit.SECONDS.toMillis(timeout));
gateway.setClientMode(true);
return gateway;
}
org.springframework.integration.ip.tcp.connection.TcpNioClientConnectionFactory may only be used by one inbound adapter
Any clue?
Use a pair of channel adapters instead of an outbound gateway. Instead of using a MessagingGateway, you can do the correlation yourself in your application, or you can use the same technique as is used in the tcp-client-server-multiplex sample app. It uses an aggregator to aggregate a copy of the outbound message with an inbound message, replying to the gateway.
It's old, and uses XML configuration, but the same techniques apply.
<publish-subscribe-channel id="input" />
<ip:tcp-outbound-channel-adapter id="outAdapter.client"
order="2"
channel="input"
connection-factory="client" /> <!-- Collaborator -->
<!-- Also send a copy to the custom aggregator for correlation and
so this message's replyChannel will be transferred to the
aggregated message.
The order ensures this gets to the aggregator first -->
<bridge input-channel="input" output-channel="toAggregator.client"
order="1"/>
<!-- Asynch receive reply -->
<ip:tcp-inbound-channel-adapter id="inAdapter.client"
channel="toAggregator.client"
connection-factory="client" /> <!-- Collaborator -->
<!-- dataType attribute invokes the conversion service, if necessary -->
<channel id="toAggregator.client" datatype="java.lang.String" />
<aggregator input-channel="toAggregator.client"
output-channel="toTransformer.client"
expire-groups-upon-completion="true"
expire-groups-upon-timeout="true"
discard-channel="noResponseChannel"
group-timeout="1000"
correlation-strategy-expression="payload.substring(0,3)"
release-strategy-expression="size() == 2" />
<channel id="noResponseChannel" />
<service-activator input-channel="noResponseChannel" ref="echoService" method="noResponse" />
<transformer input-channel="toTransformer.client"
expression="payload.get(1)"/> <!-- The response is always second -->
(This simple sample correlates on the first 3 bytes).
Gary, thanks for your guidance.
To solve this issue is important to first understand Messaging Channel types.
So, in the configurer class:
#Bean(name = REQUEST_CHANNEL)
public DirectChannel sender() {
return new DirectChannel();
}
#Bean(name = RESPONSE_CHANNEL)
public PollableChannel receiver() {
return new QueueChannel();
}
#Bean
#ServiceActivator(inputChannel = REQUEST_CHANNEL)
public TcpSendingMessageHandler outboundClient(TcpNioClientConnectionFactory connectionFactory) {
TcpSendingMessageHandler outbound = new TcpSendingMessageHandler();
outbound.setConnectionFactory(connectionFactory);
outbound.setRetryInterval(TimeUnit.SECONDS.toMillis(timeout));
outbound.setClientMode(true);
return outbound;
}
#Bean
public TcpReceivingChannelAdapter inboundClient(TcpNioClientConnectionFactory connectionFactory) {
TcpReceivingChannelAdapter inbound = new TcpReceivingChannelAdapter();
inbound.setConnectionFactory(connectionFactory);
inbound.setRetryInterval(TimeUnit.SECONDS.toMillis(timeout));
inbound.setOutputChannel(receiver());
inbound.setClientMode(true);
return inbound;
}
This scratch #Singleton class illustrates how to operate the requests and responses (considering that requests and responses contains a UID to correlate them):
#Autowired
private DirectChannel sender;
#Autowired
private PollableChannel receiver;
private BlockingQueue<Request> requestPool = new LinkedBlockingQueue<>();
private Map<String, Response> responsePool = Collections.synchronizedMap(new HashMap<>());
#PostConstruct
private void init() {
new Receiver().start();
new Sender().start();
}
/*
* It can be called as many as necessary without hanging for a response
*/
public void send(Request req) {
requestPool.add(req);
}
/*
* Check for a response until a socket timout
*/
public Response receive(String key) {
Response res = responsePool.get(key);
if (res != null) {
responsePool.remove(key);
}
return res;
}
private class Receiver extends Thread {
#Override
public void run() {
while (true) {
try {
tcpReceive();
Thread.sleep(250);
} catch (InterruptedException e) { }
}
}
private void tcpReceive() {
Response res = (Message<Response>) receiver.receive();
if (res != null) {
responsePool.put(res.getUID(), res);
}
}
}
private class Sender extends Thread {
#Override
public void run() {
while (true) {
try {
tcpSend();
Thread.sleep(250);
} catch (InterruptedException e) { }
}
}
private void tcpSend() {
Request req = requestPool.poll(125, TimeUnit.MILLISECONDS);
if (req != null) {
sender.send(MessageBuilder.withPayload(req).build());
}
}
}
UPDATED
I forgot to mention this:
#Bean
public TcpNioClientConnectionFactory clientConnectionFactory(Config config) {
// Get host properties
Host host = getHost(config);
// Create socket factory
TcpNioClientConnectionFactory factory = new TcpNioClientConnectionFactory(host.name, host.port);
factory.setSingleUse(false); // IMPORTANT FOR SINGLE CHANNEL
factory.setSoTimeout((int) TimeUnit.SECONDS.toMillis(timeout));
return factory;
}
Feel free to make any considerations.
The Spring framework support tcp connection as well , i wrote code below to setup a simple socket server , i am confused about adding below futures to my socket server :
authorizing clients based on a unique identifier ( for example a client secret received from client, maybe using TCP Connection Events )
send a message directly to specific client (based on identifier)
broadcast a message
UPDATE :
Config.sendMessage added to send message to single client
Config.broadCast added to broadcast message
authorizeIncomingConnection to authorize clients , accept or reject connections
tcpConnections static filed added to keep tcpEvent sources
Questions !
is using tcpConnections HashMap good idea ?!
is the authorization method i implemented a good one ?!
Main.java
#SpringBootApplication
public class Main {
public static void main(final String[] args) {
SpringApplication.run(Main.class, args);
}
}
Config.java
#EnableIntegration
#IntegrationComponentScan
#Configuration
public class Config implements ApplicationListener<TcpConnectionEvent> {
private static final Logger LOGGER = Logger.getLogger(Config.class.getName());
#Bean
public AbstractServerConnectionFactory AbstractServerConnectionFactory() {
return new TcpNetServerConnectionFactory(8181);
}
#Bean
public TcpInboundGateway TcpInboundGateway(AbstractServerConnectionFactory connectionFactory) {
TcpInboundGateway inGate = new TcpInboundGateway();
inGate.setConnectionFactory(connectionFactory);
inGate.setRequestChannel(getMessageChannel());
return inGate;
}
#Bean
public MessageChannel getMessageChannel() {
return new DirectChannel();
}
#MessageEndpoint
public class Echo {
#Transformer(inputChannel = "getMessageChannel")
public String convert(byte[] bytes) throws Exception {
return new String(bytes);
}
}
private static ConcurrentHashMap<String, TcpConnection> tcpConnections = new ConcurrentHashMap<>();
#Override
public void onApplicationEvent(TcpConnectionEvent tcpEvent) {
TcpConnection source = (TcpConnection) tcpEvent.getSource();
if (tcpEvent instanceof TcpConnectionOpenEvent) {
LOGGER.info("Socket Opened " + source.getConnectionId());
tcpConnections.put(tcpEvent.getConnectionId(), source);
if (!authorizeIncomingConnection(source.getSocketInfo())) {
LOGGER.warn("Socket Rejected " + source.getConnectionId());
source.close();
}
} else if (tcpEvent instanceof TcpConnectionCloseEvent) {
LOGGER.info("Socket Closed " + source.getConnectionId());
tcpConnections.remove(source.getConnectionId());
}
}
private boolean authorizeIncomingConnection(SocketInfo socketInfo) {
//Authorization Logic , Like Ip,Mac Address WhiteList or anyThing else !
return (System.currentTimeMillis() / 1000) % 2 == 0;
}
public static String broadCast(String message) {
Set<String> connectionIds = tcpConnections.keySet();
int successCounter = 0;
int FailureCounter = 0;
for (String connectionId : connectionIds) {
try {
sendMessage(connectionId, message);
successCounter++;
} catch (Exception e) {
FailureCounter++;
}
}
return "BroadCast Result , Success : " + successCounter + " Failure : " + FailureCounter;
}
public static void sendMessage(String connectionId, final String message) throws Exception {
tcpConnections.get(connectionId).send(new Message<String>() {
#Override
public String getPayload() {
return message;
}
#Override
public MessageHeaders getHeaders() {
return null;
}
});
}
}
MainController.java
#Controller
public class MainController {
#RequestMapping("/notify/{connectionId}/{message}")
#ResponseBody
public String home(#PathVariable String connectionId, #PathVariable String message) {
try {
Config.sendMessage(connectionId, message);
return "Client Notified !";
} catch (Exception e) {
return "Failed To Notify Client , cause : \n " + e.toString();
}
}
#RequestMapping("/broadCast/{message}")
#ResponseBody
public String home(#PathVariable String message) {
return Config.broadCast(message);
}
}
Usage :
Socket Request/Response Mode
notify single client
http://localhost:8080/notify/{connectionId}/{message}
broadCast
http://localhost:8080/broadCast/{message}
The TcpConnectionOpenEvent contains a connectionId property. Each message coming from that client will have the same property in the IpHeaders.CONNECTION_ID message header.
Add a custom router that keeps track of the logged-on state of each connection.
Lookup the connection id and if not authenticated, route to a challenge/response subflow.
When authenticated, route to the normal flow.
To use arbitrary messaging (rather than request/response) use a TcpReceivingChannelAdapter and TcpSendingMessageHandler instead of an inbound gateway. Both configured to use the same connection factory. For each message sent to the message handler, add the IpHeaders.CONNECTION_ID header to target the specific client.
To broadcast, send a message for each connection id.
I have a requirement where i need to look for a file continuously at unix location.Once its available then i need to parse it and convert to some json format.This needs to be done using Spring integration - DSL.
Following is the piece of code I got from spring site but it shows following exception:
o.s.integration.handler.LoggingHandler: org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'application.processFileChannel'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
Below is the code:
#SpringBootApplication
public class FileReadingJavaApplication {
public static void main(String[] args) {
new SpringApplicationBuilder(FileReadingJavaApplication.class)
.web(false)
.run(args);
}
#Bean
public IntegrationFlow fileReadingFlow() {
return IntegrationFlows
.from(s -> s.file(new File("Y://"))
.patternFilter("*.txt"),
e -> e.poller(Pollers.fixedDelay(1000)))
.transform(Transformers.fileToString())
.channel("processFileChannel")
.get();
}
}
New Code:
#SpringBootApplication
public class SpringIntegration {
public static void main(String[] args) {
new SpringApplicationBuilder(SpringIntegration.class)
.web(false)
.run(args);
}
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost("ip");
factory.setPort(port);
factory.setUser("username");
factory.setPassword("pwd");
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
#Bean
public SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sftpSessionFactory());
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.setRemoteDirectory("remote dir");
fileSynchronizer.setFilter(new SftpSimplePatternFileListFilter("*.txt"));
return fileSynchronizer;
}
#Bean
#InboundChannelAdapter(channel = "sftpChannel", poller = #Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public MessageSource ftpMessageSource() {
SftpInboundFileSynchronizingMessageSource source =
new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer());
source.setLocalFilter(new AcceptOnceFileListFilter<File>());
source.setLocalDirectory(new File("Local directory"));
return source;
}
#Bean
#ServiceActivator(inputChannel = "fileInputChannel")
public MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println("File Name : "+message.getPayload());
}
};
}
#Bean
public static StandardIntegrationFlow processFileFlow() {
return IntegrationFlows
.from("fileInputChannel").split()
.handle("fileProcessor", "process").get();
}
#Bean
#InboundChannelAdapter(value = "fileInputChannel", poller = #Poller(fixedDelay = "1000"))
public MessageSource<File> fileReadingMessageSource() {
AcceptOnceFileListFilter<File> filters =new AcceptOnceFileListFilter<>();
FileReadingMessageSource source = new FileReadingMessageSource();
source.setAutoCreateDirectory(true);
source.setDirectory(new File("Local directory"));
source.setFilter(filters);
return source;
}
#Bean
public FileProcessor fileProcessor() {
return new FileProcessor();
}
#Bean
#ServiceActivator(inputChannel = "fileInputChannel")
public AmqpOutboundEndpoint amqpOutbound(AmqpTemplate amqpTemplate) {
AmqpOutboundEndpoint outbound = new AmqpOutboundEndpoint(amqpTemplate);
outbound.setExpectReply(true);
outbound.setRoutingKey("foo"); // default exchange - route to queue 'foo'
return outbound;
}
#MessagingGateway(defaultRequestChannel = "amqpOutboundChannel")
public interface MyGateway {
String sendToRabbit(String data);
}
}
FileProcessor:
public class FileProcessor {
public void process(Message<String> msg) {
String content = msg.getPayload();
JSONObject jsonObject ;
Map<String, String> dataMap = new HashMap<String, String>();
for(int i=0;i<=content.length();i++){
String userId = content.substring(i+5,i+16);
dataMap = new HashMap<String, String>();
dataMap.put("username", username.trim());
i+=290; //each record of size 290 in file
jsonObject = new JSONObject(dataMap);
System.out.println(jsonObject);
}
}
}
Your code is correct , but an exception tells you that there is need something what will read messages from the direct channel "processFileChannel".
Please, read more about different channel types in the Spring Integration Reference Manual.
EDIT
One of first class citizen in Spring Integration is MessageChannel abstraction. See EIP for more information.
The definition like .channel("processFileChannel") mean declare DirectChannel. This kind of channel means accept message on the send and perform handling directly just in send process. In the raw Java words it may sound like: call one service from another. Throw NPE if the another hasn't been autowired.
So, if you use DirectChannel for the output, you should declare somewhere a subscriber for it. I don't know what is your logic, but that is how it works and no other choice to fix Dispatcher has no subscribers for channel.
Although you can use some other MessageChannel type. But for this purpose you should read more doc, e.g. Mark Fisher's Spring Integration in Action.