This is a new question following up on this older question and answer (specifically the comment that says "don't comment on old answers, ask a new question"), as well as these examples in GitHub.
I know the answer and examples are minimal working "trivial examples", but I don't know enough about how "things work in Spring" (or should work) to understand how to decompose those generic, trivial examples into separate servers and clients that suit my purpose. I currently have a working Spring-Boot daemon application that is client to / calls on (without any "spring integration") a legacy daemon application over a TCP Socket connection. It's all working, running in production.
But now I am tasked with migrating the legacy daemon to Spring Boot too. So I only need to configure and set up a cached/pooled TCP connection "socket listener" on the server-side. However, the "client parts" of the existing (self contained) examples confuse me. In my case the "client side" (the existing Spring Boot daemon) is not going to change and is a separate app on a separate server, I only need to set up / configure the "server-side" of the socket connection (the "legacy-daemon freshly migrated to Spring Boot" daemon).
I've copied this example configuration (exactly) into my legacy-migration project
#EnableIntegration
#IntegrationComponentScan
#Configuration
public static class Config {
#Value(${some.port})
private int port;
#MessagingGateway(defaultRequestChannel="toTcp")
public interface Gateway {
String viaTcp(String in);
}
#Bean
#ServiceActivator(inputChannel="toTcp")
public MessageHandler tcpOutGate(AbstractClientConnectionFactory connectionFactory) {
TcpOutboundGateway gate = new TcpOutboundGateway();
gate.setConnectionFactory(connectionFactory);
gate.setOutputChannelName("resultToString");
return gate;
}
#Bean
public TcpInboundGateway tcpInGate(AbstractServerConnectionFactory connectionFactory) {
TcpInboundGateway inGate = new TcpInboundGateway();
inGate.setConnectionFactory(connectionFactory);
inGate.setRequestChannel(fromTcp());
return inGate;
}
#Bean
public MessageChannel fromTcp() {
return new DirectChannel();
}
#MessageEndpoint
public static class Echo {
#Transformer(inputChannel="fromTcp", outputChannel="toEcho")
public String convert(byte[] bytes) {
return new String(bytes);
}
#ServiceActivator(inputChannel="toEcho")
public String upCase(String in) {
return in.toUpperCase();
}
#Transformer(inputChannel="resultToString")
public String convertResult(byte[] bytes) {
return new String(bytes);
}
}
#Bean
public AbstractClientConnectionFactory clientCF() {
return new TcpNetClientConnectionFactory("localhost", this.port);
}
#Bean
public AbstractServerConnectionFactory serverCF() {
return new TcpNetServerConnectionFactory(this.port);
}
}
...and the project will start on 'localhost' and "listen" on port 10000. But, when I connect to the socket from another local app and send some test text, nothing returns until I shut down the socket listening app. Only after the socket listening app starts shutting down does a response (the correct 'uppercased' result) go back to the sending app.
How do I get the "listener" to return a response to the "sender" normally, without shutting down the listener's server first?
Or can someone please provide an example that ONLY shows the server-side (hopefully annotation based) setup? (Or edit the example so the server and client are clearly decoupled?)
Samples usually contain both the client and server because it's easier that way. But there is nothing special about breaking apart the client and server sides. Here's an example using the Java DSL:
#SpringBootApplication
public class So60443538Application {
public static void main(String[] args) {
SpringApplication.run(So60443538Application.class, args);
}
#Bean
public IntegrationFlow server() {
return IntegrationFlows.from(Tcp.inboundGateway(Tcp.netServer(1234)))
.transform(Transformers.objectToString()) // byte[] -> String
.<String, String>transform(p -> p.toUpperCase())
.get();
}
}
#SpringBootApplication
public class So604435381Application {
private static final Logger LOG = LoggerFactory.getLogger(So604435381Application.class);
public static void main(String[] args) {
SpringApplication.run(So604435381Application.class, args);
}
#Bean
public IntegrationFlow client() {
return IntegrationFlows.from(Gate.class)
.handle(Tcp.outboundGateway(Tcp.netClient("localhost", 1234)))
.transform(Transformers.objectToString())
.get();
}
#Bean
#DependsOn("client")
public ApplicationRunner runner(Gate gateway) {
return args -> LOG.info(gateway.exchange("foo"));
}
}
interface Gate {
String exchange(String in);
}
2020-02-28 09:14:04.158 INFO 35974 --- [ main] com.example.demo.So604435381Application : FOO
Related
i have a problem, i do not know how to set the host dynamically and doing RPC operation on different host
Here is the situation
I have a multiple RabbitMQ running on different servers and networks (i.e 192.168.1.0/24, 192.168.2.0/24).
The behavior would be i have a list of IP address which i will perform an RPC with.
So, for each entry in the ip address list, i want to perform a convertSendAndReceive and process the reply and so on.
Tried some codes in documentation but it seems it does not work even the invalid address (addresses that don't have a valid RabbitMQ running, or is not event existing on the network, for example 1.1.1.1) gets received by a valid RabbitMQ (running on 192.168.1.1 for example)
Note: I can successfully perform RPC call on correct address, however, i can also successfully perform RPC call on invalid address which im not suppose to
Anyone has any idea about this?
Here is my source
TaskSchedulerConfiguration.java
#Configuration
#EnableScheduling
public class TaskSchedulerConfiguration {
#Autowired
private IpAddressRepo ipAddressRepo;
#Autowired
private RemoteProcedureService remote;
#Scheduled(fixedDelayString = "5000", initialDelay = 2000)
public void scheduledTask() {
ipAddressRepo.findAll().stream()
.forEach(ipaddress -> {
boolean status = false;
try {
remote.setIpAddress(ipaddress);
remote.doSomeRPC();
} catch (Exception e) {
logger.debug("Unable to Connect to licenser server: {}", license.getIpaddress());
logger.debug(e.getMessage(), e);
}
});
}
}
RemoteProcedureService.java
#Service
public class RemoteProcedureService {
#Autowired
private RabbitTemplate template;
#Autowired
private DirectExchange exchange;
public boolean doSomeRPC() throws JsonProcessingException {
//I passed this.factory.getHost() so that i will know if only the valid ip address will be received by the other side
//at this point, other side receives invalid ipaddress which supposedly will not be receive by the oher side
boolean response = (Boolean) template.convertSendAndReceive(exchange.getName(), "rpc", this.factory.getHost());
return response;
}
public void setIpAddress(String host) {
factory.setHost(host);
factory.setCloseTimeout(prop.getRabbitMQCloseConnectTimeout());
factory.setPort(prop.getRabbitMQPort());
factory.setUsername(prop.getRabbitMQUsername());
factory.setPassword(prop.getRabbitMQPassword());
template.setConnectionFactory(factory);
}
}
AmqpConfiguration.java
#Configuration
public class AmqpConfiguration {
public static final String topicExchangeName = "testExchange";
public static final String queueName = "rpc";
#Autowired
private LicenseVisualizationProperties prop;
//Commented this out since this will only be assigne once
//i need to achieve to set it dynamically in order to send to different hosts
//so put it in RemoteProcedureService.java, but it never worked
// #Bean
// public ConnectionFactory connectionFactory() {
// CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
// connectionFactory.setCloseTimeout(prop.getRabbitMQCloseConnectTimeout());
// connectionFactory.setPort(prop.getRabbitMQPort());
// connectionFactory.setUsername(prop.getRabbitMQUsername());
// connectionFactory.setPassword(prop.getRabbitMQPassword());
// return connectionFactory;
// }
#Bean
public DirectExchange exhange() {
return new DirectExchange(topicExchangeName);
}
}
UPDATE 1
It seems that, during the loop, when an valid ip is set in the CachingConnectionFactory succeeding ip addressing loop, regardliess if valid or invalid, gets received by the first valid ip set in CachingConnectionFactory
UPDATE 2
I found out that once it can establish a successfully connection, it will not create a new connection. How do you force RabbitTemplate to establish a new connection?
It's a rather strange use case and won't perform very well; you would be better to have a pool of connection factories and templates.
However, to answer your question:
Call resetConnection() to close the connection.
We are using spring cloude stream 2.0 & Kafka as a message broker.
We've implemented a circuit breaker which stops the Application context, for cases where the target system (DB or 3rd party API) is unavilable, as suggested here: Stop Spring Cloud Stream #StreamListener from listening when target system is down
Now in spring cloud stream 2.0 there is a way to manage the lifecycle of binder using actuator: Binding visualization and control
Is it possible to control the binder lifecycle from the code, means in case target server is down, to pause the binder, and when it's up, to resume?
Sorry, I misread your question.
You can auto wire the BindingsEndpoint but, unfortunately, its State enum is private so you can't call changeState() programmatically.
I have opened an issue for this.
EDIT
You can do it with reflection, but it's a bit ugly...
#SpringBootApplication
#EnableBinding(Sink.class)
public class So53476384Application {
public static void main(String[] args) {
SpringApplication.run(So53476384Application.class, args);
}
#Autowired
BindingsEndpoint binding;
#Bean
public ApplicationRunner runner() {
return args -> {
Class<?> clazz = ClassUtils.forName("org.springframework.cloud.stream.endpoint.BindingsEndpoint$State",
So53476384Application.class.getClassLoader());
ReflectionUtils.doWithMethods(BindingsEndpoint.class, method -> {
try {
method.invoke(this.binding, "input", clazz.getEnumConstants()[2]); // PAUSE
}
catch (InvocationTargetException e) {
e.printStackTrace();
}
}, method -> method.getName().equals("changeState"));
};
}
#StreamListener(Sink.INPUT)
public void listen(String in) {
}
}
I'm using Spring and I have a JMS queue to send messages from client to server. I'd like to stop the messages from being sent when the server is down, and resend them when it's back up.
I know it was asked before but I can't make it work. I created a JmsListener and gave it an ID, but I cannot get it's container in order to stop\start it.
#Resource(name="testId")
private AbstractJmsListeningContainer _probeUpdatesListenerContainer;
public void testSendJms() {
_jmsTemplate.convertAndSend("queue", "working");
}
#JmsListener(destination="queue", id="testId")
public void testJms(String s) {
System.out.println("Received JMS: " + s);
}
The container bean is never created. I also tried getting it from the context or using #Autowired and #Qualifier("testId") with no luck.
How can I get the container?
You need #EnableJms on one of your configuration classes.
You need a jmsListenerContainerFactory bean.
You can stop and start the containers using the JmsListenerEndpointRegistry bean.
See the Spring documentation.
If you use CachingConnectionFactory in your project, you need to call the resetConnection() method between stop and restart, otherwise the old physical connection will remain open, and it will be reused when you restart.
I used JmsListenerEndpointRegistry. Here's my example. I hope this will help.
Bean configuration in JmsConfiguration.java. I changed default autostart option.
#Bean(name="someQueueScheduled")
public DefaultJmsListenerContainerFactory odsContractScheduledQueueContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
ActiveMQConnectionFactory cf = new ActiveMQConnectionFactory(someActiveMQ);
Map<String, Class<?>> typeIds = new HashMap<>();
typeIds.put(SomeDTO);
factory.setMessageConverter(messageConverter(Collections.unmodifiableMap(typeIds)));
factory.setPubSubDomain(false);
factory.setConnectionFactory(cf);
factory.setAutoStartup(false);
return factory;
}
Invoke in SomeFacade.java
public class SomeFacade {
#Autowired
JmsListenerEndpointRegistry someUpdateListener;
public void stopSomeUpdateListener() {
MessageListenerContainer container = someUpdateListener.getListenerContainer("someUpdateListener");
container.stop();
}
public void startSomeUpdateListener() {
MessageListenerContainer container = someUpdateListener.getListenerContainer("someUpdateListener");
container.start();
}
}
JmsListener implementation in SomeService.java
public class SomeService {
#JmsListener(id = "someUpdateListener",
destination = "${some.someQueueName}",
containerFactory ="someQueueScheduled")
public void pullUpdateSomething(SomeDTO someDTO) {
}
}
My TCP server built using Spring integration works great. I use ByteArrayLengthHeaderSerializer as a serializer.
Once in a while, client data comes very slowly making the server respond very slowly.
I would like to wait a maximum of 5 seconds to read each byte of the data from the client. If the data byte does not come in 5 seconds, I would like to send NAK.
How to set the timeout of 5 seconds? Where should it be set?
Do I need to customize serializer?
Here is my spring context:
<int-ip:tcp-connection-factory id="crLfServer"
type="server"
port="${availableServerSocket}"
single-use="true"
so-timeout="10000"
using-nio="false"
serializer="connectionSerializeDeserialize"
deserializer="connectionSerializeDeserialize"
so-linger="2000"/>
<bean id="connectionSerializeDeserialize" class="org.springframework.integration.ip.tcp.serializer.ByteArrayLengthHeaderSerializer"/>
<int-ip:tcp-inbound-gateway id="gatewayCrLf"
connection-factory="crLfServer"
request-channel="serverBytes2StringChannel"
error-channel="errorChannel"
reply-timeout="10000"/> <!-- reply-timeout works on inbound-gateway -->
<int:channel id="toSA" />
<int:service-activator input-channel="toSA"
ref="myService"
method="prepare"/>
<int:object-to-string-transformer id="serverBytes2String"
input-channel="serverBytes2StringChannel"
output-channel="toSA"/>
<int:transformer id="errorHandler"
input-channel="errorChannel"
expression="payload.failedMessage.payload + ':' + payload.cause.message"/>
Thank you
You would need a custom deserializer; by default when the read times out (after the so-timeout) we close the socket. You would have to catch the timeout and return a partial message, with some information to tell the downstream flow to return the nack.
The deserializer does not have access to the connection so it can't send the nack itself.
You could do it in a custom subclass TcpMessageMapper, though - override toMessage().
That said, your solution might be brittle unless you close the socket anyway because the stream may still contain some data from the previous message, although with single-use true, I assume you are only sending one message per socket.
EDIT
#SpringBootApplication
public class So40408085Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext context = SpringApplication.run(So40408085Application.class, args);
context.getBean("toTcp", MessageChannel.class).send(new GenericMessage<>("foo"));
Thread.sleep(5000);
context.close();
}
#Bean
public TcpNetServerConnectionFactory server() {
TcpNetServerConnectionFactory server = new TcpNetServerConnectionFactory(1234);
server.setSoTimeout(1000);
server.setMapper(new TimeoutMapper()); // use 'mapper' attribute in XML
return server;
}
#Bean
public TcpInboundGateway inGate() {
TcpInboundGateway inGate = new TcpInboundGateway();
inGate.setConnectionFactory(server());
inGate.setRequestChannelName("inChannel");
return inGate;
}
#ServiceActivator(inputChannel = "inChannel")
public String upCase(byte[] in) {
return new String(in).toUpperCase();
}
#Bean
public TcpNetClientConnectionFactory client() {
TcpNetClientConnectionFactory client = new TcpNetClientConnectionFactory("localhost", 1234);
client.setSerializer(new ByteArrayLfSerializer()); // so the server will timeout - he's expecting CRLF
return client;
}
#Bean
#ServiceActivator(inputChannel = "toTcp")
public TcpOutboundGateway out() {
TcpOutboundGateway outGate = new TcpOutboundGateway();
outGate.setConnectionFactory(client());
outGate.setOutputChannelName("reply");
return outGate;
}
#ServiceActivator(inputChannel = "reply")
public void reply(byte[] in) {
System.out.println(new String(in));
}
public static class TimeoutMapper extends TcpMessageMapper {
#Override
public Message<?> toMessage(TcpConnection connection) throws Exception {
try {
return super.toMessage(connection);
}
catch (SocketTimeoutException e) {
connection.send(new GenericMessage<>("You took too long to send me data, sorry"));
connection.close();
return null;
}
}
}
}
Is there a way to change port used by tcp-inbound gateway on the fly? I'd like to set port and timeout used by tcp-inbound-gateway based on the configuration persisted in the database and have ability to change them on the fly without restarting an application. In order to do so I decided to use "publish-subscriber" pattern and extended TcpInboundGateway class:
public class RuntimeInboundGateway extends TcpInboundGateway implements SettingsSubscriber {
#Autowired
private Settings settings;
#PostConstruct
public void subscribe() {
settings.subscribe(this);
}
#Override
public void onSettingsChanged(Settings settings) {
this.stop();
AbstractByteArraySerializer serializer = new ByteArrayLfSerializer();
TcpNetServerConnectionFactory connectionFactory = new TcpNetServerConnectionFactory(settings.getPort());
connectionFactory.setSerializer(serializer);
connectionFactory.afterPropertiesSet();
this.setConnectionFactory(connectionFactory);
this.afterPropertiesSet();
this.start();
}
}
The settings object is a singleton bean and when it is changed the tcp inbound gateway starts indeed listening on the new port but looks like it doesn't send inbound messages further on the flow. Here is an excerpt from xml configuration:
<int-ip:tcp-connection-factory id="connFactory" type="server" port="${port}"
serializer="serializer"
deserializer="serializer"/>
<bean id="serializer" class="org.springframework.integration.ip.tcp.serializer.ByteArrayLfSerializer"/>
<bean id="inboundGateway" class="com.example.RuntimeInboundGateway">
<property name="connectionFactory" ref="connFactory"/>
<property name="requestChannel" ref="requestChannel"/>
<property name="replyChannel" ref="responseChannel"/>
<property name="errorChannel" ref="exceptionChannel"/>
<property name="autoStartup" value="true"/>
</bean>
There is logging-channel-adapter in the configuration which logs any requests to the service without any issues until the settings are changed. After that, it doesn't and I see no messages received though I'm able to connect to the new port by the telnet localhost <NEW_PORT>. Could somebody take a look and say how the desired behaviour can be achieved?
A quick look at your code indicated it should work ok, so I just wrote a quick Spring Boot app and it worked fine for me...
#SpringBootApplication
public class So40084223Application {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = SpringApplication.run(So40084223Application.class, args);
Socket socket = SocketFactory.getDefault().createSocket("localhost", 1234);
socket.getOutputStream().write("foo\r\n".getBytes());
socket.close();
QueueChannel queue = ctx.getBean("queue", QueueChannel.class);
System.out.println(queue.receive(10000));
ctx.getBean(MyInboundGateway.class).recycle(1235);
socket = SocketFactory.getDefault().createSocket("localhost", 1235);
socket.getOutputStream().write("fooo\r\n".getBytes());
socket.close();
System.out.println(queue.receive(10000));
ctx.close();
}
#Bean
public TcpNetServerConnectionFactory cf() {
return new TcpNetServerConnectionFactory(1234);
}
#Bean
public MyInboundGateway gate(TcpNetServerConnectionFactory cf) {
MyInboundGateway gate = new MyInboundGateway();
gate.setConnectionFactory(cf);
gate.setRequestChannel(queue());
return gate;
}
#Bean
public QueueChannel queue() {
return new QueueChannel();
}
public static class MyInboundGateway extends TcpInboundGateway implements ApplicationEventPublisherAware {
private ApplicationEventPublisher applicationEventPublisher;
#Override
public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) {
this.applicationEventPublisher = applicationEventPublisher;
}
public void recycle(int port) {
stop();
TcpNetServerConnectionFactory sf = new TcpNetServerConnectionFactory(port);
sf.setApplicationEventPublisher(this.applicationEventPublisher);
sf.afterPropertiesSet();
setConnectionFactory(sf);
afterPropertiesSet();
start();
}
}
}
I would turn on DEBUG logging to see if it gives you any clues.
You also might to want to explore using the new DSL dynamic flow registration instead. The tcp-dynamic-client shows how to use that technique to add/remove flow snippets on the fly. It's on the client side, but similar techniques can be used on the server side to register/unregister your gateway and connection factory.
The cause of the troubles is me. Since the deserializer is not specified in the code above the default one is used and it couldn't demarcate inbound messages from the input byte stream. Just one line connectionFactory.setDeserializer(serializer); solved the issue I spent a day on.