I have a controller with an endpoint that provide a flux like this reported below.
When the app is deployed to kubernetes, methods doOnCancel and doOnTerminate are not invoked.
Locally instead, it works like a charm (when the tab of the browser is closed as instance).
#Slf4j
#RestController
public class TestController {
...
#GetMapping(value = "/test", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> testStream() {
log.info("Requested test streaming");
return mySink.asFlux()
.startWith("INIT TEST")
.doOnCancel(() -> log.info("On cancel"))
.doOnTerminate(() -> log.info("On terminate"));
}
...
}
2022-08-06 18:25:42.115 INFO 3685 --- [ main] com.wuase.sinkdemo.SinkDemoApplication : Starting SinkDemoApplication using Java 1.8.0_252 on aniello-pc with PID 3685 (/home/pc/eclipse-workspace/sink-demo/target/classes started by pc in /home/pc/eclipse-workspace/sink-demo)
2022-08-06 18:25:42.124 INFO 3685 --- [ main] com.wuase.sinkdemo.SinkDemoApplication : No active profile set, falling back to 1 default profile: "default"
2022-08-06 18:25:44.985 INFO 3685 --- [ main] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port 8080
2022-08-06 18:25:45.018 INFO 3685 --- [ main] com.wuase.sinkdemo.SinkDemoApplication : Started SinkDemoApplication in 3.737 seconds (JVM running for 5.36)
2022-08-06 18:26:09.706 INFO 3685 --- [or-http-epoll-3] com.wuase.sinkdemo.TestController : Requested test streaming
2022-08-06 18:26:14.799 INFO 3685 --- [or-http-epoll-3] com.wuase.sinkdemo.TestController : On cancel
Has anyone encountred the same problem?
Any idea about that?
I recently wanted to add a new behavior in my project that uses spring-kafka.
The idea is really simple :
App1 create a new scenario name "SCENARIO_1" and publish this string in the topic "NEW_SCENARIO"
App1 publish some message on topic "APP2-SCENARIO_1" and "APP3-SCENARIO_1"
App2 (group-id=app2) listens on NEW_SCENARIO and creates a new consumer<Object,String> listening on a new topic "APP2-SCENARIO_1"
App3 (group-id=app3) listens on NEW_SCENARIO and creates a new consumer<Object,String> listening on a new topic "APP3-SCENARIO_1"
The goal is to create dynamically new topics and consumer. I cannot use spring kafka annotation since I need it to be dynamic so I did this :
#KafkaListener(topics = ScenarioTopics.NEW_SCENARIO)
public void receive(final String topic) {
logger.info("Get new scenario " + topic + ", creating new consumer");
TopicPartitionOffset topicPartitionOffset = new TopicPartitionOffset(
"APP2_" + topic, 1, 0L);
ContainerProperties containerProps = new ContainerProperties(topicPartitionOffset);
containerProps.setMessageListener((MessageListener<Object, String>) message -> {
// process my message
});
KafkaMessageListenerContainer<Object, String> container = new KafkaMessageListenerContainer<>(kafkaPeopleConsumerFactory, containerProps);
container.start();
}
And this does not work. I'm missing probably something, but I can't figure what.
Here I have some logs that tells me that the leader is not available, which is weird since I got the new scenario event.
2022-03-14 18:08:26.057 INFO 21892 --- [ntainer#0-0-C-1] o.l.b.v.c.c.i.k.KafkaScenarioListener : Get new scenario W4BdDBEowY, creating new consumer
2022-03-14 18:08:26.061 INFO 21892 --- [ntainer#0-0-C-1] o.a.k.clients.consumer.ConsumerConfig : ConsumerConfig values:
allow.auto.create.topics = true
[...lot of things...]
value.deserializer = class org.springframework.kafka.support.serializer.JsonDeserializer
2022-03-14 18:08:26.067 INFO 21892 --- [ntainer#0-0-C-1] o.a.kafka.common.utils.AppInfoParser : Kafka version: 3.0.0
2022-03-14 18:08:26.067 INFO 21892 --- [ntainer#0-0-C-1] o.a.kafka.common.utils.AppInfoParser : Kafka commitId: 8cb0a5e9d3441962
2022-03-14 18:08:26.067 INFO 21892 --- [ntainer#0-0-C-1] o.a.kafka.common.utils.AppInfoParser : Kafka startTimeMs: 1647277706067
2022-03-14 18:08:26.068 INFO 21892 --- [ntainer#0-0-C-1] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-people-creator-2, groupId=people-creator] Subscribed to partition(s): PEOPLE_W4BdDBEowY-1
2022-03-14 18:08:26.072 INFO 21892 --- [ -C-1] o.a.k.clients.consumer.KafkaConsumer : [Consumer clientId=consumer-people-creator-2, groupId=people-creator] Seeking to offset 0 for partition PEOPLE_W4BdDBEowY-1
2022-03-14 18:08:26.081 WARN 21892 --- [ -C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-people-creator-2, groupId=people-creator] Error while fetching metadata with correlation id 2 : {PEOPLE_W4BdDBEowY=LEADER_NOT_AVAILABLE}
2022-03-14 18:08:26.081 INFO 21892 --- [ -C-1] org.apache.kafka.clients.Metadata : [Consumer clientId=consumer-people-creator-2, groupId=people-creator] Cluster ID: ebyKy-RVSRmUDaaeQqMaQg
2022-03-14 18:18:04.882 WARN 21892 --- [ -C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-people-creator-2, groupId=people-creator] Error while fetching metadata with correlation id 5314 : {PEOPLE_W4BdDBEowY=LEADER_NOT_AVAILABLE}
2022-03-14 18:18:04.997 WARN 21892 --- [ -C-1] org.apache.kafka.clients.NetworkClient : [Consumer clientId=consumer-people-creator-2, groupId=people-creator] Error while fetching metadata with correlation id 5315 : {PEOPLE_W4BdDBEowY=LEADER_NOT_AVAILABLE}
How do I create dynamically a kafka consumer on a topic ? I think I do it very wrong, but I searched a lot and really didn't find anything.
There are several answers here about dynamically creating containers...
Trigger one Kafka consumer by using values of another consumer In Spring Kafka
Kafka Consumer in spring can I re-assign partitions programmatically?
Create consumer dynamically spring kafka
Dynamically start and off KafkaListener just to load previous messages at the start of a session
I am trying to start the JMS listener using jmsListenerEndpointRegistry.start() which was stopped using jmsListenerEndpointRegistry.stop(). But looks like it is not getting started. When I am trying to consume the messages it is not allowing me to do so as it is still stopped. Please help me how to start it back using start method.
In application.properties I have spring.jms.listener.auto-startup=true
Using Apache ActiveMQ(Version-5.16.3)
2022-01-06 16:27:54.699 INFO 28804 --- [nio-9091-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization completed in 22 ms
2022-01-06 16:27:54.726 ERROR 28804 --- [nio-9091-exec-1] com.jms.poc.controller.JmsController : --------- Trying to start JMS using jmsListenerEndpointRegistry.start()----------
2022-01-06 16:27:54.727 ERROR 28804 --- [nio-9091-exec-1] com.jms.poc.controller.JmsController : ----------jmsListenerEndpointRegistry.isRunning()-------- : false
There is no need to change the autoStartup property in startJmsListener - that property only applies when the application context is initialized.
You have no #JmsListeneners - the only one is commented out.
registry.isRunning() only returns true when at least one of its containers is running.
I have a simple Spring-Boot web application with some scheduling tasks:
#SpringBootApplication
#EnableScheduling
public class Application extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(final SpringApplicationBuilder app) {
return app.sources(Application.class);
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
And a service executing two scheduling tasks:
#Service
public class SchedulingService {
#Scheduled(fixedRate = 15000)
private void first() {
// log first
}
#Scheduled(fixedRate = 6000)
public void second() {
// log second
}
}
Is there any explanation revealing why one of the scheduling tasks has been executed before the application and servlet starts up? This happens repeatedly and the log order is always the same, so I suspect the multithreading doesn't take place - there is nearly one-second difference between the first and the second scheduling task.
Here is the log with highlighted parts of the scheduling task log:
2018-08-18 20:47:53.085 INFO 251168 --- [ost-startStop-1] o.s.b.a.w.s.WelcomePageHandlerMapping : Adding welcome page template: index
2018-08-18 20:47:53.300 INFO 251168 --- [ost-startStop-1] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2018-08-18 20:47:53.314 INFO 251168 --- [ost-startStop-1] s.a.ScheduledAnnotationBeanPostProcessor : No TaskScheduler/ScheduledExecutorService bean found for scheduled processing
2018-08-18 20:47:53.321 HERE LOGGED FIRST
2018-08-18 20:47:53.344 INFO 251168 --- [ost-startStop-1] my.appname.Application : Started Application in 5.565 seconds (JVM running for 16.93)
2018-08-18 20:47:53.396 INFO 251168 --- [ main] org.apache.coyote.ajp.AjpNioProtocol : Starting ProtocolHandler ["ajp-nio-8009"]
2018-08-18 20:47:53.400 INFO 251168 --- [ main] org.apache.catalina.startup.Catalina : Server startup in 15970 ms
2018-08-18 20:47:53.477 INFO 251168 --- [nio-8080-exec-1] o.a.c.c.C.[.[localhost].[/Rattle] : Initializing Spring FrameworkServlet 'dispatcherServlet'
2018-08-18 20:47:53.477 INFO 251168 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization started
2018-08-18 20:47:53.520 INFO 251168 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization completed in 43 ms
2018-08-18 20:47:54.103 HERE LOGGED SECOND
2018-08-18 20:47:59.335 HERE LOGGED SECOND
2018-08-18 20:48:05.334 HERE LOGGED SECOND
2018-08-18 20:48:08.334 HERE LOGGED FIRST
I just created an empty SpringBoot project, added your class and tried to reproduce the same issue.
On SpringBoot 2.0.4.RELEASE I see that both tasks run at the same time and same thread:
2018-08-18 21:16:54.145 INFO 10239 --- [pool-1-thread-1] com.test.SchedulingService : LOG FIRST
2018-08-18 21:16:54.145 INFO 10239 --- [pool-1-thread-1] com.test.SchedulingService : LOG SECOND
Then, I added a Thread.sleep(100) on the both tasks as I cannot predict the execution order.
2018-08-18 21:21:14.775 INFO 10274 --- [pool-1-thread-1] com.test.SchedulingService : LOG FIRST
2018-08-18 21:21:14.878 INFO 10274 --- [pool-1-thread-1] com.test.SchedulingService : LOG SECOND
The delay of aprox 100ms between the log entries, confirm that they run on the same thread.
You might think that setting the initialDelay = 0 on the #Scheduled might help, but it will not; everything will still be on the main thread.
The solution I found is defining a custom bean that creates a customTaskScheduler:
#Bean()
public ThreadPoolTaskScheduler taskScheduler(){
ThreadPoolTaskScheduler taskScheduler = new ThreadPoolTaskScheduler();
taskScheduler.setPoolSize(2);
taskScheduler.setThreadNamePrefix("Async-");
return taskScheduler;
}
Now, the logs show that both tasks are executed at the same time and by different threads:
2018-08-18 21:30:26.482 INFO 10383 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2018-08-18 21:30:26.486 INFO 10383 --- [ main] com.test.ProjectApplication : Started ProjectApplication in 1.767 seconds (JVM running for 2.137)
2018-08-18 21:30:26.555 INFO 10383 --- [ Async-2] com.test.SchedulingService : LOG SECOND
2018-08-18 21:30:26.555 INFO 10383 --- [ Async-1] com.test.SchedulingService : LOG FIRST
I defined a thread pool of size = 2. What will happen if I have 3 tasks to run? One of the tasks will need to wait to complete the execution of the previous 2, get the thread released and check the execution queue again.
2018-08-18 21:33:48.895 INFO 10412 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
2018-08-18 21:33:48.899 INFO 10412 --- [ main] com.test.ProjectApplication : Started ProjectApplication in 1.888 seconds (JVM running for 2.258)
2018-08-18 21:33:48.960 INFO 10412 --- [ Async-1] com.test.SchedulingService : LOG FIRST
2018-08-18 21:33:48.960 INFO 10412 --- [ Async-2] com.test.SchedulingService : LOG SECOND
2018-08-18 21:33:49.065 INFO 10412 --- [ Async-2] com.test.SchedulingService : LOG THIRD
I had same issue, but I was able to get around it by specifying the cron parameter instead of using fixedRate or fixedRateString like so #Scheduled(cron = "0 0 4 ? * SUN,THU")
More details on using the cron expression in spring-boot can be found here https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/support/CronExpression.html
I have an external OPC UA server, from which I would like to read data. I use username and password authentication, so my client is initialized like follows:
public class MyClient {
// ...
public MyClient() throws Exception {
EndpointDescription[] endpoints =
UaTcpStackClient.getEndpoints(OPCConstants.OPC_SERVER_URI).get();
// using the first endpoint
EndpointDescription endpoint = endpoints[0];
// client configuration
OpcUaClientConfig config = OpcUaClientConfig.builder()
.setApplicationName(LocalizedText.english("Example Client"))
.setApplicationUri(String.format("some:example-client:%s",
UUID.randomUUID()))
.setIdentityProvider(new UsernameProvider(USERNAME, PWD))
.setEndpoint(endpoint)
.build();
}
}
The client's request is the following:
public CompletableFuture<DataValue> getData(NodeId nodeId) {
LOGGER.debug("Sending request");
return client.readValue(60000000.0, TimestampsToReturn.Server, nodeId);
}
I call this request from the main method after initializing the client and connecting it to the server:
MyClient client = new MyClient();
NodeId requestedData = new NodeId(DATA_ID, DATA_KEY);
LOGGER.info("Sending synchronous TestStackRequest NodeId={}",
requestedData);
client.connect();
DataValue response = client.getData(requestedData).get();
LOGGER.info("Received response value={}", response.getValue());
client.disconnect();
However, this code doesn't work (the session is closed when trying to read informations from the server). I get the following output:
2018-04-12 17:43:27,765 DEBUG --- [ua-netty-event-loop-0] Recycler : -Dio.netty.recycler.maxCapacity.default: 262144
2018-04-12 17:43:27,777 DEBUG --- [ua-netty-event-loop-0] UaTcpClientAcknowledgeHandler : Sent Hello message on channel=[id: 0xfd9519e3, L:/172.20.100.54:55805 - R:/172.20.100.135:4840].
2018-04-12 17:43:27,786 DEBUG --- [ua-netty-event-loop-0] UaTcpClientAcknowledgeHandler : Received Acknowledge message on channel=[id: 0xfd9519e3, L:/172.20.100.54:55805 - R:/172.20.100.135:4840].
2018-04-12 17:43:27,793 DEBUG --- [ua-netty-event-loop-0] UaTcpClientMessageHandler : OpenSecureChannel timeout scheduled for +5s
2018-04-12 17:43:27,946 DEBUG --- [ua-shared-pool-0] UaTcpClientMessageHandler : Sent OpenSecureChannelRequest (Issue, id=0, currentToken=-1, previousToken=-1).
2018-04-12 17:43:27,951 DEBUG --- [ua-netty-event-loop-0] UaTcpClientMessageHandler : OpenSecureChannel timeout canceled
2018-04-12 17:43:27,961 DEBUG --- [ua-shared-pool-0] UaTcpClientMessageHandler : Received OpenSecureChannelResponse.
2018-04-12 17:43:27,967 DEBUG --- [ua-shared-pool-0] UaTcpClientMessageHandler : SecureChannel id=1698234671, currentTokenId=1, previousTokenId=-1, lifetime=3600000ms, createdAt=DateTime{utcTime=131680285857690000, javaDate=Thu Apr 12 19:43:05 CEST 2018}
2018-04-12 17:43:27,968 DEBUG --- [ua-netty-event-loop-0] UaTcpClientMessageHandler : 0 message(s) queued before handshake completed; sending now.
2018-04-12 17:43:27,968 DEBUG --- [ua-shared-pool-1] ClientChannelManager : Channel bootstrap succeeded: localAddress=/172.20.100.54:55805, remoteAddress=/172.20.100.135:4840
2018-04-12 17:43:27,996 DEBUG --- [ua-shared-pool-0] ClientChannelManager : disconnect(), currentState=Connected
2018-04-12 17:43:27,997 DEBUG --- [ua-shared-pool-1] ClientChannelManager : Sending CloseSecureChannelRequest...
2018-04-12 17:43:28,000 DEBUG --- [ua-netty-event-loop-0] ClientChannelManager : channelInactive(), disconnect complete
2018-04-12 17:43:28,001 DEBUG --- [ua-netty-event-loop-0] ClientChannelManager : disconnect complete, state set to Idle
2018-04-12 17:43:28,011 INFO --- [main] OpcUaClient : Eclipse Milo OPC UA Stack version: 0.2.1
2018-04-12 17:43:28,011 INFO --- [main] OpcUaClient : Eclipse Milo OPC UA Client SDK version: 0.2.1
2018-04-12 17:43:28,056 DEBUG --- [main] OpcUaClient : Added ServiceFaultListener: org.eclipse.milo.opcua.sdk.client.session.SessionFsm$FaultListener#46d59067
2018-04-12 17:43:28,066 DEBUG --- [main] OpcUaClient : Added SessionActivityListener: org.eclipse.milo.opcua.sdk.client.subscriptions.OpcUaSubscriptionManager$1#78452606
2018-04-12 17:43:28,189 INFO --- [main] CommunicationMain : Sending synchronous TestStackRequest NodeId=NodeId{ns=6, id=::opcua:opcData.outGoing.basic.cycleStep}
2018-04-12 17:43:28,189 DEBUG --- [main] ClientChannelManager : connect(), currentState=NotConnected
2018-04-12 17:43:28,190 DEBUG --- [main] ClientChannelManager : connect() while NotConnected
java.lang.Exception
at org.eclipse.milo.opcua.stack.client.ClientChannelManager.connect(ClientChannelManager.java:67)
at org.eclipse.milo.opcua.stack.client.UaTcpStackClient.connect(UaTcpStackClient.java:127)
at org.eclipse.milo.opcua.sdk.client.OpcUaClient.connect(OpcUaClient.java:313)
at com.mycompany.opcua.participants.MyClient.connect(MyClient.java:147)
at com.mycompany.opcua.participants.CommunicationMain.testClient(CommunicationMain.java:69)
at com.mycompany.opcua.participants.CommunicationMain.main(CommunicationMain.java:51)
2018-04-12 17:43:28,190 DEBUG --- [main] MyClient : Sending request
2018-04-12 17:43:28,197 DEBUG --- [ua-netty-event-loop-1] UaTcpClientAcknowledgeHandler : Sent Hello message on channel=[id: 0xd9b3f832, L:/172.20.100.54:55806 - R:/172.20.100.135:4840].
2018-04-12 17:43:28,204 DEBUG --- [ua-netty-event-loop-1] UaTcpClientAcknowledgeHandler : Received Acknowledge message on channel=[id: 0xd9b3f832, L:/172.20.100.54:55806 - R:/172.20.100.135:4840].
2018-04-12 17:43:28,205 DEBUG --- [ua-netty-event-loop-1] UaTcpClientMessageHandler : OpenSecureChannel timeout scheduled for +5s
2018-04-12 17:43:28,205 DEBUG --- [ua-shared-pool-0] UaTcpClientMessageHandler : Sent OpenSecureChannelRequest (Issue, id=0, currentToken=-1, previousToken=-1).
2018-04-12 17:43:28,208 DEBUG --- [ua-netty-event-loop-1] UaTcpClientMessageHandler : OpenSecureChannel timeout canceled
2018-04-12 17:43:28,208 DEBUG --- [ua-shared-pool-0] UaTcpClientMessageHandler : Received OpenSecureChannelResponse.
2018-04-12 17:43:28,209 DEBUG --- [ua-shared-pool-0] UaTcpClientMessageHandler : SecureChannel id=1698234672, currentTokenId=1, previousTokenId=-1, lifetime=3600000ms, createdAt=DateTime{utcTime=131680285860260000, javaDate=Thu Apr 12 19:43:06 CEST 2018}
2018-04-12 17:43:28,209 DEBUG --- [ua-netty-event-loop-1] UaTcpClientMessageHandler : 0 message(s) queued before handshake completed; sending now.
2018-04-12 17:43:28,209 DEBUG --- [ua-shared-pool-1] ClientChannelManager : Channel bootstrap succeeded: localAddress=/172.20.100.54:55806, remoteAddress=/172.20.100.135:4840
2018-04-12 17:43:28,210 DEBUG --- [ua-shared-pool-0] SessionFsm : S(Inactive) x E(CreateSessionEvent) = S'(Creating)
Exception in thread "main" java.util.concurrent.ExecutionException: UaException: status=Bad_SessionClosed, message=The session was closed by the client.
at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)2018-04-12 17:43:28,212 DEBUG --- [ua-shared-pool-1] SessionFsm : Sending CreateSessionRequest...
at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
at com.mycompany.opcua.participants.CommunicationMain.testClient(CommunicationMain.java:70)
at com.mycompany.opcua.participants.CommunicationMain.main(CommunicationMain.java:51)
Caused by: UaException: status=Bad_SessionClosed, message=The session was closed by the client.
at org.eclipse.milo.opcua.stack.core.util.FutureUtils.failedUaFuture(FutureUtils.java:100)
at org.eclipse.milo.opcua.stack.core.util.FutureUtils.failedUaFuture(FutureUtils.java:88)
at org.eclipse.milo.opcua.sdk.client.session.states.Inactive.(Inactive.java:28)
at org.eclipse.milo.opcua.sdk.client.session.SessionFsm.(SessionFsm.java:69)
at org.eclipse.milo.opcua.sdk.client.OpcUaClient.(OpcUaClient.java:159)2018-04-12 17:43:28,212 INFO --- [NonceUtilSecureRandom] NonceUtil : SecureRandom seeded in 0ms.
at com.mycompany.opcua.participants.MyClient.(MyClient.java:112)
at com.mycompany.opcua.participants.CommunicationMain.testClient(CommunicationMain.java:60)
... 1 more
I use Eclipse milo 0.2.1 as OPC UA library.
Could you please tell me hat can cause this issue and how to fix it? Can it be a race condition related to this?
I can connect to the same server using other client (UaExpert).
Thank you in advance.
All of the calls you're making (connect(), disconnect(), and readValues()) are asynchronous, so what's likely happening here is you're not connected when you attempt the read.
Make sure for these examples you block for the result before moving to the next step. (you're not doing this on connect())