I have an issue with my SpringBoot Camel App.
It works very nicely with non embedded broker. Here is its configuration:
THe URL is given as a String pointing either to standalone ActiveMQ serveur or embedded broker URL (vm://dpcgbroker?broker.persistent=false)
#Configuration
public class CamelConfiguration {
#Value("${broker.mqURL}")
String mqURL;
#Bean
CamelContextConfiguration contextConfiguration() {
return new CamelContextConfiguration() {
#Override
public void beforeApplicationStart(CamelContext context) {
ActiveMQComponent activeMQComponent = ActiveMQComponent.activeMQComponent(mqURL);
activeMQComponent.setConfiguration(getJmsConfiguration());
context.addComponent("jms", activeMQComponent);
DefaultShutdownStrategy shutdownStrategy = new DefaultShutdownStrategy();
shutdownStrategy.setTimeUnit(TimeUnit.SECONDS);
shutdownStrategy.setTimeout(20);
context.setShutdownStrategy(shutdownStrategy);
}
#Override
public void afterApplicationStart(CamelContext camelContext) {
}
};
}
public PooledConnectionFactory getPooledConnectionFactory() {
PooledConnectionFactory pooledConnectionFactory = new PooledConnectionFactory(getConnectionFactory());
pooledConnectionFactory.setMaxConnections(30);
return pooledConnectionFactory;
}
public ActiveMQConnectionFactory getConnectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(mqURL);
connectionFactory.setTrustAllPackages(true);
return connectionFactory;
}
public JmsConfiguration getJmsConfiguration() {
JmsConfiguration jmsConfiguration = new JmsConfiguration();
jmsConfiguration.setDisableTimeToLive(true);
jmsConfiguration.setTransacted(true);
jmsConfiguration.setLazyCreateTransactionManager(false);
jmsConfiguration.setConnectionFactory(getConnectionFactory());
jmsConfiguration.setCacheLevelName("CACHE_CONSUMER");
return jmsConfiguration;
}
#Bean
#Scope(value = ConfigurableBeanFactory.SCOPE_SINGLETON)
public PlatformTransactionManager getTransactionManager() {
PlatformTransactionManager platformTransactionManager = new JmsTransactionManager(getConnectionFactory());
return platformTransactionManager;
}
When I switch to embedded broker the behaviour is different and Apache ActiveMQ is starting then stopping, then restarting. I have no idea why and there are no exceptions
Here is an extract of the logs
15967 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Using Persistence Adapter: MemoryPersistenceAdapter
15991 INFO - 5063 - [JMX connector] org.apache.activemq.broker.jmx.ManagementContext - JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
16106 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:1) is starting
16111 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:1) started
16111 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - For help or more information please see: http://activemq.apache.org
16132 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.TransportConnector - Connector vm://dpcgbroker started
16205 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.TransportConnector - Connector vm://dpcgbroker stopped
16205 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:1) is shutting down
16213 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:1) uptime 0.274 seconds
16213 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:1) is shutdown
16618 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.camel.processor.aggregate.AggregateProcessor - Defaulting to MemoryAggregationRepository
16621 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.camel.processor.aggregate.AggregateProcessor - Using CompletionTimeout to trigger after 10 millis of inactivity.
16647 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.camel.spring.SpringCamelContext - Skipping starting of route jobStatusRoute as its configured with autoStartup=false
16656 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Using Persistence Adapter: MemoryPersistenceAdapter
16656 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:2) is starting
16657 INFO - 5063 - [JMX connector] org.apache.activemq.broker.jmx.ManagementContext - JMX consoles can connect to service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
16657 INFO - 5063 - [Camel (camel-1) thread #1 - file://conf/DR3/planets/] org.apache.activemq.broker.BrokerService - Apache ActiveMQ 5.15.2 (dpcgbroker, ID:Greg.local-59246-1525440325893-0:2) started
The issue is that some messages are sent in the first instance of the broker that are lost when it stops, then second instance starts with empty list of messages.
I have no idea why it is stopping.
PS: I do not want to use persistent embedded broker, this is useless workaround
After a while I understood what is happening. The whole thing is that the embedded vm broker has a special behaviour which is described in the documentation that states
Once all VM connections to the broker have been closed, the embedded broker will automatically shutdown.
My application was starting, then sending a message and then the consumers were started. When I was sending the message, the connection was closed, causing the broker to shutdown.
So the solution was simply to start the consumers first, then allow producer to publish messages.
PS: a workaround that might be useful is to use a PooledConnectionFactory. I tested and it also works, since I guess the pool maintains the connections alive. With that tricks you can start procuders and consumer in whatever order you like
Related
we are trying to communicate to an azure iothub via amqp over websocket from a java docker container inside an azure kubernetes cluster. Sadly it seems, that the container cant establish a connection while locally or even on another virtual machine (where only docker is installed) the container run successfully.
The network policies rules should allow all necessary protocols and ports to communicate with eventhub endpoint of the iot hub.
Does anybody know which switch we have to pull to "allow" the container from the cluster the communication with the iothub?
The only logs we have are this:
13:10:26.688 [main] DEBUG reactor.util.Loggers$LoggerFactory - Using Slf4j logging framework
13:10:26.851 [main] INFO com.azure.messaging.eventhubs.EventHubClientBuilder - connectionId[REDACTED]: Emitting a single connection.
13:10:26.901 [main] DEBUG com.azure.core.amqp.implementation.ReactorConnection - connectionId[REDACTED]: Connection state: UNINITIALIZED
13:10:26.903 [main] INFO com.azure.messaging.eventhubs.implementation.EventHubConnectionProcessor - namespace[REDACTED] entityPath[REDACTED]: Setting next AMQP channel.
13:10:26.903 [main] INFO com.azure.messaging.eventhubs.implementation.EventHubConnectionProcessor - namespace[REDACTED] entityPath[REDACTED]: Next AMQP channel received, updating 0 current subscribers
13:10:26.920 [main] INFO com.azure.core.amqp.implementation.ReactorConnection - connectionId[REDACTED]: Creating and starting connection to REDACTED:443
13:10:26.940 [main] INFO com.azure.core.amqp.implementation.ReactorExecutor - connectionId[REDACTED], message[Starting reactor.]
13:10:26.955 [single-1] INFO com.azure.core.amqp.implementation.handler.ConnectionHandler - onConnectionInit hostname[REDACTED], connectionId[REDACTED]
13:10:26.956 [single-1] INFO com.azure.core.amqp.implementation.handler.ReactorHandler - connectionId[REDACTED] reactor.onReactorInit
13:10:26.956 [single-1] INFO com.azure.core.amqp.implementation.handler.ConnectionHandler - onConnectionLocalOpen hostname[REDACTED:443], connectionId[REDACTED], errorCondition[null], errorDescription[null]
13:10:26.975 [main] DEBUG com.azure.core.amqp.implementation.ReactorSession - Connection state: UNINITIALIZED
13:10:26.991 [main] INFO com.azure.core.amqp.implementation.ReactorConnection - Emitting new response channel. connectionId: REDACTED. entityPath: $management. linkName: mgmt.
13:10:26.991 [main] INFO class com.azure.core.amqp.implementation.RequestResponseChannel<mgmt-session> - namespace[REDACTED] entityPath[$management]: Setting next AMQP channel.
13:10:26.991 [main] INFO class com.azure.core.amqp.implementation.RequestResponseChannel<mgmt-session> - namespace[REDACTED] entityPath[$management]: Next AMQP channel received, updating 0 current subscribers
13:10:26.993 [main] INFO com.azure.messaging.eventhubs.implementation.ManagementChannel - Management endpoint state: UNINITIALIZED
13:10:27.032 [main] INFO com.azure.messaging.eventhubs.implementation.EventHubConnectionProcessor - Upstream connection publisher was completed. Terminating processor.
13:10:27.033 [main] INFO com.azure.messaging.eventhubs.implementation.EventHubConnectionProcessor - namespace[REDACTED] entityPath[REDACTED]: AMQP channel processor completed. Notifying 0 subscribers.
13:10:27.040 [main] INFO com.azure.messaging.eventhubs.implementation.EventHubReactorAmqpConnection - connectionId[REDACTED]: Disposing of connection.
13:10:27.040 [main] INFO class com.azure.core.amqp.implementation.RequestResponseChannel<mgmt-session> - Upstream connection publisher was completed. Terminating processor.
13:10:27.040 [main] INFO class com.azure.core.amqp.implementation.RequestResponseChannel<mgmt-session> - namespace[REDACTED] entityPath[$management]: AMQP channel processor completed. Notifying 0 subscribers.
13:10:27.041 [main] INFO com.azure.core.amqp.implementation.ReactorConnection - connectionId[REDACTED]: Disposing of ReactorConnection.
13:10:27.041 [main] INFO com.azure.messaging.eventhubs.implementation.EventHubConnectionProcessor - namespace[REDACTED] entityPath[REDACTED]: Channel is disposed.
13:10:27.041 [main] INFO com.azure.core.amqp.implementation.ReactorConnection - connectionId[REDACTED]: Removing session 'mgmt-session'
13:10:27.041 [main] INFO com.azure.core.amqp.implementation.ReactorSession - sessionId[mgmt-session]: Disposing of session.
13:10:27.043 [main] INFO com.azure.core.amqp.implementation.AmqpExceptionHandler - Shutdown received: ReactorExecutor.close() was called., isTransient[false], initiatedByClient[true]
13:10:27.089 [single-1] DEBUG com.azure.core.amqp.implementation.handler.SessionHandler - onSessionLocalOpen connectionId[REDACTED], entityName[mgmt-session], condition[Error{condition=null, description='null', info=null}]
13:10:27.090 [single-1] INFO com.azure.core.amqp.implementation.handler.SendLinkHandler - onLinkLocalClose connectionId[REDACTED], linkName[mgmt:sender], errorCondition[null], errorDescription[null]
13:10:27.090 [single-1] INFO com.azure.core.amqp.implementation.handler.ReceiveLinkHandler - onLinkLocalClose connectionId[REDACTED], linkName[mgmt:receiver], errorCondition[null], errorDescription[null]
13:10:27.090 [single-1] DEBUG com.azure.core.amqp.implementation.handler.SessionHandler - onSessionLocalClose connectionId[mgmt-session], entityName[REDACTED], condition[Error{condition=null, description='null', info=null}]
13:10:27.090 [single-1] INFO com.azure.core.amqp.implementation.handler.ConnectionHandler - onConnectionLocalClose hostname[REDACTED:443], connectionId[REDACTED], errorCondition[null], errorDescription[null]
13:10:27.090 [single-1] INFO com.azure.core.amqp.implementation.handler.ConnectionHandler - onConnectionBound hostname[REDACTED], connectionId[REDACTED]
13:10:27.098 [single-1] DEBUG com.azure.core.amqp.implementation.handler.WebSocketsConnectionHandler - connectionId[REDACTED] Adding web sockets transport layer for hostname[REDACTED:443]
13:10:27.125 [single-1] DEBUG com.azure.core.amqp.implementation.handler.DispatchHandler - Running task for event: %s
13:10:27.126 [single-1] INFO com.azure.core.amqp.implementation.ReactorExecutor - connectionId[REDACTED], message[Processing all pending tasks and closing old reactor.]
13:10:27.126 [single-1] DEBUG com.azure.core.amqp.implementation.handler.SendLinkHandler - onLinkLocalOpen connectionId[REDACTED], linkName[mgmt:sender], localTarget[Target{address='$management', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, capabilities=null}]
13:10:27.126 [single-1] INFO com.azure.core.amqp.implementation.handler.ReceiveLinkHandler - onLinkLocalOpen connectionId[REDACTED], linkName[mgmt:receiver], localSource[Source{address='$management', durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null, filter=null, defaultOutcome=null, outcomes=null, capabilities=null}]
13:10:27.127 [single-1] INFO com.azure.core.amqp.implementation.ReactorExecutor - connectionId[REDACTED], message[Stopping the reactor because thread was interrupted or the reactor has no more events to process.]
The failure was not the networking at all.
My mistake was, that i assume if i can run the container manual via docker run -it * it should also work in a kubernetes cluster. But with the -it argument the container stays open und and pseudo-tty was attached. But in the kubernetes cluster this of cause will not happen, so we have to adjust the loop logic of the java application and it works after.
Thanks # all
I am running a simple Hello World kind of application in Apache Storm 1.1.0 . Application has a random integer spout and a bolt which prints the tuple output. But somehow I am not able to get it working on my windows system.
I am new to Apache storm and following a tutorial. I have looked for the answers in stack overflow, but i was not able to find any solved question regarding the same.
Following is my run Topology code:
public static void runTopology() {
//String filePath = "./src/main/resources/operations.txt";
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("randomNumberSpout", new RandomIntSpout());
builder.setBolt("printingBolt", new PrintingBolt()).shuffleGrouping("randomNumberSpout");
Config config = new Config();
config.setDebug(true);
LocalCluster cluster = new LocalCluster();
try{
cluster.submitTopology("Test", config, builder.createTopology());
}finally{
cluster.shutdown();
}
}
Bolt code
public class PrintingBolt extends BaseBasicBolt {
/**
*
*/
private static final long serialVersionUID = 1L;
public void execute(Tuple tuple, BasicOutputCollector basicOutputCollector) {
System.out.println("Printing Tupple!!!!");
System.out.println(tuple);
System.out.println("Tupple processed " + tuple.getInteger(1));
basicOutputCollector.emit(new Values(tuple.getInteger(1)));
}
public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
outputFieldsDeclarer.declare(new Fields("TestOutput"));
}
}
Spout Code
public class RandomIntSpout extends BaseRichSpout {
/**
*
*/
private static final long serialVersionUID = 1L;
private Random random;
private SpoutOutputCollector outputCollector;
/*#Override
public void open(Map<String,Object> map, TopologyContext topologyContext,
SpoutOutputCollector spoutOutputCollector) {
random = new Random();
outputCollector = spoutOutputCollector;
}*/
public void nextTuple() {
Utils.sleep(1000);
outputCollector.emit(new Values(random.nextInt(), System.currentTimeMillis()));
}
public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) {
outputFieldsDeclarer.declare(new Fields("randomInt", "timestamp"));
}
public void open(Map conf, TopologyContext context, SpoutOutputCollector collector) {
random = new Random();
outputCollector = collector;
}
}
I can provide the rest of the code as well, but I don't think that will be required. If required please mention in comments, I will provide that as well.
I get following error whenever I try to run the application.
10620 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 > watcher=org.apache.storm.shade.org.apache.curator.ConnectionState#31b0f02
10625 [main-SendThread(0:0:0:0:0:0:0:1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server > 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2000. Will not attempt to authenticate using SASL (unknown error)
10627 [main-SendThread(0:0:0:0:0:0:0:1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to > 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2000, initiating session
10627 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /> 0:0:0:0:0:0:0:1:56905
10628 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at /> 0:0:0:0:0:0:0:1:56905
10631 [main-SendThread(0:0:0:0:0:0:0:1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server > 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2000, sessionid = 0x16a8e5abd97000d, negotiated timeout = 20000
10631 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x16a8e5abd97000d with negotiated timeout 20000 for client /> 0:0:0:0:0:0:0:1:56905
10632 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
10635 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x16a8e5abd97000d type:create cxid:0x2 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/storm/blobstoremaxkeysequencenumber > Error:KeeperErrorCode = NoNode for /storm/blobstoremaxkeysequencenumber
10655 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
10657 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd97000d
10659 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
10659 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd97000d closed
10661 [main] INFO o.a.s.cluster - setup-path/blobstore/Test-1-1557166474-stormconf.ser/IBMT450PC053RLV.Corp.CVS.com:6627-1
10660 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd97000d, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
10671 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /> 0:0:0:0:0:0:0:1:56905 which had sessionid 0x16a8e5abd97000d
10746 [main] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - Starting
10747 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Initiating client connection, connectString=localhost:2000/storm sessionTimeout=20000 > watcher=org.apache.storm.shade.org.apache.curator.ConnectionState#73893ec1
10755 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Opening socket connection to server 127.0.0.1/127.0.0.1:2000. Will not > attempt to authenticate using SASL (unknown error)
10756 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Socket connection established to 127.0.0.1/127.0.0.1:2000, initiating > session
10758 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:56908
10759 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Client attempting to establish new session at > /127.0.0.1:56908
10766 [main-SendThread(127.0.0.1:2000)] INFO o.a.s.s.o.a.z.ClientCnxn - Session establishment complete on server 127.0.0.1/127.0.0.1:2000, > sessionid = 0x16a8e5abd97000e, negotiated timeout = 20000
10766 [SyncThread:0] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - Established session 0x16a8e5abd97000e with negotiated timeout 20000 for client > /127.0.0.1:56908
10767 [main-EventThread] INFO o.a.s.s.o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
10778 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
10781 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd97000e
10785 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd97000e closed
10785 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56908 > which had sessionid 0x16a8e5abd97000e
10785 [main] INFO o.a.s.cluster - setup-path/blobstore/Test-1-1557166474-stormcode.ser/IBMT450PC053RLV.Corp.CVS.com:6627-1
10786 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
10821 [main] INFO o.a.s.d.nimbus - desired replication count 1 achieved, current-replication-count for conf key = 1, current-replication-count > for code key = 1, current-replication-count for jar key = 1
11042 [main] INFO o.a.s.d.nimbus - Activating Test: Test-1-1557166474
11058 [main] INFO o.a.s.d.nimbus - Shutting down master
11064 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11066 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd970003
11068 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11069 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd970003, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
11068 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd970003 closed
11069 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56875 > which had sessionid 0x16a8e5abd970003
11069 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11072 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd970004
11074 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11075 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /> 0:0:0:0:0:0:0:1:56878 which had sessionid 0x16a8e5abd970004
11074 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd970004 closed
11077 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11079 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd970000
11081 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd970000 closed
11081 [main] INFO o.a.s.zookeeper - closing zookeeper connection of leader elector.
11082 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11082 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56866 > which had sessionid 0x16a8e5abd970000
11082 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11084 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd970001
11086 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd970001 closed
11087 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11087 [main] INFO o.a.s.d.nimbus - Shut down master
11087 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11089 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd970001, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
11089 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56869 > which had sessionid 0x16a8e5abd970001
11090 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd970006
11092 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd970006 closed
11093 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11093 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd970006, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
11094 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56884 > which had sessionid 0x16a8e5abd970006
11095 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11095 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd970008
11098 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd970008 closed
11098 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11099 [main] INFO o.a.s.d.s.ReadClusterState - Setting Thread[SLOT_1024,5,main] assignment to null
11099 [main] INFO o.a.s.d.s.ReadClusterState - Setting Thread[SLOT_1025,5,main] assignment to null
11099 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd970008, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
11099 [main] INFO o.a.s.d.s.ReadClusterState - Setting Thread[SLOT_1026,5,main] assignment to null
11099 [main] INFO o.a.s.d.s.ReadClusterState - Waiting for Thread[SLOT_1024,5,main] to be EMPTY, currently EMPTY
11099 [main] INFO o.a.s.d.s.ReadClusterState - Waiting for Thread[SLOT_1025,5,main] to be EMPTY, currently EMPTY
11099 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /> 0:0:0:0:0:0:0:1:56890 which had sessionid 0x16a8e5abd970008
11099 [main] INFO o.a.s.d.s.ReadClusterState - Waiting for Thread[SLOT_1026,5,main] to be EMPTY, currently EMPTY
11099 [main] INFO o.a.s.d.s.Supervisor - Shutting down supervisor 009e412c-0d39-400c-8302-08296524c703
11100 [Thread-10] INFO o.a.s.e.EventManagerImp - Event manager interrupted
11102 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11103 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd97000a
11105 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd97000a closed
11105 [main] INFO o.a.s.d.s.ReadClusterState - Setting Thread[SLOT_1027,5,main] assignment to null
11105 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11105 [main] INFO o.a.s.d.s.ReadClusterState - Setting Thread[SLOT_1028,5,main] assignment to null
11105 [main] INFO o.a.s.d.s.ReadClusterState - Setting Thread[SLOT_1029,5,main] assignment to null
11106 [main] INFO o.a.s.d.s.ReadClusterState - Waiting for Thread[SLOT_1027,5,main] to be EMPTY, currently EMPTY
11106 [main] INFO o.a.s.d.s.ReadClusterState - Waiting for Thread[SLOT_1028,5,main] to be EMPTY, currently EMPTY
11106 [main] INFO o.a.s.d.s.ReadClusterState - Waiting for Thread[SLOT_1029,5,main] to be EMPTY, currently EMPTY
11106 [main] INFO o.a.s.d.s.Supervisor - Shutting down supervisor 5daf8496-451f-43ca-b176-b16055d6183c
11106 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd97000a, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
11106 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /> 0:0:0:0:0:0:0:1:56896 which had sessionid 0x16a8e5abd97000a
11106 [Thread-14] INFO o.a.s.e.EventManagerImp - Event manager interrupted
11108 [Curator-Framework-0] INFO o.a.s.s.o.a.c.f.i.CuratorFrameworkImpl - backgroundOperationsLoop exiting
11109 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Processed session termination for sessionid: > 0x16a8e5abd97000c
11112 [main] INFO o.a.s.s.o.a.z.ZooKeeper - Session: 0x16a8e5abd97000c closed
11112 [main-EventThread] INFO o.a.s.s.o.a.z.ClientCnxn - EventThread shut down
11112 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN o.a.s.s.o.a.z.s.NIOServerCnxn - caught end of stream exception
org.apache.storm.shade.org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid > 0x16a8e5abd97000c, likely client has closed socket
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.shade.org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [storm-core-1.1.0.jar:1.1.0]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_60]
11113 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56902 > which had sessionid 0x16a8e5abd97000c
11114 [main] INFO o.a.s.testing - Shutting down in process zookeeper
11115 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] INFO o.a.s.s.o.a.z.s.NIOServerCnxnFactory - NIOServerCnxn factory exited run method
11116 [main] INFO o.a.s.s.o.a.z.s.ZooKeeperServer - shutting down
11116 [main] INFO o.a.s.s.o.a.z.s.SessionTrackerImpl - Shutting down
11116 [main] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - Shutting down
11117 [main] INFO o.a.s.s.o.a.z.s.SyncRequestProcessor - Shutting down
11117 [SyncThread:0] INFO o.a.s.s.o.a.z.s.SyncRequestProcessor - SyncRequestProcessor exited!
11117 [ProcessThread(sid:0 cport:-1):] INFO o.a.s.s.o.a.z.s.PrepRequestProcessor - PrepRequestProcessor exited loop!
11117 [main] INFO o.a.s.s.o.a.z.s.FinalRequestProcessor - shutdown of request processor complete
11118 [main] INFO o.a.s.testing - Done shutting down in process zookeeper
11118 [main] INFO o.a.s.testing - Deleting temporary path C:\Users\AKHAND~1\AppData\Local\Temp\ae4119b4-70b3-4d04-9aee-5bfae4c4775b
11203 [main] INFO o.a.s.testing - Deleting temporary path C:\Users\AKHAND~1\AppData\Local\Temp\a78b8c79-b9b3-438d-8df6-5d7bd74281fc
11215 [main] INFO o.a.s.testing - Unable to delete file: > C:\Users\AKHAND~1\AppData\Local\Temp\a78b8c79-b9b3-438d-8df6-5d7bd74281fc\version-2\log.1
11215 [main] INFO o.a.s.testing - Deleting temporary path C:\Users\AKHAND~1\AppData\Local\Temp\0e4fbadc-ad33-4577-9784-4cc163a778fa
11255 [main] INFO o.a.s.testing - Deleting temporary path C:\Users\AKHAND~1\AppData\Local\Temp\456d6b1d-eb21-4b76-98f1-a2bb44b2aa5e
12197 [SessionTracker] INFO o.a.s.s.o.a.z.s.SessionTrackerImpl - SessionTrackerImpl exited loop!
I am not able to understand why client socket is closed and why session is closed? I am not able to get it working. Please help.
I think you might need to add a sleep here
try{
cluster.submitTopology("Test", config, builder.createTopology());
//Sleep here
}finally{
cluster.shutdown();
}
Currently you are submitting the topology, and immediately shutting down. Unless you sleep a bit, your topology doesn't get a chance to run.
I'm using ActiveMQ 5.9 with Camel 2.10.3, and under load (during a performance test) I'm experiencing some problems, it seems that the broker is stuck trying to close a connection, and I can't understand the reason.
The configuration of the JMS system is the following: there are two broker (configured in failover mode) and many client nodes that acts both as consumer both as producer of some queues (let's take one for example: 'customer_update queue'.
I'm using PooledConnectionFactory with default configuration, 'CACHE_CONSUMER' cache level and 10 max concurrent consumer per each client node.
Broker configuration is the following: tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600
Here's the thread holding lock on the broker, and never releasing it:
"ActiveMQ Transport: tcp:///10.128.43.206:38694#61616" #5774 daemon prio=5 os_prio=0 tid=0x00007f2a4424e800 nid=0
xaba4 waiting on condition [0x00007f29fe397000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000004fd008fb0> (a java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.
java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.activemq.broker.TransportConnection.stop(TransportConnection.java:983)
at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:699)
- locked <0x000000050401eed0> (a java.lang.Object)
at org.apache.activemq.broker.jmx.ManagedTransportConnection.processAddConnection(ManagedTransportConnection.java:79)
at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:149)
at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:748)
I have more than 500 other threads on the broker that look like this:
"ActiveMQ Transport: tcp:///10.128.43.206:52074#61616" #2962 daemon prio=5 os_prio=0 tid=0x00007f2a440c9000 nid=0xa01f waiting for monitor entry [0x00007f29fc768000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:696)
- waiting to lock <0x000000050401eed0> (a java.lang.Object)
at org.apache.activemq.broker.jmx.ManagedTransportConnection.processAddConnection(ManagedTransportConnection.java:79)
at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:149)
at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
at java.lang.Thread.run(Thread.java:748)
First error I see on the broker is this one:
2018-05-16 16:36:59,336 [org.apache.activemq.broker.TransportConnection.Transport:856]
WARN - Transport Connection to: tcp://10.128.43.206:48747 failed: java.io.EOFException
On the client node that is referenced in the broker (10.128.43.206), I see this logs, it seems that the node is trying to reconnect, but right after it's disconnected again, and this happens again and again.
2018-05-16 16:36:59,322 [org.apache.activemq.transport.failover.FailoverTransport:856] WARN - Transport (tcp://10.128.43.169:61616) failed, reason: java.io.IOException, attempting to automatically reconnect
2018-05-16 16:36:59,322 [org.apache.activemq.transport.failover.FailoverTransport:856] WARN - Transport (tcp://10.128.43.169:61616) failed, reason: java.io.IOException, attempting to automatically reconnect
2018-05-16 16:36:59,375 [org.apache.activemq.transport.failover.FailoverTransport:856] INFO - Successfully reconnected to tcp://10.128.43.169:61616
2018-05-16 16:36:59,375 [org.apache.activemq.transport.failover.FailoverTransport:856] INFO - Successfully reconnected to tcp://10.128.43.169:61616
2018-05-16 16:36:59,375 [org.apache.activemq.TransactionContext:856] INFO - commit failed for transaction TX:ID:52374-1526300283331-1:1:898
javax.jms.TransactionRolledBackException: Transaction completion in doubt due to failover. Forcing rollback of TX:ID:52374-1526300283331-1:1:898
at org.apache.activemq.state.ConnectionStateTracker.restoreTransactions(ConnectionStateTracker.java:231)
at org.apache.activemq.state.ConnectionStateTracker.restore(ConnectionStateTracker.java:169)
at org.apache.activemq.transport.failover.FailoverTransport.restoreTransport(FailoverTransport.java:827)
at org.apache.activemq.transport.failover.FailoverTransport.doReconnect(FailoverTransport.java:1005)
at org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:136)
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129)
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2018-05-16 16:37:00,091 [org.apache.activemq.transport.failover.FailoverTransport:856] INFO - Successfully reconnected to tcp://10.128.43.169:61616
2018-05-16 16:37:00,091 [org.apache.activemq.transport.failover.FailoverTransport:856] INFO - Successfully reconnected to tcp://10.128.43.169:61616
2018-05-16 16:37:00,112 [org.apache.activemq.transport.failover.FailoverTransport:856] WARN - Transport (tcp://10.128.43.169:61616) failed, reason: java.io.IOException, attempting to automatically reconnect
At the end, broker reach maxConnections available (1000), and needs to be restarted.
Could it be due to the fact that one client node is acting both as a consumer and a producer, using same connection pool, generating some kind of deadlock?
Do you have some suggestions?
Thanks
Giulio
Most probably I was affected by this issue:
https://issues.apache.org/jira/browse/AMQ-5090
Updating to ActiveMQ 5.10.0 and Camel 2.13.1 solved the issue (system is much more stable, even during performance tests).
Thanks
Giulio
I'm starting to learn Apache Camel and faced with the problem.
I need to read XML file from file system, parse it and transfer some file specified in this XML to another location.
This is example of XML located in "C:/Users/JuISe/Desktop/jms".
<file>
<from>C:/Users/JuISe/Desktop/from</from>
<to>C:/Users/JuISe/Desktop/to</to>
</file>
It means transfer all files from
"C:/Users/JuISe/Desktop/from" directory to "C:/Users/JuISe/Desktop/to"
Here is my code:
public class FileShifter {
public static void main(String args[]) {
CamelContext context = new DefaultCamelContext();
try {
context.addRoutes(new MyRouteBuilder());
context.start();
Thread.sleep(10000);
context.stop();
}catch (Exception ex) {
ex.printStackTrace();
}
}
}
class MyRouteBuilder extends RouteBuilder {
private String from;
private String to;
public void configure(){
from("file:C:/Users/JuISe/Desktop/jms?noop=true")
.setHeader("from", xpath("file/from/text()").stringResult())
.setHeader("to", xpath("file/to/text()").stringResult())
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
from = exchange.getIn().getHeader("from").toString();
to = exchange.getIn().getHeader("to").toString();
}
})
.pollEnrich("file:" + from)
.to("file:" + to);
}
}
It doesn't works.
Here is logs:
[main] INFO org.apache.camel.impl.converter.DefaultTypeConverter - Loaded 216 type converters
[main] INFO org.apache.camel.impl.DefaultRuntimeEndpointRegistry - Runtime endpoint registry is in extended mode gathering usage statistics of all incoming and outgoing endpoints (cache limit: 1000)
[main] INFO org.apache.camel.impl.DefaultCamelContext - AllowUseOriginalMessage is enabled. If access to the original message is not needed, then its recommended to turn this option off as it may improve performance.
[main] INFO org.apache.camel.impl.DefaultCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[main] INFO org.apache.camel.component.file.FileEndpoint - Endpoint is configured with noop=true so forcing endpoint to be idempotent as well
[main] INFO org.apache.camel.component.file.FileEndpoint - Using default memory based idempotent repository with cache max size: 1000
[main] INFO org.apache.camel.impl.DefaultCamelContext - Route: route1 started and consuming from: Endpoint[file://C:/Users/JuISe/Desktop/jms?noop=true]
[main] INFO org.apache.camel.impl.DefaultCamelContext - Total 1 routes, of which 1 is started.
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.16.1 (CamelContext: camel-1) started in 1.033 seconds
[Camel (camel-1) thread #0 - file://C:/Users/JuISe/Desktop/jms] WARN org.apache.camel.component.file.strategy.MarkerFileExclusiveReadLockStrategy - Deleting orphaned lock file: C:\Users\JuISe\Desktop\jms\message.xml.camelLock
[Camel (camel-1) thread #0 - file://C:/Users/JuISe/Desktop/jms] INFO org.apache.camel.builder.xml.XPathBuilder - Created default XPathFactory com.sun.org.apache.xpath.internal.jaxp.XPathFactoryImpl#2308d4c8
[main] INFO org.apache.camel.impl.DefaultCamelContext - Apache Camel 2.16.1 (CamelContext: camel-1) is shutting down
[main] INFO org.apache.camel.impl.DefaultShutdownStrategy - Starting to graceful shutdown 1 routes (timeout 300 seconds)
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 300 seconds. Inflights per route: [route1 = 2]
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 299 seconds. Inflights per route: [route1 = 2]
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 298 seconds. Inflights per route: [route1 = 2]
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 297 seconds. Inflights per route: [route1 = 2]
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 296 seconds. Inflights per route: [route1 = 2]
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 295 seconds. Inflights per route: [route1 = 2]
[Camel (camel-1) thread #2 - ShutdownTask] INFO org.apache.camel.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 294 seconds. Inflights per route: [route1 = 2]
Thanks for a help!
Try using a bean with producer and consumer template , file end points directory cannot be dynamic
from("file:/Users/smunirat/apps/destination/jms?noop=true")
.setHeader("from", xpath("file/from/text()").stringResult())
.setHeader("to", xpath("file/to/text()").stringResult())
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
from = exchange.getIn().getHeader("from").toString();
to = exchange.getIn().getHeader("to").toString();
exchange.getOut().setHeader("from", from);
exchange.getOut().setHeader("to", to);
}
})
.to("log:Sundar?showAll=true&multiline=true")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
ConsumerTemplate createConsumerTemplate = exchange.getContext().createConsumerTemplate();
ProducerTemplate createProducerTemplate = exchange.getContext().createProducerTemplate();
Exchange receive = createConsumerTemplate.receive("file://"+exchange.getIn().getHeader("from"));
createProducerTemplate.sendBody("file://"+exchange.getIn().getHeader("to"),receive.getIn().getMandatoryBody());
}
})
.log("Message");
This might require a little tweaking to change the file name and delete the original file from the from location
I've a Camel process (that I run from command line) which route is similar to this one:
public class ProfilerRoute extends RouteBuilder {
#Override
public void configure() {
from("kestrel://my_queue?concurrentConsumers=10&waitTimeMs=500")
.unmarshal().json(JsonLibrary.Jackson, MyClass.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
/* Do the real processing [...] */
exchange.getIn().setBody(null);
}
})
.filter(body().isNotNull())
.to("file://nowhere");
}
}
Note that I'm trashing whatever message after having processed it, being this a pure consumer
process.
The process is run by its own. No other process is writing on the queue, the queue is empty.
However when I try to kill the process the process is not going to die.
From the logs I see the following lines (indented for readability):
[ Thread-1] MainSupport$HangupInterceptor INFO
Received hang up - stopping the main instance.
[ Thread-1] MainSupport INFO
Apache Camel stopping
[ Thread-1] GuiceCamelContext INFO
Apache Camel 2.11.1 (CamelContext: camel-1)
is shutting down
[ Thread-1] DefaultShutdownStrategy INFO
Starting to graceful shutdown 1 routes
(timeout 300 seconds)
[l-1) thread #12 - ShutdownTask] DefaultShutdownStrategy INFO
Waiting as there are still 10 inflight and
pending exchanges to complete,
timeout in 300 seconds.
And so on with decreasing timeout. At the end of the timeout I get on the logs:
[l-1) thread #12 - ShutdownTask] DefaultShutdownStrategy INFO
Waiting as there are still 10 inflight and
pending exchanges to complete,
timeout in 1 seconds.
[ Thread-1] DefaultShutdownStrategy WARN
Timeout occurred.
Now forcing the routes to be shutdown now.
[l-1) thread #12 - ShutdownTask] DefaultShutdownStrategy WARN
Interrupted while waiting during graceful
shutdown, will force shutdown now.
[ Thread-1] KestrelConsumer INFO
Stopping consumer for
kestrel://localhost:22133/my_queue?concurrentConsumers=10&waitTimeMs=500
But the process will not die anyway (even if I try to kill it at this point).
I would have expected that after the waiting time all the threads would realise that a shutdown is going on and stop.
I've read the "Graceful Shutdown" document, however I could not find something that explains the behaviour I'm facing.
As you can see from logs I'm using the 2.11.1 version of Apache Camel.
UPDATE: According to Claus Ibsen it might be a problem of the camel-kestrel component. I filed a issue on ASF Jira for Camel: CAMEL-6632
This is a bug in camel-kestrel, and a JIRA ticket has been logged to fix this: https://issues.apache.org/jira/browse/CAMEL-6632