Develop Reference

how to create a connection pool/only once for a third party Application?

I'm using JAVA/Spring MVC and I need to make a Connection Pool for a Third Party Application integration in my application becouse when i try to connect it multiple time my application and server System utilize 100% RAM.
here i have to problem, when users start to hit a specific method (callGenerationService()) multiple time, my Heap memory(RAM space) increases and becomes 100% and application going to slow becouse of it connect third party application multiple times ? here i need to create a connection only once and get it multiple times. where my connection like,
public class ClickToCallServiceImpl implements ClickToCallServiceInterface {
Client client = null;
#Override
public ClickToCall callGenerationService(ClickToCall clickToCall) {
client = new Client();
client.connect("127.0.0.1", 8021 , "password", 10); //Every time Connection Connect.
client.setEventSubscriptions("plain", "all");
// client.sendSyncApiCommand("",""); //here i run command on every hit like.
client.sendSyncApiCommand(clickToCall.command1, clickToCall.command2);
client.close();
}
}
and here 'ClickToCall' is a #Component Bean/POJO Class with variables setters and getters.
Is there, how to we create a connection (either pool or only once connect) for above connection where i connect only once and hit clickToCall.Command1 and clickToCall.Command2 multiple times and utilize less RAM? Thanks in advance.

Please note that I'm not an expert of freeswitch esl so you must check the code properly. Anyway this is what I would do.
First I create a Factory for Client
public class FreeSwitchEslClientFactory extends BasePooledObjectFactory<Client> {
#Override
public Client create() throws Exception {
//Create and connect: NOTE I'M NOT AN EXPERT OF ESL FREESWITCH SO YOU MUST CHECK IT PROPERLY
Client client = new Client();
client.connect("127.0.0.1", 8021 , "password", 10);
client.setEventSubscriptions("plain", "all");
return client;
}
#Override
public PooledObject<Client> wrap(Client obj) {
return new DefaultPooledObject<Client>(obj);
}
}
Then I create a shareable GenericObjectPool:
#Configuration
#ComponentScan(basePackages= {"it.olgna.spring.pool"})
public class CommonPoolConfig {
#Bean("clientPool")
public GenericObjectPool<Client> clientPool(){
GenericObjectPool<Client> result = new GenericObjectPool<Client>(new FreeSwitchEslClientFactory());
//Pool config e.g. max pool dimension
result.setMaxTotal(20);
return result;
}
}
Finally I use the created pool in order to get the Client obj:
#Component
public class FreeSwitchEslCommandSender {
#Autowired
#Qualifier("clientPool")
private GenericObjectPool<Client> pool;
public void sendCommand(String command, String param) throws Exception{
Client client = null;
try {
client = pool.borrowObject();
client.sendSyncApiCommand(command, param);
} finally {
if( client != null ) {
client.close();
}
pool.returnObject(client);
}
}
}
I didn't test (also because I can't) it but it should work. In any case I pray you to properly check the configuration. I don't know if it's OK to always create a Client object and connect or if it's better to connect when you want to send command
I hope it can be useful
EDIT INFORMATION
Sorry I made an error early. You must return the client to the pool
I updated my FreeSwitchEslCommandSender class
Angelo

How can I set expire time of queue message using ActiveMQ by Java Programming?

In this code, I am using setJMSExpiration(1000) for expire message of one second in queue from publisher side. But From Consumer Side, It is returning properly message after 1 second instead of null.
public class RegistrationPublisher extends Thread{
public void run() {
publisherQueue("Registration.Main.*");
}
public void publisherQueue(String server){
try {
String url="tcp://192.168.20.49:61616";
// Create a ConnectionFactory
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue(server);
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);
String text = "Test";
TextMessage message = session.createTextMessage(text);
message.setJMSExpiration(1000);// For Expire message in one second
producer.send(message);
producer.close();
session.close();
connection.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
public static void main(String args[]) throws IOException{
RegistrationPublisher registrationPublisher=new RegistrationPublisher();
registrationPublisher.start();
}
}

You do this by configuring the JMS MessageProducer to do it for you via the send method that accepts a TTL or by calling setTimeToLive on the producer which adds the same TTL to all sent messages. The JMS APIs for the message version are clear that calling the setters on the message have no effect.
void setJMSExpiration(long expiration) throws JMSException
Sets the message's expiration value.
This method is for use by JMS providers only to set this field when a message is sent. This message cannot be used by clients to configure the expiration time of the message. This method is public to allow a JMS provider to set this field when sending a message whose implementation is not its own.

I first also thought that is was possible to set expiration directly on the message in the post-processor, but as Tim Bish said above, this is not the intended way to do it, and the value will get reset to 0 afterward. I couldn't access to the producer directly neither to set a time to live, because this object was in library org.springframework.jms (I was following this documentation).
One thing I could do was to set to time to live on the jmsTemplate:
import org.springframework.jms.core.JmsTemplate;
#Service
public class MyJmsServiceImpl implements MyJmsService {
#Inject
private JmsTemplate jmsTemplate;
private void convertAndSendToResponseQueue(String targetQueueName, String correlationid, Object message) {
// Set time to live
jmsTemplate.setExplicitQosEnabled(true);
jmsTemplate.setTimeToLive(5000);
jmsTemplate.convertAndSend(targetQueueName, message, new JmsResponsePostProcessor(correlationid));
}
}

Unable to configure "Keep Alive" in Camel HTTP component

I'm having some troubles with the right setup of the HTTP component. Currently a microservice pulls JSON Content from a provider, process it and send it to the next service for further processes. The main problem is that this microservice create a ton of CLOSE_WAIT socket connections. I understand that the whole concept of "KEEP-ALIVE" shall keep the connection open until I close it, but it's possible that the server will drop the connection for some reasons and creates this CLOSE_WAIT socket.
I've created a small service for debugging / testing purposes which make a GET Call to Google, but even this connection stays open until i close the program. I've tried many different solutions:
.setHeader("Connection", constant("Close"))
-Dhttp.keepAlive=false as VM argument
Switching from Camel-Http to Camel-Http4
httpClient.soTimeout=500 (Camel-HTTP), httpClient.socketTimeout=500 and connectionTimeToLive=500 (Camel-HTTP4)
.setHeader("Connection", simple("Keep-Alive")) and
.setHeader("Keep-Alive", simple("timeout=10")) (Camel-HTTP4)
Setting via debugging the response of DefaultConnectionKeepAliveStrategy from -1 (never ending) to a specific value in Camel-HTTP4 - that works but I was not able to inject my own strategy.
but i had no success. So maybe one of you can help me:
How can i tell the Camel-HTTP that it should close a connection when a specific time is passed? For example, the service pulls every hour from the content provider. After 3-4 hours the HttpComponent should close the connection after the pull and reopen it when the next pull is there. Currently every connection would be put back into the MultiThreadedHttpConnectionManager and the socket is still open.
If it's not possible to do that with Camel-HTTP: How can i inject a HttpClientBuilder into the Creation of my route? I know that it should be possible via httpClient option but I don't understand that specific part of the documentation.
Thank you all for your help

Unfortunately none of the proposed answers solved the CLOSE_WAIT connection status on my side until the application finally was closed.
I reproduced this problem with the following test case:
public class HttpInvokationTest extends CamelSpringTestSupport {
private static final Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
#EndpointInject(uri = "mock:success")
private MockEndpoint successEndpoint;
#EndpointInject(uri = "mock:failure")
private MockEndpoint failureEndpoint;
#Override
protected AbstractApplicationContext createApplicationContext() {
return new AnnotationConfigApplicationContext(ContextConfig.class);
}
#Configuration
#Import(HttpClientSpringTestConfig.class)
public static class ContextConfig extends CamelConfiguration {
#Override
public List<RouteBuilder> routes() {
List<RouteBuilder> routes = new ArrayList<>(1);
routes.add(new RouteBuilder() {
#Override
public void configure() {
from("direct:start")
.log(LoggingLevel.INFO, LOG, CONFIDENTIAL, "Invoking external URL: ${header[ERPEL_URL]}")
.setHeader("Connection", constant("close"))
.recipientList(header("TEST_URL"))
.log(LoggingLevel.DEBUG, "HTTP response code: ${header["+Exchange.HTTP_RESPONSE_CODE+"]}")
.bean(CopyBodyToHeaders.class)
.choice()
.when(header(Exchange.HTTP_RESPONSE_CODE).isGreaterThanOrEqualTo(300))
.to("mock:failure")
.otherwise()
.to("mock:success");
}
});
return routes;
}
}
#Test
public void testHttpInvocation() throws Exception {
successEndpoint.expectedMessageCount(1);
failureEndpoint.expectedMessageCount(0);
ProducerTemplate template = context.createProducerTemplate();
template.sendBodyAndHeader("direct:start", null, "TEST_URL", "http4://meta.stackoverflow.com");
successEndpoint.assertIsSatisfied();
failureEndpoint.assertIsSatisfied();
Exchange exchange = successEndpoint.getExchanges().get(0);
Map<String, Object> headers = exchange.getIn().getHeaders();
String body = exchange.getIn().getBody(String.class);
for (String key : headers.keySet()) {
LOG.info("Header: {} -> {}", key, headers.get(key));
}
LOG.info("Body: {}", body);
Thread.sleep(120000);
}
}
and issuing netstat -ab -p tcp | grep 151.101.129.69 requests, where the IP is the one of meta.stackoverflow.com.
This gave responses like:
tcp4 0 0 192.168.0.10.52183 151.101.129.69.https ESTABLISHED 37562 2118
tcp4 0 0 192.168.0.10.52182 151.101.129.69.http ESTABLISHED 885 523
right after the invocation followeb by
tcp4 0 0 192.168.0.10.52183 151.101.129.69.https CLOSE_WAIT 37562 2118
tcp4 0 0 192.168.0.10.52182 151.101.129.69.http CLOSE_WAIT 885 523
responses until the application was closed due to the Connection: keep-alive header even with a configuration like the one below:
#Configuration
#EnableConfigurationProperties(HttpClientSettings.class)
public class HttpClientSpringTestConfig {
private final static Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
#Resource
private HttpClientSettings httpClientSettings;
#Resource
private CamelContext camelContext;
private SocketConfig httpClientSocketConfig() {
/*
socket timeout:
Monitors the time passed between two consecutive incoming messages over the connection and
raises a SocketTimeoutException if no message was received within the given timeout interval
*/
LOG.info("Creating a SocketConfig with a socket timeout of {} seconds", httpClientSettings.getSoTimeout());
return SocketConfig.custom()
.setSoTimeout(httpClientSettings.getSoTimeout() * 1000)
.setSoKeepAlive(false)
.setSoReuseAddress(false)
.build();
}
private RequestConfig httpClientRequestConfig() {
/*
connection timeout:
The time span the application will wait for a connection to get established. If the connection
is not established within the given amount of time a ConnectionTimeoutException will be raised.
*/
LOG.info("Creating a RequestConfig with a socket timeout of {} seconds and a connection timeout of {} seconds",
httpClientSettings.getSoTimeout(), httpClientSettings.getConTimeout());
return RequestConfig.custom()
.setConnectTimeout(httpClientSettings.getConTimeout() * 1000)
.setSocketTimeout(httpClientSettings.getSoTimeout() * 1000)
.build();
}
#Bean(name = "httpClientConfigurer")
public HttpClientConfigurer httpConfiguration() {
ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
#Override
public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
return 5 * 1000;
}
};
PoolingHttpClientConnectionManager conMgr =
new PoolingHttpClientConnectionManager();
conMgr.closeIdleConnections(5, TimeUnit.SECONDS);
return builder -> builder.setDefaultSocketConfig(httpClientSocketConfig())
.setDefaultRequestConfig(httpClientRequestConfig())
.setConnectionTimeToLive(5, TimeUnit.SECONDS)
.setKeepAliveStrategy(myStrategy)
.setConnectionManager(conMgr);
}
#PostConstruct
public void init() {
LOG.debug("Initializing HTTP clients");
HttpComponent httpComponent = camelContext.getComponent("http4", HttpComponent.class);
httpComponent.setHttpClientConfigurer(httpConfiguration());
HttpComponent httpsComponent = camelContext.getComponent("https4", HttpComponent.class);
httpsComponent.setHttpClientConfigurer(httpConfiguration());
}
}
or defining the settings directly on the respective HttpComponent.
On examining the respective proposed methods in the HttpClient code it gets obvious that these methods are single-shot operations and not configurations that HttpClient internally will check every few milliseconds itself.
PoolingHttpClientConnectionManager states further that:
The handling of stale connections was changed in version 4.4. Previously, the code would check every connection by default before re-using it. The code now only checks the connection if the elapsed time since the last use of the connection exceeds the timeout that has been set. The default timeout is set to 2000ms
which only occurs if an attempt is done on re-using a connection, which makes sense for a connection pool, especially if multiple messages are exchanged via the same connection. For single-shot invocations, that should more behave like a Connection: close there might not be a reuse of that connection for some time, leaving the connection open or half-closed as no further attempt is done to read from that connection and therefore recognizing itself that the connection could be closed.
I noticed that I already solved such an issue a while back with traditional HttpClients and started to port this solution to Camel, which worked out quite easily.
The solution basically consists of registering HttpClients with a service and then periodically (5 seconds in my case) call closeExpiredConnections() and closeIdleConnections(...).
This logic is kept in a singleton enum, as this is actually in a library that a couple of applications use, each running in their own JVM.
/**
* This singleton monitor will check every few seconds for idle and stale connections and perform
* a cleanup on the connections using the registered connection managers.
*/
public enum IdleConnectionMonitor {
INSTANCE;
private final static Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
/** The execution service which runs the cleanup every 5 seconds **/
private ScheduledExecutorService executorService =
Executors.newScheduledThreadPool(1, new NamingThreadFactory());
/** The actual thread which performs the monitoring **/
private IdleConnectionMonitorThread monitorThread = new IdleConnectionMonitorThread();
IdleConnectionMonitor() {
// execute the thread every 5 seconds till the application is shutdown (or the shutdown method
// is invoked)
executorService.scheduleAtFixedRate(monitorThread, 5, 5, TimeUnit.SECONDS);
}
/**
* Registers a {#link HttpClientConnectionManager} to monitor for stale connections
*/
public void registerConnectionManager(HttpClientConnectionManager connMgr) {
monitorThread.registerConnectionManager(connMgr);
}
/**
* Request to stop the monitoring for stale HTTP connections.
*/
public void shutdown() {
executorService.shutdown();
try {
if (!executorService.awaitTermination(3, TimeUnit.SECONDS)) {
LOG.warn("Connection monitor shutdown not finished after 3 seconds!");
}
} catch (InterruptedException iEx) {
LOG.warn("Execution service was interrupted while waiting for graceful shutdown");
}
}
/**
* Upon invocation, the list of registered connection managers will be iterated through and if a
* referenced object is still reachable {#link HttpClientConnectionManager#closeExpiredConnections()}
* and {#link HttpClientConnectionManager#closeIdleConnections(long, TimeUnit)} will be invoked
* in order to cleanup stale connections.
* <p/>
* This runnable implementation holds a weakly referable list of {#link
* HttpClientConnectionManager} objects. If a connection manager is only reachable by {#link
* WeakReference}s or {#link PhantomReference}s it gets eligible for garbage collection and thus
* may return null values. If this is the case, the connection manager will be removed from the
* internal list of registered connection managers to monitor.
*/
private static class IdleConnectionMonitorThread implements Runnable {
// we store only weak-references to connection managers in the list, as the lifetime of the
// thread may extend the lifespan of a connection manager and thus allowing the garbage
// collector to collect unused objects as soon as possible
private List<WeakReference<HttpClientConnectionManager>> registeredConnectionManagers =
Collections.synchronizedList(new ArrayList<>());
#Override
public void run() {
LOG.trace("Executing connection cleanup");
Iterator<WeakReference<HttpClientConnectionManager>> conMgrs =
registeredConnectionManagers.iterator();
while (conMgrs.hasNext()) {
WeakReference<HttpClientConnectionManager> weakConMgr = conMgrs.next();
HttpClientConnectionManager conMgr = weakConMgr.get();
if (conMgr != null) {
LOG.trace("Found connection manager: {}", conMgr);
conMgr.closeExpiredConnections();
conMgr.closeIdleConnections(30, TimeUnit.SECONDS);
} else {
conMgrs.remove();
}
}
}
void registerConnectionManager(HttpClientConnectionManager connMgr) {
registeredConnectionManagers.add(new WeakReference<>(connMgr));
}
}
private static class NamingThreadFactory implements ThreadFactory {
#Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setName("Connection Manager Monitor");
return t;
}
}
}
As mentioned, this singleton service spawns an own thread that invokes the two, above mentioned methods every 5 seconds. These invocations take care of closing connections that are either unused for a certain amount of time or that are IDLE for the stated amount of time.
In order to camelize this service EventNotifierSupport can be utilized in order to let Camel take care of shutting down the monitor thread once it is closing down.
/**
* This Camel service with take care of the lifecycle management of {#link IdleConnectionMonitor}
* and invoke {#link IdleConnectionMonitor#shutdown()} once Camel is closing down in order to stop
* listening for stale connetions.
*/
public class IdleConnectionMonitorService extends EventNotifierSupport {
private final static Logger LOG = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
private IdleConnectionMonitor connectionMonitor;
#Override
public void notify(EventObject event) {
if (event instanceof CamelContextStartedEvent) {
LOG.info("Start listening for closable HTTP connections");
connectionMonitor = IdleConnectionMonitor.INSTANCE;
} else if (event instanceof CamelContextStoppingEvent){
LOG.info("Shutting down listener for open HTTP connections");
connectionMonitor.shutdown();
}
}
#Override
public boolean isEnabled(EventObject event) {
return event instanceof CamelContextStartedEvent || event instanceof CamelContextStoppingEvent;
}
public IdleConnectionMonitor getConnectionMonitor() {
return this.connectionMonitor;
}
}
In order to take advantage of that service, the connection manager that is used by the HttpClient Camel uses internally needs to be registered with the service, which is done in the code block below:
private void registerHttpClientConnectionManager(HttpClientConnectionManager conMgr) {
if (!getIdleConnectionMonitorService().isPresent()) {
// register the service with Camel so that on a shutdown the monitoring thread will be stopped
camelContext.getManagementStrategy().addEventNotifier(new IdleConnectionMonitorService());
}
IdleConnectionMonitor.INSTANCE.registerConnectionManager(conMgr);
}
private Optional<IdleConnectionMonitorService> getIdleConnectionMonitorService() {
for (EventNotifier eventNotifier : camelContext.getManagementStrategy().getEventNotifiers()) {
if (eventNotifier instanceof IdleConnectionMonitorService) {
return Optional.of((IdleConnectionMonitorService) eventNotifier);
}
}
return Optional.empty();
}
Last but not least the connection manager defined in httpConfiguration inside the HttpClientSpringTestConfig in my case needed to be past to the introduced register function
PoolingHttpClientConnectionManager conMgr = new PoolingHttpClientConnectionManager();
registerHttpClientConnectionManager(conMgr);
This might not be the prettiest solution, but it does close the half-closed connections on my machine.
#edit
I just learned that you can use a NoConnectionReuseStrategy which changes the connection state to TIME_WAIT rather than CLOSE_WAIT and therefore removes the connection after a short moment. Unfortunately, the request is still issued with a Connection: keep-alive header. This strategy will create a new connection per request, i.e. if you've got a 301 Moved Permanently redirect response the redirect would occur on a new connection.
The httpClientConfigurer bean would need to change to the following in order to make use of the above mentioned strategy:
#Bean(name = "httpClientConfigurer")
public HttpClientConfigurer httpConfiguration() {
return builder -> builder.setDefaultSocketConfig(socketConfig)
.setDefaultRequestConfig(requestConfig)
.setConnectionReuseStrategy(NoConnectionReuseStrategy.INSTANCE);
}

It can be done by closing idle connections if they are idle for configured time. You can achieve same by configuring idle connection timeout for Camel Http Component.
Camel Http provide interface to do so.
Cast org.apache.camel.component.http4.HttpComponent to PoolingHttpClientConnectionManager
PoolingHttpClientConnectionManager poolingClientConnectionManager = (PoolingHttpClientConnectionManager) httpComponent
.getClientConnectionManager();
poolingClientConnectionManager.closeIdleConnections(5000, TimeUnit.MILLISECONDS);
Visit Here [http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/PoolingHttpClientConnectionManager.html#closeIdleConnections(long, java.util.concurrent.TimeUnit)]

Firstly Roman Vottner, your answer and just your sheer dedication to finding the issue helped me a truckload. I have been struggling with the CLOSE_WAIT for 2 days now and your answer was what helped. Here is what I did. Added the following code in my CamelConfiguration class which essentially tampers with CamelContext at startup.
HttpComponent http4 = camelContext.getComponent("https4", HttpComponent.class);
http4.setHttpClientConfigurer(new HttpClientConfigurer() {
#Override
public void configureHttpClient(HttpClientBuilder builder) {
builder.setConnectionReuseStrategy(NoConnectionReuseStrategy.INSTANCE);
}
});
Worked like a charm.

You can provide your own clientConnectionManager to HTTP4. Generally you should use an instance of org.apache.http.impl.conn.PoolingHttpClientConnectionManager, which you'd configure with your own org.apache.http.config.SocketConfig by passing it to setDefaultSocketConfig method of the connection manager.
If you're using Spring with Java config, you would have a method:
#Bean
PoolingHttpClientConnectionManager connectionManager() {
SocketConfig socketConfig = SocketConfig.custom()
.setSoKeepAlive(false)
.setSoReuseAddress(true)
.build();
PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setDefaultSocketConfig(socketConfig);
return connectionManager;
}
and then you'd just use it in your endpoint definition like so: clientConnectionManager=#connectionManager

Using a timing based PollingConsumer to a direct endpoint

Functionally I wish to check a URL is active before I consume from a JMS (WMQ) endpoint.
If the URL cannot be reached or a server error, then I do not want to pick up from the queue. So I want to keep trying (with unlimited retries) the URL via a polling consumer. So as soon as it is available I can pick up from JMS.
I have a RouteBuilder that is set up with a direct endpoint, that is configured to run a Processor that will ping a service.
So:
public class PingRoute extends RouteBuilder {
#Override
public void configureCamel() {
from("direct:pingRoute").routeId(PingRoute.class.getSimpleName())
.process(new PingProcessor(url))
.to("log://PingRoute?showAll=true");
}
}
In another route I am setting up my timer:
#Override
public void configureCamel() {
from(timerEndpoint).beanRef(PollingConsumerBean.class.getSimpleName(), "checkPingRoute");
...
}
And with the PollingConsumerBean I am attempting to receive the body via a consumer:
public void checkPingRoute(){
// loop to check the consumer. Check we can carry on with the pick up from the JMS queue.
while(true){
Boolean pingAvailable = consumer.receiveBody("direct:pingRoute", Boolean.class);
...
}
I add the route to the context and use a producer to send:
context.addRoutes(new PingRoute());
context.start();
producer.sendBody(TimerPollingRoute.TIMER_POLLING_ROUTE_ENDPOINT, "a body");
And I get the following IllegalArgumentException:
Cannot add a 2nd consumer to the same endpoint. Endpoint Endpoint[direct://pingRoute] only allows one consumer.
Is there a way to setup the direct route as a polling consumer?

Business logic is not quite clear, unfortunately. As I understand it - you need to wait for a response from the service. IMHO you have to use Content Enricher EIP http://camel.apache.org/content-enricher.html . pollEnrich is what you need at timer route.
.pollEnrich("direct:waitForResponce", -1) or
.pollEnrich("seda:waitForResponce", -1)
public class PingRoute extends RouteBuilder {
#Override
public void configureCamel() {
from("direct:pingRoute").routeId(PingRoute.class.getSimpleName())
.process(new PingProcessor(url))
.choice().when(body())
.to("log://PingRoute?showAll=true")
.to("direct:waitForResponce")
.otherwise()
.to("direct:pingRoute")
.end();
}
};
timer:
#Override
public void configureCamel() {
from(timerEndpoint)
.inOnly("direct:pingRoute")
.pollEnrich("direct:waitForResponce", -1)
...
}

Based on the OP's clarification of their use case, they have several problems to solve:
Consume the message from the JMS queue if, and only if, the ping to the URL is positive.
If the URL is unresponsive, the JMS message should not disappear from the queue and a retry must take place until the URL becomes responsive again, in which case the message will be ultimately consumed.
The OP has not specified if the amount of retries is limited or unlimited.
Based on this problem scenario, I suggest a redesign of their solution that leverages ActiveMQ retries, broker-side redelivery and JMS transactions in Camel to:
Return the message to the queue if the URL ping failed (via a transaction rollback).
Ensure that the message is not lost (by using JMS persistence and broker-side redeliveries, AMQ will durably schedule the retry cycle).
Be able to specify a sophisticated retry cycle per message, e.g. with exponential backoffs, maximum retries, etc.
Optionally sending the message to a Dead Letter Queue if the retry cycle was exhausted without a positive result, so that some other (possibly manual) action can be planned.
Now, implementation-wise:
from("activemq:queue:abc?transacted=true") // (1)
.to("http4://host.endpoint.com/foo?method=GET") // (2) (3)
.process(new HandleSuccess()); // (4)
Comments:
Note the transacted flag.
If the HTTP invocation fails, the HTTP4 endpoint will raise an Exception.
Since there are no configured exception handlers, Camel will propagate the exception to the consumer endpoint (activemq) which will rollback the transaction.
If the invocation succeeded, the flow will continue and the exchange body will now contain the payload returned by the HTTP server and you can handle it in whichever way you wish. Here I'm using a processor.
Next, what's important is that you configure the redelivery policy in ActiveMQ, as well as enable broker-side redeliveries. You do that in your activemq.xml configuration file:
<plugins>
<redeliveryPlugin fallbackToDeadLetter="true" sendToDlqIfMaxRetriesExceeded="true">
<redeliveryPolicyMap>
<redeliveryPolicyMap>
<redeliveryPolicyEntries>
<redeliveryPolicy queue="my.queue"
initialRedeliveryDelay="30000"
maximumRedeliveries="17"
maximumRedeliveryDelay="259200000"
redeliveryDelay="30000"
useExponentialBackOff="true"
backOffMultiplier="2" />
</redeliveryPolicyEntries>
</redeliveryPolicyMap>
</redeliveryPolicyMap>
</redeliveryPlugin>
</plugins>
And make sure that the scheduler support is enabled in the top-level <broker /> element:
<broker xmlns="http://activemq.apache.org/schema/core"
brokerName="mybroker"
schedulerSupport="true">
...
</broker>
I hope that helps.
EDIT 1: OP is using IBM WebSphere MQ as a broker, I missed that. You could use a JMS QueueBrowser to peek at messages and try their corresponding URLs before actually consuming a message, but it is not possible to selectively consume an individual message – that's not what MOM (messaging-oriented middleware) is about.
So I insist that you should explore JMS transactions, but rather than leaving it up to the broker to redeliver the message, you can start the pinging cycle to the URL within the TX body itself. With regards to Camel, you could implement it as follows:
from("jms:queue:myqueue?transacted=true")
.bean(new UrlPinger());
UrlPinger.java:
public class UrlPinger {
#EndpointInject
private ProducerTemplate template;
private Pattern pattern = Pattern.compile("^(http(?:s)?)\\:");
#Handler
public void pingUrl(#Body String url, CamelContext context) throws InterruptedException {
// Replace http(s): with http(s)4: to use the Camel HTTP4 endpoint.
Matcher m = pattern.matcher(url);
if (m.matches()) {
url = m.replaceFirst(m.group(1) + "4:");
}
// Try forever until the status code is 200.
while (getStatusCode(url, context) != 200) {
Thread.sleep(5000);
}
}
private int getStatusCode(String url, CamelContext context) {
Exchange response = template.request(url + "?method=GET&throwExceptionOnFailure=false", new Processor() {
#Override public void process(Exchange exchange) throws Exception {
// No body since this is a GET request.
exchange.getIn().getBody(null);
}
});
return response.getIn().getHeader(Exchange.HTTP_RESPONSE_CODE, Integer.class);
}
}
Notes:
Note the throwExceptionOnFailure=false option. An Exception will not be raised, therefore the loop will execute until the condition is true.
Inside the bean, I'm looping forever until the HTTP status is 200. Of course, your logic will be different.
Between attempt and attempt, I'm sleeping 5000ms.
I'm assuming the URL to ping is in the body of the incoming JMS message. I'm replacing the leading http(s): with http(s)4: in order to use the Camel HTTP4 endpoint.
Performing the pinging inside the TX guarantees that the message will only be consumed once the ping condition is true (in this case HTTP status == 200).
You might want to introduce a desist condition (you don't want to keep trying forever). Maybe introduce some backoff to not overwhelm the other party.
If either Camel or the broker goes down within a retry cycle, the message will be automatically rolled back.
Take into account that JMS transactions are Session-bound, so if you want to start many concurrent consumers (concurrentConsumers JMS endpoint option), you'll need to set cacheLevelName=CACHE_NONE for each thread to use a different JMS Session.

I am having a bit of difficulty figuring out exactly what you want to do, but it appears to me that you want to consume data from an endpoint on an interval. For this the best pattern is a polling consumer: http://camel.apache.org/polling-consumer.html
The error you are currently receiving is because you have two consumers both trying to read from the "direct://pingRoute" If this was intended you could change the direct to a seda://pingRoute so its an in memory queue your data will be in.

All the answers here pointed me on the right direction but I finally came up with a solution that managed to fit our code base and framework.
Firstly, I discovered there isn't a need to have bean to act as a polling consumer but a processor could be used instead.
#Override
public void configureCamel() {
from("timer://fnzPoller?period=2000&delay=2000").processRef(UrlPingProcessor.class.getSimpleName())
.processRef(StopStartProcessor.class.getSimpleName()).to("log://TimerPollingRoute?showAll=true");
}
Then in the UrlPingProcessor there is CXF service to ping the url and can check the response :
#Override
public void process(Exchange exchange) {
try {
// CXF service
FnzPingServiceImpl fnzPingService = new FnzPingServiceImpl(url);
fnzPingService.getPing();
} catch (WebApplicationException e) {
int responseCode = e.getResponse().getStatus();
boolean isValidResponseCode = ResponseCodeUtil.isResponseCodeValid(responseCode);
if (!isValidResponseCode) {
// Sets a flag to stop for the StopStartProcessor
stopRoute(exchange);
}
}
}
Then in the StopStartProcessor it is using a ExecutorService to stop or start a route via new thread.:
#Override
public void process(final Exchange exchange) {
// routeBuilder is set on the constructor.
final String routeId = routeBuilder.getClass().getSimpleName();
Boolean stopRoute = ExchangeHeaderUtil.getHeader(exchange, Exchange.ROUTE_STOP, Boolean.class);
boolean stopRoutePrim = BooleanUtils.isTrue(stopRoute);
if (stopRoutePrim) {
StopRouteThread stopRouteThread = new StopRouteThread(exchange, routeId);
executorService.execute(stopRouteThread);
} else {
CamelContext context = exchange.getContext();
Route route = context.getRoute(routeId);
if (route == null) {
try {
context.addRoutes(routeBuilder);
} catch (Exception e) {
String msg = "Unable to add a route: " + routeBuilder;
LOGGER.warn(msg, e);
}
}
}
}

synchronous messaging with STOMP over HornetQ

I am trying to figure out how to do syncronous messaging using stomp with hornetq, or if its even possible. I have an async stomp client working, but I can't see how I would implement a sync version.
On the server side, my acceptor looks like this:
<acceptor name="stomp-acceptor">
<factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
<param key="protocol" value="stomp" />
<param key="port" value="61613" />
</acceptor>
and my listener looks like this:
public class SimpleSyncListener extends BaseListener implements SessionAwareMessageListener<Message> {
#Override
public void onMessage(Message message, Session session) throws JMSException {
String lastMessage = "";
try {
lastMessage = ((TextMessage) message).getText();
//System.out.println("server recieved: " + lastMessage);
Destination replyDestination = message.getJMSReplyTo();
StringBuffer sb = new StringBuffer();
sb.append("reply ");
sb.append(Calendar.getInstance().getTimeInMillis());
sb.append(" ");
sb.append(lastMessage);
TextMessage replyMessage = session.createTextMessage(sb.toString());
replyMessage.setJMSCorrelationID(message.getJMSMessageID());
MessageProducer replyProducer = session.createProducer(replyDestination);
replyProducer.send(replyMessage);
} catch (JMSException e) {
throw new RuntimeException(e);
}
incrementCount();
}
I assume I need to put something in the temp queue and send it back like you do with JMS. Its just not clear to me how that works with STOMP. Do i need to open another tcp connection back on the client side that correspond to the "temp queue" on the server side?

Stomp is a simple protocol, and on this case I don't think you can have a multiplexed channel. So you will probably need a Stream to send, and a Stream to receive.

The common strategy to implement synchronous (request / response) communication with JMS - using temporary destinations - is also available with the STOMP implementations of many message brokers (ActiveMQ, Apollo, OpenMQ and RabbitMQ are examples).
However, HornetQ does not support temporary destinations in the current 2.4.0.Final version.