I have configured the re delivery related properties as follows. In fact, I wanted to retry only for HTTP Code 500x series. But I see that onException() takes only exception type. How to achieve this? My application is spring boot & camel based.
errorHandler(defaultErrorHandler().maximumRedeliveries(3).redeliveryDelay(5000).logRetryAttempted(true).logExhausted(true));
onException().retryAttemptedLogLevel(LoggingLevel.WARN)
Camel 2.23 is the version I am using(As per the comment).
Need more information, how you get resp code.
if you get response from camel route you can add condition in route:
.choice().when().simple("header.HTTP_RESPONSE_CODE >= 500").errorHandler(defaultErrorHandler().maximumRedeliveries(3).redeliveryDelay(5000).logRetryAttempted(true).logExhausted(true)).end();
Related
I have an apache camel route which is calling apache kafka topic (producer) just like below in my spring boot application (deployed on Tomcat server):
from("timer://foo?period=1000").to("kafka:myTopic?brokers=localhost:9092");
This spring boot app is a rest API that is supposed to get around 300 TPS.
Q1) I know that a single tomcat server thread serves each request coming to my spring boot app. Does the same thread will be used in the above line of code used by apache camel for invoking the myTopic? Or apache camel uses some connection pooling internally just like RestTemplate?
Q2) Because the TPS will be increasing to 500 TPS in near future, does it make any sense to introduce pooling for the above line of code? I believe if I use connection pooling, my application performance will increase. However, I am not able to find the code which I can use to enable connection pooling for the above line of code.
If anyone has any idea please let me know. Please note that I am not looking for parallel processing so seda or multicast in camel is not an option. I am only having a single call to kafka topic as shown above in the code so just looking for how to enable connection pooling in this line of code.
Thanks.
We have a spring boot application and we are using apache camel as a framework for message processing. We are trying to best optimize our application settings to make the enqueue of messages on the ActiveMQ queue fast which is received by the Logstash on the other end of the queue as consumers.
The documentation is scattered at many places and there are too many configurations available.
For example, the camel link for spring boot specifies 102 options. Similarly, the activemq apache camel link details these with much more.
This is what we have currently configured:
Application.properties:
################################################
# Spring Active MQ
################################################
spring.activemq.broker-url=tcp://localhost:61616
spring.activemq.packages.trust-all=true
spring.activemq.user=admin
spring.activemq.password=admin
Apache Camel
.to("activemq:queue:"dataQueue"?messageConverter=#queueMessageConverter");
Problem:
1 - We suspect that we have to use poolConnectionFactory and not default Spring JMS Template bean which is somehow auto picked up.
2 - We also want the process to be asynchornous. We just want to put the message on queue and dont want to wait for any ACK from activemq or do anyretry or something.
3 - We want to wait for retry only if queue is full.
4 - Where should we set the settings for ActiveMq size? and also the activemq is putting things in Dead letter queue in case no consumer availaible? We want to override that behaviour and want to keep the message in there. (Is this have to be configured in Activemq and not in Our app/apache camel)
Update
Here is we have solved it after some more investigation and based on feedback for now. Note: this does not involve retrying, for that we will try the option suggested in the answer.
For Seda queues:
producer:
.to("seda:somequeue?waitForTaskToComplete=Never");
consumer:
.from("seda:somequeue?concurrentConsumers=20");
Active MQ:
.to("activemq:queue:dataQueue?disableReplyTo=true);
Application.Properties:
#Enable poolconnection factory
spring.activemq.pool.enabled=true
spring.activemq.pool.blockIfFull=true
spring.activemq.pool.max-connections=50
Yes, you need to use pooledConnectionFactory. Especially with Camel+Spring Boot. Or look to use the camel-sjms component. The culprit is Spring's JMSTemplate. Super high latency.
Send NON_PERSISTENT and AUTO_ACK, also turn on sendAsync on the connection factory
You need to catch javax.jms.ResourceAllocationException in your route to do retries when Producer Flow Control kicks in (aka queue or broker is full)
ActiveMQ does sizing based on bytes, not message count. See the SystemUsage settings in Producer Flow Control docs and Per-Destination Policies policies for limiting queue size based on bytes.
I am using Apache camel 3.1.0 with Spring boot 1.5.6 and I have successfully created perfectly working Camel's Transactional routes. Basic route is like this
from(Source JmsEndpoint)
.process(
myDaoImplClass.myTransactionalDatabaseMethod("Save data coming in Message")
).to(Destination Endpoint)
Issue occurs when Queue Depth gets full on destination queue and due to transactional route it is not removed from Source queue. Thus source queue again sends same message and my database method gives DataIntegrityViolationException due to duplicate message.
Is there a way I can rollback persisted data or do commit only when whole route is successful.
Check out Camels onCompletion and the Unit of Work concept.
You can use it to do stuff on failure or success only.
is there anyway that can help me increase the timeout of the swagger request that i issue to my RestAPI using " Try it out" of swagger ?
i roamed the internet and disn't find something usefull, and i tried to add things to my yml conf files, but i didn't know what to write
i expected to find something like this maybe :
swagger.timeout=5000
i have a spring boot application using swagger in order to test my webservices in a restController. i have a treatement that takes about 30 seconds, the thing that makes swagger throw a 500 timeout error for me.
thanks !
As far as I know, Swagger registers its endpoints as normal Spring controllers so use the underlying web container to serve its traffic.
Try to increase the global connection timeout:
server.connection-timeout=120000
Time that connectors wait for another HTTP request before closing the connection. When not set, the connector's container-specific default is used. Use a value of -1 to indicate no (that is, an infinite) timeout.
solved by adding this to ZUUL's (API Gateway)
ribbon:
ReadTimeout: 120000
ConnectTimeout: 120000
I am attempting to create a camel route using blueprint that sends a message on an activeMQ queue then listens to the response on the temporary queue created in the request. This seems pretty basic, but I can't find an example that utilizes it.
I have tried searching and reading the docs and here's what I've found:
http://camel.apache.org/jms.html
http://camel.apache.org/exchange-pattern.html
http://camel.apache.org/request-reply.html
https://access.redhat.com/documentation/en-US/Red_Hat_JBoss_Fuse/6.0/html/EIP_Transaction_Guide/files/FMRTxnJMSSynchronous.html
http://kosalads.blogspot.com/2014/04/ApacheCamelRequestReplyPatternWithJavaDS.html
http://grokbase.com/t/camel/users/128n88xeva/how-to-use-request-reply-in-jms
http://camel.465427.n5.nabble.com/ExchangePattern-InOut-I-Can-t-get-any-response-td5056301.html
https://examples.javacodegeeks.com/enterprise-java/apache-camel/apache-camel-exchange-example/
Which is frustrating.
I have my activeMQ component set up such:
<to pattern="InOut" uri="activemq:queue:tripRequest.updateStatus.v1.0?useMessageIDAsCorrelationID=true"/>
<log message="Update Status responded ${out.body}"/>
The log shows the input XML, which surprised me. After checking the docs, I created a new activeMQ instance that listens to the same queue and dumps to a log, but this threw errors and it keeps mixing up my log and unmarshal objects on my other route.
How can I accomplish this?
Check the answer in the link below. It should give you hints on how to build your active-mq uri for a request/reply scenario.
Implement Request-Reply pattern using ActiveMQ, Camel and Spring