http request - camel's concurrency - java

Does any method of camel's consumer exist which is able to process >20 http requests per second? I try to work with restlet and jetty components, but failed with both.
For example, I set this config for a jetty component:
<bean id="jetty" class="org.apache.camel.component.jetty.JettyHttpComponent">
<property name="httpClientMinThreads" value="10"/>
<property name="httpClientMaxThreads" value="254" />
<property name="minThreads" value="10"/>
<property name="maxThreads" value="254" />
</bean>
and hoped that everything will be OK, but nothing.
My route config:
from("jetty:http://0.0.0.0:8888" + linkRequest+"?matchOnUriPrefix=true")
.onException(Exception.class)
.log(LoggingLevel.ERROR, "${exception.stacktrace}")
.useOriginalMessage()
.handled(true)
.setBody(simple("Something went wrong"))
.end()
.process(new MyFirstProcessor())//here I get httpHeaders,create entity A
.to("jpa:RequestEntity")
.process(new MySecondProcessor())//set some filed in entity A and send it
.to("bean:service?method=process")//here I recieve entity A and create entity B
.to("jpa:ResponseEntity")
.process(new MyThirdProcessor())//here response is created;
Please explain to me how I can configurate parameters of camel (I glance at the threading model configuration), jetty-component or restlet component - so that my router can handle all incoming requests.
UPDATE
These problems were caused by the settings of the connection pool to the database.

Related

In a CXF based Camel route, unable to return the response body from a remote service after post-processing

What i'm trying to do is create a proof-of-concept Camel route, that exposes a cxfrs service endpoint. Requests to this endpoint are routed to another service on a different server using cxf client. After i get the xml response, i need to do some stuff with it, lets say save the response body to a DB for example. And of course the original requestor needs to receive the response as well.
If i don't do any post-processing of the response, then i get the response xml in the browser as expected. But any time i try to add another step to my route for processing the response, the browser gets a response that is empty. As you can see in the commented out lines, it doesn't matter which camel component i use the call the bean. I tried bean, process, and to. Even if i comment out all the code from the bean so it does nothing, the result is the same.
Here's my route:
<cxf:rsServer address="{{base.url}}/employeeservicecxf" id="restServiceCxf">
<cxf:serviceBeans>
<bean class="com.kf.camel.sample.EmployeeServiceResource"/>
</cxf:serviceBeans>
</cxf:rsServer>
<cxf:rsClient
address="http://{{remote.server}}/adminrest/jaxrs/projects/10475/products"
id="rsClient" loggingFeatureEnabled="true" />
<bean class="com.kf.camel.sample.CamelProcessor" id="processor"/>
<bean class="com.kf.camel.sample.CamelResponseProcessor" id="responseProcessor"/>
<camelContext id="_camelContext1" trace="true" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="properties" location="app.properties"/>
<route id="_route1">
<from id="_from1" uri="cxfrs://bean://restServiceCxf"/>
<process id="_process1" ref="processor"/>
<setHeader headerName="CamelHttpMethod" id="_setHeader1">
<constant>GET</constant>
</setHeader>
<to id="_to1" uri="cxfrs://bean://rsClient"/>
<!-- to id="_to3" uri="bean://com.kf.camel.sample.CamelResponseProcessor?method=process"/-->
<bean id="_bean1" ref="responseProcessor" method="process"/>
<!-- process id="_process2" ref="responseProcessor"/-->
</route>
</camelContext>
</beans>
Response Headers
Response Body with content length mismatch error
Have you tried to enable stream caching?
Sounds like any first operation you do on the response is consuming the stream and any further attempt to read the stream again gets an empty result.
When you want to read a stream multiple times in Camel you have to enable stream caching.

spring integration callback on message received

Still new to spring and spring integration so please bear with me. =)
I have set up a client to connect to a remote server using TCP. The server sends a message as soon as a connection is established. Using ngrep I have verified that the connection is up and that the message is sent from the server.
By using the gateway interface "gw" I can successfully receive the message. However what I would like to do is to trigger the com.example.Client.onMessage method when a message is received. My understanding is that this should be possible using a ServiceActivator as shown below. Is this true or do I have to use my own dedicated thread doing a blocking receive? Thanks.
Configuration
<bean id="javaDeserializer"
class="org.springframework.integration.ip.tcp.serializer.ByteArrayLfSerializer" />
<int-ip:tcp-connection-factory id="client"
type="client" host="localhost" port="12000"
single-use="false" so-timeout="10000" so-keep-alive="true" deserializer="javaDeserializer"
serializer="javaSerializer"/>
<int:gateway id="gw" service-interface="com.example.Interface"
default-request-channel="input" default-reply-channel="replies" />
<int:channel id="input" />
<int:channel id="replies">
<int:queue />
</int:channel>
<int-ip:tcp-outbound-channel-adapter
id="outboundClient" channel="input" connection-factory="client" />
<int-ip:tcp-inbound-channel-adapter
id="inboundClient" channel="replies" connection-factory="client"
client-mode="true" retry-interval="10000" auto-startup="true" />
<int:service-activator input-channel="input" output-channel="replies" ref="com.example.Client" method="onMessage" />
ServiceActivator
#EnableIntegration
#IntegrationComponentScan
#MessageEndpoint
public class Client {
#ServiceActivator
public void onMessage(byte[] received) {
//Not called
}
}
#EnableIntegration and #IntegrationComponentScan must be configured on the #Configuration. The general Spring annotations configuration principles: http://docs.spring.io/spring/docs/current/spring-framework-reference/html/beans.html#beans-java. Although having an XML config you don't need those annotations at all.
If you want to receive messages from TCP to the <service-activator>, you have to configure its input-channel to the replies.
Right now you have a mess in your config: several subscribers for the input channel. In this case they receive incoming messages via round-robin manner.
If I understand correctly you should remove all the <int:gateway> staff and just perform step #2. Although it isn't clear how you are going to send messages to the input channel...

How to alter MQ headers using Spring JmsTemplate

I'm using simple Spring JmsTemplate to send messages onto a MQ.
<beans:bean id="myJMSTemplate" class="org.springframework.jms.core.JmsTemplate">
<beans:property name="connectionFactory">
<beans:ref bean="cachedConnectionFactory" />
</beans:property>
<beans:property name="pubSubDomain">
<beans:value>false</beans:value>
</beans:property>
<beans:property name="receiveTimeout">
<beans:value>1000</beans:value>
</beans:property>
</beans:bean>
<int-jms:outbound-channel-adapter id="sendMessageToAvengers" channel="antEventChannel" jms-template="myJMSTemplate" destination-name='com.marvel.avengers.antMan'/>
This works fine, however, My client application isn't able to process the message as the message format is in 'MQHRF2' by default.
How can I alter my MQ headers so as to send explicitly MQSTR format.
Thanks in advance.
You must set the property targetcllient=1 to send the msg as MQSTR.
To do this in sending part of java code, change the queue name as below
String senderQ = "queue:///MYQUEUENAME?targetClient=1";
jmsTemplate.send(senderQ, new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
TextMessage message = session.createTextMessage(text);
message.setJMSReplyTo(replyToQ);
return message;
}
});
Alternatively you can try setting this in jmstemplate bean
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<!-- set format to MQSTR with "targetClient=1" parameter -->
<property name="defaultDestinationName" value="queue:///default?targetClient=1" />
<property name="connectionFactory" ref="mqConnectionFactory" />
</bean>
The IBM MQ JMS interface has a property called TARGCLIENT which when set to the value MQ will not add the MQRFH2 header. I don't know whether the Spring interface allows it to be used.
Alternatively, the queue definition on the queue manager can be configured to remove the header for those applications that can't process it. This is a better way to remove the MQRFH2 header as it removes it at get time instead of at put time, thus meaning that if an application is able to process the MQRFH2 header, it is still there, but for those applications that cannot process it, it is removed for them.
To make the queue operate in this way, issue the following MQSC command on your queue manager:
ALTER QLOCAL(q-name) PROPCTL(NONE)
Additional Reading
Properties of IBM MQ classes for JMS objects > TARGCLIENT
PROPCTL queue options
DEFINE queues - see PROPCTL attribute

Synchronous & Asynchronous Message Receiver in 1 queue (Spring w/ ActiveMQ) JMS API

I am using Spring JMS with ActiveMQ as the broker and running the application on Tomcat.
I have one queue, let's say queue.a. In my web app, I already have a MessageListener running whenever I start my web app. The only thing is, I want to add some kind of queue consumer but synchronously. I already try using JmsTemplate etc. But when both of my consumer (listener async & consumer synchronous) is up and I trigger the .receive() method, the message sent to the queue always sucked up to the message listener that have been always online since the web app started. After the end of the timeout,the synchronous receiver did not consume any message at all.
But,when I comment out the messageListener, the synchronous customer run well.
I'm still a newbie,do any of you have any way to make what I want possible? Thanks! Sorry for my bad english :(
<bean id="someQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="TEST.QUEUE?consumer.priority=10" />
</bean>
and then, set it to your listener/receiver bean:
<bean id="someReceiver" class="blah.blah.SomeReceiver">
<property name="destination" ref="someQueue" />
<property name="jmsTemplate" ref="jmsTemplate" />
</bean>
Does this solve your problem?

jms error handler: receive a callback only when jms gives up

I am using spring-jms with active mq.
I have a ErrorHandler listener set up, that receives a callback each time a jms message delivery fails - i.e. the method that processes the message throws an exception instead of returning gracefully.
However my jms is configured to retry several times until the jms delivery finally succeeds. And my callback is notified of all of the failures.
What I want is a listener that receives a notification only when all the retries finally fail. The motivation is to bring the issue up for the admin's attention. But I don't want spurious notifications in the admin's console.
<bean abstract="true" id="abstractDestinationListener"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsFactory" />
<property name="errorHandler" ref="destinationErrorHandler"/>
<property name="sessionTransacted" value="true"/>
</bean>
What you might consider is using a per-destination dead-letter queue (individualDeadLetterStrategy - https://activemq.apache.org/message-redelivery-and-dlq-handling.html) for the destinations in question. When the maximum redelivery count has been hit, the message is moved out to the DLQ. You can then set up a consumer on that queue with a listener that emails the administrator.
Another way to do it might be to wrap your listener with a try-catch, and rethrow any exceptions only if message.getIntProperty("JMSXDeliveryCount") < MAX_REDELIVERY_COUNT, and email the admin otherwise. However, that option means placing your redelivery limit in two places - the broker config and the code.
if you need more fine-grained control/flexibility/routing with JMS messaging/handling, then you should just use Apache Camel's error handling...

Categories

Resources