Environment :Camel 2.20.0
Java 1.8
Hit rate : 150hits/second
We are trying to upgrade our Camel version to 2.20.0 but we encouter some troubles and memory leak between camel-cxf 2.19.5 and 2.20.0.
We have a Camel route (exposing a CXF Endpoint) which write the payload in a file thanks log4j and move it every 15s in another folder.
The second Camel route read the file from the other folder and split it (and do some Processor).
In our test, the split just log the message "****** FOO BAR ****".
Due to the upgrade of the cxf library, our server crash outOfMemory and the garbage Collector isn't efficient as in the 2.19.5 version.
Is there any change between the 2 versions ?
Is there any configuration to change to keep our server stable ?
We can see these troubles (in 2.20.0) in VisualVM that the memory graph increase until the maximum memory and crash.
Thanks in advance.
First route exposing the CXF
<cxf:cxfEndpoint id="frontalSupervision" address="/supervision" endpointName="s:supervisionSOAP" serviceName="s:supervision"
wsdlURL="wsdl/supervision.wsdl" xmlns:s="http://*****/supervision/">
<cxf:properties>
<entry key="dataFormat" value="PAYLOAD" />
<entry key="receiveTimeout" value="${ws.cxf.receiveTimeout}"/>
</cxf:properties>
<cxf:inInterceptors>
<ref bean="supervision-authentication" />
</cxf:inInterceptors>
</cxf:cxfEndpoint>
<routeContext id="webservice-to-log4j2-route-context" xmlns="http://camel.apache.org/schema/spring">
<route id="webservice-to-log4j2-route">
<from uri="cxf:bean:frontalSupervision" />
<!-- permet de logger le contenu de la requete -->
<to uri="log:body?level=DEBUG" />
<process ref="log4j2LoggerProcessor" />
</route>
</routeContext>
-
The seconde route :
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<routeContext id="log4j2-to-bdd-route-context" xmlns="http://camel.apache.org/schema/spring">
<route id="log4j2-to-bdd-route">
<from
uri="file:///{{messages.log.directory}}/{{messages.log.rolling.relative.directory}}?delete={{messages.files.delete}}&antInclude={{messages.log.filename}}*.log&maxMessagesPerPoll={{messages.maxmessageperpoll}}&sortBy={{messages.sort.by}}&delay={{messages.delay}}" />
<split streaming="true" parallelProcessing="true"
executorServiceRef="splitExecutor">
<tokenize token="{{messages.xmltype.name}}" xml="true" regex="true"/>
<log message="******* FOO BAR *****" />
</split>
</route>
</routeContext>
</beans>
Related
I'm trying to migrate from Camel 2.X to 3.X and have run in to a question about logging the routing trace.
Previously I have configured it like this in my application context xml-file:
<bean id="camelTracer" class="org.apache.camel.processor.interceptor.Tracer">
<property name="traceExceptions" value="false" />
<property name="traceInterceptors" value="true" />
<property name="logLevel" value="DEBUG" />
<property name="logName" value="com.mycompany.routing.trace" />
</bean>
<bean id="traceFormatter" class="org.apache.camel.processor.interceptor.DefaultTraceFormatter">
<property name="showBody" value="true" />
<property name="maxChars" value="0" />
</bean>
But that obviously does not work anymore.
From the migration guide on the Camel website:
"A new tracer has been implemented and the old tracer has been removed. The new tracer logs messages at the org.apache.camel.Tracing logger name which is hardcoded. The format of the output is also updated to make it better. The tracer can be customized."
If I set .tracing() at the start of my routes it does log the trace. The name is hardcoded which is fine, but I would like to change the level from INFO to DEBUG among other things.
Does anyone know where to find information on how to configure this "new" tracer (preferrably in an applicationContext.xml file)? Or anywhere else, maybe in the Java DSL route? Or if it is even possible?
Thanks!
Logging level of DefaultTracer cannot be changed by configuration. You need to implement customized Tracer and bind this implementation to registry.
Tracer:
public class TracerCustom extends DefaultTracer {
private static final Logger LOG = LoggerFactory.getLogger("com.stackoverflow.camel.TracerCustom");
#Override
protected void dumpTrace(String out) {
LOG.debug(out);
}
// Customize other methods if needed
}
Spring context:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<bean class="com.stackoverflow.camel.TracerCustom" />
<camelContext id="tracerCamelContext" xmlns="http://camel.apache.org/schema/spring">
<route trace="true">
<from uri="timer:test"/>
<to uri="log:test"/>
</route>
</camelContext>
</beans>
How can I add default WS-addressing to the xml?
<cxf:cxfEndpoint id="endpoint" xmlns:s="http://tempuri.org/"
address="https://uslugaterytws1test.stat.gov.pl/TerytWs1.svc"
endpointName="s:custom"
serviceName="s:TerytWs1"
wsdlURL="classpath:/wsdl/terytws1.wsdl">
<cxf:properties>
<entry key="schema-validation-enabled" value="false" />
</cxf:properties>
<cxf:inInterceptors>
</cxf:inInterceptors>
<cxf:inFaultInterceptors>
</cxf:inFaultInterceptors>
<cxf:outInterceptors>
</cxf:outInterceptors>
<cxf:outFaultInterceptors>
</cxf:outFaultInterceptors>
</cxf:cxfEndpoint>
<cxf:cxfEndpoint id="poxyEndpoint" xmlns:s="http://tempuri.org/"
address="http:localhost:5678/myproxy"
endpointName="s:customProxy"
serviceName="s:TerytWs1Proxy"
wsdlURL="classpath:/wsdl/terytws1Proxy.wsdl">
<cxf:properties>
<entry key="schema-validation-enabled" value="false" />
</cxf:properties>
<cxf:inInterceptors>
</cxf:inInterceptors>
<cxf:inFaultInterceptors>
</cxf:inFaultInterceptors>
<cxf:outInterceptors>
<ref component-id="wssOutInterceptor" />
</cxf:outInterceptors>
<cxf:outFaultInterceptors>
<ref component-id="wssOutInterceptor" />
</cxf:outFaultInterceptors>
</cxf:cxfEndpoint>
<camelContext id="proxyTerytContext" xmlns="http://camel.apache.org/schema/blueprint">
<route id="route-TerytWs1">
<from id="inbound" uri="cxf:bean:proxyEndpoint?dataFormat=CXF_MESSAGE" />
<to id="outbound" uri="cxf:bean:endpoint?dataFormat=CXF_MESSAGE" />
</route>
</camelContext>
When I send request to http:localhost:5678/myproxy then I get:
<faultcode xmlns:a="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">a:InvalidSecurity</faultcode>
<faultstring xml:lang="en-US">An error occurred when verifying security for the message.</faultstring>
I have read many similar questions and examples but haven't found the solution for pure cxf xml. I have been trying to solve this for 2 days. Now I'm crying.
EDIT:
This is an original wsdl: https://uslugaterytws1test.stat.gov.pl/terytws1.svc?wsdl
and this is my proxy to it: https://github.com/woblak/training/blob/master/teryt_testProxy.wsdl
user: TestPubliczny
pass: 1234abcd
There are some examples in the Camel unit tests which test CXF endpoints with WS-Addressing enabled; WSAddressingTest-context.xml seems like it might be relevant to your question?
Here, WS-Addressing has been enabled on a CXF endpoint by adding the wsa:addressing element in features:
<cxf:cxfEndpoint...>
<cxf:features>
<wsa:addressing xmlns:wsa="http://cxf.apache.org/ws/addressing" />
</cxf:features>
</cxf:cxfEndpoint>
The error seems to be related to security. You have an interceptor configured (wssOutInterceptor) but there's no source code. Perhaps you should look there to see if you're setting the auth details.
I would also add message logging so you can see the content of the payload sent to the target and verify that it contains your credentials.
Or, if you're using the Camel CXF namespace ( xmlns:cxf="http://camel.apache.org/schema/cxf") you can use:
<cxf:cxfEndpoint ... loggingFeatureEnabled="true">
...
</cxf:cxfEndpoint>
I am using Apache camel and jboss fuse, I have created a sample route blue print listed below in, i have sucessfully handling exceptions in all my routes now the problem is i can not find any example of throttling in routes as i have defined. in apache camel documentation, they have given simple DSL throttling and in stackoverflow i have found rabbitMq throttling which is not my case. how to throttle routes like this in apache camel
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:camel="http://camel.apache.org/schema/blueprint"
xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<cxf:rsServer address="/testservice" id="testserver" serviceClass="com.company.HelloBean">
<camelContext id="testContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="testRoute" >
<throttle timePeriodMillis="10000">
<constant>3</constant>
<from id="_from1" uri="cxfrs:bean:testserver"/>
<bean beanType="com.company.HelloBean"
id="_bean1" method="hello"/>
</throttle>
</route>
</camelContext>
</blueprint>
this give error when in deploy application in jboss fuse. that can not find service
hy, all you need is you are currently defining your from endpoint in throttle tag which is wrong you need to define the throttling tag only in TO tag like this
<throttle id="_throttle1" rejectExecution="true" timePeriodMillis="10000">
<constant>1</constant>
<bean beanType="com.company.HelloBean"
id="_bean1" method="hello"/>
</throttle>
when requests come to an end point from, you will throttle while requests are going TO another end point you are using beans so, you can do something like this
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:camel="http://camel.apache.org/schema/blueprint"
xmlns:cxf="http://camel.apache.org/schema/blueprint/cxf"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd">
<cxf:rsServer address="/testservice" id="testserver" serviceClass="com.evampsaanga.gethomepage.GetHomePageDataLand">
<cxf:providers>
<bean class="com.evampsaanga.restresponses.ExceptionHandler" id="securityException"/>
</cxf:providers>
</cxf:rsServer>
<!-- <bean class="com.saanga.servicetest.THR" id="myPolicy"/> -->
<camelContext id="testContext" trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<route id="testRoute" >
<from id="_from1" uri="cxfrs:bean:testserver"/>
<log id="_log1" message="header : ${headers}"/>
<setHeader headerName="headerbalance" id="_setHeader1">
<simple>${headers}</simple>
</setHeader>
<setBody id="_setBody1">
<simple>${body}</simple>
</setBody>
<throttle id="_throttle1" rejectExecution="true" timePeriodMillis="10000">
<constant>1</constant>
<bean beanType="com.evampsaanga.gethomepage.GetHomePageDataLand"
id="_bean1" method="Get"/>
</throttle>
</route>
</camelContext>
</blueprint>
I've referred to the JMS page of the Camel documentation and many related SO questions such as this one, but I'm unable to find a comprehensive list on the implementation.
I'm using Spring XML along with Camel and Weblogic for the server. I've made a test queue with the following names:
Server: TestJMSServer, Module: TestJMSModule, Queue: TestJMSQueue, CF: TestConnectionFactory.
According to the Camel documentation, my route should look something like this:
<camel:route id="test">
<camel:from uri="jms:TestJMSQueue" />
<camel:to uri="file:/Users/...." />
</camel:route>
This gives me an error saying "connectionFactory must be specified". So exactly what else do I need to add to my applicationContext.xml in order to listen to this queue?
You need to tell Camel's jms-component which JMS connection factory to use. Most likely you'll get that from jndi if you're using WebLogic.
In the example below i am looking up the connection factory using spring's jee:jndi-lookup (i believe that might even be a name you can use in WebLogic). The looked up factory is then made available as a spring bean with id myConnectionFactory.
This connection factory bean is then used for the connectionFactory property for camel's JmsComponent. Notice the id attribute: jms. This defines the camel endpoint uri scheme to be used in your routes.
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:jee="http://www.springframework.org/schema/jee"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<jee:jndi-lookup id="myConnectionFactory" jndi-name="jms/connectionFactory"/>
<route id="test" xmlns="http://camel.apache.org/schema/spring">
<from uri="jms:TestJMSQueue"/>
<to uri="file:/Users/...."/>
</route>
<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="myConnectionFactory"/>
<!-- more configuration required based on your requirements -->
</bean>
<!--
example uses invm amq broker:
<bean id="anothercnf" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://mybroker"/>
</bean>
-->
</beans>
Important Note: You will need to tune this further (setup transactions, setup concurrent consumers, possible configure a spring jms connection pool)
I'm dealing with a problem how to read a file with Camel in SMX 4.4.2 but only in case when
the read operation cannot be performed in Camel route from.
I need to read data from the file transform it into XML and then return it to the requestor (the requestor hits SMX via REST Service).
I'm having serious problems with reading the file after the REST invokation.
Routes are like these:
<jaxrs:server id="restService" address="http://localhost:9000/REST/"
staticSubresourceResolution="true">
<jaxrs:serviceBeans>
<ref bean="restFtpBean" />
</jaxrs:serviceBeans>
</jaxrs:server>
<bean id="restFtpBean" class="poc.rest.RESTFtpProxyService" />
<cxf:rsServer id="rsServer" address="http://localhost:9000/REST/"
serviceClass="poc.rest.RESTFtpProxyService" />
<camel:camelContext xmlns="http://camel.apache.org/schema/spring">
<endpoint id="ftpProvider" uri="ftp://localhost:21/?fileName=test.xml" />
<route>
<from uri="cxfrs://bean://rsServer" />
<to ref="ftpProvider" />
</route>
</camel:camelContext>
In this scenario Camel tries to write to file !
What am I doing wrong ?
Its always advised to learn and get familiar with the EIPs
http://camel.apache.org/content-enricher.html
Your use-case you need to consume a file from within a route, and there is an EIP pattern for that: http://camel.apache.org/content-enricher.html. See the section about using pollEnrich to consume/poll the file.