Develop Reference

How to specify a ReadConcern to use with Spring's MongoTemplate?

With the vanilla java MongoClient you can specify a read concern for a particular query for example like this:
var result = mongoClient.getDatabase("somedb")
.getCollection("collection")
.withReadConcern(ReadConcern.MAJORITY)
.find(..)
But I cannot seem to find a way to set the ReadConcern using Spring's MongoTemplate. Preferably I'd like to set it for a specific query, but if this is not possible I'm fine with configuring something equivalent to a WriteConcernResolver (but for read concerns).
Is this possible, and if so how?
I'm using spring-data-mongodb version 3.2.6.

It seems that you maybe able to set this on the TransactionOptions and MongoTransactionManager:
https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/MongoTransactionManager.html
Example:
TransactionOptions transactionOptions = TransactionOptions.builder().readConcern(ReadConcern.LOCAL).writeConcern(WriteConcern.W1).build();
return new MongoTransactionManager(dbFactory.getMongoDatabaseFactory(), transactionOptions);
=== Edited ===
How about we take it the MongoClient route, since you stated it works on it? We could create a MongoTemplate from a given MongoClient instance:
Example:
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import com.mongodb.ReadConcern;
import com.mongodb.ReadPreference;
import com.mongodb.WriteConcern;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import org.springframework.context.annotation.Bean;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.core.MongoTemplate;
#Configuration
public class DemoApplication extends AbstractMongoClientConfiguration {
#Bean
public MongoTemplate mongoTemplate() {
return new MongoTemplate(mongoClient(), "someDbName");
}
#Override
public MongoClient mongoClient() {
final ConnectionString connectionString = new ConnectionString("mongodb://localhost:27017/someDbName");
MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
.applyConnectionString(connectionString)
.readConcern(ReadConcern.DEFAULT)
.writeConcern(WriteConcern.MAJORITY)
.readPreference(ReadPreference.primary()).build();
return MongoClients.create(mongoClientSettings);
}
#Override
protected String getDatabaseName() {
return "someDbName";
}
}

Spring batch, errorr when adding new reader

I'm working on projet spring batch simple process read, process write to csv file. There are already many readers ans sql scripts in this projet. The problem when i add new reader and sql script and i start the batch, i got an error . However, when i use the new reader to and old script sql, it works and the same thing if I use an existing reader to select the new sql script, it works fine but not i use both.
These are the elements:
/**
* Reader
*/
import org.springframework.stereotype.Component;
import javax.sql.DataSource;
import java.io.IOException;
#Component
public class BillDefaultReader extends AbstractBillReader {
public BillDEfaultReader(final DataSource dataSource) throws IOException {
super(dataSource);
this.setPreparedStatementSetter(s -> s.setString(1, "DEFAULT"));
}
#Override
protected String getSqlPath() {
return "/sql/get_script1.sql"; -> SQL
}
}
// The job config
#Bean
public Job batchExJob(final Step batchExStep, final ReportGeneratorJobListener reportGeneratorJobListener) {
return this.jobs.get("batchExJob").start(batchExStep).listener(reportGeneratorJobListener).build();
}
#Bean
public Step batchExStep(final ItemReader<BillTransmissionDTO> compositeBillReader,
final ItemProcessor<BillTransmissionDTO, BillTransmissionDTO> compositeProcessor,
final ItemWriter<BillTransmissionDTO> billTransmissionWriter) {
return this.stepBuilderFactory.get("batchExStep").<BillTransmissionDTO, BillTransmissionDTO>chunk(this.commitInterval).reader(compositeBillReader)
.processor(compositeProcessor)
.writer(billTransmissionWriter)
.faultTolerant()
.skipPolicy(this::skipExceptionsPolicy)
.build();
}
#StepScope
#Bean
public ItemProcessor<BillTransmissionDTO, BillTransmissionDTO> compositeProcessor(final IntegrateBillProcessor integrateBillProcessor,
final InitialisationPayEndowmentProcessor initialisationPaiementDotationProcessor) {
final CompositeItemProcessor<BillTransmissionDTO, BillTransmissionDTO> processor = new CompositeItemProcessor<>();
processor.setDelegates(Arrays.asList(integrateBillProcessor, initializePaiementEndowmentProcessor));
return processor;
}
So this is a part of code (it's translated), This is the same process and same code for all readers and sql , any idea please?
Thank you!

Camel SCR component tries to add component to context twice if route builder includes an errorHandler

I am using Camel in Karaf using SCR to process messages from ActiveMQ
Versions:
Camel: 2.16.0
Karaf: 4.0.7
ActiveMQ 5.14.1
When I deploy the following Camel route to Karaf all works fine:
package com.test;
import org.apache.camel.builder.RouteBuilder;
public class TestRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
from("activemq:queue:TEST.IN")
.routeId("test-route")
.log("Message picked up from IN queue");
}
}
Here is my SCR Runner class:
package com.test;
import java.util.ArrayList;
import java.util.List;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.activemq.camel.component.ActiveMQComponent;
import org.apache.activemq.pool.PooledConnectionFactory;
import org.apache.camel.RoutesBuilder;
import org.apache.camel.component.jms.JmsConfiguration;
import org.apache.camel.scr.AbstractCamelRunner;
import org.apache.camel.spi.ComponentResolver;
import org.apache.felix.scr.annotations.Component;
import org.apache.felix.scr.annotations.Properties;
import org.apache.felix.scr.annotations.Property;
import org.apache.felix.scr.annotations.Reference;
import org.apache.felix.scr.annotations.ReferenceCardinality;
import org.apache.felix.scr.annotations.ReferencePolicy;
import org.apache.felix.scr.annotations.ReferencePolicyOption;
import org.apache.felix.scr.annotations.References;
import org.osgi.framework.BundleContext;
#Component(label = TestRunner.COMPONENT_LABEL, description = TestRunner.COMPONENT_DESCRIPTION, immediate = true, metatype = true)
#Properties({
#Property(name = "camelContextId", value = "test-context"),
#Property(name = "active", value = "true"),
})
#References({
#Reference(name = "camelComponent",referenceInterface = ComponentResolver.class,
cardinality = ReferenceCardinality.MANDATORY_MULTIPLE, policy = ReferencePolicy.DYNAMIC,
policyOption = ReferencePolicyOption.GREEDY, bind = "gotCamelComponent", unbind = "lostCamelComponent")
})
public class TestRunner extends AbstractCamelRunner {
public static final String COMPONENT_LABEL = "TestRunner";
public static final String COMPONENT_DESCRIPTION = "This is the description for the test runner";
#Override
protected List<RoutesBuilder> getRouteBuilders() {
List<RoutesBuilder> routesBuilders = new ArrayList<RoutesBuilder>();
routesBuilders.add(new TestRoute());
return routesBuilders;
}
#Override
protected void setupCamelContext(BundleContext bundleContext, String camelContextId)throws Exception{
super.setupCamelContext(bundleContext, camelContextId);
// Add Active MQ connection factory
ActiveMQConnectionFactory amqConnectionFactory = new ActiveMQConnectionFactory("tcp://c3m-activemq:61616");
amqConnectionFactory.setUserName("admin");
amqConnectionFactory.setPassword("admin");
// Create Pooled Connection Factory
PooledConnectionFactory amqPooledConnectionFactory = new PooledConnectionFactory(amqConnectionFactory);
amqPooledConnectionFactory.setMaxConnections(5);
amqPooledConnectionFactory.setMaximumActiveSessionPerConnection(5);
// Create JMS Configuration
JmsConfiguration consumerJmsConfig = new JmsConfiguration(amqPooledConnectionFactory);
consumerJmsConfig.setConcurrentConsumers(5);
// Create the ActiveMQ Component
ActiveMQComponent activemq = ActiveMQComponent.activeMQComponent();
activemq.setConfiguration(consumerJmsConfig);
// Add activeMQ component to the Camel Context
getContext().addComponent("activemq", activemq);
// Use MDC logging
getContext().setUseMDCLogging(true);
// Use breadcrumb logging
getContext().setUseBreadcrumb(true);
}
}
However, if I add an errorHandler to my routeBuilder then things fail.
Here's the same route with the errorHandler added:
public void configure() throws Exception {
errorHandler(deadLetterChannel("activemq:queue:TEST.DLQ").useOriginalMessage());
from("activemq:queue:TEST.IN")
.routeId("test-route")
.log("Message picked up from IN queue");
}
What happens:
- When installing the bundle on Karaf the following error is given:
2016-12-20 09:49:58,248 | ERROR | nsole user karaf | router | 124 - com.test.router - 1.1.0.SNAPSHOT | [com.test.TestRunner(7)] The activate method has thrown an exception
java.lang.IllegalArgumentException: Cannot add component as its already previously added: activemq
at org.apache.camel.impl.DefaultCamelContext.addComponent(DefaultCamelContext.java:369)
at com.test.TestRunner.setupCamelContext(TestRunner.java:75)[124:com.test.router:1.1.0.SNAPSHOT]
at org.apache.camel.scr.AbstractCamelRunner.prepare(AbstractCamelRunner.java:90)[72:org.apache.camel.camel-scr:2.16.0]
at org.apache.camel.scr.AbstractCamelRunner.activate(AbstractCamelRunner.java:79)[72:org.apache.camel.camel-scr:2.16.0]
...
And then the Camel route is NOT deployed in Karaf.
I'll proceed with some more troubleshooting, but perhaps someone understand more fully what's going wrong here

In your own TestRunner class then only add the component if its not already registered, you can use
if (context.hasComponent("activemq") != null) {
... add component
}

In the end I solved the problem with the following hack: If the component already exists, I first remove it and then add it back.
Here's the code:
// If activemq component already exists, remove it
// Note: This is a bit of a hack, but if we keep the one that is there
// Camel throws a security exception.
if (getContext().hasComponent("activemq") != null) {
getContext().removeComponent("activemq");
}
// Create the ActiveMQ Component
ActiveMQComponent activemq = ActiveMQComponent.activeMQComponent();
activemq.setConfiguration(consumerJmsConfig);
getContext().addComponent("activemq", activemq);
Not pretty, but if I don't remove it, and deploy the route, camel gives a security exception, almost as if the existing component "lost" the credentials of the broker.
Thanks for the help Claus!

Adding REST route to an existing Jetty endpoint in Camel at runtime

I have been inventing a way how to work around the problem of adding consumers to a jetty endpoint (it does not allow multiple consumers). The way we do it in our company is to build our own router and a broadcasting endpoint which consumes from jetty and routes requests to underlying "subscriptions". Only one of them will eventually process the request. It kind of works but it's not completely ok, since recently when updating to latest Camel we have found our custom built component to leak memory and in general I consider using built-in functionality over custom hacks.
I started investigating the Camel REST API and found it very nice and pretty much replacing our home-grown component apart from one thing - you cannot re-configure it at runtime - you have to stop the context basically for this to work. Below I include my unit test with a happy path and the path that fails. Frankly I think is a bug, but if there is a legitimate way to achieve what I want, I'd like to hear sound advice:
package com.anydoby.camel;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.commons.io.IOUtils;
import org.junit.Before;
import org.junit.Test;
/**
* Test tries to add/remove routes at runtime.
*/
public class RoutesTest {
private DefaultCamelContext ctx;
#Before
public void pre() throws Exception {
ctx = new DefaultCamelContext();
new RouteBuilder(ctx) {
#Override
public void configure() throws Exception {
restConfiguration("jetty").host("localhost").port(8080);
rest("/")
.get("/issues/{isin}").route().id("issues")
.process(e -> e.getOut().setBody("Here's your issue " + e.getIn().getHeader("isin"))).endRest()
.get("/listings").route().id("listings").process(e -> e.getOut().setBody("some listings"));
}
}.addRoutesToCamelContext(ctx);
ctx.start();
}
#Test
public void test() throws IOException {
{
InputStream stream = new URL("http://localhost:8080/issues/35").openStream();
assertEquals("Here's your issue 35", IOUtils.toString(stream));
}
{
InputStream stream = new URL("http://localhost:8080/listings").openStream();
assertEquals("some listings", IOUtils.toString(stream));
}
}
#Test
public void disableRoute() throws Exception {
ctx.stopRoute("issues");
ctx.removeRoute("issues");
try (InputStream stream = new URL("http://localhost:8080/issues/35").openStream()) {
fail();
} catch (Exception e) {
}
new RouteBuilder(ctx) {
#Override
public void configure() throws Exception {
rest().get("/issues/{isin}/{sedol}").route().id("issues")
.process(e -> e.getOut()
.setBody("Here's your issue " + e.getIn().getHeader("isin") + ":" + e.getIn().getHeader("sedol")))
.endRest();
}
}.addRoutesToCamelContext(ctx);
{
InputStream stream = new URL("http://localhost:8080/issues/35/65").openStream();
assertEquals("Here's your issue 35:65", IOUtils.toString(stream));
}
}
}
The disableRoute() test fails since I cannot add another consumer to an existing endpoint.
So my question is - "is there a way to add a new URL mapping to a restful camel-jetty endpoint"? If you do it during first configuration it works fine, but when later you want to reconfigure one of the routes the error is:
org.apache.camel.FailedToStartRouteException: Failed to start route because of Multiple consumers for the same endpoint is not allowed: jetty:http://localhost:8080/issues/%7Bisin%7D/%7Bsedol%7D?httpMethodRestrict=GET

how to create datasource using camel?

I have just started learning Apache Camel. I understood the basics of Routes and Components. Now I want to give a try by connecting to Oracle database, reading records from one particular table and write those records to a file using File component. To read from database I assume I need to use JDBC component and give the dataSourceName.
However, I couldn't find any info on how to create a dataSource using camel. All info that I found related to this topic uses Spring DSL examples. I don't use Spring and I just need to test this using simple standalone Java application.
I am using JDK7u25 with Apache Camel 2.12.1.
Can someone please post a sample to read from the oracle table and write to a file?
[EDIT]
After checking several solutions on the web, I came to know about following two approaches:
Camel to run as standalone. Here is my code:
import javax.sql.DataSource;
import org.apache.camel.main.Main;
import org.apache.camel.builder.RouteBuilder;
import org.apache.commons.dbcp.BasicDataSource;
public class JDBCExample {
private Main main;
public static void main(String[] args) throws Exception {
JDBCExample example = new JDBCExample();
example.boot();
}
public void boot() throws Exception {
// create a Main instance
main = new Main();
// enable hangup support so you can press ctrl + c to terminate the JVM
main.enableHangupSupport();
String url = "jdbc:oracle:thin:#MYSERVER:1521:myDB";
DataSource dataSource = setupDataSource(url);
// bind dataSource into the registery
main.bind("myDataSource", dataSource);
// add routes
main.addRouteBuilder(new MyRouteBuilder());
// run until you terminate the JVM
System.out.println("Starting Camel. Use ctrl + c to terminate the JVM.\n");
main.run();
}
class MyRouteBuilder extends RouteBuilder {
public void configure() {
String dst = "C:/Local Disk E/TestData/Destination";
from("direct:myTable")
.setBody(constant("select * from myTable"))
.to("jdbc:myDataSource")
.to("file:" + dst);
}
}
private DataSource setupDataSource(String connectURI) {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setUsername("sa");
ds.setPassword("devon1");
ds.setUrl(connectURI);
return ds;
}
}
Using the approach mentioned by Claus lbsen. Here is the code again:
import javax.sql.DataSource;
import org.apache.camel.CamelContext;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.impl.SimpleRegistry;
import org.apache.camel.main.Main;
import org.apache.camel.builder.RouteBuilder;
import org.apache.commons.dbcp.BasicDataSource;
public class JDBCExample {
private Main main;
public static void main(String[] args) throws Exception {
String url = "jdbc:oracle:thin:#MYSERVER:1521:myDB";
DataSource dataSource = setupDataSource(url);
SimpleRegistry reg = new SimpleRegistry() ;
reg.put("myDataSource",dataSource);
CamelContext context = new DefaultCamelContext(reg);
context.addRoutes(new JDBCExample().new MyRouteBuilder());
context.start();
Thread.sleep(5000);
context.stop();
}
class MyRouteBuilder extends RouteBuilder {
public void configure() {
String dst = "C:/Local Disk E/TestData/Destination";
from("direct:myTable")
.setBody(constant("select * from myTable"))
.to("jdbc:myDataSource")
.to("file:" + dst);
}
}
private static DataSource setupDataSource(String connectURI) {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setUsername("sa");
ds.setPassword("devon1");
ds.setUrl(connectURI);
return ds;
}
}
But in both the cases I am getting below exception:
Caused by: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: jdbc://myDataSource due to: No component found with scheme: jdbc
at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:534)
at org.apache.camel.util.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:63)
at org.apache.camel.model.RouteDefinition.resolveEndpoint(RouteDefinition.java:192)
at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:106)
at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:112)
at org.apache.camel.model.SendDefinition.resolveEndpoint(SendDefinition.java:61)
at org.apache.camel.model.SendDefinition.createProcessor(SendDefinition.java:55)
at org.apache.camel.model.ProcessorDefinition.makeProcessor(ProcessorDefinition.java:500)
at org.apache.camel.model.ProcessorDefinition.addRoutes(ProcessorDefinition.java:213)
at org.apache.camel.model.RouteDefinition.addRoutes(RouteDefinition.java:909)
... 12 more
[Thread-0] INFO org.apache.camel.main.MainSupport$HangupInterceptor - Received hang up - stopping the main instance.

There is the SQL example which shows how to setup a DataSource
http://camel.apache.org/sql-example.html
Yes that examples using Spring XML. But how you setup the DataSource can be done in Java code also. Then you need to register the DataSource in the Camel Registry.
For example you can use a JndiRegistry or the SimpleRegistry. The latter is easier.
Here is some pseudo code showing the principle, of creating a registry, add your beans into this registry, and then providing the registry to the constructor of DefaultCamelContext.
SimpleRegistry registry = new SimpleRegistry();
// code to create data source here
DateSource ds = ...
registry.put("myDataSource", ds);
CamelContext camel = new DefaultCamelContext(registry);

So silly me! I had not included camel-jdbc-2.12.1.jar in the CLASSPATH. Now above examples work.

Spring was mentioned there just because it is very useful paradigm of working with DB (mainly because of templates introduced by Spring Framework.) Of course you can hook up the standard JDBC connection and implement DAO by yourself - nothing wrong with that.