Trying to unzip a file using Apache Camel, I tried the example given in http://camel.apache.org/zip-file-dataformat.html but I can't find UnZippedMessageProcessor class. Here's the code:
import java.util.Iterator;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.dataformat.zipfile.ZipFileDataFormat;
public class TestRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
ZipFileDataFormat zipFile = new ZipFileDataFormat();
zipFile.setUsingIterator(true);
from("file:src/test/resources/org/apache/camel/dataformat/zipfile/")
.unmarshal(zipFile).split(body(Iterator.class)).streaming()
.process(new UnZippedMessageProcessor()).end();
}
}
Anyone tried to do this or have another way to unzip a file through a Camel route?
Thank you in advance!
You can also define the route like this, you can find the ZipSplitter inside of camel-zipfile.
from("file:src/test/resources/org/apache/camel/dataformat/zipfile?consumer.delay=1000&noop=true")
.split(new ZipSplitter())
.streaming().convertBodyTo(String.class).to("mock:processZipEntry")
.end()
This would be a lot easier to figure out if the documentation wasn't so sparse. First, like someone else mentioned, the docs assume that you'll write your own Processor implementation. A simple one looks like this:
public class ZipEntryProcessor implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
}
If you debug the process method, you'll see that the body of the input message is of type ZipInputStreamWrapper, which extends the Java class BufferedInputStream. That's useful information because it tells you that you could probably use Camel's built-in data transformations so that you don't have to write your own processor.
So here's how you consume a zip file and extract all of its entries to a directory on the file system:
from("file:src/test/resources/org/apache/camel/dataformat/zipfile/")
.split(new ZipSplitter())
.streaming()
.to("file://C:/Temp/")
.end();
It's actually ridiculously simple. Also, you have to make sure you understand the file component URI syntax properly too. That tripped me up the most. Here's an excellent blog post about that topic.
Related
I'm trying to implement a custom endpoint in a Spring Boot application.
Goal is to use routes as: from("...").process("...").to("my:...");
Now, I have 3 classes: a DefaultConsumer, a DefaultEndpoint, a DefaultComponent:
package com.my.endpoint;
import org.apache.camel.Consumer;
import org.apache.camel.Processor;
import org.apache.camel.Producer;
import org.apache.camel.support.DefaultEndpoint;
public class MyEndpoint extends DefaultEndpoint {
public MyEndpoint(String uri, MyComponent myComponent) {
}
...
}
package com.my.endpoint;
import org.apache.camel.Endpoint;
import org.apache.camel.Processor;
import org.apache.camel.support.DefaultConsumer;
public class MyConsumer extends DefaultConsumer {
public MyConsumer(Endpoint endpoint, Processor processor) {
super(endpoint, processor);
}
}
package com.my.endpoint;
import org.apache.camel.CamelContext;
import org.apache.camel.Endpoint;
import org.apache.camel.spi.annotations.Component;
import org.apache.camel.support.DefaultComponent;
import java.util.Map;
#Component("my")
public class MyComponent extends DefaultComponent {
public MyComponent(CamelContext camelContext) {
super(camelContext);
}
...
}
Now: how can I register?
In a Spring configuration class, I have:
#Override
public void configure() throws Exception {
camelContext.addComponent("my", new MyComponent(camelContext));
But is not working:
Caused by: org.apache.camel.NoSuchEndpointException: No endpoint could be found for: my, please check your classpath contains the needed Camel component jar.
So, I added the META-INF file in services/org/apache/camel/component/my:
class=com.my.endpoint.MyComponent
But also this, is not working.
There is no complete tutorial on how to implement this.
Any help?
Note: I'm trying to implement an Endpoint because I need to integrate my systems using my data types. I tried using Transformer but failed because of this: Set a custom DataType in Apache Camel Processor
Before, I tried using data type converter, but failed because of this (marked duplicate because people are too lazy to really understand questions): Enforce type conversion on Rest consumer in Apache Camel
I've FULLY read "Apache Camel In Action, Second Edition" but, at the moment, I can't continue with my project because of?
This is because custom component must be annotated by #UriEndpoint annotation.
Another way to solve this problem: Set EndpointUri via Constructor or by implementing createEndpointUri() in MyEndpoint.
So easiest way might be changing your constructor to:
public MyEndpoint(String uri, MyComponent myComponent) {
super(uri, myComponent);
}
I have a very simple use case in which I want to put together a collection of stings and trying to use aggregator EIP for this. However when trying to start up the route it complains it cannot find an Aggregator Strategy:
java.lang.IllegalArgumentException: AggregationStrategy or AggregationStrategyRef must be set on Aggregate
below is how I can reproduce the issue:
import org.apache.camel.RoutesBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;
import java.util.Arrays;
import java.util.List;
public class AggregatorTest extends CamelTestSupport {
private static final List<String> LIST = Arrays.asList(new String[] {"one", "two", "three"});
#Override
protected RoutesBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("direct:start")
.split().body()
.setHeader("cheese", constant("camembert"))
.aggregate(constant("all"))
.to("mock:end");
}
};
}
#Test
public void shouldAggregateStrings() throws Exception {
sendBody("direct:start", LIST);
}
}
Of course a very easy way to fix this would be to create an Aggregation Strategy implementation and configure my route to use it. However I would like to understand why the way is set up now does not work. According to the camel documentation on here :
By default Camel uses DefaultAggregationCollection and
UseLatestAggregationStrategy, so this simple example will just keep
the latest received exchange for the given correlation Expression
I also noticed DefaultAggregationCollection is no longer part of the camel core. So what I am missing here?
That is the old documentation. The correct documentation is at: http://camel.apache.org/aggregator2
eg the list of EIPs has links to the correct documentation: http://camel.apache.org/eip
You can find examples from those links, and as well in this little example: https://github.com/apache/camel/blob/master/examples/camel-example-aggregate/README.md
And the Camel in Action books has an EIP chapter where the aggregator is covered in much more details as well: http://camel.apache.org/books
I made a small video to demonstrate this EIP pattern using SpringBoot and Camel.
Have a look here: https://youtu.be/IdGuGGVv51Q
I thought I would use the new ResourceBundleControlProvider framework in Java 8 to fix something which Oracle themselves will never fix - the default encoding used when reading resource bundles.
So I made a control:
package com.acme.resources;
import java.io.IOException;
import java.util.Locale;
import java.util.ResourceBundle;
public class AcmeResourceBundleControl extends ResourceBundle.Control
{
#Override
public ResourceBundle newBundle(String baseName, Locale locale, String format,
ClassLoader loader, boolean reload)
throws IllegalAccessException, InstantiationException, IOException
{
throw new UnsupportedOperationException("TODO");
}
}
Then I made a provider:
package com.acme.resources;
import java.util.ResourceBundle;
import java.util.spi.ResourceBundleControlProvider;
public class AcmeResourceBundleControlProvider implements ResourceBundleControlProvider
{
private static final ResourceBundle.Control CONTROL = new AcmeResourceBundleControl();
#Override
public ResourceBundle.Control getControl(String baseName)
{
if (baseName.startsWith("com.acme."))
{
return CONTROL;
}
else
{
return null;
}
}
}
Then in META-INF/services/java.util.spi.ResourceBundleControlProvider:
com.acme.resources.AcmeResourceBundleControlProvider
Then I just tried to run our application from IDEA and I find that it never loads my provider (otherwise the exception would be raised.)
I have checked the names and they all seem to match up. I have checked the compiler output directory IDEA is using and it does contain the service file. I wrote a simple test program which just tries to look up the service:
public static void main(String[] args)
{
for (ResourceBundleControlProvider provider :
ServiceLoader.load(ResourceBundleControlProvider.class))
{
System.out.println(provider.getClass());
}
}
This does print out one entry which is the name of my implementation class. So the issue is not in the service file.
If I breakpoint inside ResourceBundle, I seem to be able to access the custom provider class. Initial forays into the debugger show that ServiceLoader isn't finding any implementations, but I can't figure out why. I'm sure there is some dodgy class loader magic going on which results in not loading my class. :(
Some scary documentation on the Javadoc makes it sound like it might have to be installed as a global extension. If that really is the case, it's a bit of a shame, because it seemed like a useful way to override the default (and in my opinion broken) behaviour. But I also read the tutorial on the matter and it didn't seem to be describing anything like that (unless the good behaviour was pulled out of Java 8 at the very last minute and the docs are out of date!)
The tutorial does state that the JAR containing the ResourceBundleControlProvider must be in the JVM's system extension directory. Section 6 of the tutorial describes the requirement:
java -Djava.ext.dirs=lib -cp build RBCPTest
When you install a Java extension, you typically put the JAR file of the extension in the lib/ext directory of your JRE. However, this command specifies the directory that contains Java extensions with the system property java.ext.dirs.
The JavaDoc for ServiceLoader.loadInstalled() also states that providers on the application's class path are ignored.
Your problem is that the java.util.ResourceBundle that comes with the JVM does a ServiceLoader.loadInstalled(ResourceBundleControlProvider.class) to obtain a list of providers in the static initializer, and uses the thus obtained list ever after.
I have got a corrupted property file from customer. The file is modified by application to update version number. The code uses apache commons configuration. When I tested, the library always seems to write files in iso-8859-1 format.
Code is simplified to below. What is the possibility of following code write bad file?
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.PropertiesConfiguration;
import java.io.FileWriter;
import java.io.IOException;
public class TestConfig {
public void editVersionInfo() throws ConfigurationException, IOException {
String filename = "C:\\temp\\VersionProperties\\Version.properties";
PropertiesConfiguration config = new PropertiesConfiguration(filename);
config.setProperty("application.version", "2011");
config.save(new FileWriter(filename));
}
public static void main(String[] args) throws IOException, ConfigurationException {
TestConfig tc = new TestConfig();
tc.editVersionInfo();
}
}
Just in case - the bad file looks like below. It does not look like in any encoding. The file originally was normal property file with keys and values all in English(ascii chars).
F????Co?aR??m??E?3#?? =
h\u00BD5j\u00B3\u00E0\u0096\u001D\u0081fe\u00BEo\b\u00A3\u0001\u00FE\u00A4\u00DE\u0000\u00FBi\"\u009C{\u00FC\u00D9\u00E2?c\u00F6\u00FF%B\u00A47\u00195\u001EXv\u0097/\u00D7x\u0099\u000E\u00A2gIX\u0014\u0097]k\u00882\u0003\u0014\u0097\u00BC\u00C3\u00AE\u00B4\u001E\u00B3R\u00E4\u00DE&\u0000\u0016\u009B\"7\u0085'\"\u00DCT*v'\u0092\u0007\u0091A\u00BD\u00ACl6~\u0097\u00C0\u00B1\u00D1\u00EB\u00FF\u00A8\u00F3\u0001'\u00BF\u0006\u001F\u009C\fk\u009F\u00C2\u00D9L^_\u0004J4\u00AF\u00D8\u00DAW\u00C4\u00CDj\u00E3\u0095\u00D1+\u00CE?\u0004>Z]\u00D7\u000B\u0098\u0016\u0095\u00AC\u00F7\u00E7\u009ATF\u0019\f)\u00A3\u00A9\u00DC\u00AD\u00ACtq5\u0085\u008E-\u00A3oH\u0000\u00C2\u0092\u00B5\u00F2\u008AG\u008F&\u00F5\u0017H\u0003!\u0083\u00B4\u008AV=\u00E0\u00EDj\u00F0\u00D0J\u00DB\u00CC\u00F2O\u00CE\u00BE\u00F0*4\u0006y~\u00C3\u00B7\"\u000B\u00E4\u00C0$>\u00F3\u00F2~\u00CE\u0097#\u00BAc\u00EC#\u00B4\u00AD\u009A\u00BAX\fF\u0083]\u00C2\u00D4\u00AB\u00F3\u009DQ\u0092\u00854z\u0097\u00FDG\t\u0095\u00E3}ty\u0082I\u00C3`\u009E
??
Edit: The customer environment is japanese. How ever the application is always run with
-Dfile.encoding=UTF8
I suspect your customer has a different default character encoding to what you have. Check their setting of the property file.encoding (counterintuitively named, I know).
An alternative possibility is that you have two threads writing that property file. I don't know, but I suspect the Apache library won't be thread-safe by default.
Edit1
I am not sure if the title is best for the problem so if any have some more orinted title please suggest
i am trying my hands on camel where i have to fetch some csv file from a file system and needs to convert it to xml format and place it on some other system
i am using camel for this and here is my sample POC code
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.Message;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import com.poc.convertor.CSVConverterBean;
public class TestPOC {
public static void main(String args[]) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("file:data/csv?noop=true").unmarshal().csv().bean(new CSVConverterBean(),"processCSVInvoice").to("file:data/csvoutput?fileName=test.xml").marshal("jaxb");
}
});
context.start();
Thread.sleep(1000);
context.stop();
}
}
In this approach camel csv unmarshaller will covert the csv file in to java list List<List<String>>
i have written a java converter CSVConverterBean which will iterate the list and set the values in the respective java objects being generated by jaxb 2.x , final object is being marshaled in to xml and file is being saved.
Everything is being working properly with only one issue , if in future there arises any request to change mapping we need to do the modification in CSVConverterBean and than this java file needs to be recompiled and need to be redistributed which we want to avoid.
my question is, Is there any way by which we can map the values from the java List being given by Camel to the respective java classes being generated by the JaxB so that the need to recompile java code can be avooided.
You can provide a "from-to" kind of configuration file to map the columns of your CSV data with your Bean properties, and code an algorithm to read that file and process the convertion.
You could do with a .properties file:
# mapping.properties
column0=propertyOne
column1=propertyTwo
For each column in your CSV, you get the value from the property file and find which property you should set the value on.
int columnIndex = 0;
for(String column : csvColumns) {
String property = findProperty(columnIndex);
reflectionUtil.setValue(object, property, column);
columnIndex++;
}
This may give you some hint.
Whenever your data changes, you will only need to change the property file, not the class.
Use Bruno's idea, but read the property names from the header row in the csv file.
I have solved this problem using dom4j.camel gave me back csv as list> and firstly i read the headers and than made these headers and the XML tags and the values as there respected values on run time.