I am trying to get JSON data from jetty endpoint (another service), create output data and send them to one or more CVS files. I have 2 routes - first one creates files for current date, based on cron settings, second one exposes jetty endpoint to create files for any specified date on GET request. They are exactly the same except starting point, I also tried to send messages from second endpoint to first one... In both cases CSV files are created but second route gives me org.apache.camel.TypeConversionException. My route is:
from(httpServer + "/lineups?throwExceptionOnFailure=false?httpMethodRestrict=GET")
.routeId("manualStart")
.setExchangePattern(ExchangePattern.InOnly)
.setHeader(Exchange.HTTP_URI, simple(apiEndpoint + "/lineups"))
.setHeader("target_date", simple("${in.header.date}"))
.setHeader(Exchange.HTTP_QUERY, simple("date=${in.header.date}"))
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.to("https://dummyhost")
.process(new MappingProcessor())
.split(body())
.setHeader("prefix", simple("${body.name}"))
.process(new FileNameProcessor())
.marshal(bindy)
.aggregate(header("prefix"), new FileAggregationStrategy())
.completionTimeout(60000L)
.to("file:" + fileLocation + "?fileName=Nielsen.${in.header.prefix}.${in.header.target_date}.txt");
I get following exception:
16:41:50.493 [qtp1583020257-49] ERROR o.a.c.c.j.CamelContinuationServlet - Error processing request
org.apache.camel.TypeConversionException: Error during type conversion from type: java.lang.String to the required type: java.io.InputStream with value
[com...beans.MyOutput#7c9c5406, com...beans.MyOutput#6e3e3511, com.... [Body clipped after 1000 chars, total length is 23865] due Failed to convert from type [java.util.ArrayList<?>] to type [java.io.InputStream] for value ...
...
com....beans.MyOutput#6f0c8349]';
nested exception is org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [java.util.ArrayList<?>] to type [java.io.InputStream]
at org.apache.camel.impl.converter.BaseTypeConverterRegistry.createTypeConversionException(BaseTypeConverterRegistry.java:610)
at org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(BaseTypeConverterRegistry.java:137)
at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:72)
at org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:47)
at org.apache.camel.http.common.DefaultHttpBinding.doWriteDirectResponse(DefaultHttpBinding.java:396)
at org.apache.camel.http.common.DefaultHttpBinding.doWriteResponse(DefaultHttpBinding.java:332)
at org.apache.camel.http.common.DefaultHttpBinding.writeResponse(DefaultHttpBinding.java:264)
at org.apache.camel.component.jetty.CamelContinuationServlet.service(CamelContinuationServlet.java:227)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:821)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1685)
at org.apache.camel.component.jetty.CamelFilterWrapper.doFilter(CamelFilterWrapper.java:45)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1158)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1090)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:119)
at org.eclipse.jetty.server.Server.handleAsync(Server.java:567)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:325)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:242)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:261)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:75)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:213)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.core.convert.ConversionFailedException: Failed to convert from type [java.util.ArrayList<?>] to type [java.io.InputStream] for value
My data format for CSV formatting:
DataFormat bindy = new BindyCsvDataFormat(MyOutput.class);
This is FileAggregationStrategy:
public class FileAggregationStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
return newExchange;
}
String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
String body = oldBody + newBody;
oldExchange.getIn().setBody(body);
return oldExchange;
}
}
I tried to add .log() after each step and I see that exception is thrown on .aggregate.
What could be wrong ? Another route started from
from("quartz://start/api_cron/?cron=" + cronExpression + "&fireNow=true")
works without any exceptions.
Its the HTTP response that is being attempted to be converted from X to InputStream. You need to set some response to return, either an empty value or something you want to return to the HTTP client.
Even if you set the MEP to InOnly then Jetty will send back a response. You can use wireTap if you want to process and aggregate the message independent of the Jetty route.
Something along the lines of
from jetty
wiretap direct:foo
transform constant "ok"
from direct:foo
// put in all that stuff from your route here
Related
I just realized that choice() in Apache Camel is for dynamic routing, so every choice() needs a to(). It is not equivalent to if in Java.
But, does that mean that I cannot conditionally set header to my camel exchange?
I want to do something like this:
from("direct:eventHttpChoice") // last step returns a composite POJO with config details and the actual message and token, let's call it MyCompositePojo
.log(....) // I see this message in log
.setHeader("Authorization", simple("Bearer ${body.token}"))
.setHeader(Exchange.HTTP_METHOD, simple("${body.httpMethod.name}"))
.choice()
.when(simple("${body.httpMethod.name} in 'PUT,DELETE'"))
.setHeader(Exchange.HTTP_PATH, simple("${body.newEvent.number}"))
.endChoice()
.end()
.choice()
.when(simple("${body.httpMethod.name} in 'POST,PUT'"))
.setHeader(HttpHeaders.CONTENT_TYPE, constant(MediaType.APPLICATION_JSON))
.setBody(simple("${body.newEvent}")).marshal().json(JsonLibrary.Jsonb) // marshall here as toD() needs InputStream; and I believe here it converts my message to MyMessagePojo, the actual payload to send
.endChoice()
.otherwise() // DELETE
.when(simple("${body.configDetail.http.deleteSomeField} == 'true' && ${body.newEvent.someField} != null && ${body.newEvent.someField} != ''"))
.setHeader(Exchange.HTTP_QUERY, simple("someField=${body.newEvent.someField}&operationId=${body.newEvent.operationId}"))
.endChoice()
.otherwise()
.setHeader(Exchange.HTTP_QUERY, simple("operationId=${body.newEvent.operationId}"))
.endChoice()
.endChoice()
.end()
.log(LoggingLevel.INFO, "Sending to this url: ${body.configDetail.url}") // I don't see this log
.toD("${body.configDetail.url}", 10) // only cache at most 10 urls; I still need MyCompositePojo here
But I receive error:
2022-12-14 10:44:49,213 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (Camel (camel-1) thread #6 - JmsConsumer[my.queue]) Failed delivery for (MessageId: A9371D97F55900C-0000000000000001 on ExchangeId: A9371D97F55900C-0000000000000001). Exhausted after delivery attempt: 1 caught: org.apache.camel.language.bean.RuntimeBeanExpressionException: Failed to invoke method: configDetail on null due to: org.apache.camel.component.bean.MethodNotFoundException: Method with name: configDetail not found on bean: [B#330cd22d of type: [B on the exchange: Exchange[A9371D97F55900C-0000000000000001]
MyCompositePojo has this field. But I don't know where I get it wrong.
If you think I am doing marshall() too early, but if not like this, how can I set body? Because without .marshal() I see this error:
2022-12-14 12:25:41,772 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (Camel (camel-1) thread #7 - JmsConsumer[page.large.sm.provisioning.events.online]) Failed delivery for (MessageId: 65FF01C9FC61E66-0000000000000011 on ExchangeId: 65FF01C9FC61E66-0000000000000011). Exhausted after delivery attempt: 1 caught: org.apache.camel.language.bean.RuntimeBeanExpressionException: Failed to invoke method: configDetail on null due to: org.apache.camel.component.bean.MethodNotFoundException: Method with name: configDetail not found on bean: MyPojo{xxx=xxx, ...} of type: com.example.MyPojo on the exchange: Exchange[65FF01C9FC61E66-0000000000000011]
So, it means without .marshal(), it is changing the body to my MessagePojo; but I don't want it, I just need body to be part of my original body, and when it's HTTP DELETE, I don't want to set body. And, later in the route, I still need my composite pojo. I mean, I want to set the HTTP body and only conditionally, I don't want to change the exchange body.
So, how to conditionally set header and send to dynamic URL and set body?
An alternative would be to replace the (Camel) choice logic by a custom (Java) processor.
from("direct:demo")
.process( e -> setDynamicUri(e) );
.toD("${headers.nextUri}");
private void setDynamicUri(Exchange e) {
String httpMethod = e.getMessage().getHeader("...", String.class);
String endpointUrl = ( Arrays.asList("PUT", "DELETE").contains(httpMethod) ? "url1" : "url2" );
e.getMessage().setHeader("nextUri", endpointUrl);
}
I want to write a simple app to save messages in the Iota-Tangle and read them again.
How should I configure the API so that the messages are visible in the mainnet or devnet (Tangle Explorer) after I send them ?
that means, I just want to send a message / transaction, and then see it in the tangle at https://thetangle.org/...
i hope there is someone here who knows about iota-
this Api-config does not work..
IotaAPI api = new IotaAPI.Builder()
.protocol("https")
.host("thetangle.org")
.port(443)
.localPoW(new PearlDiverLocalPoW())
.build();
SendTransferResponse response = api.sendTransfer( ...... );
And I get this Error
Execution of the API call raised exception. IOTA Node not reachable?
com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 1 path $
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1597)
at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1404)
at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:593)
at com.google.gson.stream.JsonReader.peek(JsonReader.java:425)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:206)
at retrofit2.converter.gson.GsonResponseBodyConverter.convert(GsonResponseBodyConverter.java:39)
at retrofit2.converter.gson.GsonResponseBodyConverter.convert(GsonResponseBodyConverter.java:27)
at retrofit2.OkHttpCall.parseResponse(OkHttpCall.java:225)
at retrofit2.OkHttpCall.execute(OkHttpCall.java:188)
at org.iota.jota.connection.HttpConnector.wrapCheckedException(HttpConnector.java:244)
at org.iota.jota.connection.HttpConnector.getTransactionsToApprove(HttpConnector.java:333)
at org.iota.jota.IotaAPICore.getTransactionsToApprove(IotaAPICore.java:449)
at org.iota.jota.IotaAPI.sendTrytes(IotaAPI.java:344)
at org.iota.jota.IotaAPI.sendTransfer(IotaAPI.java:1126)
at com.example.iota_experiment_3.Main.main(Main.java:60)
Exception in thread "main" java.lang.IllegalStateException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 1 path $
at org.iota.jota.connection.HttpConnector.wrapCheckedException(HttpConnector.java:272)
at org.iota.jota.connection.HttpConnector.getTransactionsToApprove(HttpConnector.java:333)
at org.iota.jota.IotaAPICore.getTransactionsToApprove(IotaAPICore.java:449)
at org.iota.jota.IotaAPI.sendTrytes(IotaAPI.java:344)
at org.iota.jota.IotaAPI.sendTransfer(IotaAPI.java:1126)
at com.example.iota_experiment_3.Main.main(Main.java:60)
I am sending a file to the server using MultipartEntityBuilder When the server not respond or any other exception occurs the file should go to the deadLetterChennal endpoint and save as a file.Everything working fine but deadLetterChennal showing conversion issue.
from("direct:sendFileExchange")
.errorHandler(deadLetterChannel("file:/home/r2/Desktop/ofBizFile/errorFinalServer")
.asyncDelayedRedelivery()
.maximumRedeliveries(3)
.redeliveryDelay(1000)
.retryAttemptedLogLevel(LoggingLevel.WARN))
.startupOrder(3)
.process(new ProcessorSetFileRequest())
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader(Exchange.HTTP_QUERY,constant("USERNAME=__&PASSWORD=_"))
.to("https4://someAddress/uploadAndImportFileFromCSVFile?throwExceptionOnFailure=false")
.to("stream:out");
The deadLetter file not coming in directory.
Where I can set the type converter in my route ?
Error I am getting:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot store file: /home/r2/Desktop/ofBizFile/errorFinalServer/orders-02-01-2018.csv
Caused by: [org.apache.camel.NoTypeConversionAvailableException - No type converter available to convert from type:
org.apache.http.entity.mime.MultipartFormEntity to the required type: java.io.InputStream with value org.apache.http.entity.mime.MultipartFormEntity#ddf0916]
You can do something like this
from("direct:sendFileExchange")
.errorHandler(deadLetterChannel("direct:savefile").useOriginalMessage()
.asyncDelayedRedelivery()
.maximumRedeliveries(3)
.redeliveryDelay(1000)
.retryAttemptedLogLevel(LoggingLevel.WARN))
.startupOrder(3)
.process(new ProcessorSetFileRequest())
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader(Exchange.HTTP_QUERY,constant("USERNAME=__&PASSWORD=_"))
.to("https4://someAddress/uploadAndImportFileFromCSVFile?throwExceptionOnFailure=false")
.to("stream:out");
from("direct:savefile").process(exchange -> {
MultipartFormEntity body = (MultipartFormEntity) exchange.getIn().getBody();
exchange.getOut().setBody(body.getContent());
}).to("file:/home/r2/Desktop/ofBizFile/errorFinalServer");
I'm trying to call 2 xsl transforms in my route via recipientList. I have one wrapped in a choice step, and the other added as normal:
from("direct:cosTransform")
.routeId(TransformerConstants.TRANSFORM_XSLT_ROUTE)
.process(exchange -> {
// Get the xml payload from the exchange body
final String xml = exchange.getIn().getBody(String.class);
// Determine LOB and set as header. If the LOB is invalid, the xslt step will fail.
final String lob = TransformerUtil.getLOBFromAcordXml(xml);
if (null == lob) {
throw new GeneralException("Could not derive the LOB from the ACORD Form. ACORD Form["
+ TransformerUtil.getAcordFormFromAcordXml(xml) + "]");
}
exchange.getIn().setHeader("lineOfBusiness", lob);
// Market should be NI or BI. If the market is invalid, the xslt step will fail.
final String market = TransformerUtil.getMarketFromAccordXml(xml);
if (!StringUtils.equals(market, TransformerConstants.BI_MARKET)
&& !StringUtils.equals(market, TransformerConstants.NI_MARKET)) {
throw new GeneralException("Missing or invalid market[" + market + "].");
}
exchange.getIn().setHeader("market", market);
})
.log("Executing an xsl transform for Market=${header.market} and LOB=${header.lineOfBusiness}")
.choice()
.when(header("market").isEqualTo("NI"))
.recipientList(simple("xslt:./xsl/${header.market}/Common.xsl?saxon=true&contentCache=false")).id("commonTransformNI")
.log("after common :${body}")
.endChoice()
.recipientList(simple("xslt:./xsl/${header.market}/${header.lineOfBusiness}.xsl?saxon=true&contentCache=false"))
.log("after lob : ${body}")
.choice().id("postTransform")
.when(header("market").isEqualTo("NI"))
.process(exchange -> {
String xml = exchange.getIn().getBody(String.class);
xml = TransformerUtil.setUniqueIds(xml);
exchange.getIn().setBody(xml);
})
.endChoice()
.end();
}
`
When the exchange with a header of to be transformed hits the first recipientList call(Common.xsl), the xml is transformed and it works fine. When it hits the second call, I get the following in the console:
[ #0 - seda://transform-receive] [CLM] [CID=UNKNOWN] o.a.c.impl.ProcessorEndpoint$1.doStart DEBUG Starting producer: Producer[xslt://./xsl/NI/CGL.xsl?contentCache=false&saxon=true]
[ #0 - seda://transform-receive] [CLM] [CID=UNKNOWN] o.a.camel.impl.ProducerCache .doGetProducer DEBUG Adding to producer cache with key: Endpoint[xslt://./xsl/NI/CGL.xsl?contentCache=false&saxon=true] for producer: Producer[xslt://./xsl/NI/CGL.xsl?contentCache=false&saxon=true]
[ #0 - seda://transform-receive] [CLM] [CID=UNKNOWN] o.a.c.b.xml.XsltUriResolver .resolve DEBUG Resolving URI from classpath:: classpath:./xsl/NI/CGL.xsl
[ #0 - seda://transform-receive] [CLM] [CID=UNKNOWN] o.a.c.p.DefaultErrorHandler .log DEBUG Failed delivery for (MessageId: ID-LIBP03P-QK70A9V-60563-1487254928941-1-7 on ExchangeId: ID-LIBP03P-QK70A9V-60563-1487254928941-1-8). On delivery attempt: 0 caught: java.lang.NullPointerException
[ #0 - seda://transform-receive] [CLM] [CID=UNKNOWN] TRANSFORM.COR.XSLT.ROUTE .log ERROR null
[ #0 - seda://transform-receive] [CLM] [CID=UNKNOWN] o.a.c.processor.SendProcessor .process DEBUG >>>> Endpoint[log://showException=true] Exchange[ID-LIBP03P-QK70A9V-60563-1487254928941-1-8]
I've tried a few tests, adding in a mock endpoint after the second call and checking for a message, using log messages to track how far along the route the exchange gets to and it never seems to make it to the second call, it always throws an exception when heading to the second xslt step. Even when I send in an exchange that won't match the when condition, it doesn't hit the second xslt step either. The paths in the xsl strings are correct, when i remove the choice logic and second xslt step, the xsl runs fine, whether the string is for the common.xsl file or a line of business xsl file
Note: Due to certain templates in the xsl, I have to split the transforms out into their own steps, i.e. i can't import one of the files in, so the transforms have to be called separately
EDIT: I have implemented a processor to run the xslt manually for now. I'd still like to get to the bottom of this though, even when using a .to call and hardcoding the parameters I still get the same issue. Is it something around the XSLT component in camel?
Application.yaml:
request.delay.ms: 200
A spring component:
#Value("${request.delay.ms}")
private long requestDelay;
Now, as strange as it sounds, it looks like spring can successfully resolve the property if it's value is a string, i.e. if I change 200 to 200a. It does throw an error
Caused by: java.lang.NumberFormatException: For input string: "200a" but that's expected.
If I change the value back to 200 I get
Caused by: org.springframework.beans.TypeMismatchException: Failed to convert value of type 'java.lang.String' to required type 'long'; nested exception is java.lang.NumberFormatException: For input string: "${request.delay.ms}"
And if I run tomcat in debug mode and step through the spring code, it works fine with no errors.
String #Value(s) are resolved fine in the same component all the time.
If you are using YAML instead of .properties you should write it in the following manner:
request:
delay:
ms: 200
which will be transformed to properties:
request.delay.ms=200 but you are mixing both files
See more here