I am consuming data from REST endpoints (order of 1000+) which all have the same structure:
<server uri>/v1/source/<some ID>
I am using RouteBuilder components like this connecting to the individual endpoint <ID>:
#Component
public class Route_to_<ID> extends RouteBuilder {
#Override
public void configure() throws Exception {
from("timer:mytimer?repeatCount=1") //
.setBody(simple("${null}")) //
.setHeader(Exchange.CONTENT_TYPE, simple("text/event-stream"))
.setHeader("CamelHttpMethod", simple("GET"))
.to(
<server uri>/v1/source/<ID>
+ _deviceName + "::" + _deviceProperty //
+ "?disableStreamCache=true" //
) //
.process(data -> {
... do same stuff for all endpoints ...
});
}
}
The corresponding SpringBootApplicationlooks like this:
#SpringBootApplication
#ComponentScan(basePackages = "my.package.where.components.reside")
public class MyRouteHandler {
}
Is there an elegant way to start all the individual routes to endpoints <ID>in one go using one single SpringBootApplication? Or does every route need an individual SpringBootApplication which is to be started individually?
You could use toD with dynamic uri that gets the version, source and id from message body, headers or exchange properties. You can also use property-placeholders to define host, port and other configs.
Since the REST endpoints all use the same structure you can change the version, source and id and use the same URI for most if not all the REST API calls.
Example:
public class ExampleTest extends CamelTestSupport {
static final String API_DYNAMIC_URI = "https://{{api.uri}}:{{api.port}}/{{api.version}}"
+ "/${exchangeProperty.source}/${exchangeProperty.id}"
+ "?disableStreamCache=true";
#Test
public void exampleTest() throws Exception {
context.adviceWith(context.getRouteDefinition("exampleRoute"),
new AdviceWithRouteBuilder(){
#Override
public void configure() throws Exception {
weaveById("apiEndpoint")
.replace()
.toD("mock:${exchangeProperty.source}/${exchangeProperty.id}")
.setBody().simple("Source: ${exchangeProperty.source} id: ${exchangeProperty.id}");
}
});
Map<String, Object> body1 = new HashMap<>();
body1.put("source", "source1");
body1.put("id", "A");
Map<String, Object> body2 = new HashMap<>();
body2.put("source", "source2");
body2.put("id", "B");
MockEndpoint source1MockEndpoint = getMockEndpoint("mock:source1/A");
source1MockEndpoint.expectedMessageCount(1);
MockEndpoint source2MockEndpoint = getMockEndpoint("mock:source2/B");
source2MockEndpoint.expectedMessageCount(2);
startCamelContext();
template.sendBody("direct:start", body1);
template.sendBody("direct:start", body2);
template.sendBody("direct:start", body2);
source1MockEndpoint.assertIsSatisfied();
source2MockEndpoint.assertIsSatisfied();
}
#Override
protected RoutesBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("direct:start")
.routeId("exampleRoute")
.setProperty("source").simple("${body['source']}")
.setProperty("id").simple("${body['id']}")
.toD(API_DYNAMIC_URI).id("apiEndpoint")
.log("Received: ${body}");
}
};
}
#Override
protected Properties useOverridePropertiesWithPropertiesComponent() {
Properties properties = new Properties();
properties.put("api.uri", "localhost");
properties.put("api.port", "3000");
properties.put("api.version", "v1");
return properties;
}
#Override
public boolean isUseAdviceWith() {
return true;
}
}
Related
I wrote my first spring integration application which uses the mqtt broker to subscribe messages from different topics which are coming from a device. The device is publishing the messages and the client(Code) is accessing those messages using same topics.
I added a handler for accessing the messages coming from the broker and use it further in classes. Now, in my case, I want to have different handlers for different topics so that they can all be mapped to different VO classes and use it further in business logic.
As I know, I want to create only one connection to the broker, one channel but different topics can come and they should be handled in different handlers for the same connection. How Can I achieve that?
#SpringBootApplication
public class MqttJavaApplication {
public static void main(String[] args) {
// SpringApplicationBuilder springApplicationBuilder = new SpringApplicationBuilder(MqttJavaApplication.class);
SpringApplication.run(MqttJavaApplication.class,args);
}
#Bean
public MessageChannel mqttInputChannel() {
return new DirectChannel();
}
#Bean
public MqttPahoMessageDrivenChannelAdapter inbound() {
String clientId = "uuid-" + UUID.randomUUID().toString();
MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter("tcp://localhost:1883", clientId,"camera/status");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
// adapter.setOutputChannelName("mqttInputChannel");
adapter.setOutputChannel(mqttInputChannel());
return adapter;
}
#Bean
public IntegrationFlow mqttInFlow() {
System.out.println(Arrays.stream(SubScribeMessages.class.getMethods()).findFirst());
return IntegrationFlows.from(inbound())
.transform(p -> p)
.handle("addTopics","handlHere")
.get();
}
#Component
public class MyService{
#Autowired
MqttPahoMessageDrivenChannelAdapter adapter;
#Bean
public String addTopics()
{
if(adapter.getTopic().length>0)
{
adapter.addTopic("camera/+/counts"); //new topic
adapter.addTopic("camera/+/live_counts"); //new topic
}
return "";
}
// topic "camera/+/counts" is handled here but other messages also come here, how do we handle other topics in separate handlers?
#ServiceActivator(inputChannel = "mqttInputChannel")
public void handleHere(#Payload Object mess) throws JsonProcessingException {
String[] topics = adapter.getTopic();
for(String topic:topics)
System.out.println(topic); // How can I get topic name which is using a wildcard?
ObjectMapper objectMapper = new ObjectMapper();
String json=mess.toString();
System.out.println(json);
CountVo countVo = objectMapper.readValue(json, CountVo.class);
if (!countVo.equals(null))
System.out.println(countVo.getIrisysCounts().get(0).getName());
}
}
}
Additional Question
How Can I get the full topic name when using a wildcard? The actual topic which was published but caught by wildcard.
Please help.
Add a router (.route(...)); you can route on the MqttHeaders.RECEIVED_TOPIC header (which contains the topic name) to different flows for each topic.
https://docs.spring.io/spring-integration/docs/current/reference/html/message-routing.html#messaging-routing-chapter
EDIT
The simplest router is to simply map the topic names to channel names. Here is an example:
#SpringBootApplication
public class So67391175Application {
public static void main(String[] args) {
SpringApplication.run(So67391175Application.class, args);
}
#Bean
public DefaultMqttPahoClientFactory pahoClientFactory() {
DefaultMqttPahoClientFactory pahoClientFactory = new DefaultMqttPahoClientFactory();
MqttConnectOptions connectionOptions = new MqttConnectOptions();
connectionOptions.setServerURIs(new String[] { "tcp://localhost:1883" });
pahoClientFactory.setConnectionOptions(connectionOptions);
return pahoClientFactory;
}
#Bean
public IntegrationFlow mqttInFlow(DefaultMqttPahoClientFactory pahoClientFactory) {
return IntegrationFlows.from(
new MqttPahoMessageDrivenChannelAdapter("testClient",
pahoClientFactory, "topic1", "topic2"))
.route("headers['" + MqttHeaders.RECEIVED_TOPIC + "']")
.get();
}
#Bean
public IntegrationFlow flow1() {
return IntegrationFlows.from("topic1")
.handle((payload, headers) -> {
System.out.println("message from topic1 " + payload + ": " + headers);
return null;
})
.get();
}
#Bean
public IntegrationFlow flow2() {
return IntegrationFlows.from("topic2")
.handle((payload, headers) -> {
System.out.println("message from topic2 " + payload + ": " + headers);
return null;
})
.get();
}
}
message from topic1 test: {mqtt_receivedRetained=false, mqtt_id=1, mqtt_duplicate=false, id=1d950bce-aa47-7e3b-1a0d-e4d01ed707de, mqtt_receivedTopic=topic1, mqtt_receivedQos=1, timestamp=1620250633090}
message from topic2 test: {mqtt_receivedRetained=false, mqtt_id=2, mqtt_duplicate=false, id=7e9c3f51-c148-2b18-3588-ed27e93dae19, mqtt_receivedTopic=topic2, mqtt_receivedQos=1, timestamp=1620250644602}
Thanks Gary! I think the answer you gave on routing can only take the defined topics, not on wildcards or for that matter any other regex. I could not understand how dynamic routing would help me.
Turns out, I can add wildcards when initializing bean and can handle using the service activator on the inputchannel using the adapter.
Like this:
#SpringBootApplication
public class MqttJavaApplication {
public static void main(String[] args) {
SpringApplication.run(MqttJavaApplication.class,args);
}
#Bean
public MessageChannel mqttInputChannel() {
return new DirectChannel();
}
#Bean
public MqttPahoMessageDrivenChannelAdapter inbound() {
String clientId = "uuid-" + UUID.randomUUID().toString();
MqttPahoMessageDrivenChannelAdapter adapter =
new MqttPahoMessageDrivenChannelAdapter("tcp://localhost:1883", clientId, "irisys/V4D-20230143/status" );
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
adapter.setOutputChannel(mqttInputChannel());
return adapter;
}
#Component
public class MyService{
#Autowired
MqttPahoMessageDrivenChannelAdapter adapter;
#Bean
public String addTopics()
{
if(adapter.getTopic().length>0)
{
adapter.addTopic("camera/+/counts");
}
return "";
}
#Bean
#ServiceActivator(inputChannel = "mqttInputChannel")
public MessageHandler handler() {
return new MessageHandler() {
#SneakyThrows
#Override
public void handleMessage(Message<?> message) throws MessagingException {
System.out.println(message.getPayload());
System.out.println(message.getHeaders());
if(message.getHeaders().get("mqtt_receivedTopic").toString().contains("counts"))
{
ObjectMapper objectMapper = new ObjectMapper();
String json=message.getPayload().toString();
System.out.println(json);
CountVo countVo = objectMapper.readValue(json, CountVo.class);
if (!countVo.equals(null))
System.out.println(countVo.getIrisysCounts().get(0).getName());
}
}
};
}
}
}
Do you think that there is better way than this? I couldn't think of anything other than this.
I have the following code base:
#Component
public class DummyRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
rest("/upload").post().to("file://rest_files");
}
#Bean
public ServletRegistrationBean servletRegistrationBean() {
SpringServerServlet serverServlet = new SpringServerServlet();
ServletRegistrationBean regBean = new ServletRegistrationBean( serverServlet, "/rest/*");
Map<String,String> params = new HashMap<>();
params.put("org.restlet.component", "restletComponent");
regBean.setInitParameters(params);
return regBean;
}
#Bean
public org.restlet.Component restletComponent() {
return new org.restlet.Component();
}
#Bean
public RestletComponent restletComponentService() {
return new RestletComponent(restletComponent());
}
}
I upload file using postman:
It is actually usual csv.
But when I open file my application stored - I see file with following content:
Obvious that file contains full request information.
How can I save only file without other data from http request?
P.S.
I tried to register callback:
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange);
final MultipartFile mandatoryBody = exchange.getIn().getBody(MultipartFile.class);
but it returns null
I'm attempting to use a http proxy with Camel's http4 component. The proxy works when testing using Intellij's HTTP Proxy "Check Connection" option.
However I don't know how to configure it correctly via Camel. When running the following integration test a "ConnectException: Connection timed out" is thrown. Can anyone clarify how to set the proxy details correctly please?
public class SimpleHttpProxyIT extends CamelTestSupport {
public static final String DIRECT_START = "direct:start";
public static final String MOCK_RESULT = "mock:result";
#Produce(uri = DIRECT_START)
protected ProducerTemplate basic;
#EndpointInject(uri = MOCK_RESULT)
protected MockEndpoint resultEndpoint;
#Test
public void testBasic() throws Exception {
basic.sendBody(null);
resultEndpoint.setExpectedMessageCount(1);
resultEndpoint.assertIsSatisfied();
}
#Override
public RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from(DIRECT_START)
.id("SunriseTest")
.log(LoggingLevel.INFO, "About to hit sunrise")
.setHeader(Exchange.HTTP_URI, simple("http://api.sunrise-sunset.org/json?lat=36.7201600&lng=-4.4203400"))
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getProperties().put("http.proxyAuthHost", "myproxy.company.org");
exchange.getProperties().put("http.proxyAuthPort", "10000");
exchange.getProperties().put("http.proxyAuthMethod", "Basic");
exchange.getProperties().put("http.proxyAuthUsername", "myusername");
exchange.getProperties().put("http.proxyAuthPassword", "mypassword");
}
})
.recipientList(simple("http4:dummyhost"))
.log(LoggingLevel.INFO, "Done")
.to(MOCK_RESULT);
}
};
}
}
I would think it should should be exchange.setProperty(...)
Setting the properties in the URI worked. I misread the documentation on "Using Proxy Settings Outside of the URI" (http://camel.apache.org/http4.html) as this was referring to setting them on the context, not the exchange.
public class SimpleHttpProxyIT extends CamelTestSupport {
public static final String DIRECT_START = "direct:start";
public static final String MOCK_RESULT = "mock:result";
#Produce(uri = DIRECT_START)
protected ProducerTemplate basic;
#EndpointInject(uri = MOCK_RESULT)
protected MockEndpoint resultEndpoint;
#Test
public void testBasic() throws Exception {
basic.sendBody(null);
resultEndpoint.setExpectedMessageCount(1);
resultEndpoint.assertIsSatisfied();
}
#Override
public RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from(DIRECT_START)
.id("SunriseTest")
.log(LoggingLevel.INFO, "About to hit sunrise")
.setHeader(Exchange.HTTP_URI, simple("http://api.sunrise-sunset.org/json?lat=36.7201600&lng=-4.4203400"))
.recipientList(simple("http4:dummyhost?proxyAuthHost=myproxy.company.org&proxyAuthPort=10000&proxyAuthUsername=myusername&proxyAuthPassword=mypassword"))
.log(LoggingLevel.INFO, "Done")
.to(MOCK_RESULT);
}
};
}
}
In javax websockets we can use something like the follows
Session.getAsyncRemote().sendText(String text)
Session.getBasicRemote().sendText();
How can we send an asynchronous messages using spring websocket.
From WebSocketSession of spring webscockets can we extract RemoteEndPoint and send an async messages
PS Note: I am using Basic Spring websockets...
The configuration and code is as follows:
#Configuration
#EnableWebMvc
#EnableAspectJAutoProxy
#EnableWebSocket
public class WebMVCConfig extends WebMvcConfigurerAdapter implements WebSocketConfigurer {
private static final String ENDPOINT_URL = "/echo";
#Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(socketHandler(), ENDPOINT_URL).setAllowedOrigins("*");
}
#Bean
public WebSocketHandler socketHandler() {
return new WebSocketTestHandler();
}
#Override
public void configureDefaultServletHandling(DefaultServletHandlerConfigurer configurer) {
configurer.enable();
}
#Bean
public DefaultHandshakeHandler handshakeHandler() {
WebSocketPolicy policy = new WebSocketPolicy(WebSocketBehavior.SERVER);
policy.setInputBufferSize(8192);
policy.setIdleTimeout(600000);
return new DefaultHandshakeHandler(new JettyRequestUpgradeStrategy(new WebSocketServerFactory(policy)));
}
public class SpringMVCInitializer extends AbstractAnnotationConfigDispatcherServletInitializer {
#Override
protected Class<?>[] getRootConfigClasses() {
return new Class[] { ApplicationConfig.class, RabbitMQConfig.class, RabbitConnectionFactory.class,
WebPropertyPlaceHolderConfig.class};
}
#Override
protected Class<?>[] getServletConfigClasses() {
return null;
}
#Override
protected String[] getServletMappings() {
return new String[] { "/" };
}
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
super.onStartup(servletContext);
}
#Configuration
public class WebSocketTestHandler extends TextWebSocketHandler {
#Override
public void afterConnectionEstablished(WebSocketSession session) throws Exception {
log.info("Connection is established to Server....:: Session Open : {}", session.isOpen());
}
#Override
public void handleTextMessage(WebSocketSession session, TextMessage message) {
}
#Override
public void afterConnectionClosed(WebSocketSession curSession, CloseStatus status) throws Exception {
}
}
So inside handleTextMessage(WebSocketSession session,TextMessage message) {
Inside this method am creating multiple threads And sending same session Object and some other parameters..Inside each thread am not modifying any session object related parameters but am trying to execute
TextMessage socketMessage = new TextMessage(message);
session.sendMessage(socketMessage);
}
So each thread is trying to send messages using same session Object..But am facing the following error
java.lang.IllegalStateException: Blocking message pending 10000 for BLOCKING
at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.lockMsg(WebSocketRemoteEndpoint.java:130) ~[websocket-common-9.3.8.v20160314.jar:9.3.8.v20160314]
at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.sendString(WebSocketRemoteEndpoint.java:379) ~[websocket-common-9.3.8.v20160314.jar:9.3.8.v20160314]
at org.springframework.web.socket.adapter.jetty.JettyWebSocketSession.sendTextMessage(JettyWebSocketSession.java:188) ~[spring-websocket-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.socket.adapter.AbstractWebSocketSession.sendMessage(AbstractWebSocketSession.java:105) ~[spring-websocket-4.2.4.RELEASE.jar:4.2.4.RELEASE]
So is it possible to send asynchronous messages using spring websockets?
If yes please let me know what configuration changes are required in the above code..Or Can we extract the core AsyncRemoteEndPoint and BasicRemoteEndpoint from spring Websocket Session and can we send asynchronous messages..or if not both the above cases ..move the code to common place and put synchonized(sessionObject)
{
sendmessage
}..Sorry if the framing of question is not clear or already a duplicate question
Please note I am not using any Stomp client or anyother features over spring websocket..Am using plain spring websockets..And is it possible to do without using Future(java feature)(If yes..it would be better)?
I used ConcurrentWebSocketSessionDecorator on the session.
according to:
https://jira.spring.io/browse/SPR-13602
The decorator "enforces sending messages one at a time with a send buffer and send time limit per session. That helps quite a bit to limit the impact of slow clients"
I have got problem to define custom error message when login failed, so now when my login fail i get http 400 with payload:
{"error":"invalid_grant","error_description":"Bad credentials"}
How to customize this message, and return own json?
I am using spring boot (1.3.2.RELEASE) and spring security OAuth2 (2.0.8.RELEASE).
First off, create a new exception that extends the Oauth2Exception. For example, we have a CustomOauthException like following:
#JsonSerialize(using = CustomOauthExceptionSerializer.class)
public class CustomOauthException extends OAuth2Exception {
public CustomOauthException(String msg) {
super(msg);
}
}
Here we're gonna use CustomOauthExceptionSerializer for serializing CustomOauthExceptions to JSON strings:
public class CustomOauthExceptionSerializer extends StdSerializer<CustomOauthException> {
public CustomOauthExceptionSerializer() {
super(CustomOauthException.class);
}
#Override
public void serialize(CustomOauthException value, JsonGenerator gen, SerializerProvider provider) throws IOException {
gen.writeStartObject();
gen.writeStringField("custom_error", value.getOAuth2ErrorCode());
gen.writeStringField("custom_error_description", value.getMessage());
if (value.getAdditionalInformation()!=null) {
for (Map.Entry<String, String> entry : value.getAdditionalInformation().entrySet()) {
String key = entry.getKey();
String add = entry.getValue();
gen.writeStringField(key, add);
}
}
gen.writeEndObject();
}
}
Finally we need to register a WebResponseExceptionTranslator in our AuthorizationServerConfigurerAdapter to translate spring security's Oauth2Exceptions to our CustomOauthExceptions. Here we are:
#Configuration
#EnableAuthorizationServer
public class AuthorizationServerConfig extends AuthorizationServerConfigurerAdapter {
#Override
public void configure(AuthorizationServerEndpointsConfigurer endpoints) throws Exception {
endpoints
// other endpoints
.exceptionTranslator(e -> {
if (e instanceof OAuth2Exception) {
OAuth2Exception oAuth2Exception = (OAuth2Exception) e;
return ResponseEntity
.status(oAuth2Exception.getHttpErrorCode())
.body(new CustomOauthException(oAuth2Exception.getMessage()));
} else {
throw e;
}
});
}
// rest of the authorization server config
}
After all these, you would see the customized JSON response:
{"custom_error":"invalid_grant", "custom_error_description":"Bad credentials"}