I have created an sample app for producer-processor-consumer scenario using Spring Cloud Scenario. Here, I have used legacy annotation based approach.
In unit tests, I wanted to test simple scenario of producing a message and asserting consumed message after it undergoes transformation. But I am not receiving message at consumer binding end. Please let me know what could be missing here.
Producer.java
#EnableBinding(MyProcessor.class)
public class Producer {
#Bean
#InboundChannelAdapter(value = MyProcessor.OUTPUT, poller = #Poller(fixedDelay = "1000", maxMessagesPerPoll = "1"))
public MessageSource<String> produceMessage() {
return () -> new GenericMessage<>("Hello Spring Cloud World >>> " + Instant.now());
}
}
TransformProcessor.java
#EnableBinding(MyProcessor.class)
public class TransformProcessor {
#Transformer(inputChannel = MyProcessor.OUTPUT, outputChannel = MyProcessor.INPUT)
public String transform(String message) {
System.out.println("Transforming the message: " + message);
return message.toUpperCase();
}
}
Consumer.java
#EnableBinding(MyProcessor.class)
public class Consumer {
#StreamListener(MyProcessor.INPUT)
public void consume(String message) {
System.out.println("Consuming transformed message: " + message);
}
}
MyProcessor.java
public interface MyProcessor {
String INPUT = "my-input";
final static String OUTPUT = "my-output";
#Input(INPUT)
SubscribableChannel anInput();
#Output(OUTPUT)
MessageChannel anOutput();
}
SpringCloudStreamLegacyApplicationTests.java
#SpringBootTest
class SpringCloudStreamLegacyApplicationTests {
#Autowired
private MyProcessor myProcessor;
#Autowired
private MessageCollector messageCollector;
#Test
public void testConsumer() {
myProcessor.anOutput().send(new GenericMessage<byte[]>("hello".getBytes()));
Message<?> poll = messageCollector.forChannel(myProcessor.anInput()).poll();
System.out.println("Received: " + poll.getPayload());
}
}
Here, I am expecting a message to be received in handleMessage method.
Note, I am using following dependency for tests:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-test-support</artifactId>
<scope>test</scope>
</dependency>
Related
I have a method for sending kafka message like this:
#Async
public void sendMessage(String topicName, Message message) {
ListenableFuture<SendResult<String, Message >> future = kafkaTemplate.send(topicName, message);
future.addCallback(new ListenableFutureCallback<>() {
#Override
public void onSuccess(SendResult<String, Message > result) {
//do nothing
}
#Override
public void onFailure(Throwable ex) {
log.error("something wrong happened"!);
}
});
}
And now I am writing unit tests for it. I would like to test also the two callback methods onSuccess and onFailure methods, so my I idea is to mock the KafkaTemplate, something like :
KafkaTemplate kafkaTemplate = Mockito.mock(KafkaTemplate.class);
But now I am getting stuck on the mocking result for these two cases:
when(kafkaTemplate.send(anyString(), any(Message.class))).thenReturn(????);
what should I put in the thenReturn method for the case success and for the case failure? Does anyone have an idea please? Thank you very much!
You can mock the template but it's better to mock the interface.
Sender sender = new Sender();
KafkaOperations template = mock(KafkaOperations.class);
SettableListenableFuture<SendResult<String, String>> future = new SettableListenableFuture<>();
when(template.send(anyString(), any(Message.class))).thenReturn(future);
sender.setTemplate(template);
sender.send(...);
future.set(new SendResult<>(...));
...or...
future.setException(...
EDIT
Updated to CompletableFuture (Spring for Apache Kafka 3.0.x and later)...
public class Sender {
private KafkaOperations<String, String> template;
public void setTemplate(KafkaOperations<String, String> template) {
this.template = template;
}
public void send(String topic, Message<?> data) {
CompletableFuture<SendResult<String, String>> future = this.template.send(data);
future.whenComplete((result, ex) -> {
if (ex == null) {
System.out.println(result);
}
else {
System.out.println(ex.getClass().getSimpleName() + "(" + ex.getMessage() + ")");
}
});
}
}
#ExtendWith(OutputCaptureExtension.class)
public class So57475464ApplicationTests {
#Test
public void test(CapturedOutput captureOutput) {
Message message = new GenericMessage<>("foo");
Sender sender = new Sender();
KafkaOperations template = mock(KafkaOperations.class);
CompletableFuture<SendResult<String, String>> future = new CompletableFuture<>();
given(template.send(any(Message.class))).willReturn(future);
sender.setTemplate(template);
sender.send("foo", message);
future.completeExceptionally(new RuntimeException("foo"));
assertThat(captureOutput).contains("RuntimeException(foo)");
}
}
I have Spring boot app
It is receiving Json formatted messages from ActiveMQ
#Component class Receiver {
#JmsListener(destination = "queue")
public void receiveMessage(BusMessage message) {
System.out.println("Received <" + message + ">");
}
The problem, is what sometimes Json can arrive without proper header
In this case i get exception
org.springframework.messaging.converter.MessageConversionException: Cannot convert from [java.lang.String] to [cam.melexis.minipcs2socketioserver.BusMessage] for org.springframework.jms.listener.adapter.AbstractAdaptableMessageListener$MessagingMessageConverterAdapter$LazyResolutionMessage#38bd4ff9, failedMessage=org.springframework.jms.listener.adapter.AbstractAdaptableMessageListener$MessagingMessageConverterAdapter$LazyResolutionMessage#38bd4ff9
One solution can be is to get message as string and deserialise it
#Component class Receiver {
#JmsListener(destination = "queue")
public void receiveScannerMessage(Message message, Session session) {
System.out.println("Received <" + message + ">");
}
But i want to do something more graceful
To simplify all #JmsListener
And to make my learning curve in Spring more curved :)
Knowing, what all messages will arrive in Json format, can i add missing header to all incoming messages?
Or it is another "better" way exist?
This made a job
#EnableJms
#Configuration class JmsListenerConfig implements JmsListenerConfigurer {
Logger logger = LoggerFactory.getLogger(Receiver.class);
#Bean
public DefaultMessageHandlerMethodFactory handlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(messageConverter());
return factory;
}
#Bean
public MessageConverter messageConverter() {
return new MappingJackson2MessageConverter();
}
#Override
public void configureJmsListeners(JmsListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(handlerMethodFactory());
}
}
But i have no idea how
Maybe anybody can help me with this?
I am new to spring boot and am trying to use the sample example from the spring integration in order to subscribe and publish using MQTT. I manage to integrate it with Thingsboard and the logger in the code below is able to receive the published message from Thingsboard.
public static void main(String[] args) {
SpringApplication.run(MqttTest.class);
}
#Bean
public MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory factory = new DefaultMqttPahoClientFactory();
MqttConnectOptions options = new MqttConnectOptions();
options.setServerURIs(new String[] { "URI HERE" });
options.setUserName("ACCESS TOKEN HERE");
factory.setConnectionOptions(options);
return factory;
}
// consumer
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p)
.handle(logger())
.get();
}
private LoggingHandler logger() {
LoggingHandler loggingHandler = new LoggingHandler("INFO");
loggingHandler.setLoggerName("LoggerBot");
return loggingHandler;
}
#Bean
public MessageProducerSupport mqttInbound() {
MqttPahoMessageDrivenChannelAdapter adapter = new MqttPahoMessageDrivenChannelAdapter("Consumer",
mqttClientFactory(), "v1/devices/me/rpc/request/+");
adapter.setCompletionTimeout(5000);
adapter.setConverter(new DefaultPahoMessageConverter());
adapter.setQos(1);
return adapter;
}
This is the console output. I am able to receive the published json message that was sent from the thingsboard dashboard. I am wondering if there is a call method to retrieve the json message string so that I can process it further. Thank you.
2019-02-01 14:06:23.590 INFO 13416 --- [ Call: Consumer] LoggerBot : {"method":"setValue","params":true}
2019-02-01 14:06:24.840 INFO 13416 --- [ Call: Consumer] LoggerBot : {"method":"setValue","params":false}
To handle the published messages, subscribe message handles to the flow to consume the messages.
MessageHandler
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p)
.handle( mess -> {
System.out.println("mess"+mess);
})
.get();
}
ServiceActivator
#Bean
public IntegrationFlow mqttInFlow() {
return IntegrationFlows.from(mqttInbound())
.transform(p -> p)
.handle("myService","handleHere")
.handle(logger())
.get();
}
#Component
public class MyService {
#ServiceActivator
public Object handleHere(#Payload Object mess) {
System.out.println("payload "+mess);
return mess;
}
}
Note: As we discussed, there are lot of different ways of achieving it.
This is just a sample for your understanding.
I'm working on simple chat module for my application using Spring WebFlux with ReactiveMongoRepository on backend and Angular 4 on front. I'm able to receive data through WebSocketSession but after streaming all messages from db i want to keep the connection so i could update message list. Can anyone give me clues how to achieve that, or maybe i'm following wrong assumptions ?
Java Backend responsible for WebSocket, my subscriber only logs current state, nothing relevant there:
WebFluxConfiguration:
#Configuration
#EnableWebFlux
public class WebSocketConfig {
private final WebSocketHandler webSocketHandler;
#Autowired
public WebSocketConfig(WebSocketHandler webSocketHandler) {
this.webSocketHandler = webSocketHandler;
}
#Bean
#Primary
public HandlerMapping webSocketMapping() {
Map<String, Object> map = new HashMap<>();
map.put("/websocket-messages", webSocketHandler);
SimpleUrlHandlerMapping mapping = new SimpleUrlHandlerMapping();
mapping.setOrder(10);
mapping.setUrlMap(map);
return mapping;
}
#Bean
public WebSocketHandlerAdapter handlerAdapter() {
return new WebSocketHandlerAdapter();
}
}
WebSocketHandler Implementation
#Component
public class MessageWebSocketHandler implements WebSocketHandler {
private final MessageRepository messageRepository;
private ObjectMapper mapper = new ObjectMapper();
private MessageSubscriber subscriber = new MessageSubscriber();
#Autowired
public MessageWebSocketHandler(MessageRepository messageRepository) {
this.messageRepository = messageRepository;
}
#Override
public Mono<Void> handle(WebSocketSession session) {
session.receive()
.map(WebSocketMessage::getPayloadAsText)
.map(this::toMessage)
.subscribe(subscriber::onNext, subscriber:: onError, subscriber::onComplete);
return session.send(
messageRepository.findAll()
.map(this::toJSON)
.map(session::textMessage));
}
private String toJSON(Message message) {
try {
return mapper.writeValueAsString(message);
} catch (JsonProcessingException e) {
throw new RuntimeException(e);
}
}
private Message toMessage(String json) {
try {
return mapper.readValue(json, Message.class);
} catch (IOException e) {
throw new RuntimeException("Invalid JSON:" + json, e);
}
}
}
and MongoRepo
#Repository
public interface MessageRepository extends
ReactiveMongoRepository<Message,String> {
}
FrontEnd Handling:
#Injectable()
export class WebSocketService {
private subject: Rx.Subject<MessageEvent>;
constructor() {
}
public connect(url): Rx.Subject<MessageEvent> {
if (!this.subject) {
this.subject = this.create(url);
console.log('Successfully connected: ' + url);
}
return this.subject;
}
private create(url): Rx.Subject<MessageEvent> {
const ws = new WebSocket(url);
const observable = Rx.Observable.create(
(obs: Rx.Observer<MessageEvent>) => {
ws.onmessage = obs.next.bind(obs);
ws.onerror = obs.error.bind(obs);
ws.onclose = obs.complete.bind(obs);
return ws.close.bind(ws);
});
const observer = {
next: (data: Object) => {
if (ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify(data));
}
}
};
return Rx.Subject.create(observer, observable);
}
}
in other service i'm mapping observable from response to my type
constructor(private wsService: WebSocketService) {
this.messages = <Subject<MessageEntity>>this.wsService
.connect('ws://localhost:8081/websocket-messages')
.map((response: MessageEvent): MessageEntity => {
const data = JSON.parse(response.data);
return new MessageEntity(data.id, data.user_id, data.username, data.message, data.links);
});
}
and finally subscribtion with send function which i can't use because of closed connection:
ngOnInit() {
this.messages = [];
this._ws_subscription = this.chatService.messages.subscribe(
(message: MessageEntity) => {
this.messages.push(message);
},
error2 => {
console.log(error2.json());
},
() => {
console.log('Closed');
}
);
}
sendTestMessage() {
this.chatService.messages.next(new MessageEntity(null, '59ca30ac87e77d0f38237739', 'mickl', 'test message angular', null));
}
Assuming your chat messages are being persisted to the datastore as they're being received, you could use the tailable cursors feature in Spring Data MongoDB Reactive (see reference documentation).
So you could create a new method on your repository like:
public interface MessageRepository extends ReactiveSortingRepository< Message, String> {
#Tailable
Flux<Message> findWithTailableCursor();
}
Note that tailable cursors have some limitations: you mongo collection needs to be capped and entries are streamed in their order of insertion.
Spring WebFlux websocket support does not yet support STOMP nor message brokers, but this might be a better choice for such a use case.
The Spring framework support tcp connection as well , i wrote code below to setup a simple socket server , i am confused about adding below futures to my socket server :
authorizing clients based on a unique identifier ( for example a client secret received from client, maybe using TCP Connection Events )
send a message directly to specific client (based on identifier)
broadcast a message
UPDATE :
Config.sendMessage added to send message to single client
Config.broadCast added to broadcast message
authorizeIncomingConnection to authorize clients , accept or reject connections
tcpConnections static filed added to keep tcpEvent sources
Questions !
is using tcpConnections HashMap good idea ?!
is the authorization method i implemented a good one ?!
Main.java
#SpringBootApplication
public class Main {
public static void main(final String[] args) {
SpringApplication.run(Main.class, args);
}
}
Config.java
#EnableIntegration
#IntegrationComponentScan
#Configuration
public class Config implements ApplicationListener<TcpConnectionEvent> {
private static final Logger LOGGER = Logger.getLogger(Config.class.getName());
#Bean
public AbstractServerConnectionFactory AbstractServerConnectionFactory() {
return new TcpNetServerConnectionFactory(8181);
}
#Bean
public TcpInboundGateway TcpInboundGateway(AbstractServerConnectionFactory connectionFactory) {
TcpInboundGateway inGate = new TcpInboundGateway();
inGate.setConnectionFactory(connectionFactory);
inGate.setRequestChannel(getMessageChannel());
return inGate;
}
#Bean
public MessageChannel getMessageChannel() {
return new DirectChannel();
}
#MessageEndpoint
public class Echo {
#Transformer(inputChannel = "getMessageChannel")
public String convert(byte[] bytes) throws Exception {
return new String(bytes);
}
}
private static ConcurrentHashMap<String, TcpConnection> tcpConnections = new ConcurrentHashMap<>();
#Override
public void onApplicationEvent(TcpConnectionEvent tcpEvent) {
TcpConnection source = (TcpConnection) tcpEvent.getSource();
if (tcpEvent instanceof TcpConnectionOpenEvent) {
LOGGER.info("Socket Opened " + source.getConnectionId());
tcpConnections.put(tcpEvent.getConnectionId(), source);
if (!authorizeIncomingConnection(source.getSocketInfo())) {
LOGGER.warn("Socket Rejected " + source.getConnectionId());
source.close();
}
} else if (tcpEvent instanceof TcpConnectionCloseEvent) {
LOGGER.info("Socket Closed " + source.getConnectionId());
tcpConnections.remove(source.getConnectionId());
}
}
private boolean authorizeIncomingConnection(SocketInfo socketInfo) {
//Authorization Logic , Like Ip,Mac Address WhiteList or anyThing else !
return (System.currentTimeMillis() / 1000) % 2 == 0;
}
public static String broadCast(String message) {
Set<String> connectionIds = tcpConnections.keySet();
int successCounter = 0;
int FailureCounter = 0;
for (String connectionId : connectionIds) {
try {
sendMessage(connectionId, message);
successCounter++;
} catch (Exception e) {
FailureCounter++;
}
}
return "BroadCast Result , Success : " + successCounter + " Failure : " + FailureCounter;
}
public static void sendMessage(String connectionId, final String message) throws Exception {
tcpConnections.get(connectionId).send(new Message<String>() {
#Override
public String getPayload() {
return message;
}
#Override
public MessageHeaders getHeaders() {
return null;
}
});
}
}
MainController.java
#Controller
public class MainController {
#RequestMapping("/notify/{connectionId}/{message}")
#ResponseBody
public String home(#PathVariable String connectionId, #PathVariable String message) {
try {
Config.sendMessage(connectionId, message);
return "Client Notified !";
} catch (Exception e) {
return "Failed To Notify Client , cause : \n " + e.toString();
}
}
#RequestMapping("/broadCast/{message}")
#ResponseBody
public String home(#PathVariable String message) {
return Config.broadCast(message);
}
}
Usage :
Socket Request/Response Mode
notify single client
http://localhost:8080/notify/{connectionId}/{message}
broadCast
http://localhost:8080/broadCast/{message}
The TcpConnectionOpenEvent contains a connectionId property. Each message coming from that client will have the same property in the IpHeaders.CONNECTION_ID message header.
Add a custom router that keeps track of the logged-on state of each connection.
Lookup the connection id and if not authenticated, route to a challenge/response subflow.
When authenticated, route to the normal flow.
To use arbitrary messaging (rather than request/response) use a TcpReceivingChannelAdapter and TcpSendingMessageHandler instead of an inbound gateway. Both configured to use the same connection factory. For each message sent to the message handler, add the IpHeaders.CONNECTION_ID header to target the specific client.
To broadcast, send a message for each connection id.