I am trying to send messages via rabbitmq to an axon4 spring boot based system. The message is received but no events are triggered. I am very sure I am missing an essential part, but up to now I wasn't able to figure it out.
Here the relevant part of my application.yml
axon:
amqp:
exchange: axon.fanout
transaction-mode: publisher_ack
# adding the following lines changed nothing
eventhandling:
processors:
amqpEvents:
source: in.queue
mode: subscribing
spring:
rabbitmq:
username: rabbit
password: rabbit
From the docs I found that I am supposed to create a SpringAMQPMessageSource bean:
import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;
import org.axonframework.extensions.amqp.eventhandling.AMQPMessageConverter;
import org.axonframework.extensions.amqp.eventhandling.spring.SpringAMQPMessageSource;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
#Slf4j
#Configuration
public class AxonConfig {
#Bean
SpringAMQPMessageSource inputMessageSource(final AMQPMessageConverter messageConverter) {
return new SpringAMQPMessageSource(messageConverter) {
#RabbitListener(queues = "in.queue")
#Override
public void onMessage(final Message message, final Channel channel) {
log.debug("received external message: {}, channel: {}", message, channel);
super.onMessage(message, channel);
}
};
}
}
If I send a message to the queue from the rabbitmq admin panel I see the log:
AxonConfig : received external message: (Body:'[B#13f7aeef(byte[167])' MessageProperties [headers={}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=, receivedRoutingKey=in.queue, deliveryTag=2, consumerTag=amq.ctag-xi34jwHHA__xjENSteX5Dw, consumerQueue=in.queue]), channel: Cached Rabbit Channel: AMQChannel(amqp://rabbit#127.0.0.1:5672/,1), conn: Proxy#11703cc8 Shared Rabbit Connection: SimpleConnection#581cb879 [delegate=amqp://rabbit#127.0.0.1:5672/, localPort= 58614]
Here the Aggregate that should receive the events:
import lombok.extern.slf4j.Slf4j;
import org.axonframework.commandhandling.CommandHandler;
import org.axonframework.config.ProcessingGroup;
import org.axonframework.eventsourcing.EventSourcingHandler;
import org.axonframework.modelling.command.AggregateIdentifier;
import org.axonframework.spring.stereotype.Aggregate;
import pm.mbo.easyway.api.app.order.commands.ConfirmOrderCommand;
import pm.mbo.easyway.api.app.order.commands.PlaceOrderCommand;
import pm.mbo.easyway.api.app.order.commands.ShipOrderCommand;
import pm.mbo.easyway.api.app.order.events.OrderConfirmedEvent;
import pm.mbo.easyway.api.app.order.events.OrderPlacedEvent;
import pm.mbo.easyway.api.app.order.events.OrderShippedEvent;
import static org.axonframework.modelling.command.AggregateLifecycle.apply;
#ProcessingGroup("amqpEvents")
#Slf4j
#Aggregate
public class OrderAggregate {
#AggregateIdentifier
private String orderId;
private boolean orderConfirmed;
#CommandHandler
public OrderAggregate(final PlaceOrderCommand command) {
log.debug("command: {}", command);
apply(new OrderPlacedEvent(command.getOrderId(), command.getProduct()));
}
#CommandHandler
public void handle(final ConfirmOrderCommand command) {
log.debug("command: {}", command);
apply(new OrderConfirmedEvent(orderId));
}
#CommandHandler
public void handle(final ShipOrderCommand command) {
log.debug("command: {}", command);
if (!orderConfirmed) {
throw new IllegalStateException("Cannot ship an order which has not been confirmed yet.");
}
apply(new OrderShippedEvent(orderId));
}
#EventSourcingHandler
public void on(final OrderPlacedEvent event) {
log.debug("event: {}", event);
this.orderId = event.getOrderId();
orderConfirmed = false;
}
#EventSourcingHandler
public void on(final OrderConfirmedEvent event) {
log.debug("event: {}", event);
orderConfirmed = true;
}
#EventSourcingHandler
public void on(final OrderShippedEvent event) {
log.debug("event: {}", event);
orderConfirmed = true;
}
protected OrderAggregate() {
}
}
So the problem is that the messages are received by the system but no events are triggered. The content of the messages seem to be irrelevant. Whatever I send to the queue I only get a log message from my onMessage method.
JavaDoc of SpringAMQPMessageSource says this:
/**
* MessageListener implementation that deserializes incoming messages and forwards them to one or more event processors.
* <p>
* The SpringAMQPMessageSource must be registered with a Spring MessageListenerContainer and forwards each message
* to all subscribed processors.
* <p>
* Note that the Processors must be subscribed before the MessageListenerContainer is started. Otherwise, messages will
* be consumed from the AMQP Queue without any processor processing them.
*
* #author Allard Buijze
* #since 3.0
*/
But up to now I couldn't find out where or how to register it.
The axon.eventhandling entries in my config and #ProcessingGroup("amqpEvents") in my Aggregate are already from testing. But having those entries in or not made no difference at all. Also tried without the mode=subscribing.
Exact versions: Spring Boot 2.1.4, Axon 4.1.1, axon-amqp-spring-boot-autoconfigure 4.1
Any help or hints highly appreciated.
Update 23.04.19:
I tried to write my own class like this:
import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;
import org.axonframework.common.Registration;
import org.axonframework.eventhandling.EventMessage;
import org.axonframework.extensions.amqp.eventhandling.AMQPMessageConverter;
import org.axonframework.messaging.SubscribableMessageSource;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.amqp.rabbit.listener.api.ChannelAwareMessageListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.function.Consumer;
#Slf4j
#Component
public class RabbitMQSpringAMQPMessageSource implements ChannelAwareMessageListener, SubscribableMessageSource<EventMessage<?>> {
private final List<Consumer<List<? extends EventMessage<?>>>> eventProcessors = new CopyOnWriteArrayList<>();
private final AMQPMessageConverter messageConverter;
#Autowired
public RabbitMQSpringAMQPMessageSource(final AMQPMessageConverter messageConverter) {
this.messageConverter = messageConverter;
}
#Override
public Registration subscribe(final Consumer<List<? extends EventMessage<?>>> messageProcessor) {
eventProcessors.add(messageProcessor);
log.debug("subscribe to: {}", messageProcessor);
return () -> eventProcessors.remove(messageProcessor);
}
#RabbitListener(queues = "${application.queues.in}")
#Override
public void onMessage(final Message message, final Channel channel) {
log.debug("received external message: {}, channel: {}", message, channel);
log.debug("eventProcessors: {}", eventProcessors);
if (!eventProcessors.isEmpty()) {
messageConverter.readAMQPMessage(message.getBody(), message.getMessageProperties().getHeaders())
.ifPresent(event -> eventProcessors.forEach(
ep -> ep.accept(Collections.singletonList(event))
));
}
}
}
The result is the same and the log now proofs that the eventProcessors are just empty.
eventProcessors: []
So the question is, how to register the event processors correctly. Is there a way how to do that properly with spring?
Update2:
Also no luck with this:
#Slf4j
#Component("rabbitMQSpringAMQPMessageSource")
public class RabbitMQSpringAMQPMessageSource extends SpringAMQPMessageSource {
#Autowired
public RabbitMQSpringAMQPMessageSource(final AMQPMessageConverter messageConverter) {
super(messageConverter);
}
#RabbitListener(queues = "${application.queues.in}")
#Override
public void onMessage(final Message message, final Channel channel) {
try {
final var eventProcessorsField = this.getClass().getSuperclass().getDeclaredField("eventProcessors");
eventProcessorsField.setAccessible(true);
final var eventProcessors = (List<Consumer<List<? extends EventMessage<?>>>>) eventProcessorsField.get(this);
log.debug("eventProcessors: {}", eventProcessors);
} catch (NoSuchFieldException | IllegalAccessException e) {
e.printStackTrace();
}
log.debug("received message: message={}, channel={}", message, channel);
super.onMessage(message, channel);
}
}
axon:
eventhandling:
processors:
amqpEvents:
source: rabbitMQSpringAMQPMessageSource
mode: SUBSCRIBING
Registering it programmatically in addition to above also didn't help:
#Autowired
void configure(EventProcessingModule epm,
RabbitMQSpringAMQPMessageSource rabbitMessageSource) {
epm.registerSubscribingEventProcessor("rabbitMQSpringAMQPMessageSource", c -> rabbitMessageSource);
epm.assignProcessingGroup("amqpEvents", "rabbitMQSpringAMQPMessageSource");// this line also made no difference
}
Of course #ProcessingGroup("amqpEvents") is in place in my class that contains the #EventSourcingHandler annotated methods.
Update 25.4.19:
see accepted answer from Allard. Thanks a lot pointing me at the mistake I made: I missed that EventSourcingHandler don't receive messages from outside. This is for projections. Not for distributing Aggregates! ups
Here the config/classes that are receiving events from rabbitmq now:
axon:
eventhandling:
processors:
amqpEvents:
source: rabbitMQSpringAMQPMessageSource
mode: SUBSCRIBING
import com.rabbitmq.client.Channel;
import lombok.extern.slf4j.Slf4j;
import org.axonframework.extensions.amqp.eventhandling.AMQPMessageConverter;
import org.axonframework.extensions.amqp.eventhandling.spring.SpringAMQPMessageSource;
import org.springframework.amqp.core.Message;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
#Slf4j
#Component("rabbitMQSpringAMQPMessageSource")
public class RabbitMQSpringAMQPMessageSource extends SpringAMQPMessageSource {
#Autowired
public RabbitMQSpringAMQPMessageSource(final AMQPMessageConverter messageConverter) {
super(messageConverter);
}
#RabbitListener(queues = "${application.queues.in}")
#Override
public void onMessage(final Message message, final Channel channel) {
log.debug("received message: message={}, channel={}", message, channel);
super.onMessage(message, channel);
}
}
import lombok.extern.slf4j.Slf4j;
import org.axonframework.config.ProcessingGroup;
import org.axonframework.eventhandling.EventHandler;
import org.axonframework.queryhandling.QueryHandler;
import org.springframework.stereotype.Service;
import pm.mbo.easyway.api.app.order.events.OrderConfirmedEvent;
import pm.mbo.easyway.api.app.order.events.OrderPlacedEvent;
import pm.mbo.easyway.api.app.order.events.OrderShippedEvent;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
#Slf4j
#ProcessingGroup("amqpEvents")
#Service
public class OrderedProductsEventHandler {
private final Map<String, OrderedProduct> orderedProducts = new HashMap<>();
#EventHandler
public void on(OrderPlacedEvent event) {
log.debug("event: {}", event);
String orderId = event.getOrderId();
orderedProducts.put(orderId, new OrderedProduct(orderId, event.getProduct()));
}
#EventHandler
public void on(OrderConfirmedEvent event) {
log.debug("event: {}", event);
orderedProducts.computeIfPresent(event.getOrderId(), (orderId, orderedProduct) -> {
orderedProduct.setOrderConfirmed();
return orderedProduct;
});
}
#EventHandler
public void on(OrderShippedEvent event) {
log.debug("event: {}", event);
orderedProducts.computeIfPresent(event.getOrderId(), (orderId, orderedProduct) -> {
orderedProduct.setOrderShipped();
return orderedProduct;
});
}
#QueryHandler
public List<OrderedProduct> handle(FindAllOrderedProductsQuery query) {
log.debug("query: {}", query);
return new ArrayList<>(orderedProducts.values());
}
}
I removed the #ProcessingGroup from my Aggregate of course.
My logs:
RabbitMQSpringAMQPMessageSource : received message: ...
OrderedProductsEventHandler : event: OrderShippedEvent...
In Axon, Aggregates do not receive events from "outside". The Event Handlers inside Aggregates (more specifically, they are EventSourcingHandlers) only handle events that have been published by that same aggregate instance, so that it can reconstruct its prior state.
It is only external event handlers, for example the ones that update projections, that will receive events from external sources.
For that to work, your application.yml should mention the bean name as a processors' source instead of the queue name. So in your first example:
eventhandling:
processors:
amqpEvents:
source: in.queue
mode: subscribing
Should become:
eventhandling:
processors:
amqpEvents:
source: inputMessageSource
mode: subscribing
But again, this only works for event handlers defined on components, not on Aggregates.
Related
I have tried to implement a TCP server socket with spring integration in an allready existing spring boot application, but I am facing a problem and this problem drives me crazy...
The client is sending a message (a byte array) to the server and timesout. That's it.
I am not receiving any exceptions from the server. It seems I have provided the wrong port or somthing but after checking the port, I am sure it is the right one.
This is my annotation based configuration class:
import home.brew.server.socket.ServerSocketHandler;
import lombok.extern.log4j.Log4j2;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.ip.dsl.Tcp;
#Log4j2
#Configuration
#EnableIntegration
public class TcpServerSocketConfiguration {
#Value("${socket.port}")
private int serverSocketPort;
#Bean
public IntegrationFlow server(ServerSocketHandler serverSocketHandler) {
TcpServerConnectionFactorySpec connectionFactory =
Tcp.netServer(socketPort)
.deserializer(new CustomSerializerDeserializer())
.serializer(new CustomSerializerDeserializer())
.soTcpNoDelay(true);
TcpInboundGatewaySpec inboundGateway =
Tcp.inboundGateway(connectionFactory);
return IntegrationFlows
.from(inboundGateway)
.handle(serverSocketHandler::handleMessage)
.get();
}
#Bean
public ServerSocketHandler serverSocketHandler() {
return new ServerSocketHandler();
}
}
I wanted to make the receive functionality work before I try to send an answer, so that's why have a minimal configuration.
And the following class should process the received message from the server socket
import lombok.extern.log4j.Log4j2;
import org.springframework.messaging.Message;
import org.springframework.messaging.MessageHeaders;
import org.springframework.messaging.MessagingException;
#Log4j2
public class ServerSocketHandler {
public String handleMessage(Message<?> message, MessageHeaders messageHeaders) {
log.info(message.getPayload());
// TODO implement something useful to process the incoming message here...
return message.getPayload().toString();
}
}
The handler method from above was never invoked even once!
I have googled for some example implementations or tutorials but I haven't found anyhing what worked for me.
I allready tried the implementations of these sites:
https://vispud.blogspot.com/2019/03/how-to-implement-simple-echo-socket.html
https://docs.spring.io/spring-integration/docs/current/reference/html/ip.html#note-nio
Spring Boot TCP Client
and a bunch of sites more... but nothing helped me :-(
UPDATE 1
I have implemented a custom serializer/deserializer:
import lombok.Data;
import lombok.extern.log4j.Log4j2;
import org.springframework.core.serializer.Deserializer;
import org.springframework.core.serializer.Serializer;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
#Log4j2
#Data
public class CustomSerializerDeserializer implements Serializer<byte[]>,
Deserializer<byte[]> {
#Override
public byte[] deserialize(InputStream inputStream) throws IOException {
return inputStream.readAllBytes();
}
#Override
public void serialize(byte[] object, OutputStream outputStream) throws IOException {
outputStream.write(object);
}
}
After the client have sent a message, the custom serializer is invoked but the content ist always empty. I have no idea why.... The serializer needs a lot of time to read all bytes from the stream and in the end it is empty. The procedure is repeating all the time, so I think I have build an infinty loop by accident...
UPDATE 2
I have captured the communication between Client and server socket:
It looks like I am stuck in the handshake and therefore there is no payload...
So if anybody could help me out with this, I would be very thankful and if you need some more information, just let me know.
Thanks in advance!
Well, after a few days of analysing and coding, I found the best solution for me to handle TCP socket communications using spring integration. For other developers who are struggling with the same problems. Here is what I've done so far.
This class contains a - for me working - annotation based TCP socket connection configuration
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.IntegrationComponentScan;
import org.springframework.integration.annotation.MessagingGateway;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.ip.IpHeaders;
import org.springframework.integration.ip.tcp.TcpInboundGateway;
import org.springframework.integration.ip.tcp.TcpOutboundGateway;
import org.springframework.integration.ip.tcp.connection.AbstractClientConnectionFactory;
import org.springframework.integration.ip.tcp.connection.AbstractServerConnectionFactory;
import org.springframework.integration.ip.tcp.connection.TcpNetClientConnectionFactory;
import org.springframework.integration.ip.tcp.connection.TcpNetServerConnectionFactory;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.handler.annotation.Header;
import org.springframework.web.context.request.RequestContextListener;
/**
* Spring annotation based configuration
*/
#Configuration
#EnableIntegration
#IntegrationComponentScan
public class TcpServerSocketConfiguration {
public static final CustomSerializerDeserializer SERIALIZER = new CustomSerializerDeserializer();
#Value("${socket.port}")
private int socketPort;
/**
* Reply messages are routed to the connection only if the reply contains the ip_connectionId header
* that was inserted into the original message by the connection factory.
*/
#MessagingGateway(defaultRequestChannel = "toTcp")
public interface Gateway {
void send(String message, #Header(IpHeaders.CONNECTION_ID) String connectionId);
}
#Bean
public MessageChannel fromTcp() {
return new DirectChannel();
}
#Bean
public MessageChannel toTcp() {
return new DirectChannel();
}
#Bean
public AbstractServerConnectionFactory serverCF() {
TcpNetServerConnectionFactory serverCf = new TcpNetServerConnectionFactory(socketPort);
serverCf.setSerializer(SERIALIZER);
serverCf.setDeserializer(SERIALIZER);
serverCf.setSoTcpNoDelay(true);
serverCf.setSoKeepAlive(true);
// serverCf.setSingleUse(true);
// final int soTimeout = 5000;
// serverCf.setSoTimeout(soTimeout);
return serverCf;
}
#Bean
public AbstractClientConnectionFactory clientCF() {
TcpNetClientConnectionFactory clientCf = new TcpNetClientConnectionFactory("localhost", socketPort);
clientCf.setSerializer(SERIALIZER);
clientCf.setDeserializer(SERIALIZER);
clientCf.setSoTcpNoDelay(true);
clientCf.setSoKeepAlive(true);
// clientCf.setSingleUse(true);
// final int soTimeout = 5000;
// clientCf.setSoTimeout(soTimeout);
return clientCf;
}
#Bean
public TcpInboundGateway tcpInGate() {
TcpInboundGateway inGate = new TcpInboundGateway();
inGate.setConnectionFactory(serverCF());
inGate.setRequestChannel(fromTcp());
inGate.setReplyChannel(toTcp());
return inGate;
}
#Bean
public TcpOutboundGateway tcpOutGate() {
TcpOutboundGateway outGate = new TcpOutboundGateway();
outGate.setConnectionFactory(clientCF());
outGate.setReplyChannel(toTcp());
return outGate;
}
This class contains a custom serializer and deserialiser
import lombok.extern.log4j.Log4j2;
import org.jetbrains.annotations.NotNull;
import org.springframework.core.serializer.Deserializer;
import org.springframework.core.serializer.Serializer;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
/**
* A custom serializer for incoming and/or outcoming messages.
*/
#Log4j2
public class CustomSerializerDeserializer implements Serializer<byte[]>, Deserializer<byte[]> {
#NotNull
#Override
public byte[] deserialize(InputStream inputStream) throws IOException {
byte[] message = new byte[0];
if (inputStream.available() > 0) {
message = inputStream.readAllBytes();
}
log.debug("Deserialized message {}", new String(message, StandardCharsets.UTF_8));
return message;
}
#Override
public void serialize(#NotNull byte[] message, OutputStream outputStream) throws IOException {
log.info("Serializing {}", new String(message, StandardCharsets.UTF_8));
outputStream.write(message);
outputStream.flush();
}
}
In the following classes you can implement some buisness logic to process incoming ...
import lombok.extern.log4j.Log4j2;
import org.springframework.integration.annotation.MessageEndpoint;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.stereotype.Component;
#Log4j2
#Component
#MessageEndpoint
public class ClientSocketHandler {
#ServiceActivator(inputChannel = "toTcp")
public byte[] handleMessage(byte[] msg) {
// TODO implement some buisiness logic here
return msg;
}
}
and outgoing messages.
import lombok.extern.log4j.Log4j2;
import org.springframework.integration.annotation.MessageEndpoint;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.stereotype.Component;
#Log4j2
#Component
#MessageEndpoint
public class ClientSocketHandler {
#ServiceActivator(inputChannel = "toTcp")
public byte[] handleMessage(byte[] msg) {
// implement some business logic here
return msg;
}
}
Hope it helps. ;-)
How are you communicating with this server? By default the connection factory is configured to require the input to be terminated by CRLF (e.g. Telnet). You have to configure a different deserializer if your client uses something else to indicate a message end.
Also, your method signature is incorrect; it should be:
public String handleMessage(byte[] message, MessageHeaders messageHeaders) {
String string = new String(message);
System.out.println(string);
return string.toUpperCase();
}
This works fine for me with Telnet:
$ telnet localhost 1234
Trying ::1...
Connected to localhost.
Escape character is '^]'.
foo
FOO
^]
telnet> quit
Connection closed.
And here is a version that works with just LF (e.g. netcat):
#Bean
public IntegrationFlow server(ServerSocketHandler serverSocketHandler) {
return IntegrationFlows.from(Tcp.inboundGateway(
Tcp.netServer(1234)
.deserializer(TcpCodecs.lf())
.serializer(TcpCodecs.lf())))
.handle(serverSocketHandler::handleMessage)
.get();
}
$ nc localhost 1234
foo
FOO
^C
Overview:
I'm totally new to Elastic search testing and I'm gonna add proper unit tests. The project compatibilities are as follow:
Java 8
Elasticsearch 6.2.4
Project uses low level rest client for fetching data from ES
More info about ES configurations is as follow:
import static java.net.InetAddress.getByName;
import static java.util.Arrays.stream;
import java.net.UnknownHostException;
import java.util.Map;
import java.util.Objects;
import javax.inject.Inject;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import au.com.api.util.RestClientUtil;
import lombok.extern.slf4j.Slf4j;
#Slf4j
#Configuration
public class ElasticConfiguration implements InitializingBean{
#Value(value = "${elasticsearch.hosts}")
private String[] hosts;
#Value(value = "${elasticsearch.httpPort}")
private int httpPort;
#Value(value = "${elasticsearch.tcpPort}")
private int tcpPort;
#Value(value = "${elasticsearch.clusterName}")
private String clusterName;
#Inject
private RestClientUtil client;
#Bean
public RestHighLevelClient restHighClient() {
return new RestHighLevelClient(RestClient.builder(httpHosts()));
}
#Bean
#Deprecated
public RestClient restClient() {
return RestClient.builder(httpHosts()).build();
}
/**
* #return TransportClient
* #throws UnknownHostException
*/
#SuppressWarnings("resource")
#Bean
public TransportClient transportClient() throws UnknownHostException{
Settings settings = Settings.builder()
.put("cluster.name", clusterName).build();
return new PreBuiltTransportClient(settings).addTransportAddresses(transportAddresses());
}
#Override
public void afterPropertiesSet() throws Exception {
log.debug("loading search templates...");
try {
for (Map.Entry<String, String> entry : Constants.SEARCH_TEMPLATE_MAP.entrySet()) {
client.putInlineSearchTemplateToElasticsearch(entry.getKey(), entry.getValue());
}
} catch (Exception e) {
log.error("Exception has occurred in putting search templates into ES.", e);
}
}
private HttpHost[] httpHosts() {
return stream(hosts).map(h -> new HttpHost(h, httpPort, "http")).toArray(HttpHost[]::new);
}
private TransportAddress[] transportAddresses() throws UnknownHostException {
TransportAddress[] transportAddresses = stream(hosts).map(h -> {
try {
return new TransportAddress(getByName(h), tcpPort);
} catch (UnknownHostException e) {
log.error("Exception has occurred in creating ES TransportAddress. host: '{}', tcpPort: '{}'", h, tcpPort, e);
}
return null;
}).filter(Objects::nonNull).toArray(TransportAddress[]::new);
if (transportAddresses.length == 0) {
throw new UnknownHostException();
}
return transportAddresses;
}
}
Issue:
I don't know how to Mock ES or how to test ES without running an standalone ES on my machine. Please use the following class as an example and let me know how could I write a testcase (unit test not integration) for getSearchResponse method:
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.transport.NoNodeAvailableException;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.script.ScriptType;
import org.elasticsearch.script.mustache.SearchTemplateRequestBuilder;
import org.elasticsearch.search.Scroll;
import org.elasticsearch.search.aggregations.Aggregation;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.MessageSource;
import org.springframework.stereotype.Repository;
#Slf4j
#Repository
#NoArgsConstructor
public abstract class NewBaseElasticsearchRepository {
#Autowired
protected NewIndexLocator newIndexLocator;
#Value(value = "${elasticsearch.client.timeout}")
private Long timeout;
#Autowired
protected TransportClient transportClient;
#Autowired
protected ThresholdService thresholdService;
#Autowired
protected MessageSource messageSource;
/**
* #param script the name of the script to be executed
* #param templateParams a map of the parameters to be sent to the script
* #param indexName the index to target (an empty indexName will search all indexes)
*
* #return a Search Response object containing details of the request results from Elasticsearch
*
* #throws NoNodeAvailableException thrown when the transport client cannot connect to any ES Nodes (or Coordinators)
* #throws Exception thrown for all other request errors such as parsing and non-connectivity related issues
*/
protected SearchResponse getSearchResponse(String script, Map<String, Object> templateParams, String... indexName) {
log.debug("transport client >> index name --> {}", Arrays.toString(indexName));
SearchResponse searchResponse;
try {
searchResponse = new SearchTemplateRequestBuilder(transportClient)
.setScript(script)
.setScriptType(ScriptType.STORED)
.setScriptParams(templateParams)
.setRequest(new SearchRequest(indexName))
.execute()
.actionGet(timeout)
.getResponse();
} catch (NoNodeAvailableException e) {
log.error(ELASTIC_SEARCH_EXCEPTION_NOT_FOUND, e.getMessage());
throw new ElasticSearchException(ELASTIC_SEARCH_EXCEPTION_NOT_FOUND);
} catch (Exception e) {
log.error(ELASTIC_SEARCH_EXCEPTION, e.getMessage());
throw new ElasticSearchException(ELASTIC_SEARCH_EXCEPTION);
}
log.debug("searchResponse ==> {}", searchResponse);
return searchResponse;
}
So, I would be grateful if you could have a look on the example class and share your genuine solutions with me here about how could I mock TransportClient and get a proper response from SearchResponse object.
Note:
I tried to use ESTestCase from org.elasticsearch.test:framework:6.2.4 but faced jar hell issue and could't resolve it. In the meantime, I could't find any proper docs related to that or Java ES unit testing, in general.
I am using Akka websockets to push data to some client.
This is what I have done so far:
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.util.concurrent.CompletionStage;
import java.util.concurrent.TimeUnit;
import akka.NotUsed;
import akka.actor.ActorSystem;
import akka.http.javadsl.ConnectHttp;
import akka.http.javadsl.Http;
import akka.http.javadsl.ServerBinding;
import akka.http.javadsl.model.HttpRequest;
import akka.http.javadsl.model.HttpResponse;
import akka.http.javadsl.model.ws.Message;
import akka.http.javadsl.model.ws.WebSocket;
import akka.japi.Function;
import akka.stream.ActorMaterializer;
import akka.stream.Materializer;
import akka.stream.javadsl.Flow;
import akka.stream.javadsl.Sink;
import akka.stream.javadsl.Source;
public class Server {
public static HttpResponse handleRequest(HttpRequest request) {
System.out.println("Handling request to " + request.getUri());
if (request.getUri().path().equals("/greeter")) {
final Flow<Message, Message, NotUsed> greeterFlow = greeterHello();
return WebSocket.handleWebSocketRequestWith(request, greeterFlow);
} else {
return HttpResponse.create().withStatus(404);
}
}
public static void main(String[] args) throws Exception {
ActorSystem system = ActorSystem.create();
try {
final Materializer materializer = ActorMaterializer.create(system);
final Function<HttpRequest, HttpResponse> handler = request -> handleRequest(request);
CompletionStage<ServerBinding> serverBindingFuture = Http.get(system).bindAndHandleSync(handler,
ConnectHttp.toHost("localhost", 8080), materializer);
// will throw if binding fails
serverBindingFuture.toCompletableFuture().get(1, TimeUnit.SECONDS);
System.out.println("Press ENTER to stop.");
new BufferedReader(new InputStreamReader(System.in)).readLine();
} finally {
system.terminate();
}
}
public static Flow<Message, Message, NotUsed> greeterHello() {
return Flow.fromSinkAndSource(Sink.ignore(),
Source.single(new akka.http.scaladsl.model.ws.TextMessage.Strict("Hello!")));
}
}
At the client side, I am successfully receiving a 'Hello!' message.
However, now I want to send data dynamically (preferably from an Actor), something like this:
import akka.actor.ActorRef;
import akka.actor.UntypedActor;
public class PushActor extends UntypedActor {
#Override
public void onReceive(Object message) {
if (message instanceof String) {
String statusChangeMessage = (String) message;
// How to push this message to a socket ??
} else {
System.out.println(String.format("'%s':\nReceived unknown message '%s'!", selfActorPath, message));
}
}
}
I am unable to find any example regarding this online.
The following is the software stack being used:
Java 1.8
akka-http 10.0.10
One - not necessarily very elegant - way of doing this is to use Source.actorRef and send the materialized actor somewhere (maybe a router actor?) depending on your requirements.
public static Flow<Message, Message, NotUsed> greeterHello() {
return Flow.fromSinkAndSourceMat(Sink.ignore(),
Source.actorRef(100, OverflowStrategy.fail()),
Keep.right()).mapMaterializedValue( /* send your actorRef to a router? */);
}
Whoever receives the actorRefs of the connected clients must be responsible for routing messages to them.
Main class for Subscriber: Application.java
package com.mynamespace;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.ComponentScan;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import akka.actor.Props;
import akka.contrib.pattern.DistributedPubSubExtension;
import akka.contrib.pattern.DistributedPubSubMediator;
import com.mynamespace.actors.SubscriberActor;
#SpringBootApplication
#ComponentScan(basePackages = "com.mynamespace.*")
public class Application {
public static void main(String[] args) throws InterruptedException {
ApplicationContext ctx = SpringApplication.run(Application.class, args);
// get hold of the actor system
ActorSystem system = ctx.getBean(ActorSystem.class);
ActorRef mediator = DistributedPubSubExtension.get(system).mediator();
ActorRef subscriber = system.actorOf(
Props.create(SubscriberActor.class), "subscriber");
// subscribe to the topic named "content"
mediator.tell(new DistributedPubSubMediator.Put(subscriber), subscriber);
// subscriber.tell("init", null);
System.out.println("Running.");
Thread.sleep(5000l);
}
}
Subscriber actor: SubscriberActor.java
package com.mynamespace.actors;
import java.util.ArrayList;
import java.util.List;
import akka.actor.UntypedActor;
import com.mynamespace.message.CategoryServiceRequest;
import com.mynamespace.message.CategoryServiceResponse;
public class SubscriberActor extends UntypedActor {
#Override
public void onReceive(Object msg) throws Exception {
if (msg instanceof CategoryServiceRequest) {
System.out.println("Request received for GetCategories.");
CategoryServiceResponse response = new CategoryServiceResponse();
List<String> categories = new ArrayList<>();
categories.add("Food");
categories.add("Fruits");
response.setCatgories(categories);
getSender().tell(response, getSelf());
} else if (msg instanceof String && msg.equals("init")) {
System.out.println("init called");
} else {
System.out
.println("Unhandelled message received for getCategories.");
}
}
}
Application.conf for subscriber
akka {
loglevel = INFO
stdout-loglevel = INFO
loggers = ["akka.event.slf4j.Slf4jLogger"]
extensions = ["akka.contrib.pattern.DistributedPubSubExtension"]
actor {
provider = "akka.cluster.ClusterActorRefProvider"
}
remote {
enabled-transports = ["akka.remote.netty.tcp"]
netty.tcp {
hostname = "127.0.0.1"
port = 0
}
}
cluster {
seed-nodes = [
"akka.tcp://mynamespace-actor-system#127.0.0.1:2551",
"akka.tcp://mynamespace-actor-system#127.0.0.1:2552"]
auto-down-unreachable-after = 10s
}
}
Main class for publisher: Application.java
package com.mynamespace;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.ComponentScan;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import akka.actor.Props;
import akka.contrib.pattern.DistributedPubSubExtension;
import akka.contrib.pattern.DistributedPubSubMediator;
import com.mynamespace.actors.PublisherActor;
#SpringBootApplication
#ComponentScan(basePackages = "com.mynamespace.*")
public class Application {
public static void main(String[] args) throws InterruptedException {
ApplicationContext ctx = SpringApplication.run(Application.class, args);
// get hold of the actor system
ActorSystem system = ctx.getBean(ActorSystem.class);
ActorRef mediator = DistributedPubSubExtension.get(system).mediator();
ActorRef publisher = system.actorOf(Props.create(PublisherActor.class),
"publisher");
mediator.tell(new DistributedPubSubMediator.Put(publisher), publisher);
Thread.sleep(5000);
publisher.tell("hi", publisher);
System.out.println("Running.");
}
}
PublisherActor.java
package com.mynamespace.actors;
import scala.concurrent.Future;
import akka.actor.ActorRef;
import akka.actor.UntypedActor;
import akka.contrib.pattern.DistributedPubSubExtension;
import akka.contrib.pattern.DistributedPubSubMediator;
import akka.dispatch.Mapper;
import akka.pattern.Patterns;
import akka.util.Timeout;
import com.mynamespace.message.CategoryServiceRequest;
import com.mynamespace.message.CategoryServiceResponse;
public class PublisherActor extends UntypedActor {
// activate the extension
ActorRef mediator = DistributedPubSubExtension.get(getContext().system())
.mediator();
public void onReceive(Object msg) {
if (msg instanceof String) {
Timeout timeOut = new Timeout(50000l);
mediator.tell(new DistributedPubSubMediator.Send(
"/user/subscriber", new CategoryServiceRequest()),
getSelf());
Future<Object> response = Patterns.ask(mediator,
new DistributedPubSubMediator.Send("/user/subscriber",
new CategoryServiceRequest()), timeOut);
Future<CategoryServiceResponse> finalresponse = response.map(
new Mapper<Object, CategoryServiceResponse>() {
#Override
public CategoryServiceResponse apply(Object parameter) {
CategoryServiceResponse responseFromRemote = (CategoryServiceResponse) parameter;
System.out.println("received:: list of size:: "
+ responseFromRemote.getCatgories().size());
return responseFromRemote;
}
}, getContext().system().dispatcher());
} else if (msg instanceof DistributedPubSubMediator.SubscribeAck) {
System.out.println("subscribbed.......");
} else {
unhandled(msg);
}
}
}
Application conf for publisher is same as of subscriber. Both are running on different ports on the same system.
I have two seed nodes defined and running on my local system. Somehow I am not able to ASK/TELL subscriber from producer (both running on different nodes) via DistributedPubSub Mediator.
After running Subscriber then publisher: I don't get any exceptions or any dead letter references printed in stdout/logs.
Is it possible to be able to view what actor references my mediator holds?
Need help to find issues or possible issues.
I had the same problem, after the comments from #spam and my own experiments the thing I can recommend is to use Publish/Subscribe with groups and sendOneMessageToEachGroup=true.
Is it supposed that the Send only works locally? if so the documentation doesn't explicit that. But I can also tell by the code there that this specific part of the documentation has been overlooked apparently (as change the Class names but then don't invoke those, invoke the previous ones on the previous examples)
Hope this helps anyone that has this issue, as the docs are a bit misleading apparently
I am trying to use Akka future with play framework to connect to a remote akka system
. After running the system the akka future gives me a warning that one argument is left.
the code are below :
this is the [lay controller code:
p
ackage controllers;
import com.typesafe.config.ConfigFactory;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import akka.actor.Props;
import play.*;
import play.mvc.*;
import views.html.*;
public class Application extends Controller {
public static Result index() throws InterruptedException {
System.out.println(" Local Node Called0");
ActorSystem csystem = ActorSystem.create("Application", ConfigFactory.load().getConfig("LocalNode"));
ActorRef localNode = csystem.actorOf(new Props(LocalNode.class));
System.out.println(" Local Node Called1");
localNode.tell("Hello");
System.out.println(" Local Node Called2");
Thread.sleep(5000);
csystem.shutdown();
return ok(index.render("I am OK"));
}
}
this is the play framework local actor node
package controllers;
import akka.actor.;
import akka.dispatch.Await;
import akka.dispatch.Future;
import akka.event.Logging;
import akka.event.LoggingAdapter;
import akka.util.Duration;
import akka.util.Timeout;
import akka.pattern.;
public class LocalNode extends UntypedActor {
LoggingAdapter log = Logging.getLogger(getContext().system(), this);
Timeout timeout = new Timeout(Duration.parse("20 seconds"));
ActorRef masterActor;
public void preStart()
{
/* Get reference to Master Node*/
masterActor =
getContext().actorFor("akka://MasterNode#127.0.0.1:2552/user/masterActor");
}
#Override
public void onReceive(Object message) throws Exception {
System.out.println(" Future called ");
Future<Object> future = Patterns.ask(masterActor , message.toString(), timeout);
String result = (String) Await.result(future, timeout.duration());
log.info("Messagefrom Server", result.toString());
}
}
this is the remote akka system master nide
package Rubine_Cluster;
import com.typesafe.config.ConfigFactory;
import akka.actor.ActorRef;
import akka.actor.ActorSystem;
import akka.actor.Props;
import akka.actor.UntypedActor;
import akka.kernel.Bootable;
/**
* Hello world!
*
*/
public class MasterNode implements Bootable
{
final ActorSystem system;
public MasterNode() {
system = ActorSystem.create("MasterNode", ConfigFactory.load()
.getConfig("masterNode"));
ActorRef masterActor = system.actorOf(new Props(MasterActor.class),"masterActor");
System.out.println(" Master Node is called ");
}
public void startup() {
}
public void shutdown() {
system.shutdown();
}
}
this is the akka remote system MasterActor
package Rubine_Cluster;
import akka.actor.*;
public class MasterActor extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof String) {
// Get reference to the message sender and reply back
getSender().tell(message + " got something");
}
}
}
this is the message from the play framewrok
[INFO] [09/30/2012 16:47:25.669] [Application-akka.actor.default-dispatcher-1] [
akka://Application/user/$a] Messagefrom Server WARNING arguments left: 1
help is needed cos I am suppose to meet the assignment deadline
thanks to all
The "arguments left: 1" warning is due to the following line:
log.info("Messagefrom Server", result.toString())
You're passing unused params to the logger. It should be this instead:
log.info("Messagefrom Server {}", result.toString())
Try to give a name to your actor in Play:
ActorRef localNode = csystem.actorOf(new Props(LocalNode.class), "localNode");
And also note that if you need to access the Akka system from Play, it is better to use:
ActorSystem csystem = Akka.system();
instead of:
ActorSystem csystem = ActorSystem.create("Application", ConfigFactory.load().getConfig("LocalNode"));