I am coming from Sidekiq and am now moving to a Java solution for distributed jobs. I came across RabbitMQ and ActiveMQ, but it seems those brokers use plaintext or raw byte[] messages. I was wondering if it's possible to send custom messages using these frameworks?
Ideally I would just define a Java class for each specific message type and use that in both worker and producer. Is such a thing possible? Or should I look at other types of middleware?
MyOwnMessageFormat message = new MyOwnMessageFormat(content)
channel.send(message)
Message message = channel.receive()
if (message.class == MyOwnMessageFormat)
{
doSomething();
}
Exchanging messages through message brokers in Java is much easier when done through a service bus like Camel. You dont loose the flexibility of configuring your broker endpoint and still your code is isolated from the particular transport used or the message format. E.g. you can deploy ActiveMQ and later switch to RabbitMQ without having to update your code - just the service bus configuration. Or you can switch from plain Java serialization to JSON when sending messages to the broker by adding a message transformer. Again your business layer does not have to be modified.
Here's a sample that uses POJO producing/consuming where the producer calls a regular Java interface and the consumer implements the interface. The sample assumes that the sender/receiver are instantiated with Spring in order for the Camel endpoints to be injected
Message sender:
interface MyService {
MyResult addTask(MyTask task);
}
class Sender {
#Produce(uri="activemq:queue:myservice")
MyService service;
public void run() {
MyTask task = new MyTask();
MyResult result = service.addTask(task);
}
Message receiver:
class Receiver {
#Consume(uri="activemq:queue:myservice")
public MyResult addTask(MyTask task) {
return new MyResult();
}
}
MyTask & MyResult need to be serializable.
I think learning the Camel framework is not very hard while it can be very rewarding.
In JMS,as long as your custom message object is Serializable, you can use an ObjectMessage,Stream Message or Map Message according to your requirement like below to send(Object Message):
MessageProducer producer = session.createProducer( destination );
ObjectMessage message = session.createObjectMessage( getMyObject() );
producer.send( message );
For receiving:
Message message = consumer.receive();
if (message instanceof ObjectMessage) {
Object object = ((ObjectMessage) message).getObject();
Hope this helps you!
Yes, Before publishing the message to RabbitMQ, convert your message to a Json String.
Now on the receiving end, when the message is received, Parse it to convert into the same format.
In ruby this can be done with
while sending => Message.to_json
on Receiving => message = JSON.parse(received_msg)
To get the class of data, you can send a variable specifying the class of the data along with the data.
This is what worked for me with the latest version of ActiveMQ
...
MessageConsumer consumer=session.createConsumer(destination);
while(true) {
javax.jms.Message message=consumer.receive();
ActiveMQObjectMessage queueMessage=(ActiveMQObjectMessage)message;
Object payload=queueMessage.getObject();
if(payload instanceof NotificationMessage) {
this.sendMessage((NotificationMessage)payload);
}
}
The NotificationMessage object extends Serializable and looks like this
public static class NotificationMessage implements Serializable {
/**
*
*/
private static final long serialVersionUID = 1631373969001850200L;
public String to;
public String data;
}
Related
In my little microservice, I created a Producer Kafka to send the messages with errors (messages having errors in the JSON format) inside the DeadLetter in this way :
#Component
public class KafkaProducer {
#Autowired
private KafkaTemplate<String, String> kafkaTemplate;
public void sendDeadLetter(String message) {
kafkaTemplate.send("DeadLetter", message);
}
}
I would like to create a JUnitTest for the completeness of the project, but I have no idea how to create the eventuality of a possible JSON error in order to create the test. I thank everyone for any possible help and advice
To create a JUnitTest consistent with your code. I should recreate the case where you pass it a warped or invalid JSON. In your case, I would opt to configure a MockConsumer from which to read any message that the logic of your code will be invited to the dead letter.
To have a usable test structure, I recommend something like this:
#KafkaListener(topics = "yourTopic")
public void listen(String message) {
messages.add(message);
}
For testing a basic structure could be
#Test
public void testDeadLetter(){
//Set up a mockConsumer
MockConsumer<String,String> yourMockConsumer = new MockConsumer<String,String> (OffsetResetStrategy.EARLIEST);
yourMockConsumer.subscribe(Collections.singletonList("yourTopic"));
//Sending message on embedded Kafka broker
String error = "ERRORE";
kafkaTemplate.send("yourTopic", error);
//Reading the message may take a second
Thread.sleep(1000);
//Create an Assert that checks you that the message is equal to the error specified above
}
I hope it will be useful to you!
You can create Kafka topic using testcontainers and write your tests on top of that.
Sharing an example on how to use testcontainers https://github.com/0001vrn/testcontainers-example
I am trying to build a simple cloud stream application with kafka binding. Let me describe the set up.
1. I have a producer producing to topic topic_1.
2. There's a stream binder, binding topic_1 after some processing into topic_2.
#StreamListener(MyBinder.INPUT)
#SendTo(MyBinder.OUTPUT_2)
public String handleIncomingMsgs(String s) {
logger.info(s); // prints all the messages
return s;
}
When the producer produces messages, the StreamListner handleIncomingMsgs gets all the messages.
After receiving, it should forward the messages to some other channel.
#Service
#EnableBinding(MyBinder.class)
public class LogMsg {
#StreamListener(MyBinder.OUTPUT_2)
public void handle(String board) {
logger.info("Received payload: " + board); //prints every alternate messages
}
Here is my binder
public interface ViewsStreams {
String INPUT = "input";
String OUTPUT_1 = "output_1";
String OP_USERS = "output_2";
#Autowired
#Input(INPUT)
SubscribableChannel job_board_views();
#Autowired
#Output(OUTPUT_1)
MessageChannel outboundJobBoards();
#Autowired
#Output(OUTPUT_2)
MessageChannel outboundUsers();
}
I am new in these technologies. Unable to figure out what is going wrong here. Can someone please help?
Your guess is correct; you have two consumers on the OUTPUT_2 channel - the listener and the binding which sends out the message.
They each get alternate messages.
I am trying to do the following with Play Framework 2.6:
The browser targets the server and a WebSocket is created
Later on (after some other request is performed), the servers sends a message to the browser via the WebSocket previously created
Point 1 can be easily done with a route:
public WebSocket socket() {
return WebSocket.Text.accept(request -> {
// Log events to the console
Sink<String, ?> in = Sink.foreach(System.out::println);
// Send a single 'Hello!' message and then leave the socket open
Source<String, ?> out = Source.single("Hello!").concat(Source.maybe());
return Flow.fromSinkAndSource(in, out);
});
}
and the WebSocket can be saved server side.
But then how can I send data via the WebSocket? (triggered server side)
This was easy to do with 2.5 but the documentation is not very helpful for Play 2.6.
I've managed to implement websocket with help of Akka actors. At first step define actor that will handle messages
public class WebSocketActor extends AbstractActor {
private final ActorRef out;
#Inject
public WebSocketActor(ActorRef out) {
this.out = out;
}
#Override
public Receive createReceive() {
return receiveBuilder()
.match(String.class, message ->
out.tell("Sending message at " + LocalDateTime.now().toString(), self())
)
.build();
}
public static Props props(final ActorRef out) {
return Props.create(WebSocketActor.class, out);
}
}
This actor will be created per client. ActorRef out will send message to connected client. In this example response is send to client on each string message passed to WebSocketActor.
Now define API endpoint to open access to websocket for clients. Define ActorFlow that will create new instance of actor on new connection
public WebSocket ws() {
return WebSocket.Text.accept(request -> ActorFlow.actorRef((out) -> WebSocketActor.props(out), actorSystem, materializer));
}
According to source code ActorFlow creates actors with flowActor name. So to send message to websockets somewhere in the code we can find actors by their path. This will broadcast message to all connected clients
actorSystem.actorSelection("/user/*/flowActor").tell("Hello", ActorRef.noSender());
Unfortunately I didn't find easy way to change ActorFlow default name but maybe this answer may help you play-scala-akka-websockets-change-actor-path.
Also you can check play-java-websocket-example project from playframework examples.
I am developing a client and server communication system using Netty NIO in Java. My code can be found in the following repository. Currently I am having one server and two clients and I am sending information from server to the clients and the opposite.
What I am trying to figure out, when I am receiving a message form the first client to the server, how can i send that message to the second client (and the opposite from client 2 to client 1). How can I send a message to a specific client?
I have noticed that my issues arised because of the way that I am trying to send the messages from the server. My code in serverHandler is the following:
for (Channel ch : channels1) {
responseData.setIntValue(channels1.size());
remoteAddr.add(ch.remoteAddress().toString());
future = ch.writeAndFlush(responseData);
//future.addListener(ChannelFutureListener.CLOSE);
System.out.println("the requested data from the clients are: "+requestData);
responseData1.setStringValue(requestData.toString());
future = ch.writeAndFlush(responseData1);
System.out.println(future);
}
By default am sending a message about the number of the connections, but also when I am receiving message from the client 1 or 2 I want to send it back to 2 and 1. So I want to perform the communication between the two components. How can I send from the server to a specific client? I am not sure how can I send the messages back to the clients.
General approach
Let's describe an approach to the problem.
When receiving data on the server side, use the remote address of the channel (the java.net.SocketAddress Channel.remoteAddress() method) to identify the client.
Such identification may be done using a map like: Map<SocketAddress, Client>, where the Client class or interface should contain the appropriate client connection (channel) associated context, including its Channel. Be sure to keep the map up-to-date: handle the «client connected» and «client disconnected» events appropriately.
After a client is identified, you may just send the appropriate messages to the clients, except the current sending client, using the client connection (channel) map.
Additionally, I would like to recommend you to find a good implementation of a chat application using Netty and to take a look at it.
Netty-specific solution
Let's consider the server side implementation, in particular, the implementation of the ProcessingHandler class.
It already manages the active channels by representing them as the channel group:
static final ChannelGroup channels1 =
new DefaultChannelGroup(GlobalEventExecutor.INSTANCE);
Keeping the channel group up-to-date
The current implementation handles the «channel becomes active» event to keep the channel group up-to-date:
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
channels1.add(ctx.channel());
// ...
}
But this is only a half: it is necessary to handle the «channel becomes inactive» event symmetrically as well. The implementation should look like:
#Override
public void channelInactive(final ChannelHandlerContext ctx) throws Exception {
channels1.remove(ctx.channel());
}
Broadcasting: Sending the received message to all channels, except the current one
To implement the desired behaviour, just update the implementation by introducing the appropriate check as follows:
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
// ...
for (Channel ch : channels1) {
// Does `ch` represent the channel of the current sending client?
if (ch.equals(ctx.channel())) {
// Skip.
continue;
}
// Send the message to the `ch` channel.
// ...
}
// ...
}
Sending and receiving string problem
Currently, the functionality around the ResponseData class is not present (not implemented).
The following draft changes are required to make both the client and the server work.
The ResponseData class: the getStringValue and toString methods should be corrected:
String getStringValue() {
return this.strValue;
}
#Override
public String toString() {
return intValue + ";" + strValue;
}
The ResponseDataEncoder class: it should use the string value:
private final Charset charset = Charset.forName("UTF-8");
#Override
protected void encode(final ChannelHandlerContext ctx, final ResponseData msg, final ByteBuf out) throws Exception {
out.writeInt(msg.getIntValue());
out.writeInt(msg.getStringValue().length());
out.writeCharSequence(msg.getStringValue(), charset);
}
The ResponseDataDecoder class: it should use the string value:
private final Charset charset = Charset.forName("UTF-8");
#Override
protected void decode(final ChannelHandlerContext ctx, final ByteBuf in, final List<Object> out) throws Exception {
ResponseData data = new ResponseData();
data.setIntValue(in.readInt());
int strLen = in.readInt();
data.setStringValue(in.readCharSequence(strLen, charset).toString());
out.add(data);
}
The ClientHandler class: it should correctly receive and handle the message:
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
final ResponseData responseData = (ResponseData) msg;
System.out.println("The message sent from the server " + responseData);
update.accept(responseData.getIntValue());
}
Additional references
«SecureChat ‐ an TLS-based chat server, derived from the Telnet example», Netty Documentation. In particular, the implementation of the SecureChatServerHandler class.
«Netty in Action», Norman Maurer, Marvin Allen Wolfthal (ISBN-13: 978-1617291470), «Part 3 — Network protocols», the «12.2 Our example WebSocket application» subchapter. Covers implementation of «a browser-based chat application».
I'd like to exchange messages with a customer over a jms queue. Something like:
User updated Address; Field Street; OldValue: foo; NewValue: bar;
What is the best interface to Exchange with the customer? I Could define it like the text above and send as TextMessage. But this is not really an interface I would prefer.
Shall I exchange an API with the java classes? Or use Soap over JMS?
What's the best way?
You can send text or a java objects (that must implement java.io.Serializable interface). If you will use simple text try to make it structured in a structred format xml or Json.
Otherwise use java object transmission
You can send text or a java objects (that must implement java.io.Serializable interface). If you will use simple text try to make it structured in a structred format xml or Json.
If you want to use java objects this may work also. Here is a very simple example :
/*the object you will be sent to consumer*/
public class MSObject implements java.io.Serializable{
......
}
/*sending method*/
private void sendObjectMessage() throws Exception {
ActiveMQSession mysession = (ActiveMQSession) session;
ObjectMessage msg = session.createObjectMessage();
msg.setObject( new MSObject("first name","Jimmy") );
msg.setJMSType(this.CMD_TYPE_MESSAGETYPE_OBJECT);
producer.send(msg);
if (transacted) {
session.commit();
}
System.out.println("The object message has been sent to " + this.subject
+ " queue.");
Thread.sleep(sleepTime);
}
/*receiving method*/
public void onMessage(Message message) {
if (message instanceof ObjectMessage) {
System.out.println("recevied a Object Message");
ObjectMessage msg = (ObjectMessage)message;
MSObject obj = (MSObject)msg.getObject();
System.out.println("info: ");
System.out.println("key: "+obj.getKey());
System.out.println("value: "+obj.getValue());
}
}
Soap sound like the simplest, any xml wrapping would work.
I would go for google protobufs instead though, then you could send binary data.