I would like to know how my Selenium framework can dequeue a message sitting in a message queue. I have built an application to send a JSON string containing k/v pairs to a message queue.
My architecture is as follows and separate apps:
A JSP Web Application exists accepting parameters resulting in a JSON string
A message sender exists and takes the JSON string and publishes it to a Queue
A message consumer exists and consumes the Messages. Its basically just sitting here
A Selenium Java Framework exists, but I would like to process the messages and for each message it will interpret the k/v pairs and kicks off the script.
I would like to use the messages already in the queue and process these messages within the selenium framework, how can I achieve this?
I will appreciate the help. I have edited the question with the code
This is the code snippet to send the JSON Message
public class MessageSender {
public static void main(String[] args) throws IOException {
SingleNumberLogin generateLogin = new SingleNumberLogin();
//function call to build the JSON object
String jsonQueue = generateLogin.buildJASONObject();
ConnectionFactory conFactory = new ConnectionFactory();
try {
Connection connInterface = conFactory.newConnection();
Channel mqChannel = connInterface.createChannel();
mqChannel.queueDeclare("MyQueue",false,false,false,null);
//Just assigning json to another string, then publish the message
String myMessage = jsonQueue;
mqChannel.basicPublish("","MyQueue",false ,false, null,myMessage.getBytes());
}catch (
IOException | TimeoutException e)
{
System.out.println(e.getStackTrace());
}
conFactory.setUsername("guest");
conFactory.setPassword("guest");
conFactory.setVirtualHost("/");
conFactory.setHost("localhost");
conFactory.setPort(5672);
}
}
code snippet for consumer code that I have inserted into the startup function of the automation script, so if a message arrives a single test case is executed
#BeforeTest
public static void initializeTestBaseSetup() throws Exception, IOException, TimeoutException {
ConnectionFactory conFactory = new ConnectionFactory();
Connection connInterface = conFactory.newConnection();
Channel mqChannel = connInterface.createChannel();
mqChannel.queueDeclare("MyQueue",false,false,false,null);
mqChannel.basicConsume("MyQueue", true, (consumerTag, message) -> {
//convert to byte array
String m = new String (message.getBody(), "UTF-8");
System.out.println("Message received" + m);
}, consumerTag -> {
});
}
Output JSON
JSON Message received 2020-08-28T20:39:30.845{
"NUMBER": "0000011111",
"Type": "BAU",
"User": "MyUser ",
"Email": "riidonesh#gmail.com",
}
When tested in isolation, it works perfectly fine, what I mean is that I send the message and check that the consumer receives it, adding the consumer code to my framework is where i am stuck.
I would suggest you don't think about what you have as a "selenium framework" - think of it as a "java framework".
Selenium is a set of libraries that allow you automate the web browser at a GUI level. The framework is the coded solution to facilitate creation and management of your test suite - it doesn't have to be limited to selenium and chances that's already just one of its components.
Trying to answer your question directly:
SELENIUM cannot read messages
JAVA can read messages
If your rabbitmq has a web front end then you may be able to use selenium for it, but this isn't a very efficient or a logical solution.
What you might want to consider, and what i would do, is extending your framework to use the rabbitmq libraries to process messages as you need. These libraries are designed for this task.
You say:
I would like to process the messages and for each message it will
interpret the k/v pairs and kicks off the script.
I understand this to mean that the messages are the pre-req data for the tests. If you want to read the values of a message before the test you can either:
Place the get/read in a generic #Before method
or if it's a specific message per test case, add it into the start of the test.
You're working in java so you can do whatever you want really.
To get you started, the rabbitmq tutorial starts here.
This is there hello world example for reading messages from the queue:
public class Recv {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
}
}
Related
I'm experiencing communication issues while testing between two Netty socket applications (the main application, and the integration test application), where I've been receiving an unusual amount of partial messages.
One pattern being noticed, is that the first message being sent from the test application (application executes sending the message from outside the pipeline, using a sharable handler) tends to always be partial. This is also noticed during times of latency, where another issue occurs.
The other issue is that when a partial message is receive at times, the decoder seems to be trapped in a loop, where it continues try to read the partial message indefinitely. I have a unit test to simulate the partial message using EmbeddedChannel, but the unit test is not replicating what I am seeing during the integration test.
The main application is using the following pipeline:
ch.pipeline().addLast(<HeaderTrailerFrameDecoder>, <NettyMessageDecoder>, <HeaderTrailerFrameEncoder>, <NettyMessageEncoder>);
ch.pipeline().addLast(<IdleStateHandler>,<EventHandler>, <ChannelHandler>);
where:
HeaderTrailerFrameDecoder - Removes single-byte frame from beginning / end of packet
NettyMessageDecoder - Converts message from ByteBuf to domain object
HeaderTrailerFrameEncoder - Appends frame to message packets
NettyMessageEncoder - Converts message from domain object to ByteBuf
IdleStateHandler - Netty class, detects stale / idle connections
EventHandler - Sharable handler to send messages from outside of the pipeline
ChannelHandler - Main handler for all business logic
I'm thinking the problem could be with all of the encoders / decoders, and not releasing my ByteBuf objects perhaps? I'm only seeing the issue from the first decoder, the HeaderTrailerFrameDecoder, so I'll provide a snippet below. For every connection, the first message coming through from a series of messages being sent by the test application, first produces the log msg="could not find trailer".
#Slf4j // Lombok logging
public class HeaderTrailerFrameDecoder extends ByteToMessageDecoder {
private final byte header;
private final byte trailer;
HeaderTrailerFrameDecoder(byte header, byte trailer) {
this.header = header;
this.trailer = trailer;
}
#Override
protected void decode(
final ChannelHandlerContext ctx,
final ByteBuf buf,
final List<Object> out) {
log.trace("msg=\"decoding message with header and trailer\", buf={}", buf);
// Find header
int headerIndex = buf.forEachByte(value -> value != header);
if (headerIndex < 0) {
log.error("msg=\"could not find header\", payload=\"{}\"", payload);
buf.skipBytes(buf.readableBytes());
return;
}
int beforeHeaderLen = headerIndex - buf.readerIndex();
buf.skipBytes(beforeHeaderLen + 1);
// Find trailer
int trailerIndex = buf.forEachByte(value -> value != trailer);
if (trailerIndex < 0) {
String payload = debug(buf);
log.error("msg=\"could not find trailer\"");
buf.resetReaderIndex();
return;
}
int insideFrameLen = trailerIndex - buf.readerIndex();
ByteBuf frame = buf.readBytes(insideFrameLen);
buf.skipBytes(1);
// Pass message through
out.add(frame);
}
}
This is probably some silly mistake I'm missing, but here is the issue:
I am trying to insert a simple "hello" message into a Rabbit queue, with a predefined exchange and routing key.
This is the code that I am using:
private static void send_equity_task_to_rabbitmq(ConnectionFactory factory) throws IOException,TimeoutException{
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare("b", false, false, false, null);
channel.exchangeDeclare("b", "direct");
channel.basicPublish("b","b",null, "hello".getBytes());
channel.close();
connection.close();
}
public static void main(String[] argv) throws TimeoutException,IOException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("127.0.0.1");
Date start_time= Calendar.getInstance().getTime();
Long start_time_timestamp=System.currentTimeMillis();
System.out.println("[INFO] Starting connection to queue at:"+start_time);
send_equity_task_to_rabbitmq(factory);
Long end_time_timestamp=System.currentTimeMillis();
System.out.println("[INFO] Message sent and processed successfully after:"+ (end_time_timestamp-start_time_timestamp)+" miliseconds");
}
}
The code runs without any error. However, when I check the amount of records inside the "b" queue, I get:
$ rabbitmqctl list_queues
Listing queues ...
b 0
...done.
I don't have consumers for this queue at the moment, so I assume since it has 0 records, that I am using basicPublish badly.
What could be wrong?
Thank you.
I think you need to bind the queue to the exchange. You've created a queue called "b" and an exchange called "b". The exchange will distribute messages to queues that are bound to it, using the "b" routingKey, but as the "b" queue isn't bound to the "b" exchange, the "b" exchange doesn't publish to that queue.
I have an ActimeMQ consumer which expects a message in javax.jms.ObjectMessage format.
This message pojo has 5 string elements.
Now I am trying to write a message producer for this consumer in NodeJs.
I am using stompit module
My current NodeJs code is
stompit.connect(connectOptions, function(error, client) {
if (error) {
console.log('connect error ' + error.message);
return;
} else {
console.log("connected");
}
var sendHeaders = {
'destination': '/queue/test',
'transformation': 'jms-object-json'
};
var msg = new Object();
msg.val1 = "12";
msg.val2 = "test";
msg.val3 = "1";
msg.val4 = "1";
msg.val5 = "Y";
var frame = client.send(sendHeaders);
frame.write(JSON.stringify(msg));
frame.end();
});
Java consumer is able to get the message but throws the exception
org.apache.activemq.command.ActiveMQTextMessage cannot be cast to javax.jms.ObjectMessage
I have read this page from activeMQ which says that
Currently, ActiveMQ comes with a transformer that can transform XML/JSON text to Java objects, but you can add your own transformers as well
I didn't quite understand this part on how to convert data.
I have added xstream-1.4.10.jar and jettison-1.3.8.jar in apache-activemq-5.15.0\lib and restarted the ActiveMq server.
But still I get the error in the consumer.
Also in the ActiveMQ console -> Queues -> message properties, it shows transformation-error
Please let me know how I can convert this ActiveMQTextMessage type to javax.jms.ObjectMessage before it reaches the consumer
There isn't a transformer in ActiveMQ that will convert any random JSON string into and ObjectMessages, you'd have to write you own to handle whatever format you are sending. The converter in ActiveMQ will convert some basic types that Map from the JSON but it's tricky and not necessarily reliable. You are better off handling the TextMessage and doing something meaningful with the JSON yourself.
ActiveMQTextMessage and ObjectMessage are different , they can't cast to each other.
From ActiveMQTextMessage , you can get the true message content as a String, then you have to trans it to a json object yourself.
Am using apache camel, With Polling consumer, when poll my mail is mark as read.
options : delete=false&peek=false&unseen=true
After polling , when i am processing the attachment, if any error occurs , i want to make the mail as "unread". So that i can pool again later.
public void process(Exchange exchange) throws Exception {
Map<String, DataHandler> attachments = exchange.getIn().getAttachments();
Message messageCopy = exchange.getIn().copy();
if (messageCopy.getAttachments().size() > 0) {
for (Map.Entry<String, DataHandler> entry : messageCopy.getAttachments().entrySet()) {
DataHandler dHandler = entry.getValue();
// get the file name
String filename = dHandler.getName();
// get the content and convert it to byte[]
byte[] data =
exchange.getContext().getTypeConverter().convertTo(byte[].class, dHandler.getInputStream());
log.info("Downloading attachment, file name : " + filename);
InputStream fileInputStream = new ByteArrayInputStream(data);
try {
// Processing attachments
// if any error occurs here, i want to make the mail mark as unread
} catch (Exception e) {
log.info(e.getMessage());
}
}
}
}
I noticed the option peek, by setting it to true, It will not make the mail mark as read during polling, in that case is there any option to make it mark as read after processing.
To get the result that you want you should have options
peek=true&unseen=true
The peek=true option is supposed to ensure that messages remain the exact state on the mail server as they where before polling even if there is an exception. However, currently it won't work. This is actually a bug in Camel Mail component. I've submitted a patch to https://issues.apache.org/jira/browse/CAMEL-9106 and this will probably be fixed in a future release.
As a workaround you can set mapMailMessages=false but then you will have to work with the email message content yourself. In Camel 2.15 onward you also have postProcessAction option and with that you could probably remove the SEEN flags from messages with processing errors. Still, I would recommend waiting for the fix though.
We can set the mail unread flag with the following code
public void process(Exchange exchange) throws Exception {
final Message mailMessage = exchange.getIn(MailMessage.class).getMessage();
mailMessage.setFlag(Flag.SEEN, false);
}
Given the following publisher in node.js and the following subscriber in java (this setup is fully functional) I have the following two questions:
What should I use as the third argument in queueBind and why? Why does it works as is ("test" is a random pick)?
Is there a way to specify queue in addition to exchange in rabbit.js? If yes then how? If not then why and which module should I use instead (code example would be welcome)?
// node.js
var context = require("rabbit.js").createContext();
var pub = context.socket('PUB');
pub.connect(config.exchange);
server.post("/message/:msg", function(req, res) {
pub.write(req.params.msg, 'utf8');
res.end();
});
// java
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(host);
try {
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.exchangeDeclare(exchange, "fanout");
String queueName = channel.queueDeclare().getQueue();
channel.queueBind(queueName, exchange, "test"); // Question1: what should I use as the third argument and why?
// Question2: is there a way to configure rabbit.js with a queue name instead?
//channel.queueDeclare(queueName, false, false, false, null);
QueueingConsumer consumer = new QueueingConsumer(channel);
channel.basicConsume(queueName, true, consumer);
try {
while (true) {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
String message = new String(delivery.getBody());
LOG.info("Received message: " + message);
}
} catch (InterruptedException e) {
LOG.catching(e);
} finally {
channel.close();
connection.close();
}
} catch (IOException e) {
LOG.catching(e);
}
Own answer, what I've digged up so far:
The third argument, the routing key, is what is known as topic in rabbit.js. By supplying test I am only subscribing to messages send to the test topic or without a topic set (default in rabbit.js). If I were to use topic in the publisher as well, I could use pub.publish(topic, message, encoding) instead of pub.write(message, encoding) or supply it to the connect method
Does not look so and still do not know why really. The argument goes that rabbit.js is a higher-level library and it, therefore, makes certain simplifications. Why exactly this simplification is made I do not know. However, I primarily wanted to use a single exchange for multiple communication threads, which I can also achieve by using topics/routing keys. So not a big deal.