I have a Java process which is listening for messages from a queue hosted by ActiveMQ and calls webservices if received status is COMPLETE.
I was also thinking of handling webservices calls using ActiveMQ as well. Is there a way I could make best use of ActiveMQ using maybe another queue?
This might help me in handling a scenario if one or more webservice calls fails in first attempt. But I'm trying to think what could I do to achieve something like this.
Do I need to forward webservice parameters to ActiveMQ queue and then look for something?
#Autowired
private JavaMailSender javaMailSender;
// Working Code with JMS 2.0
#JmsListener(destination = "MessageProducer")
public void processBrokerQueues(String message) throws DaoException {
...
if(receivedStatus.equals("COMPLETE")) {
// Do something inside COMPLETE
// Need to call 8-10 webservices (shown only 2 below for breviety)
ProcessBroker obj = new ProcessBroker();
ProcessBroker obj1 = new ProcessBroker();
// Calling webservice 1
try {
System.out.println("Testing 1 - Send Http POST request #1 ");
obj.sendHTTPPOST1();
} finally {
obj.close();
}
// Calling webservice 2.
try {
System.out.println("Testing 2 - Send Http POST request #2");
obj1.sendHTTPPOST2();
} finally {
obj1.close();
}
} else {
// Do something
}
}
What I'm looking for:
Basically If there is a possibility of creating an additional message queue, submit 8 messages to the queue, one corresponding to each webservice end point and have a queue listener dedicated to the message queue. If the message could not be delivered successfully, it could be put back on the queue for later delivery.
Related
Let's say I have a Load Balancer (LB) in front of 1..n VertX (V) instances, each VertX instance is connected to a queue (Q), and I have 1..m Backends (BE).
A user clicks on a button which makes a post request or even opens a web socket, the load balancer forwards the request to one of the VertX instances, which fires a request to the queue, one of the Backends consumes the message and sends a response back; if the correct VertX instance consumes it, it can lookup the response handler and write a response to the user, if the wrong VertX instance consumes it, there won't be a response handler to write a response to and the user will wait indefinitely for a response.
See this sketch:
Alternatively, V2 dies and the load balancer reconnects the user to V1 which means even if I could send it back to the exact same one that made the request, it's not guaranteed to still be there once the response comes back, but the user might still be there awaiting a response via another VertX instance.
What I'm currently doing is to generate a GUID for each new connection, then as soon as the websocket connects, store the websocket handler inside a hashmap against the GUID and then when the BE wants to respond, it does a fanout to all 1..n VertX instances, the one that currently has the correct GUID in its hashmap can then write a response to the user.
Same for handling POST / GET in this manner.
Pseudocode:
queue.handler { q ->
q.handler {
val handler = someMap.get(q.guid)
// only respond if handler exists
if (handler != null){
handler.writeResponse(someresponsemessagehere)
}
}
}
vertx.createHttpServer().websocketHandler { ws ->
val guid = generateGUID()
someMap.put(guid, ws)
ws.writeFinalTextFrame("guid=${guid}")
ws.handler {
val guid = extractGuid(it)
// send request to BE including generated GUID
sendMessageToBE(guid, "blahblah")
}
}.requestHandler { router.accept(it) }.listen(port)
This does however mean that if I have a 1000 VertX applications running, that the backend will need to fanout its message to a 1000 frontend instances of which only one will make use of the message.
VertX seems like it already takes care of async operations very well, is there a way in VertX to identify each websocket connection instead of having to maintain a map of GUIDs mapped to websocket handlers / post handlers?
Also, referring to the picture, is there a way for V3 to consume the message, but still be able to write a response back to the websocket handler that's currently connected to V2?
What you're missing from your diagram is the Vertx EventBus.
Basically you can assume that your V1...Vn are interconnected:
V1<->V2<->...<->Vn
Let's assume that Va receives your outbound Q message (the red line), that is intended for Vb.
It then should send it to Vb using EventBus:
eventBus.send("Vb UUID", "Message for Vb, also containing WebSocket UUID", ar -> {
if (ar.succeeded()) {
// All went well
}
else {
// Vb died or has other problems. Your choice how to handle this
}
});
I've read Netty Guide, it does not explain much on ChannelFuture. I find ChannelFuture is a complex idea when applying it.
What I am trying to do is to write message to a context after it's initial response. Different from typical request/response flow. I need a flow like this:
Client send request -> Server (netty)
Server send a response with ctx.writeAndFlush(msg);
Server send some more message to that ctx after step 2 is complete.
The problem is that if I do something like this, the second write will not send out:
ctx.writeAndFlush(response);
Message newMsg = createMessage();
ctx.writeAndFlush(newMsg); //will not send to client
Then I try to use ChannelFuture, it works, but I am not sure if I am logically correct:
ChannelFuture msgIsSent = ctx.writeAndFlush(response);
if(msgIsSent.isDone())
{
Message newMsg = createMessage();
ctx.writeAndFlush(newMsg); //this works
}
or should I use a ChannelFutureListener() instead?
ChannelFuture msgIsSent = ctx.writeAndFlush(response);
msgIsSent.addListener(new ChannelFutureListener(){
#Override
public void operationComplete(ChannelFuture future)
{
Message newMsg = createMessage();
ctx.writeAndFlush(newMsg);
}
});
Will this also works?
Which one is the best practice approach? Is there any potential problem using method 2?
Of course, this depends too on your "protocol" (meaning for instance if you use HTTP, sending 2 answears for the same request is not supported by HTTP protocol). But let say your protocol allows you to send multiple response parts:
Netty add messages to send to the pipeline, respecting the order.
So in your first example, I'm a bit surprised it does not work:
ctx.writeAndFlush(response);
Message newMsg = createMessage();
ctx.writeAndFlush(newMsg); // should send the message
However it could be lead by your protocol. For instance, this could happen:
response in message queue to send
flush not yet done
newMsg in message queue to send
flush now come but protocol does not support 2 messages so only send first one
So if your protocol must admit that first message is sent already, then you have to wait for the first, so doing something like:
ctx.writeAndFlush(response).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) {
if (future.isDone()) {
Message newMsg = createMessage();
ctx.writeAndFlush(newMsg);
} else { // an error occurs, do perhaps something else
}
}
});
So your last proposal (I've just don't create a ChannelFuture but directly used the result of writeAndFlush, but both are equals). Just take care of the case where operationComplete does not mean it is in success.
Try this:
ctx.channel().writeAndFlush(response);
Message newMsg = createMessage();
ctx.channel().writeAndFlush(newMsg);
Channel.write() always starts from the tail of the ChannelPipeline.
ChannelHandlerContext.write() starts from the current position of the ChannelHandler.
#2 looks better but make sure to test if the operation was successful. If not, use future.getCause() to access the exception. Not that it will change the functionality, but you can shorten the code by simply adding the listener directly on the result of the write call, I.e. you don't need to declare the future itself since it will be provided in the callback.
I have a EJB Singleton that constantly listens for messages as a String and when a message arrives it processes the message for example storing to the database and doing other things.
Once I receive the message and after furthing processing, I need to asycnhrnously send the message. Java provides the ExecuteService to asynchronusly process the message. EJB also provides asynchronously processing through the #Asynchrnous annotation. The listener works by checking every couple of seconds to see if the message has arrived. Once the asynchronous method has been sent, a reply would come back in the form of an InputStream. I have two queries:
I have sample code:
#Startup
#Singleton
public class Processor {
#PostConstruct
public void init() {
while(true) {
Thread.sleep(1000);
if(!portIsOpen()) {
openPort();
listen();
}
else {
listen();
}
}
}
public void listen() {
// listens for messages, once the message arrives process them and then send asynchronously
Future<InputStream> inputStream = processAsynchrnously();
while(!inputStream.isDone()) {
// carry on with repeated requests
}
inputStream = inputStream.get();
// read this inputStream and turn into XML
}
#Asynchronous
public Future<InputStream> processAsynchronosly() {
// make an http connection and send the request of with the data as XML
return new AsychResult<InputStream>(httpUrlConnection.getInputStream());
}
}
The above is what I am trying to achive. However, the aim is to send the request asynchrnously and not wait but continue to receive other requests repeatly. The problem is the while(!inputStream.isDone()) this means it is blocked.
1) How can this be implemented in a different way so the while loop is not required.
2) When the data is available so the input stream is returned with the XML, in EJB is there a listener that can be invoked so that I will implement a method that will read the contents of the input stream? I am aware there are Inceptors but an Inceptor will not work in this case.
EDIT:
Please note the application is a TCIIP application, so it listens for messages on a port. It is not a RMI-IIOP application. This Singleton acts as a listener, so when the message arrives it processes it.
If MDB is the only solution how will MDB help? Could you show a code sample?
I'm using Atmosphere in my Spring MVC app to facilitate push, using a streaming transport.
Throughout the lifecycle of my app, the client will subscribe and unsubscribe for many different topics.
Atmosphere seems to use a single http connection per subscription - ie., every call to $.atmosphere.subscribe(request) creates a new connection. This quickly exhausts the number of connections allowed from the browser to the atmosphere server.
Instead of creating a new resource each time, I'd like to be able to add and remove the AtmosphereResource to broadcasters after it's initial creation.
However, as the AtmosphereResource is a one-to-one representation of the inbound request, each time the client sends a request to the server, it arrives on a new AtomsphereResource, meaning I have no way to reference the original resource, and append it to the topic's Broadcaster.
I've tried using both $.atmosphere.subscribe(request) and calling atmosphereResource.push(request) on the resource returned from the original subscribe() call. However, this made no difference.
What is the correct way to approach this?
Here's how I got it working:
First, when the client does their initial connect, ensure that the atmosphere-specific headers are accepted by the browser before calling suspend():
#RequestMapping("/subscribe")
public ResponseEntity<HttpStatus> connect(AtmosphereResource resource)
{
resource.getResponse().setHeader("Access-Control-Expose-Headers", ATMOSPHERE_TRACKING_ID + "," + X_CACHE_DATE);
resource.suspend();
}
Then, when the client sends additional subscribe requests, although they come in on a different resource, they contain the ATMOPSHERE_TRACKING_ID of the original resource. This allows you to look it up via the resourceFactory:
#RequestMapping(value="/subscribe", method=RequestMethod.POST)
public ResponseEntity<HttpStatus> addSubscription(AtmosphereResource resource, #RequestParam("topic") String topic)
{
String atmosphereId = resource.getResponse().getHeader(ATMOSPHERE_TRACKING_ID);
if (atmosphereId == null || atmosphereId.isEmpty())
{
log.error("Cannot add subscription, as the atmosphere tracking ID was not found");
return new ResponseEntity<HttpStatus>(HttpStatus.BAD_REQUEST);
}
AtmosphereResource originalResource = resourceFactory.find(atmosphereId);
if (originalResource == null)
{
log.error("The provided Atmosphere tracking ID is not associated to a known resource");
return new ResponseEntity<HttpStatus>(HttpStatus.BAD_REQUEST);
}
Broadcaster broadcaster = broadcasterFactory.lookup(topic, true);
broadcaster.addAtmosphereResource(originalResource);
log.info("Added subscription to {} for atmosphere resource {}",topic, atmosphereId);
return getOkResponse();
}
I've created a mock XMPP server that processes PLAIN encryption stanzas. I'm able to use Pidgin and go through the entire session creation, to the point where Pidgin thinks the user is on an actually XMPP server and is sending regular pings.
However, it seems like not all messages are processed correctly and when I do get a successful login, it was just luck. I'm talking, maybe 1/10th of the time I actually get connected. The other times it seems like Pidgin missed a message or I dumped messages to fast on the transport.
If I enable Pidgin's XMPP Console plugin, the first connection is ALWAYS successful, but a second user fails to make it through, typically dying when Pidgin requests Service Discovery.
My Mina code is something like this:
try
{
int PORT = 20600;
IoAcceptor acceptor = null;
acceptor = new NioSocketAcceptor();
acceptor.getFilterChain().addFirst("codec", new ProtocolCodecFilter( new ProtocolCodecFactoryImpl()));
acceptor.getFilterChain().addLast("executor", new ExecutorFilter(IoEventType.MESSAGE_RECEIVED));
acceptor.setHandler( new SimpleServerHandler());
acceptor.getSessionConfig().setIdleTime(IdleStatus.BOTH_IDLE, 10);
acceptor.bind( new InetSocketAddress(PORT));
}
catch (Exception ex)
{
System.out.println(ex.getMessage());
}
and the SimpleServerHandler is responsible for message/stanza processing and session creation. The messageReceived function looks like:
#Override
public void messageReceived(IoSession session, Object msg) throws Exception
{
String str = msg.toString();
System.out.println("MESSAGE: " + str);
process(session, str);
}
and finally, process is in charge of parsing the message out, and writing the response. I do use sychonized on my write:
public void sessionWrite(IoSession session, String buf)
{
synchronized(session)
{
WriteFuture future = session.write(buf);
}
}
I have omitted my processing code for brevity, but it simply looks for certain pieces of data, crafts a response and calls sessionWrite(...)
My question is, will this pattern work? And if not, should I consider shoving received messages in a Queue and simply processing the Queue from say a Timer?
It turns out, Pidgin would send two IQ stanzas, but I wasn't handling them correctly. My decoder now determines the end of a stanza and only writes a stanza to the buffer I read from.
Works like a dream now!