I have JMS Queue which has hundreds of jobs running and have multiple consumers on the same jvm which are processing to Queue to pick the job and process them, the system is crashing due to some performance issue and so I want to deploy JMS Consumers on different JVM, am new to deploying application or part of application between different JVM and would really appreciate if you can provide me some working example of how to do it and how to implement then it would be very helpful to me.
Update:
Now there are two ways to consume messages from JMS Queues either by creating standalone jms consumers or by creating message driven beans, now I need to have multiple consumers on different jvms that will listen to the queue and process the messages, with standalone jms consumers, i can run multiple of those consumers ( just run different jms consumers as standard java program as they have main function) but my question is how can i run multiple of message driven beans on different jvms, meaning how can i run multiple onMessage() functions on different jvms, hope am making some sense here. Kindly advise or point me in the right direction.
Any guidance would be highly appreciated.
Thanks.
Related
I’m working on an application that often queries a very large number of actors and hence sends / receives a very large number of messages. When the application is ran on a single machine this is not an issue because the messages are sent within the boundaries of a single JVM which is quite fast. However, when I run the application on multiple nodes (using akka-cluster) each node hosts part of these actors and the messages go over the network which becomes extremely slow.
One solution that I came up with is to have a ManagerActor on each node where the application is ran. This will greatly minimize the number of messages exchanged (i.e. instead of sending thousands of messages to each of the actors, if we run the application on 3 nodes we send 3 messages - one for each ManagerActor which then sends messages within the current JVM to the other (thousands of) actors which is very fast). However, I’m fairly new to Akka and I’m not quite sure that such a solution makes sense. Do you see any drawbacks of it? Any other options which are better / more native to Akka?
You could use Akka's Distributed Publish-Subscribe to achieve that. That way you simply start a manager actor on each node the usual way, have them subscribe to a topic, and then publish messages to them using that topic topic. There is a simple example of this in the docs linked above.
We are backend processor and doing programming with JMS MQ. We have 2 queues. one is used to get message and another one is used to send message. All the banking users will put messages to Q1 through their IB, MB etc. We receive messages from Q1 and process it and we send message to Q2.
Currently we do not use multi threading for this. Can we able to use multi threading for this or single thread is enough to do this. because we are getting messages from Q1 one by one and process it.
Kindly revert back to me if question is not understandable. Please someone help me.
Yes, JMS allows for multiple readers on the same queue. You can do this by multi-threading, multiple application instances, or a dispatch layer that fetches messages and then passes them to a handler through a callback or other mechanism.
The application must support that, however. For example, if two messages are related and must be processed in order, the order is not preserved if there is more than one listener on the queue. This is one reason why async messaging patterns strongly prefer messages not to have order dependencies or affinities.
If you use multi-threading, it is important to make sure to preserve transactionality. If multiple threads use the same connection and one issues a COMMIT then that commits all outstanding messages across all the threads sharing that connection.
I'm working in a Java EE application and I want that some WebServices are executed in parallel.
I would like to know the pros and cons of 2 different approaches:
Use JMS queues and MDBs, so each message I put in the queue would be executed in parallel. This way the application part that put the message into the queue would have a while, that waits the MDBs to response in a RS Queue.
Use the java concurrent API (Future / Callable).
ADDED
This is what the application needs to do:
The application already does it via an MDB, but I was thinking about a refactoring.
TODAY'S SCENARIO:
//CALLER CLASS
FOREACH INTEGRATION
PUT MESSAGE INTO A QUEUE AND STORE AN ARRAY OF CORRELATION_IDs
END
THREAD.SLEEP(X) // SOMETIME FOR INTEGRATION TO FINISH
WHILE (true){
GET RESPONSE FROM THE RESPONSE QUEUE FOR EACH INTEGRATION USING THE CORRELATION PREVIOUSLY STORED
}
//MDB CLASS
HAS A HUGE SWITCH CASE THAT PROCESS EACH INTEGRATION
RETURN THE RESULT INTO THE RESPONSE QUEUE;
Questions:
Is it ok to use the concurrent API in java? In my opinion using the concurrent API will eliminate a layer of failure (JMS).
My deployment environment is Websphere. Is it a good practice to create your own threads with the concurrent java API.
Thanks in advance
Whatever solution you go with, you will eventually need to cope with a burst of traffic. The JMS/MDB the burst is controlled by the queue effectively. Also a point to consider is that the queue can be made persistence, so it will survive a server restart. Also a queue can be distributed across many servers, giving you horizontal scalability.
The thread approach is of course quicker to develop, test and deploy. However, I would consider using a BlockingQueue so that your threads do not run amock.
jms pros: you can have persistence, you can connect to existing infrastructure
jms cons: seems to heavy to be used only as a dispatcher
manual concurrency cons: well, it's manual. and parallel programming is difficult. some webservers (especially clouds) may forbid to create your own threads
not sure what exactly you want to do but webserver by default processes requests in parallel, so maybe you don't need anything else?
I have a module which runs standalone in a JVM (no containers) and communicates with other modules via JMS.
My module is both a producer in one queue and a consumer in a different queue.
I have then need to cluster this module, both for HA reasons and for workload reasons, and I'm probably going to go with Terracotta+Hibernate for clustering my entities.
Currently when my app starts it launches a thread (via Executors.newSingleThreadExecutor()) which serves as the consumer (I can attach actual code sample if relevant and neccessary).
What I understood from reading questions here is that if I just start up my module on N different JVMs then N different subscribers will be created and each message in the queue will arrive to N subscribers.
What I'd like to do is have only one of them (let's currently say that which one is not important) process that message and so in actuality enable me to process N messages at a time.
How can/should this be done? Am I way off the track?
BTW, I'm using OpenMQ as my implementation but I don't know if that's relevant.
Thanks for any help
A classic case of message handling in clustered environment. This is what I would do.
Use Broadcast message (Channel based) in place of Queue. Queue being useful for point to point communication is not very effective. Set validity of message till the time it is consumed by one of the consumer. This way, other consumers wont even see the message and only one consumer will consume it.
Take a look at JGroups. You may consider implementing your module/subscribers to use jgroups for the kind of synchronization you need. JGroups provide Reliable Multicast Communication.
I have a java class Processor that is listening to a jms topic and it is struggling to keep up with the speed in which messages are arriving so we've decided to go concurrent:
A single class listening to the topic who's job is to hand the messages out to a pool of worker threads, effectively being a load balancer. It also has to prevent 2 workers processing messages for the same customer.
I expected there to be quite a lot of information on the internet about this but everything seems to suggest the use of EJBs where the app server manages the pool and does the balancing. I'm sure this must be a really common problem, but can't seem to find any libraries or design patterns to assist. Am I making more out of it than it is and should just delve in and write my own code?
Why don't you just use a queue instead of a topic and have several instances of the same application handle messages from this queue ?
This is an easy problem to solve with a pool of listeners. That's what the app server would be doing for you.
I'd get a good app server and use its MDBs to solve this quickly. Size the pool to keep up and you'll be fine.
If you insist on writing your own code, get a good open source pool implementation and use it.
If it must be non-EJB, consider Spring. It has message driven POJOs that could be just what you need.