What is a good solution for communication via message broker that supports both (C)Python and Java/JMS applications? My particular requirements are:
open source solution
Available on Linux-based systems
No rendezvous between sender and receiver required (i.e. uses a message broker)
Multiple producers and consumers supported for a single event queue (only one consumer receives each message)
Unit of work support with two-phase commit (XA support nice to have)
Support for persistent messages (i.e. that survive a restart of the broker)
Supports JMS for Java clients
No component is "fringe", meaning at risk of falling out of maintenance due to lack of community support/interest
If there is a Python client that manages to "speak JMS" that would be awesome, but an answer including a task to write my own Python JMS layer is acceptable
I have had a surprisingly hard time finding a solution for this. Apache's ActiveMQ has no Python support out of the box. ZeroMQ requires a rendezvous. RabbitMQ does not appear to support JMS. The best candidate I have found is a combination of ActiveMQ and the pyactivemq library. But the first and last release of pyactivemq was in 2008, so it would appear that that fails my "no fringe" requirement.
The ideal answer will be the names of one or more well-supported and well-documented open source packages, that you have personally used to communicate between a Java/JMS and Python application, and that do not require a lot of integration work to get started. An answer that includes an "easy" (up to a few days of work) implementation of additional glue code to meet all the requirements above, would be acceptable. A commercial solution, in the absence of a good open source candidate, would be acceptable also.
Also, Jython is out. (If only I could...) The same Python applications will need to use modules only available in CPython.
JMS is a specification not implementation . RabbitMQ is a really option .
I have also happily used HornetQ http://www.jboss.org/hornetq from Jboss as with every thing it is more aligned with every thing Java EE but RabbitMQ would be choice espcially if you are using Spring as well
I have had a surprisingly hard time finding a solution for this.
Apache's ActiveMQ has no Python support out of the box.
ActiveMQ brokers fully support using the Stomp protocol out of the box. Stomp is a text based protocol for messaging that has clients for many platforms and languages.
ActiveMQ's documentation should contain information on how to set up a connector for stomp. In its simplest form, enabling a connector would look something like:
<transportConnectors>
<transportConnector name="stomp" uri="stomp://localhost:61613"/>
</transportConnectors>
Once enabled on the broker side, you can then use any python library that supports stomp. You can then use Stomp on the python side and JMS on the java side for communication with the broker and sending/receiving from specific destinations.
You might want to take a look at OpenAMQ and another look at RabbitMQ.
The underlying messaging technology used by RabbitMQ and OpenAMQ is AMQP. You should be able to easily find Python and Java clients that work against both of these brokers (and ostensibly any other spec-compliant broker).
If JMS is a must-have, then you might be able to find a JMS client out there implemented on top of AMQP (OpenAMQ provided such a client at one time, but I am unsure of its current status).
We had been using GlassFish Message Queue (formerly Sun Java MQ) - it is inherited from OpenMQ
It satisfies most of your requirements, if not all.
We had been using fail over-clustered brokers in Red Hat Linux (RHEL) - it is reliable for heavy usage. Though some 'quirks' lurk here and there.
Related
I Know the theoretical difference like :
Amazon MQ provides a managed message broker service that takes care of
operating ActiveMQ, including broker set up, monitoring, maintenance,
and provisioning the underlying infrastructure for high availability
and durability. You may want to consider Amazon MQ when you want to
offload operational overhead and associated costs.
What i am asking is there any difference in making connection, fetching data with active MQ and Amazon MQ.(On coding side mainly in java)
I think there is no difference.
No there is no difference on the code side. Both of them uses same underlying framework and libraries.
Only difference you might see on the code side is amazonMQ is SSL secured which makes it mandate to pass username and password in the code.
AmazonMQ works on version equal to or greater than <5.15.8> version for activemq related dependencies.
Other differences are obviously with the kind of services AWS provides like better monitoring, logs analysis, configuration management/version management of config etc.
I've learned a respectable amount about networking protocols in Grad School and in professional experience and sent HTTP requests programmatically using AJAX and such.
The project on which I work professionally uses JMS to communicate and I'm curious about how it works.
When using REST (for instance) one makes an HTTP request with parameters in either the URI or the message header in order to invoke a service and further describe its needs.
A mentor of mine at work and I were discussing how JMS works and I'm struggling to understand at an application level how messages are actually sent. As far as I understand JMS in general (I realize there are many implementations of JMS) it is a specification for how to format data being sent.
Is the message itself still sent via HTTP(S)? Could it be SMTP?
Without going excruciatingly deep I would like to understand how one would, at a code level, send a JMS message from one service to another?
Am I even thinking about this correctly?
Can it be done any number of different ways?
Is there a convention that's used in the industry?
If someone could shed some light on JMS for me I would appreciate it.
Thanks!
JMS is not a protocol, it's an API specification. It's not something like TCP or HTTP protocol. Simply put the JMS specification defines signature of messaging APIs. How the APIs are internally implemented and what protocols they use to communicate with the messaging provider is vendor specific.
The vendor specific JMS implementations know how to communicate with their own messaging provider, but not with any other vendors messaging providers. For example IBM's MQ JMS implementation uses it's own protocol to communicate with IBM MQ Queue Manager, similarly Oracle JMS, Active MQ implementations with their own messaging provider.
Hope this helped.
I am currently working on a project that needs to work with WebSockets.
The clients will be the browsers only, so not interested in any message broker feature of these two.
Although I don't think that it would be much a difference, but would be interested if I can get a few comments on STOMP vs WAMP as a subprotocol using Spring-WebSockets.
And I might get a few hints that may help choose one over the other.
Cheers,
EDITED (27-02-2014):
As, programming directly with webSockets is low-level, so its suggested in many places (one the spring documentation) to use some sub-protocol.
Also, the use of a sub-protocol over websocket, is a type of native webSocket security; like you can do a sub-protocol validation.
There are many other sub-protocol that can be used instead of STOMP or WAMP like XMPP, AMQP.
I couldn't find much information related to these and webSockets, most of it is related to message brokers only.
The one reason (obvious) that I should use STOMP over any other sub-protocol, is that I am using spring-webSockets so for STOMP I don't need add any additional library at the server side, for others I would need to.
At the clients there are many JS libraries that I can use like socket.io for STOMP and similar JS for WAMP and others.
Since, my main concern is at the server side: So, STOMP it is.
I will update this post if I find out something new while implementing my solution.
I am trying to understand the basics of Message Queues. I see that there are many implementations available as libraries for MQs (ActiveMQ, RabbitMQ, ZeroMQ etc). Also J2EE enabled servers provide such support I think.
What I fail to understand about the topic, is how are these kind of constructs used by real software. I mean what kind of messages are usually being exchanged? Strings? Binary data?
If I understand correctly one can configure the transport protocol, but what is usually the application data format?
Is it a new way of communication, like e.g. SOAP WS or REST WS or RPC etc where each has a different application msg format?
Message queues usually using for application integration. In enterprise it is usually used to implement ESB, but nowadays there are smaller application systems that utilize similar patterns.
Concerning data being transmitted - usually it is XML messages, but actually depends on applications and MQ software - some of them are able to handle binary messages, some are not.
For example imagine you have two applications which require data interchange. If you integrate them using some kind of messaging software such as ActiveMQ, for example, then it will give you some benefits like routing, fault-tolerance, balancing, etc, out-of-the-box.
You may integrate your applications using MQ directly, but ESBs usually
give you ability to use web services: app just calls ws of ESB and knows nothing about underlying architecture.
Also MQs and ESBs gives you a level of abstraction: you may switch your apps in a system absolutely transparent, as long as data exchange interface is preserved.
MQs are mainly used for interprocess communication, or for inter-thread communication within the same process. They provide an asynchronous communications protocol, meaning that the sender and receiver of the message do not need to interact with the message queue at the same time. Messages placed onto the queue are stored until the recipient retrieves them.
wikipedia can be good intro to topic. http://en.wikipedia.org/wiki/Message_queue#Standards_and_protocols
Also, to understand diff between webservice and mq, read this thread: Message Queue vs. Web Services?
I will like to know:
I have a scenario. If a user adds a product to the system (I'm developing), there's a listener that sends a notification to the user's client base notifying of a new product added by the user.
I've read this thread and (seeing I've never used JMS nor ThreadPool before) I was wondering whether I should use JMS or ThreadPooling.
I am using Tomcat 5.5 and higher and JBoss 5 and higher (depending on company last resort) to deploy my web application.
If I use JMS, do I use Apache ActiveMQ or JBoss Messaging? Are they both compatible to run on both platforms (Tomcat and JBoss)?
Thanks in advance.
For communicating between applications, JMS is a very good solution, especially for events and notifications. JMS allows for such notifications to be sent and received using what is known as asynchronous messaging whereby the sender and receiver have no knowledge of one another and no requirement to be available at the same time.
ActiveMQ is a very widely used message broker that provides client APIs for Java, C/C++, C#, Perl, PHP, Python, Ruby and more. This allows the use of JMS with applications written in Java and other languages.
I have implemented JMS messaging many, many times for a large variety of business situations to handle events and notifications. The vast majority of these times, I have recommended and/or used Spring JMS no matter what message broker is being used. Spring JMS is incredibly easy to use, extremely robust and highly scalable. Spring JMS removes the complexity of creating your own message producers and message consumers, which can save you a tremendous amount of time.
To see how easy it is to send messages using Spring JMS, check out a blog post I wrote recently titled Using the Spring JmsTemplate to Send JMS Messages. I'm also working on a blog post about receiving messages using Spring JMS.
If you have any further questions, let me know.
Bruce
I had a similar requirement once, and we used JMS. Then main problem was how to deal with errors because SMTP is indeed not transactional:
is it ok if some email are lost?
is it ok if some email are sent twice?
We decided it was better to send the message twice, and here is more or less the design we had:
We relied on container managed transaction and if for some reason the email can not be sent, we decided to rollback the JMS transaction; the message would be redelivered later by JMS and an new attempt to send the message was done.
If the JMS message delivery transaction failed after the email was sent (e.g. because of a problem with JMS), the transaction would be rolled back automatically and the message was redelivered later. In this case, the email was sent twice because STMP is not transactional.
Even if the email can be sent (from point of view of code), the SMTP server can still have problem later. In this case, the JMS have been delivered and consumed, so we had no way to know which email had been processed and how to re-send them manually.
But we were already using JMS. I would not introduce JMS just for that given that the main argument is that JMS is transactional, but SMTP isn't anyway.
I would go for something lighter -- possibly with a ThreadPool -- and store the state in a database to know which email need to be sent or has been sent. If there are some problem, you can look at the database and take the ad-hoc decisions.
I know that this reply is very late to this dicsussion, but I hope it will still be valuable for folks seeking info on integrating ActiveMQ and Tomcat.
I've had many people ask me for help with issues they have had integrating ActiveMQ and Tomcat so I decided to write some articles about it. Not only is this topic covered in ActiveMQ in Action (see chapter 8), but I also published a series of articles on it titled ActiveMQ and Tomcat: Perfect Partners. Hopefully people will find this helpful.
I would go for a persistent JMS (I have used only WLS JMS and Websphere MQ so can't compare AQ vs JBoss, whichever offers a better guarantee for delivery). Also, I would seriously consider making the email engine a completely separate application, depending on how much you expect the traffic to grow.