I haven't touched any J2EE stuff in years and I need to whip up a quick JMS client for a demo.
I'm using Eclipse, on OS X and I can't even get started because I can't seem to figure out how to get the required libraries.
This is supposed to be a simple stand alone application (not running in a container) that pulls messages from a topic.
Every JMS implementation has its own set of libraries that specify how you get the initial connection factory. If you have an existing server from which to pull messages, you need to examine the documentation of that server to determine where to find the libraries to put in your classpath and how to create your initial connection factory. If you want to create a server for the purposes of the demonstration, I recommend using an embedded Active MQ broker.
Once you have your connection factory, polling for messages from a topic is pretty straightforward. Here is some example code which can be called to drain a topic of its current messages.
// Implementation specific code
public void drainTopic(TopicConnectionFactory factory, String topicName, String subscriberId)
// set factory properties like the server ID
Connection conn = factory.createConnection();
conn.start();
Session session = conn.createSession(false, Session.AUTO_ACKNOWLEDGE);
Topic topic = session.createTopic(topicName);
MessageConsumer consumer = session.createDurableSubscriber(topic, subscriberId);
Message message;
while (null != (message = consumer.receive(1000))) {
// do work on the message
}
conn.close();
}
Note the use of a durable subscriber. This means that I don't have to try to maintain a single connection all the time and handle the errors if it times out somehow. But because the subscription is durable, the server knows to retain any messages the topic receives while I'm not connected and provide them on my next connection. This code would be the same regardless of the host OS. The only tricky part is the creation of the provider specific connection factory.
Related
I'm so stuck about this.
I want to subscribe an ActiveMQ topic. ActiveMQ works on Centos machine, NOT LOCALHOST. I can consume messages with tcp, http protocols. Code;
public static void main(String[] args) throws JMSException {
PropertyUtils.loadPropertyFile();
Properties receiverProperties = PropertyUtils.getreceiverProperties();
// URL of the JMS server
String url = (String) receiverProperties.get("receiver.connection.url");
// Name of the queue we will receive messages from
String subject = (String) receiverProperties.get("receiver.topic.name");
// Getting JMS connection from the server
ConnectionFactory connectionFactory = new ActiveMQConnectionFactory(url);
Connection connection = connectionFactory.createConnection();
connection.start();
// Creating session for getting messages
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Getting the topic
Destination destination = session.createTopic(subject);
// MessageConsumer is used for receiving (consuming) messages
MessageConsumer consumer = session.createConsumer(destination);
Message message = null;
// Here we receive the message.
while (true) {
message = consumer.receive();
if (message instanceof TextMessage) {
TextMessage textMessage = (TextMessage) message;
System.out.println("Received message '" + textMessage.getText() + "'");
}
}
// We will be using TestMessage in our example. MessageProducer sent us a
// TextMessage
// so we must cast to it to get access to its .getText() method.
// connection.close();
}
I want to use wss protocol. This is a must for me. When I changed url with wss://host:port getting;
Could not create Transport. Reason: java.io.IOException: createTransport() method not implemented!
So I checked the alternatives. People figure this out with Stomp over WS. My first achievement is wss connection.
Any recommendation will be appreciated!
The exception you're seeing is expected because the OpenWire JMS client you're using doesn't support WebSocket connections, and it doesn't really need to. WebSocket connections are really only relevant for clients running in a limited environment like a web browser. Web browsers don't support running Java clients.
If you really want to use STOMP over WebSockets then you'll have to use a STOMP client implementation that supports WebSockets (most do).
Keep in mind that ActiveMQ is a broker. It does not supply clients for all the protocols it supports. It only provides a JMS client because it is required in order to implement JMS. STOMP, for example, is a standardized protocol and anyone can implement a client which will work with any broker implementing STOMP. There are lots of STOMP client implementations available written in many different languages for many different platforms. Any good search engine should help you find one to fit your needs.
So, i used concurrency in spring jms 50-100, allowing max connections upto 200. Everything is working as expected but if i try to retrieve 100k messages from queue, i mean there are 100k messages on my sqs and i reading them through the spring jms normal approach.
#JmsListener
Public void process (String message) {
count++;
Println (count);
//code
}
I am seeing all the logs in my console but after around 17k it starts throwing exceptions
Something like : aws sdk exception : port already in use.
Why do i see this exception and how do. I get rid of it?
I tried looking on the internet for it. Couldn't find anything.
My setting :
Concurrency 50-100
Set messages per task :50
Client acknowledged
timestamp=10:27:57.183, level=WARN , logger=c.a.s.j.SQSMessageConsumerPrefetch, message={ConsumerPrefetchThread-30} Encountered exception during receive in ConsumerPrefetch thread,
javax.jms.JMSException: AmazonClientException: receiveMessage.
at com.amazon.sqs.javamessaging.AmazonSQSMessagingClientWrapper.handleException(AmazonSQSMessagingClientWrapper.java:422)
at com.amazon.sqs.javamessaging.AmazonSQSMessagingClientWrapper.receiveMessage(AmazonSQSMessagingClientWrapper.java:339)
at com.amazon.sqs.javamessaging.SQSMessageConsumerPrefetch.getMessages(SQSMessageConsumerPrefetch.java:248)
at com.amazon.sqs.javamessaging.SQSMessageConsumerPrefetch.run(SQSMessageConsumerPrefetch.java:207)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.SdkClientException: Unable to execute HTTP request: Address already in use: connect
Update : i looked for the problem and it seems that new sockets are being created until every sockets gets exhausted.
My spring jms version would be 4.3.10
To replicate this problem just do the above configuration with the max connection as 200 and currency set to 50-100 and push some 40k messages to the sqs queue.. One can use https://github.com/adamw/elasticmq this as a local stack server which replicates Amazon sqs.. After being done till here. Comment jms listener and use soap ui load testing and call the send message to fire many messages. Just because you commented #jmslistener annotation, it won't consume messages from queue. Once you see that you have sent 40k messages, stop. Uncomment #jmslistener and restart the server.
Update :
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setDestinationResolver(new DynamicDestinationResolver());
factory.setErrorHandler(Throwable::printStackTrace);
factory.setConcurrency("50-100");
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return factory;
Update :
SQSConnectionFactory connectionFactory = new SQSConnectionFactory( new ProviderConfiguration(), amazonSQSclient);
Update :
Client configuration details :
Protocol : HTTP
Max connections : 200
Update :
I used cache connection factory class and it seems. I read on stack overflow and in their official documentation to not use cache connection factory class and default jms listener container factory.
https://stackoverflow.com/a/21989895/5871514
It's gives the same error that i got before though.
update
My goal is to get a 500 tps, i.e i should be able to consume that much.. So i tried this method and it seems I can reach 100-200, but not more than that.. Plus this thing is a blocker at high concurrency .. If you use it.. If you have some better solution to achieve it.. I am all ears.
**updated **
I am using amazonsqsclient
Starvation on the Consumer
One possible optimization that JMS clients tend to implement, is a message consumption buffer or "prefetch". This buffer is sometimes tunable via the number of messages or by a buffer size in bytes.
The intention is to prevent the consumer from going to the server every single time it receives a messages, rather than pulling multiple messages in a batch.
In an environment where you have many "fast consumers" (which is the opinionated view these libraries may take), this prefetch is set to a somewhat high default in order to minimize these round trips.
However, in an environment with slow message consumers, this prefetch can be a problem. The slow consumer is holding up messaging consumption for those prefetched messages from the faster consumer. In a highly concurrent environment, this can cause starvation quickly.
That being the case the SQSConnectionFactory has a property for this:
SQSConnectionFactory sqsConnectionFactory = new SQSConnectionFactory( new ProviderConfiguration(), amazonSQSclient);
sqsConnectionFactory.setNumberOfMessagesToPrefetch(0);
Starvation on the Producer (i.e. via JmsTemplate)
It's very common for these JMS implementations to expect be interfaced to the broker via some intermediary. These intermediaries actually cache and reuse connections or use a pooling mechanism to reuse them. In the Java EE world, this is usually taken care of a JCA adapter or other method on a Java EE server.
Because of the way Spring JMS works, it expects an intermediary delegate for the ConnectionFactory to exist to do this caching/pooling. Otherwise, when Spring JMS wants to connect to the broker, it will attempt to open a new connection and session (!) every time you want to do something with the broker.
To solve this, Spring provides a few options. The simplest being the CachingConnectionFactory, which caches a single Connection, and allows many Sessions to be opened on that Connection. A simple way to add this to your #Configuration above would be something like:
#Bean
public ConnectionFactory connectionFactory(AmazonSQSClient amazonSQSclient) {
SQSConnectionFactory sqsConnectionFactory = new SQSConnectionFactory(new ProviderConfiguration(), amazonSQSclient);
// Doing the following is key!
CachingConnectionFactory connectionfactory = new CachingConnectionFactory();
connectionfactory.setTargetConnectionFactory(sqsConnectionFactory);
// Set the #connectionfactory properties to your liking here...
return connectionFactory;
}
If you want something more fancy as a JMS pooling solution (which will pool Connections and MessageProducers for you in addition to multiple Sessions), you can use the reasonably new PooledJMS project's JmsPoolConnectionFactory, or the like, from their library.
Currently i'm working on a standalone Java apps that connects to a Websphere MQ to send and receive messages.
The flow is in asynchronous mode, which we implemented using MessageListener class to retrieve the messages from the queue when they are ready. The code to initialize the consumer with the listener is as follow:
if(connection == null)
connection = getJmsConnection();
try {
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
if (isTopic) {
destination = session.createTopic(destinationName);
} else {
destination = session.createQueue(destinationName);
}
consumer = session.createConsumer(destination);
consumer.setMessageListener(listener);
} catch (JMSException e) {
e.printStackTrace();
}
The getJmsConnection() method will return a connection from a pool, implemented using Apache Commons Pool library.
My question is, will the connection assign to the listener from the pool be active and tied to that listener as long as the program is running? Or the connection is used intermittently and can be reuse by other processes? Our idea is to have the sending and receiving process to reuse the connection from the pool, but i'm not sure how the MessageListener deal with the connection they are assigned with.
Thank you.
The key object here is the session rather than the connection; the session is on the one that will be doing the primary work here with the message consumption (async or otherwise).
It is advisable to try and share out the connection as widely as possible. Temporary destinations are scoped on the connection level. So the use of pooling is a good idea; it will be perfectly possible to share that connection around.
However I would also say that it might be worth considering pooling the sessions. With the code here a new session will be created, each time through that code, that will mean a new connection to the WebSphere MQ queue manager will be created. It's not clear what the scope of that will be, but if that is closed quickly it could become a bottleneck.
I haven't been able to figure this one out from Google alone. I am connecting to a non-durable EMS topic, which publishes updates to a set of data. If I skip a few updates, it doesn't matter, as the following update will overwrite it anyway.
The number of messages being published on the EMS topic is quite high, and occasionally for whatever reason the consumer lags behind. Is there a way, on the client connection side, to determine a 'time to live' for messages? I know there is on other brokers, but specifically on Tibco I have been unable to figure out whether it's possible or not, only that this parameter can definitely be set on the server side for all clients (this is not an option for me).
I am creating my connection factory and then creating an Apache Camel jms endpoint with the following code:
TibjmsConnectionFactory connectionFactory = new TibjmsConnectionFactory();
connectionFactory.setServerUrl(properties.getProperty(endpoints.getServerUrl()));
connectionFactory.setUserName(properties.getProperty(endpoints.getUsername()));
connectionFactory.setUserPassword(properties.getProperty(endpoints.getPassword()));
JmsComponent emsComponent = JmsComponent.jmsComponent(connectionFactory);
emsComponent.setAsyncConsumer(true);
emsComponent.setConcurrentConsumers(Integer.parseInt(properties.getProperty("jms.concurrent.consumers")));
emsComponent.setDeliveryPersistent(false);
emsComponent.setClientId("MyClient." + ManagementFactory.getRuntimeMXBean().getName() + "." + emsConnectionNumber.getAndIncrement());
return emsComponent;
I am using tibjms-6.0.1, tibjmsufo-6.0.1, and various other tib***-6.0.1.
The JMSExpiration property can be set per message or, more globally, at the destination level (in which case the JMSExpiration of all messages received in this destination is overridden). It cannot be set per consumer.
One option would be to create a bridge from the topic to a custom queue that only your consumer application will listen to, and set the "expiration" property of this queue to 0 (unlimited). All messages published on the topic will then be copied to this queue and won't ever expire, whatever their JMSExpiration value.
I am having a problem about JMS. The problem is, I have an application and it is trying to send a message through JMS , but after JMS server restart, it throws exception as when the server was down time. It is not reconnecting.
It is completely fine without a restart of JMS server and I am using weblogic 10.x.
Is it a problem about JMS configuration?
Thanks
Can you post your code how you are sending messages?
If you attempt to send a message when the JMS server is down, you will probably have an exception, and have to deal with that. When you attempt to send a message the next time, when the JMS server is restarted and running, you probably create a new connection from your connection factory. Reconnection will happend then:
// Let's say, you inject CF and Dest.
#Resource(lookup = "jms/ConnectionFactory")
private static ConnectionFactory cf;
#Resource(lookup = "jms/MyQueue")
private static Destination dest;
public void sendMessage(){ // called every time you need to send a message.
try{
Connection con = cf.createConnection(); // will reconnect, otherwise pooled.
Session sess = con.createSession(false,Session.AUTO_ACKNOWLEDGE);
MessageProducer prod = sess.createProducer(dest);
Message msg = sess.createTextMessage("Hello, World");
prod.send(msg);
sess.close();
con.close();
}catch(Exception e){
// handle errors
}
}
However, there are some built in failover/reconnecting features in Weblogic JMS, take a look at this page:
Oracle code listing and documentation about reconnecting JMS producers