Develop Reference

MQ Message in the Request and Reply store in Database

Based on multiple analysis through google I created a Java based MQ JMS client . Basically I am new to MQ and
got few doubts whether the code which I created will work properly for the below Request and Reply.
Request and Reply message:
REQUEST(SERVICE,10,CREATE_TEST,MSGID,15,FGD024049364194,TESTID,4,
USMQ,SRID,8,#MSTD,EMPID,5,8104,LOC,4,QR,AT-RP,4,QR,RTR,
7,2624931,UVT-ORD-SYS,4,CHAT,UVT-REQ,1,S,UVT-ORD,9,QT0046259,VTRD-2)RETURN();
REPLY(MSGID,15,FGD024049364194,DESTID,4,TRMQ,EMPID,5,8104,LOC,4,RTR,VTCT,0,
,UVT-DELCMNT,0,,UVT-DEL-REA,0,,UVT-DLVRY-FLG,1,N,UVT-DLVRY-STUS,1,
10,CREATE_TEST,TKT-NBR,7,2624931,USERID,8,#AMSATD)
MESSAGE(INFO,TEST-GROUP,5,PS,INFO,UVTTS,49,+00 INVALID/NORMAL,VICE,62,
00000 UPDATE SUCCESSFUL 3734931,INFO,STSUTITMEOUT,60,+0000 INVALID/OUT OF WORLD.);
My requirement is store the above Request message in a table in the oracle database and I want to
read the message from the table and put in the Queue which will interact the other system(third party).
The other systeme will reply the message as above and I need to store the reply message along with the Message Id in the same read table.
Please clarify My doubt and correct me what i need to change in the code in case wrong:
1) In the Request there is a MSG Id availbale and also in the Reply there is a messge ID. How it will works in my scenario
I read in some site "like a MessgeID is automatically generated for you, and you can't change that behaviour".
so in my scenario as in the below code what message id will get.Is it correct?
2) When I read the message do I get complete above Reply message or only the Message mentioned in Reply.
Please clarify my above doubts:
code for Write method:
public void write(List<Createbean> createbeanList) throws MQException
{
try {
MQQueueManager qMgr = new MQQueueManager(qManager, env);
// Set up the options on the queue we wish to open
int openOptions = MQC.MQOO_INPUT_AS_Q_DEF | MQC.MQOO_OUTPUT;
// Now specify the queue that we wish to open and the open options
log.info("Accessing queue: "+qName);
MQQueue queue = qMgr.accessQueue(qName, openOptions);
// Define a simple WebSphere MQ Message ...
MQMessage msg = new MQMessage();
msg.format = MQC.MQFMT_STRING;
msg.format = MQC.MQFMT_STRING;
msg.feedback = MQC.MQFB_NONE;
msg.messageType = MQC.MQMT_DATAGRAM;
for (Createbean createBean : createbeanList) {
String createMessage =createBean.getMessage();
msg.writeString(createMessage);
}
// Specify the default put message options
MQPutMessageOptions pmo = new MQPutMessageOptions();
// Put the message to the queue
queue.put(msg, pmo);
// Close the queue
queue.close();
// Disconnect from the QueueManager
// logger.debug(CLASS, methodName, "Disconnecting from the Queue
// Manager");
qMgr.disconnect();
// logger.debug(CLASS, methodName, "Done!");
Read method:
private void read() throws MQException
{
int openOptions = MQC.MQOO_FAIL_IF_QUIESCING | MQC.MQOO_INPUT_SHARED | MQC.MQOO_BROWSE;
MQQueue queue = _queueManager.accessQueue( inputQName,
openOptions,
null, // default q manager
null, // no dynamic q name
null ); // no alternate user id
log.info("MQRead v1.0 connected.\n");
int depth = queue.getCurrentDepth();
log.info("Current depth: " + depth + "\n");
if (depth == 0)
{
return;
}
MQGetMessageOptions getOptions = new MQGetMessageOptions();
//getOptions.options = MQC.MQGMO_NO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING + MQC.MQGMO_CONVERT;
getOptions.options=MQC.MQGMO_WAIT | MQC.MQGMO_BROWSE_FIRST;
getOptions.matchOptions=MQC.MQMO_NONE;
getOptions.waitInterval=5000;
long messageCount = 0;
boolean thereAreMessages=true;
while(thereAreMessages)
{
if(messageCount >0){
MQMessage message = new MQMessage();
try
{
message.messageId = MQC.MQMI_NONE;
queue.get(message, getOptions);
log.info(" MsgId : ");
String messageID= dumpHexId(message.messageId);
String msg = message.readString(message.getMessageLength());
log.info("Browsed message: " + msg);
log.info("Actually get message?");
byte[] b = new byte[message.getMessageLength()];
message.readFully(b);
createDAO rmdao = new createDAO();
rmdao.updateCreate(new String(b),messageID);
log.info(new String(b));
message.clearMessage();
/************************************************/
/* Reset the options to browse the next message */
/************************************************/
getOptions.options= MQC.MQGMO_WAIT | MQC.MQGMO_BROWSE_NEXT;
}
Thanks in advance

Get ClusterName of MQ Queue using Java

I'm building a java application that connects to a MQQueueManager and extracts information about queues. I'm able to get data like QueueType, MaximumMessageLength and more. However, I also want the name of the cluster the queue might be in. There is no function that comes with the MQQueue that gives me this information. After searching the internet I found several things pointing in this direction, but no examples.
A part of my function that gives me the MaximumDepth is:
queueManager = makeConnection(host, portNo, qMgr, channelName);
queue = queueManager.accessQueue(queueName, CMQC.MQOO_INQUIRE);
maxQueueDepth = queue.getMaximumDepth();
(makeConnection is not shown here, it is the function that makes the actual connection to the QueueManager; I also left out the try/catch/finally for less clutter)
How do I get ClusterName and perhaps other data, that doesn't have a function like queue.getMaximumDepth()?

There are two ways to get information about a queue.
The API Inquire call gets operational status of a queue. This includes things like the name the MQOpen call resolved to or the depth if the queue is local. Much of the q.inquire functionality has been superseded with getter and setter functions on the queue. If you are not using the v8.0 client with the latest functionality, you are highly advised to upgrade. It can access all versions of QMgr.
The following code is from Getting and setting attribute values in WebSphere MQ classes for Java
// inquire on a queue
final static int MQIA_DEF_PRIORITY = 6;
final static int MQCA_Q_DESC = 2013;
final static int MQ_Q_DESC_LENGTH = 64;
int[] selectors = new int[2];
int[] intAttrs = new int[1];
byte[] charAttrs = new byte[MQ_Q_DESC_LENGTH]
selectors[0] = MQIA_DEF_PRIORITY;
selectors[1] = MQCA_Q_DESC;
queue.inquire(selectors,intAttrs,charAttrs);
System.out.println("Default Priority = " + intAttrs[0]);
System.out.println("Description : " + new String(charAttrs,0));
For things that are not part of the API Inquire call, a PCF command is needed. Programmable Command Format, commonly abbreviated as PCF, is a message format used to pass messages to the command queue and for reading messages from the command queue, event queues and others.
To use a PCF command the calling application must be authorized with +put on SYSTEM.ADMIN.COMMAND.QUEUE and for +dsp on the object being inquired upon.
IBM provides sample code.
On Windows, please see: %MQ_FILE_PATH%\Tools\pcf\samples
In UNIX flavors, please see: /opt/mqm/samp/pcf/samples
The locations may vary depending on where MQ was installed.
Please see: Handling PCF messages with IBM MQ classes for Java. The following snippet is from the PCF_DisplayActiveLocalQueues.java sample program.
public static void DisplayActiveLocalQueues(PCF_CommonMethods pcfCM) throws PCFException,
MQDataException, IOException {
// Create the PCF message type for the inquire.
PCFMessage pcfCmd = new PCFMessage(MQConstants.MQCMD_INQUIRE_Q);
// Add the inquire rules.
// Queue name = wildcard.
pcfCmd.addParameter(MQConstants.MQCA_Q_NAME, "*");
// Queue type = LOCAL.
pcfCmd.addParameter(MQConstants.MQIA_Q_TYPE, MQConstants.MQQT_LOCAL);
// Queue depth filter = "WHERE depth > 0".
pcfCmd.addFilterParameter(MQConstants.MQIA_CURRENT_Q_DEPTH, MQConstants.MQCFOP_GREATER, 0);
// Execute the command. The returned object is an array of PCF messages.
PCFMessage[] pcfResponse = pcfCM.agent.send(pcfCmd);
// For each returned message, extract the message from the array and display the
// required information.
System.out.println("+-----+------------------------------------------------+-----+");
System.out.println("|Index| Queue Name |Depth|");
System.out.println("+-----+------------------------------------------------+-----+");
for (int index = 0; index < pcfResponse.length; index++) {
PCFMessage response = pcfResponse[index];
System.out.println("|"
+ (index + pcfCM.padding).substring(0, 5)
+ "|"
+ (response.getParameterValue(MQConstants.MQCA_Q_NAME) + pcfCM.padding).substring(0, 48)
+ "|"
+ (response.getParameterValue(MQConstants.MQIA_CURRENT_Q_DEPTH) + pcfCM.padding)
.substring(0, 5) + "|");
}
System.out.println("+-----+------------------------------------------------+-----+");
return;
}
}

After more research I finally found what I was looking for.
This example of IBM: Getting and setting attribute values in WebSphere MQ classes helped me to set up the inquiry.
The necessary values I found in this list: Constant Field Values.
I also needed to expand the openOptionsArg of accessQueue(), else cluster queues cannot be inquired.
Final result:
(without makeConnection())
public class QueueManagerServices {
final static int MQOO_INQUIRE_TOTAL = CMQC.MQOO_FAIL_IF_QUIESCING | CMQC.MQOO_INPUT_SHARED | CMQC.MQOO_INQUIRE;
MQQueueManager queueManager = null;
String cluster = null;
MQQueue queue = null;
public String getcluster(String host, int portNo, String qMgr, String channelName){
try{
queueManager = makeConnection(host, portNo, qMgr, channelName);
queue = queueManager.accessQueue(queueName, MQOO_INQUIRE_TOTAL);
int MQCA_CLUSTER_NAME = 2029;
int MQ_CLUSTER_NAME_LENGTH = 48;
int[] selectors = new int[1];
int[] intAttrs = new int[1];
byte[] charAttrs = new byte[MQ_CLUSTER_NAME_LENGTH];
selectors[0] = MQCA_CLUSTER_NAME;
queue.inquire(selectors, intAttrs, charAttrs);
cluster = new String (charAttrs);
} catch (MQException e) {
System.out.println(e);
} finally {
if (queue != null){
queue.close();
}
if (queueManager != null){
queueManager.disconnect();
}
}
return cluster;
}
}

java websphere MQ

My aim is to put n number of messages in a for loop to a WebSphere MQ queue using WebSphere MQ java programming.
My java program will run as a standalone program.
If any exception in between , I need to rollback all the messages.
If no exception then I should commit all the messages .
The outside world should not see my messages in the queue until I complete fully.
How do I achieve this?
Updated with sample code as per reply from T.Rob:
Please check if sample code is fine ?
Does setting MQGMO_SYNCPOINT is only related to my program's invocation ?
(because similar programs running parallely will also be putting messages on the same queue and those messages should not gett affected by my program's SYNCPOINT.)
public void sendMsg() {
MQQueue queue = null;
MQQueueManager queueManager = null;
MQMessage mqMessage = null;
MQPutMessageOptions pmo = null;
System.out.println("Entering..");
try {
MQEnvironment.hostname = "x.x.x.x";
MQEnvironment.channel = "xxx.SVRCONN";
MQEnvironment.port = 9999;
queueManager = new MQQueueManager("XXXQMANAGER");
int openOptions = MQConstants.MQOO_OUTPUT;
queue = queueManager.accessQueue("XXX_QUEUENAME", openOptions, null, null, null);
pmo = new MQPutMessageOptions();
pmo.options = CMQC.MQGMO_SYNCPOINT;
String input = "testing";
System.out.println("sending messages....");
for (int i = 0; i < 10; i++) {
input = input + ": " + i;
mqMessage = new MQMessage();
mqMessage.writeString(input);
System.out.println("Putting message: " + i);
queue.put(mqMessage, pmo);
}
queueManager.commit();
System.out.println("Exiting..");
} catch (Exception e) {
e.printStackTrace();
try {
System.out.println("rolling back messages");
if (queueManager != null)
queueManager.backout();
} catch (MQException e1) {
e1.printStackTrace();
}
} finally {
try {
if (queue != null)
queue.close();
if (queueManager != null)
queueManager.close();
} catch (MQException e) {
e.printStackTrace();
}
}
}

WMQ supports both local and global (XA) units of work. The local units of work are available simply by specifying the option. Global XA transactions require a transaction manager, as mentioned by keithkreissl in another answer.
For what you described, a POJO doing messaging under syncpoint, specify MQC.MQGMO_SYNCPOINT in your MQGetMessageOptions. When you are ready to commit, issue the MQQManager.commit() or MQQManager.backout() call.
Note that the response and doc provided by ggrandes refers to the JMS and not Java classes. The Java classes use Java equivalents of the WMQ procedural API, can support many threads (doc) and even provide connection pooling (doc). Please refer to the Java documentation rather than the JMS documentation for the correct behavior. Also, I've linked to the WMQ V7.5 documentation which goes with the latest WMQ Java V7.5 client. The later clients have a lot more local functionality (tracing, flexible install path, MQClient.ini, etc.) and work with back-level QMgrs. It is highly recommended to be using the latest client and the download is free.

you only need to create a session with transaction enabled.
Session session;
// ...
boolean transacted = true;
session = connection.createSession(transacted, Session.AUTO_ACKNOWLEDGE);
try {
// ...do things...
session.commit();
} catch (Exception e) {
session.rollback();
}
// ...
WARN-NOTE: Sessions are not thread-safe ;-)
Doc Websphere MQ/JMS

If you have access to a transaction manager and more importantly an XATransaction wired up to your MQ access, you can start a transaction at the beginning of your message processing put all the messages on the queue then commit the transaction. Using the XATransactions it will not put any messages until the transaction commits. If you don't have access to that, you can do a little more plumbing by placing your messages in a local data object, wrap your code in a try/catch if no exceptions iterate through the local data object sending the messages. The issue with the later approach is that it will commit all your other processing but if a problem occurs in the sending of messages your other processing will not be rolled back.

Remove a JMS message from MQ Queue using JMSMessageID

Is there a way to remove a JMS message from an IBM MQ Queue using JMSMessageId ina Java application(not using tools)? Also are such operations vendor-specific?
Looked through the API for receive operations which are used to remove messages, but for removing specific messages, do we need to filter using MessageSelector and remove appropriately, or is there a more simple way? [checking for any available method which can be directly used]
Can you please provide tutorials/examples [can be links too] to show the API usage for such operations?

When you use JMSMessageID as the only message property in a selector, WMQ optimizes the lookup to be the same as a native WMQ API get by MQMD.MessageID which is an indexed field in the queue. Please see the JMS Message Selection topic for more details.
QueueReceiver rcvr = sess.createReceiver(inputQ, "JMSCorrelationID = '"+msgId+"'")
You can also do the same thing using native WMQ API calls using Java native code. You would do a normal GET operation but specify the message ID in the MQMD structure.
myMsg.messageId = someMsgID;
MQGetMessageOptions gmo = new MQGetMessageOptions();
myQueue.get(myMsg, gmo);

How to delete specific message form queue by using messageid?
I also have like your problem, I provide the resuable function. You just need to pass MessageId and Queue name. It is ok for me.
private void deleteMessage(String messageId, String queueName) {
try {
JMXServiceURL url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi");
JMXConnector jmxc = JMXConnectorFactory.connect(url);
MBeanServerConnection conn = jmxc.getMBeanServerConnection();
ObjectName name = new ObjectName("org.apache.activemq:type=Broker,brokerName=localhost");
BrokerViewMBean proxy = (BrokerViewMBean)MBeanServerInvocationHandler.newProxyInstance(conn, name, BrokerViewMBean.class, true);
for (ObjectName queue : proxy.getQueues()) {
QueueViewMBean queueBean = (QueueViewMBean) MBeanServerInvocationHandler.newProxyInstance(conn, queue, QueueViewMBean.class, true);
if(queueBean.getName().equals(queueName)) {
System.out.println("Deleted : " + messageId);
queueBean.removeMessage(messageId);
return;
}
}
} catch(Exception e) {
e.printStackTrace();
}
}
I use activemq-all-5.8.0.jar.

JavaMail IMAP over SSL quite slow - Bulk fetching multiple messages

I am currently trying to use JavaMail to get emails from IMAP servers (Gmail and others). Basically, my code works: I indeed can get the headers, body contents and so on. My problem is the following: when working on an IMAP server (no SSL), it basically takes 1-2ms to process a message. When I go on an IMAPS server (hence with SSL, such as Gmail) I reach around 250m/message. I ONLY measure the time when processing the messages (the connection, handshake and such are NOT taken into account).
I know that since this is SSL, the data is encrypted. However, the time for decryption should not be that important, should it?
I have tried setting a higher ServerCacheSize value, a higher connectionpoolsize, but am seriously running out of ideas. Anyone confronted with this problem? Solved it one might hope?
My fear is that the JavaMail API uses a different connection each time it fetches a mail from the IMAPS server (involving the overhead for handshake...). If so, is there a way to override this behavior?
Here is my code (although quite standard) called from the Main() class:
public static int connectTest(String SSL, String user, String pwd, String host) throws IOException,
ProtocolException,
GeneralSecurityException {
Properties props = System.getProperties();
props.setProperty("mail.store.protocol", SSL);
props.setProperty("mail.imaps.ssl.trust", host);
props.setProperty("mail.imaps.connectionpoolsize", "10");
try {
Session session = Session.getDefaultInstance(props, null);
// session.setDebug(true);
Store store = session.getStore(SSL);
store.connect(host, user, pwd);
Folder inbox = store.getFolder("INBOX");
inbox.open(Folder.READ_ONLY);
int numMess = inbox.getMessageCount();
Message[] messages = inbox.getMessages();
for (Message m : messages) {
m.getAllHeaders();
m.getContent();
}
inbox.close(false);
store.close();
return numMess;
} catch (MessagingException e) {
e.printStackTrace();
System.exit(2);
}
return 0;
}
Thanks in advance.

after a lot of work, and assistance from the people at JavaMail, the source of this "slowness" is from the FETCH behavior in the API. Indeed, as pjaol said, we return to the server each time we need info (a header, or message content) for a message.
If FetchProfile allows us to bulk fetch header information, or flags, for many messages, getting contents of multiple messages is NOT directly possible.
Luckily, we can write our own IMAP command to avoid this "limitation" (it was done this way to avoid out of memory errors: fetching every mail in memory in one command can be quite heavy).
Here is my code:
import com.sun.mail.iap.Argument;
import com.sun.mail.iap.ProtocolException;
import com.sun.mail.iap.Response;
import com.sun.mail.imap.IMAPFolder;
import com.sun.mail.imap.protocol.BODY;
import com.sun.mail.imap.protocol.FetchResponse;
import com.sun.mail.imap.protocol.IMAPProtocol;
import com.sun.mail.imap.protocol.UID;
public class CustomProtocolCommand implements IMAPFolder.ProtocolCommand {
/** Index on server of first mail to fetch **/
int start;
/** Index on server of last mail to fetch **/
int end;
public CustomProtocolCommand(int start, int end) {
this.start = start;
this.end = end;
}
#Override
public Object doCommand(IMAPProtocol protocol) throws ProtocolException {
Argument args = new Argument();
args.writeString(Integer.toString(start) + ":" + Integer.toString(end));
args.writeString("BODY[]");
Response[] r = protocol.command("FETCH", args);
Response response = r[r.length - 1];
if (response.isOK()) {
Properties props = new Properties();
props.setProperty("mail.store.protocol", "imap");
props.setProperty("mail.mime.base64.ignoreerrors", "true");
props.setProperty("mail.imap.partialfetch", "false");
props.setProperty("mail.imaps.partialfetch", "false");
Session session = Session.getInstance(props, null);
FetchResponse fetch;
BODY body;
MimeMessage mm;
ByteArrayInputStream is = null;
// last response is only result summary: not contents
for (int i = 0; i < r.length - 1; i++) {
if (r[i] instanceof IMAPResponse) {
fetch = (FetchResponse) r[i];
body = (BODY) fetch.getItem(0);
is = body.getByteArrayInputStream();
try {
mm = new MimeMessage(session, is);
Contents.getContents(mm, i);
} catch (MessagingException e) {
e.printStackTrace();
}
}
}
}
// dispatch remaining untagged responses
protocol.notifyResponseHandlers(r);
protocol.handleResult(response);
return "" + (r.length - 1);
}
}
the getContents(MimeMessage mm, int i) function is a classic function that recursively prints the contents of the message to a file (many examples available on the net).
To avoid out of memory errors, I simply set a maxDocs and maxSize limit (this has been done arbitrarily and can probably be improved!) used as follows:
public int efficientGetContents(IMAPFolder inbox, Message[] messages)
throws MessagingException {
FetchProfile fp = new FetchProfile();
fp.add(FetchProfile.Item.FLAGS);
fp.add(FetchProfile.Item.ENVELOPE);
inbox.fetch(messages, fp);
int index = 0;
int nbMessages = messages.length;
final int maxDoc = 5000;
final long maxSize = 100000000; // 100Mo
// Message numbers limit to fetch
int start;
int end;
while (index < nbMessages) {
start = messages[index].getMessageNumber();
int docs = 0;
int totalSize = 0;
boolean noskip = true; // There are no jumps in the message numbers
// list
boolean notend = true;
// Until we reach one of the limits
while (docs < maxDoc && totalSize < maxSize && noskip && notend) {
docs++;
totalSize += messages[index].getSize();
index++;
if (notend = (index < nbMessages)) {
noskip = (messages[index - 1].getMessageNumber() + 1 == messages[index]
.getMessageNumber());
}
}
end = messages[index - 1].getMessageNumber();
inbox.doCommand(new CustomProtocolCommand(start, end));
System.out.println("Fetching contents for " + start + ":" + end);
System.out.println("Size fetched = " + (totalSize / 1000000)
+ " Mo");
}
return nbMessages;
}
Do not that here I am using message numbers, which is unstable (these change if messages are erased from the server). A better method would be to use UIDs! Then you would change the command from FETCH to UID FETCH.
Hope this helps out!

You need to add a FetchProfile to the inbox before you iterate through the messages.
Message is a lazy loading object, it will return to the server for each message and for each
field that doesn't get provided with the default profile.
e.g.
for (Message message: messages) {
message.getSubject(); //-> goes to the imap server to fetch the subject line
}
If you want to display like an inbox listing of say just From, Subject, Sent, Attachement etc.. you would use something like the following
inbox.open(Folder.READ_ONLY);
Message[] messages = inbox.getMessages(start + 1, total);
FetchProfile fp = new FetchProfile();
fp.add(FetchProfile.Item.ENVELOPE);
fp.add(FetchProfileItem.FLAGS);
fp.add(FetchProfileItem.CONTENT_INFO);
fp.add("X-mailer");
inbox.fetch(messages, fp); // Load the profile of the messages in 1 fetch.
for (Message message: messages) {
message.getSubject(); //Subject is already local, no additional fetch required
}
Hope that helps.

The total time includes the time required in cryptographic operations. The cryptographic operations need a random seeder. There are different random seeding implementations which provide random bits for use in the cryptography. By default, Java uses /dev/urandom and this is specified in your java.security as below:
securerandom.source=file:/dev/urandom
On Windows, java uses Microsoft CryptoAPI seed functionality which usually has no problems. However, on unix and linux, Java, by default uses /dev/random for random seeding. And read operations on /dev/random sometimes block and takes long time to complete. If you are using the *nix platforms then the time spent in this would get counted in the overall time.
Since, I dont know what platform you are using, I can't for sure say that this could be your problem. But if you are, then this could be one of reasons why your operations are taking long time. One of the solution to this could be to use /dev/urandom instead of /dev/random as your random seeder, which does not block. This can be specified with the system property "java.security.egd". For example,
-Djava.security.egd=file:/dev/urandom
Specifying this system property will override the securerandom.source setting in your java.security file. You can give it a try. Hope it helps.