Develop Reference

Java Mail IMAP taking lot of time to getting messages in mail box, 25s to get 20 messages

There is a way to improve the performance for getting messages from the Javamail IMAP server?
i found some tips, but couldn't improve the performance.
This method will get about 20 messages from a folder, and returning the formatted messages to a DTO, I tried to comment the lines to get attachments, and getting the recipients, but it doensnt change anything in the performance.
/**
* Get all the emails that are inside the param folder
*
* #param folderId folder to return the emails from the informed folder.
* #param pageToken pageToken to be used as pagination
* #param maxResults maxResults to create a limit to the number of registers
* #return {#link EmailsDTO} with all the messages
*/
public EmailsDTO getMessages(final String folderId, final String pageToken, final Integer maxResults) {
final Properties properties = getServerInputProperties();
final Session session = Session.getDefaultInstance(properties);
try {
final EmailsDTO emailsDTO = new EmailsDTO();
final List<EmailDTO> emailDTOList = new ArrayList<>();
// connects to the message store
final Store store = session.getStore(IMAP);
store.connect(email, password);
// opens the folder to search the messages
final Folder folder = store.getFolder(folderId);
folder.open(Folder.READ_ONLY);
//set a pagination used to get the results in JavaMail
final int pageStart = nonNull(pageToken) ? Integer.valueOf(pageToken) : 1;
final int pageEnd = (pageStart + maxResults) > folder.getMessageCount() ? folder.getMessageCount() : pageStart + maxResults;
// fetches new messages from server, starts with 1
final Message[] messages = folder.getMessages(pageStart, pageEnd);
final FetchProfile fetchProfile = new FetchProfile();
fetchProfile.add(FetchProfile.Item.ENVELOPE);
folder.fetch(messages, fetchProfile); // Load the profile of the messages in 1 fetch.
for (final Message msg : messages) {
final EmailDTO emailDTO = new EmailDTO();
emailDTO.setId(msg.getHeader(MESSAGE_ID)[0]);
emailDTO.setCc(this.parseAddresses(msg.getRecipients(Message.RecipientType.CC)));
emailDTO.setTo(this.parseAddresses(msg.getRecipients(Message.RecipientType.TO)));
emailDTO.setBcc(this.parseAddresses(msg.getRecipients(Message.RecipientType.BCC)));
emailDTO.setFrom(this.getFrom(msg.getFrom()));
emailDTO.setRead(msg.getFlags().contains(Flags.Flag.SEEN));
emailDTO.setHasAttachments(this.hasAttachments(msg));
emailDTO.setSubject(msg.getSubject());
emailDTO.setMessage(this.getMessageContent(msg));
emailDTO.setAttachments(this.getAttachements(msg));
emailDTO.setDate(msg.getReceivedDate().toString());
emailDTO.setParentFolder(msg.getFolder().getName());
//TODO: see what need to put here emailDTO.setLabels();
emailDTOList.add(emailDTO);
}
// disconnect
folder.close(false);
store.close();
emailsDTO.setEstimatedNumberOfMessages(folder.getMessageCount());
emailsDTO.setNextPageToken(String.valueOf(emailDTOList.size()));
emailsDTO.setMessages(emailDTOList);
return emailsDTO;
} catch (final MessagingException | IOException e) {
throw new RuntimeException(String.format("couldn't get messages from the folder with id: %s", folderId), e);
}
}

In for (final Message msg : messages), change messages to folder.getMessages().
messages -> loads multiple times
folder.getMessages() -> loads in a single fetch, because you have used fetchProfile in this.

CRM Dynamics Online 2016 Azure SDK for Java receive BrokeredMessages with 500 error code

I have a problem with CRM Dynamics Online 2016 Azure SDK for Java.
I can connect to Azure Service Bus, I can see queues and message count in queues, but cannot receive messages. Message Id is null and message body contain 500 error
500The server was unable to process the
request; please retry the operation. If the problem persists, please
contact your Service Bus administrator and provide the tracking id.
TrackingId:acf8a543-33c9-486d-b13b-443823e6c394_G9,TimeStamp:4/13/2016
7:26:22 AM. If the problem persists, please contact
your Service Bus administrator and provide the tracking id.
TrackingId:acf8a543-33
Is there any working sample on the Internet to solve the problem?
Test code:
#Test
public void readAllExistedMessagesFromAllQueue() {
try {
ServiceBusContract serviceBusContract = ServiceBusConfiguration.configureWithConnectionString(null, Configuration.load(), ASB_CONNECTION_STRING).create(ServiceBusContract.class);
ReceiveMessageOptions opts = ReceiveMessageOptions.DEFAULT;
opts.setReceiveMode(ReceiveMode.PEEK_LOCK);
ListQueuesResult result = serviceBusContract.listQueues();
if (result != null && result.getItems().size() > 0) {
for (QueueInfo queueInfo : result.getItems()) {
logger.debug("queu: " + queueInfo.getPath() + " MessageCount: " + queueInfo.getMessageCount());
for (int i = 0; i < result.getItems().size(); i++) {
BrokeredMessage message = serviceBusContract.receiveQueueMessage(queueInfo.getPath(),
opts).getValue();
if (message == null) {
continue;
}
System.out.print("__________________________________________");
System.out.println("MessageID: " + message.getMessageId());
System.out.print("From queue: ");
byte[] b = new byte[200];
String s = null;
int numRead = message.getBody().read(b);
while (-1 != numRead) {
s = new String(b);
s = s.trim();
System.out.print(s);
numRead = message.getBody().read(b);
}
System.out.println();
}
}
}
} catch (IOException e) {
logger.error(e);
} catch (ServiceException e) {
logger.error(e);
}
}

Per my experience, according to the source code and javadocs of Service Bus, the ServiceBusContract is a Java interface, that you can't directly create an instance of the interface ServiceBusContract.
So please try to use the code below from the section Create a queue of the document "How to use Service Bus queues".
Configuration config =
ServiceBusConfiguration.configureWithSASAuthentication(
"<your-servicebus-namespace>",
"RootManageSharedAccessKey",
"<SAS-key-value>",
".servicebus.windows.net"
);
ServiceBusContract serviceBusContract = ServiceBusService.create(config);
Update
You can find the SharedAccessKeyName & SharedAccessKey in the connection string via click the button below at the bottom of your service bus page.
Then, show the view below and copy the CONNECTION STRING.
The connection string like this below.
Endpoint=sb://<your-servicebus-namespace>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<SAS-key-value>
Please copy the correct part of connection string instead of the related part of the code.

Get ClusterName of MQ Queue using Java

I'm building a java application that connects to a MQQueueManager and extracts information about queues. I'm able to get data like QueueType, MaximumMessageLength and more. However, I also want the name of the cluster the queue might be in. There is no function that comes with the MQQueue that gives me this information. After searching the internet I found several things pointing in this direction, but no examples.
A part of my function that gives me the MaximumDepth is:
queueManager = makeConnection(host, portNo, qMgr, channelName);
queue = queueManager.accessQueue(queueName, CMQC.MQOO_INQUIRE);
maxQueueDepth = queue.getMaximumDepth();
(makeConnection is not shown here, it is the function that makes the actual connection to the QueueManager; I also left out the try/catch/finally for less clutter)
How do I get ClusterName and perhaps other data, that doesn't have a function like queue.getMaximumDepth()?

There are two ways to get information about a queue.
The API Inquire call gets operational status of a queue. This includes things like the name the MQOpen call resolved to or the depth if the queue is local. Much of the q.inquire functionality has been superseded with getter and setter functions on the queue. If you are not using the v8.0 client with the latest functionality, you are highly advised to upgrade. It can access all versions of QMgr.
The following code is from Getting and setting attribute values in WebSphere MQ classes for Java
// inquire on a queue
final static int MQIA_DEF_PRIORITY = 6;
final static int MQCA_Q_DESC = 2013;
final static int MQ_Q_DESC_LENGTH = 64;
int[] selectors = new int[2];
int[] intAttrs = new int[1];
byte[] charAttrs = new byte[MQ_Q_DESC_LENGTH]
selectors[0] = MQIA_DEF_PRIORITY;
selectors[1] = MQCA_Q_DESC;
queue.inquire(selectors,intAttrs,charAttrs);
System.out.println("Default Priority = " + intAttrs[0]);
System.out.println("Description : " + new String(charAttrs,0));
For things that are not part of the API Inquire call, a PCF command is needed. Programmable Command Format, commonly abbreviated as PCF, is a message format used to pass messages to the command queue and for reading messages from the command queue, event queues and others.
To use a PCF command the calling application must be authorized with +put on SYSTEM.ADMIN.COMMAND.QUEUE and for +dsp on the object being inquired upon.
IBM provides sample code.
On Windows, please see: %MQ_FILE_PATH%\Tools\pcf\samples
In UNIX flavors, please see: /opt/mqm/samp/pcf/samples
The locations may vary depending on where MQ was installed.
Please see: Handling PCF messages with IBM MQ classes for Java. The following snippet is from the PCF_DisplayActiveLocalQueues.java sample program.
public static void DisplayActiveLocalQueues(PCF_CommonMethods pcfCM) throws PCFException,
MQDataException, IOException {
// Create the PCF message type for the inquire.
PCFMessage pcfCmd = new PCFMessage(MQConstants.MQCMD_INQUIRE_Q);
// Add the inquire rules.
// Queue name = wildcard.
pcfCmd.addParameter(MQConstants.MQCA_Q_NAME, "*");
// Queue type = LOCAL.
pcfCmd.addParameter(MQConstants.MQIA_Q_TYPE, MQConstants.MQQT_LOCAL);
// Queue depth filter = "WHERE depth > 0".
pcfCmd.addFilterParameter(MQConstants.MQIA_CURRENT_Q_DEPTH, MQConstants.MQCFOP_GREATER, 0);
// Execute the command. The returned object is an array of PCF messages.
PCFMessage[] pcfResponse = pcfCM.agent.send(pcfCmd);
// For each returned message, extract the message from the array and display the
// required information.
System.out.println("+-----+------------------------------------------------+-----+");
System.out.println("|Index| Queue Name |Depth|");
System.out.println("+-----+------------------------------------------------+-----+");
for (int index = 0; index < pcfResponse.length; index++) {
PCFMessage response = pcfResponse[index];
System.out.println("|"
+ (index + pcfCM.padding).substring(0, 5)
+ "|"
+ (response.getParameterValue(MQConstants.MQCA_Q_NAME) + pcfCM.padding).substring(0, 48)
+ "|"
+ (response.getParameterValue(MQConstants.MQIA_CURRENT_Q_DEPTH) + pcfCM.padding)
.substring(0, 5) + "|");
}
System.out.println("+-----+------------------------------------------------+-----+");
return;
}
}

After more research I finally found what I was looking for.
This example of IBM: Getting and setting attribute values in WebSphere MQ classes helped me to set up the inquiry.
The necessary values I found in this list: Constant Field Values.
I also needed to expand the openOptionsArg of accessQueue(), else cluster queues cannot be inquired.
Final result:
(without makeConnection())
public class QueueManagerServices {
final static int MQOO_INQUIRE_TOTAL = CMQC.MQOO_FAIL_IF_QUIESCING | CMQC.MQOO_INPUT_SHARED | CMQC.MQOO_INQUIRE;
MQQueueManager queueManager = null;
String cluster = null;
MQQueue queue = null;
public String getcluster(String host, int portNo, String qMgr, String channelName){
try{
queueManager = makeConnection(host, portNo, qMgr, channelName);
queue = queueManager.accessQueue(queueName, MQOO_INQUIRE_TOTAL);
int MQCA_CLUSTER_NAME = 2029;
int MQ_CLUSTER_NAME_LENGTH = 48;
int[] selectors = new int[1];
int[] intAttrs = new int[1];
byte[] charAttrs = new byte[MQ_CLUSTER_NAME_LENGTH];
selectors[0] = MQCA_CLUSTER_NAME;
queue.inquire(selectors, intAttrs, charAttrs);
cluster = new String (charAttrs);
} catch (MQException e) {
System.out.println(e);
} finally {
if (queue != null){
queue.close();
}
if (queueManager != null){
queueManager.disconnect();
}
}
return cluster;
}
}

Delete top message from MQQueue

I am constructing a messaging system using MQSeries. For some reason, when I perform q.get(...), I am getting an exception thrown (I don't know the specific MQException). Below is the code causing the error:
private static MQGetMessageOptions GMO = new MQGetMessageOptions();
private static int GMO_OPTIONS = MQC.MQGMO_SYNCPOINT | MQC.MQGMO_WAIT;
GMO.options = GMO.options | GMO_OPTIONS;
GMO.waitInterval = MQC.MQWI_UNLIMITED;
MQEnvironment.hostname = args[0];
MQEnvironment.channel = args[2];
MQEnvironment.port = Integer.parseInt(args[1]);
MQQueueManager queueManager = new MQQueueManager(args[3])
MQMessage msg = new MQMessage();
MQQueue q = queueManager.accessQueue("qName1",MQC.MQOO_OUTPUT);
q.get(msg, GMO);
My plan is, when this error occurs, skip the message and delete it. To perform the delete I will call the following function:
private void deleteMsg(MQQueueManager queueManager, String queueName) throws MQException {
MQGetMessageOptions tempGmo = new MQGetMessageOptions();
tempGmo.options |= MQC.MQGMO_WAIT;
tempGmo.waitInterval = 1000;
MQQueue remover = queueManager.accessQueue(queueName, MQC.MQOO_INPUT_AS_Q_DEF);
remover.get(new MQMessage(), tempGmo);
queueManager.commit();
}
Would the remover.get() in my deleteMsg function also, in this specific scenario, fail for the same reason? Or does the option used to construct the MQQueue(MQC.MQOO_INPUT_AS_Q_DEF vs MQC.MQOO_OUTPUT) prevent it from also failing? If I am having trouble accessing my queue's message, how do I discard the top message and move to the next?
To shorten my question:
If I am unable to perform a get() on a given queue to retrieve a message, how can we delete that corrupt message on the same queue?
Thank you!

OMG!
MQQueue q = queueManager.accessQueue("qName1",MQC.MQOO_OUTPUT);
q.get(msg, GMO);
Your are opening a queue for output (writing) but you are trying to get a message. You have your shoes on the wrong feet!! Secondly, why aren't you catching the MQException that MQ would be throwing?? The exception would have included the reason code which would have given you the exact explanation to your issue.
Here's how you should be opening the queue for reading:
try
{
int oo = MQC.MQOO_INPUT_SHARED + MQC.MQOO_FAIL_IF_QUIESCING;
MQQueue q = queueManager.accessQueue("qName1",oo);
MQGetMessageOptions gmo = new MQGetMessageOptions();
gmo.options = MQC.MQGMO_NO_WAIT + MQC.MQGMO_FAIL_IF_QUIESCING;
q.get(msg, gmo);
}
catch (MQException e)
{
System.err.println(e.getLocalizedMessage() );
System.err.println("CC = " + e.completionCode + " - RC = " + e.reasonCode);
}
Also, make sure you use the appropriate "Fail if quiescing" option for the particular MQ API call.
Finally, look up "backout queue". If your application is having an issue with a message then the message should be moved to a backout queue and not simply deleted.

I do not why what you are doing does not work for you but I wonder why are you using proprietary API of MQ Series instead of using JMS API. In JMS terms remove top message just means receive the message, so call of session.receieve() does the work.
Using common JMS API has a lot of advantages. The main of them is that you can easily move from MQ Series to any other messaging solution without changing even one line of your code.

I wonder if the program compiled because there is no option called GMO_OPTIONS. All MQ constants are prefixed MQC

How can I send and receive messages on quickfix?

I have created two classes: Initiator and Acceptor. I want to send messages from the initiator to the acceptor and then process the received messages. I can't send message.
This is my initiator.java
SocketInitiator socketInitiator = null;
String fileName = "conf/intiator.cfg";
try {
SessionSettings initiatorSettings = new SessionSettings(new FileInputStream(fileName));
Application initiatorApplication = new Initiator();
FileStoreFactory fileStoreFactory = new FileStoreFactory(
initiatorSettings);
FileLogFactory fileLogFactory = new FileLogFactory(
initiatorSettings);
MessageFactory messageFactory = new DefaultMessageFactory();
socketInitiator = new SocketInitiator(initiatorApplication, fileStoreFactory, initiatorSettings, fileLogFactory, messageFactory);
socketInitiator.start();
Message msg = new Message();
msg.setString(1, "Hello this is test Message");
SessionID sessionId = (SessionID) socketInitiator.getSessions().get(0);
Session.lookupSession(sessionId).logon();
initiatorApplication.onLogon(sessionId);
initiatorApplication.toApp(msg, sessionId);
} catch (Exception e) {
e.printStackTrace();
}
Here is its overRide message of Application Interface.
public void toApp(Message message, SessionID sessionId) throws DoNotSend {
try {
boolean result = quickfix.Session.sendToTarget(message, sessionId);
if (result) {
System.out.println("Message ahs send :)");
} else {
System.out.println("Not Send");
}
} catch (Exception e) {
e.printStackTrace();
}
}
this is initiator initiator.cfg file
[default]
StartTime=00:00:01
EndTime=23:59:59
HeartBtInt=10
SocketUseSSL=N
MillisecondsInTimeStamp=Y
FileIncludeMilliseconds=Y
CheckLatency=N
SocketTcpNoDelay=Y
[session]
BeginString=FIX.4.4
ConnectionType=initiator
DisableQuickFixReconnLogic=Y
AdapterUserIndex=0
SocketConnectHost=127.0.0.1
Timezone=America/New_York
SocketConnectPort=3000
UseDataDictionary=Y
DataDictionary=conf/resources/FIX44.xml
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
LogonTimeout=10
FileStorePath=conf/connector
FileLogPath=conf/connector/logs
FileLogBackupPath=conf/connector
ResetOnLogout=Y
ResetOnDisconnect=N
SendResetSeqNumFlag=Y
RawData=fxall123
#SessionQualifier=FXallStream
MillisecondsInTimeStamp=Y
FileIncludeMilliseconds=Y
[session]
BeginString=FIX.4.4
ConnectionType=initiator
DisableQuickFixReconnLogic=Y
AdapterUserIndex=1
SenderCompID=initiator-id
#SenderSubID=trader1
#TargetCompID=target-id
#TargetSubID=qftrade
SocketConnectHost=127.0.0.1
Timezone=America/New_York
#SocketConnectPort=443
SocketConnectPort=3000
UseDataDictionary=Y
DataDictionary=conf/resources/FIX44.xml
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
LogonTimeout=5
FileStorePath=conf/connector
FileLogPath=conf/connector
FileLogBackupPath=conf/connector/backup
ResetOnLogout=Y
ResetOnLogon=Y
ResetOnDisconnect=N
SendResetSeqNumFlag=Y
RawData=fxall123
#SessionQualifier=FXallTrade
Acceptor.java
String fileName = "conf/acceptor.cfg";
SocketAcceptor socketAcceptor = null;
try {
FileInputStream is = new FileInputStream(fileName);
SessionSettings executorSettings = new SessionSettings(is);
Application application = new Acceptor();
FileStoreFactory fileStoreFactory = new FileStoreFactory(
executorSettings);
MessageFactory messageFactory = new DefaultMessageFactory();
FileLogFactory fileLogFactory = new FileLogFactory(executorSettings);
socketAcceptor = new SocketAcceptor(application, fileStoreFactory,
executorSettings, fileLogFactory, messageFactory);
socketAcceptor.start();
SessionID sessionId = (SessionID) socketAcceptor.getSessions().get(0);
application.onLogon(sessionId);
int[] i = {1, 2, 3, 4, 5};
// application.fromApp(new Message(i), sessionId);
} catch (Exception e) {
e.printStackTrace();
}
acceptor.cfg
[default]
StartTime=00:00:00
EndTime=23:50:00
HeartBtInt=10
ReconnectInterval=6
SocketUseSSL=N
MillisecondsInTimeStamp=Y
CheckLatency=N
SocketTcpNoDelay=N
SocketAcceptAddress=127.0.0.1
SocketAcceptPort=3000
[session]
BeginString=FIX.4.4
ConnectionType=acceptor
#DisableQuickFixReconnLogic=Y
AdapterUserIndex=0
SenderCompID=target-id
#SenderSubID=qfstream
#TargetCompID=inttest
#TargetSubID=trader1
Timezone=America/New_York
UseDataDictionary=Y
DataDictionary=conf/resources/FIX44.xml
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
LogonTimeout=5
FileStorePath=conf/Acceptor
FileLogPath=conf/Acceptor/logs
ResetOnLogout=Y
ResetOnDisconnect=N
SendResetSeqNumFlag=Y
Can anyone tell me where is the problem. Either in configuration or in code?

I will just point out the obvious (there is a lot wrong with this code, see the examples to understand how quickfixj works). The Message that you are trying to send is not a valid FIX message and so will be rejected by the engine before it is sent. You will need to create a real FIX message for quickfix to send it. Just as importantly I need to mention that toApp is an event handler that gets called when a message is sent. If you put a call to quickfix.Session.sendToTarget in it it will then call toApp again when it gets sent. Since there is no control statement in your toApp this results in an infinite loop. The configuration is also slightly wrong, and other bits of code look hinkey, but those are your biggest problems. Please look at the documentation and examples.

you need to make changes in xml file. I guess a field that you are using is not declared in fix.4.4 file. Check your xml file and include that in following way.
i-e you want to send QuoteID with Currency, but in Currency QuoteID is not declared. In that case you will not be able to send message.
you need to decleared that field(QuoteID) in fix.4.4/or any version you are using.
<Currency>
//
//
<field name="QuoteID" required="N" /> // N or Y depend on your requirement
//
//
</currency>
also check log, You will find error message there.
May be you are not including require fields that can also create problem.
use Following link to check which fields are required.
http://www.fixprotocol.org/FIXimate3.0/