Develop Reference

make mail mark as unread after polling - Apache Camel

Am using apache camel, With Polling consumer, when poll my mail is mark as read.
options : delete=false&peek=false&unseen=true
After polling , when i am processing the attachment, if any error occurs , i want to make the mail as "unread". So that i can pool again later.
public void process(Exchange exchange) throws Exception {
Map<String, DataHandler> attachments = exchange.getIn().getAttachments();
Message messageCopy = exchange.getIn().copy();
if (messageCopy.getAttachments().size() > 0) {
for (Map.Entry<String, DataHandler> entry : messageCopy.getAttachments().entrySet()) {
DataHandler dHandler = entry.getValue();
// get the file name
String filename = dHandler.getName();
// get the content and convert it to byte[]
byte[] data =
exchange.getContext().getTypeConverter().convertTo(byte[].class, dHandler.getInputStream());
log.info("Downloading attachment, file name : " + filename);
InputStream fileInputStream = new ByteArrayInputStream(data);
try {
// Processing attachments
// if any error occurs here, i want to make the mail mark as unread
} catch (Exception e) {
log.info(e.getMessage());
}
}
}
}
I noticed the option peek, by setting it to true, It will not make the mail mark as read during polling, in that case is there any option to make it mark as read after processing.

To get the result that you want you should have options
peek=true&unseen=true
The peek=true option is supposed to ensure that messages remain the exact state on the mail server as they where before polling even if there is an exception. However, currently it won't work. This is actually a bug in Camel Mail component. I've submitted a patch to https://issues.apache.org/jira/browse/CAMEL-9106 and this will probably be fixed in a future release.
As a workaround you can set mapMailMessages=false but then you will have to work with the email message content yourself. In Camel 2.15 onward you also have postProcessAction option and with that you could probably remove the SEEN flags from messages with processing errors. Still, I would recommend waiting for the fix though.

We can set the mail unread flag with the following code
public void process(Exchange exchange) throws Exception {
final Message mailMessage = exchange.getIn(MailMessage.class).getMessage();
mailMessage.setFlag(Flag.SEEN, false);
}

Getting error "Can't read input file" on production server but not locally

I make a POST to a request with a File included in the request body.
In my method I retrieve this File
if(request.body.file("imageFile").getOrElse(null) != null) {
request.body.file("imageFile").map{ case FilePart(key, name, contentType, content) =>
try{
val in:InputStream = new BufferedInputStream(new ByteArrayInputStream(content))
image = ImageIO.read(in)
} catch {
case e => Logger.debug(e.printStackTrace.toString); throw new Exception(e.getMessage)
}
}
}
If a File is included in the request body it tries to get it, else it just tries to get a file from S3.
else {
try{
val in:InputStream = new BufferedInputStream(new ByteArrayInputStream(S3Storage.retrieveS3File("facebook.jpg").content))
image = ImageIO.read(in)
} catch {
case e:IOException => Logger.debug("Failed to retrieve facebook image"); throw new IOException(e.getMessage)
}
All this works fine when I run it on my computer, but when I check in this and test it on the amazon server the image = ImageIO.read(in) gives me an error; Can't read input file!.
For me this makes no sense since the file is either in the request body or it's grabbed from a S3 bucket.
I've debugged this code and in the production environment there is a file available there when the "read" is done.
Why cannot the file be read from the production environment?
regards

One suggestion would be not to swallow the original exception and stack trace.
Use constructor new Exception(message, catchedException) in your catch blocks.

How can I send and receive messages on quickfix?

I have created two classes: Initiator and Acceptor. I want to send messages from the initiator to the acceptor and then process the received messages. I can't send message.
This is my initiator.java
SocketInitiator socketInitiator = null;
String fileName = "conf/intiator.cfg";
try {
SessionSettings initiatorSettings = new SessionSettings(new FileInputStream(fileName));
Application initiatorApplication = new Initiator();
FileStoreFactory fileStoreFactory = new FileStoreFactory(
initiatorSettings);
FileLogFactory fileLogFactory = new FileLogFactory(
initiatorSettings);
MessageFactory messageFactory = new DefaultMessageFactory();
socketInitiator = new SocketInitiator(initiatorApplication, fileStoreFactory, initiatorSettings, fileLogFactory, messageFactory);
socketInitiator.start();
Message msg = new Message();
msg.setString(1, "Hello this is test Message");
SessionID sessionId = (SessionID) socketInitiator.getSessions().get(0);
Session.lookupSession(sessionId).logon();
initiatorApplication.onLogon(sessionId);
initiatorApplication.toApp(msg, sessionId);
} catch (Exception e) {
e.printStackTrace();
}
Here is its overRide message of Application Interface.
public void toApp(Message message, SessionID sessionId) throws DoNotSend {
try {
boolean result = quickfix.Session.sendToTarget(message, sessionId);
if (result) {
System.out.println("Message ahs send :)");
} else {
System.out.println("Not Send");
}
} catch (Exception e) {
e.printStackTrace();
}
}
this is initiator initiator.cfg file
[default]
StartTime=00:00:01
EndTime=23:59:59
HeartBtInt=10
SocketUseSSL=N
MillisecondsInTimeStamp=Y
FileIncludeMilliseconds=Y
CheckLatency=N
SocketTcpNoDelay=Y
[session]
BeginString=FIX.4.4
ConnectionType=initiator
DisableQuickFixReconnLogic=Y
AdapterUserIndex=0
SocketConnectHost=127.0.0.1
Timezone=America/New_York
SocketConnectPort=3000
UseDataDictionary=Y
DataDictionary=conf/resources/FIX44.xml
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
LogonTimeout=10
FileStorePath=conf/connector
FileLogPath=conf/connector/logs
FileLogBackupPath=conf/connector
ResetOnLogout=Y
ResetOnDisconnect=N
SendResetSeqNumFlag=Y
RawData=fxall123
#SessionQualifier=FXallStream
MillisecondsInTimeStamp=Y
FileIncludeMilliseconds=Y
[session]
BeginString=FIX.4.4
ConnectionType=initiator
DisableQuickFixReconnLogic=Y
AdapterUserIndex=1
SenderCompID=initiator-id
#SenderSubID=trader1
#TargetCompID=target-id
#TargetSubID=qftrade
SocketConnectHost=127.0.0.1
Timezone=America/New_York
#SocketConnectPort=443
SocketConnectPort=3000
UseDataDictionary=Y
DataDictionary=conf/resources/FIX44.xml
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
LogonTimeout=5
FileStorePath=conf/connector
FileLogPath=conf/connector
FileLogBackupPath=conf/connector/backup
ResetOnLogout=Y
ResetOnLogon=Y
ResetOnDisconnect=N
SendResetSeqNumFlag=Y
RawData=fxall123
#SessionQualifier=FXallTrade
Acceptor.java
String fileName = "conf/acceptor.cfg";
SocketAcceptor socketAcceptor = null;
try {
FileInputStream is = new FileInputStream(fileName);
SessionSettings executorSettings = new SessionSettings(is);
Application application = new Acceptor();
FileStoreFactory fileStoreFactory = new FileStoreFactory(
executorSettings);
MessageFactory messageFactory = new DefaultMessageFactory();
FileLogFactory fileLogFactory = new FileLogFactory(executorSettings);
socketAcceptor = new SocketAcceptor(application, fileStoreFactory,
executorSettings, fileLogFactory, messageFactory);
socketAcceptor.start();
SessionID sessionId = (SessionID) socketAcceptor.getSessions().get(0);
application.onLogon(sessionId);
int[] i = {1, 2, 3, 4, 5};
// application.fromApp(new Message(i), sessionId);
} catch (Exception e) {
e.printStackTrace();
}
acceptor.cfg
[default]
StartTime=00:00:00
EndTime=23:50:00
HeartBtInt=10
ReconnectInterval=6
SocketUseSSL=N
MillisecondsInTimeStamp=Y
CheckLatency=N
SocketTcpNoDelay=N
SocketAcceptAddress=127.0.0.1
SocketAcceptPort=3000
[session]
BeginString=FIX.4.4
ConnectionType=acceptor
#DisableQuickFixReconnLogic=Y
AdapterUserIndex=0
SenderCompID=target-id
#SenderSubID=qfstream
#TargetCompID=inttest
#TargetSubID=trader1
Timezone=America/New_York
UseDataDictionary=Y
DataDictionary=conf/resources/FIX44.xml
ValidateFieldsOutOfOrder=N
ValidateFieldsHaveValues=N
ValidateUserDefinedFields=N
LogonTimeout=5
FileStorePath=conf/Acceptor
FileLogPath=conf/Acceptor/logs
ResetOnLogout=Y
ResetOnDisconnect=N
SendResetSeqNumFlag=Y
Can anyone tell me where is the problem. Either in configuration or in code?

I will just point out the obvious (there is a lot wrong with this code, see the examples to understand how quickfixj works). The Message that you are trying to send is not a valid FIX message and so will be rejected by the engine before it is sent. You will need to create a real FIX message for quickfix to send it. Just as importantly I need to mention that toApp is an event handler that gets called when a message is sent. If you put a call to quickfix.Session.sendToTarget in it it will then call toApp again when it gets sent. Since there is no control statement in your toApp this results in an infinite loop. The configuration is also slightly wrong, and other bits of code look hinkey, but those are your biggest problems. Please look at the documentation and examples.

you need to make changes in xml file. I guess a field that you are using is not declared in fix.4.4 file. Check your xml file and include that in following way.
i-e you want to send QuoteID with Currency, but in Currency QuoteID is not declared. In that case you will not be able to send message.
you need to decleared that field(QuoteID) in fix.4.4/or any version you are using.
<Currency>
//
//
<field name="QuoteID" required="N" /> // N or Y depend on your requirement
//
//
</currency>
also check log, You will find error message there.
May be you are not including require fields that can also create problem.
use Following link to check which fields are required.
http://www.fixprotocol.org/FIXimate3.0/

javamail mark gmail message as read

Note: added after answer:
Thanks.. Yeah I had tried the Flag.SEEN to true and saveChanges.. I also had read getContent marks it read. I tried using it in the for statement that loops through the messages. But I got the messages again from the folder anyways in the next loop. I was assuming the folder was live, so grabbing the content, then grabbing the messages again from the folder with the filter to not get any seen should work, but I was still getting the same message. I could try closing the folder and reopen as a test to see if it's marked. Also if I go over to my client and click the message, then my code stops seeing it even in the loop, so I was hoping to do the same in the code.
original:
I'm using javamail to get email from a gmail account, it's working great, when I get the message I'd like to mark it as read, can anyone give me some direction? Here is my current code:
Properties props = System.getProperties();
props.setProperty("mail.store.protocol", "imaps");
try {
Session session = Session.getDefaultInstance(props, null);
Store store = session.getStore("imaps");
store.connect("imap.gmail.com", eUserName, ePassWord);
// Get folder
Folder folder = store.getFolder("INBOX");
if (folder == null || !folder.exists()) {
return null;
}
folder.open(Folder.READ_ONLY);
// Only pull unread
FlagTerm ft = new FlagTerm(new Flags(Flags.Flag.SEEN), false);
Message messages[]; // = folder.search(ft);
for(int x = 0; x < timeOutInSeconds; x++) {
log.reportMessage("looking for emails");
try {
folder.getMessages();
messages = folder.search(ft);
if (messages.length > 0) {
for (Message message : messages) {
//log.reportMessage("found message: should not see again, marking read");
// want to mark as read
}
}
Thread.sleep(1000);
}
catch(Exception ex) {
}
}
// Close connection
folder.close(false);
store.close();
return null;
}
catch (NoSuchProviderException ex) {
return null;
}
catch (MessagingException ex) {
return null;
}
}

First of all, you can't mark a message as read if you are using a POP3 server - the POP3 protocol doesn't support that. However, the IMAP v4 protocol does.
You might think the way to do this is to get the message, set the Flags.Flag.SEEN flag to true, and then call message.saveChanges(). Oddly, this is not the case.
Instead, the JavaMail API Design Specification, Chapter 4, section "The Flags Class" states that the SEEN flag is implicitly set when the contents of a message are retrieved. So, to mark a message as read, you can use the following code:
myImapFolder.open(Folder.READ_WRITE);
myImapFolder.getMessage(myMsgID).getContent();
myImapFolder.close(false);
Or another way is to use the MimeMessage copy constructor, ie:
MimeMessage source = (MimeMessage) folder.getMessage(1)
MimeMessage copy = new MimeMessage(source);
When you construct the copy, the seen flag is implicitly set for the message referred to by source.

One liner that will do it WITHOUT downloading the entire message:
single message:
folder.setFlags(new Message[] {message}, new Flags(Flags.Flag.SEEN), true);
all messages:
folder.setFlags(messages, new Flags(Flags.Flag.SEEN), true);
Other methods of calling getContent() or creating a copy with new MimeMessage(original) cause the client to download the entire message, and creates a huge performance hit.
Note that the inbox must be opened for READ_WRITE:
folder.open(Folder.READ_WRITE);

Well this post is old but the easiest solution hasn´t been posted yet.
You are accessing the Message.
message.setFlag(Flag.SEEN, true);

for (Message message : messages) {
message.setFlag(Flags.Flag.SEEN,true);
}
and change the below line
folder.open(Folder.READ_ONLY);
to this
folder.open(Folder.READ_WRITE);

You may also consider having a public static int max_message_number, and storing in it the message[i].getMessageNumber(); as soon as you read a message. Then before reading any message just check if the max_message_number < message[i].getmessageNumber(). If true then don't print this message (as it has been already read)

message.setFlag( Flag.SEEN,true ) give "cannot find symbol"
message.setFlag( Flags.Flag.SEEN,true ) seems good.

If you are using a for loop to read or check a mail one by one, the code can be as follows to mark a gmail message as read:
Message[] unreadMessages = inbox.search(new FlagTerm(new Flags(Flag.SEEN), false));
for (int q = 0; q < unreadMessages.length; q++) {
unreadMessages[q].setFlag(Flag.SEEN, true);
}
What this code does is that it makes it unread one by one.
And also folder/inbox needs to be READ_WRITE, instead of READ_ONLY:
folder.open(Folder.READ_WRITE);

You can also try
head over to the gmail settings > Forwarding and POP/IMAP
from the dropdown of When messages are accessed with POP
select mark Gmail's copy as read and save the changes

To mark the mail as read, you just have to call the mailmessage.getContent() method.
Whether you use IMAP or POP, calling that method on a specific mail mark it as Read

The easiest way to do that is set the folder to be read or written into or from. Means like this...
Folder inbox = null;
inbox.open(Folder.READ_WRITE);
the Folder class should be imported.

Emailing Large Attachments

Are there any packages available that will efficiently send emails with large attachments (each file is capped at 10mb, but could include multiple files). If not, any suggestions on an appropriate design that wouldn't result in out of memory exceptions causing issues across applications deployed on the same server?
Files are delivered to the application server by ftp. Once transmission is complete, a web service is invoked (metadata for the transaction). Based on business rules, this service may need need to email the files.
My initial thoughts were a putting the request on a message queue (so the service can return immediately), and having a synchronized method process the request (so multiple requests at or around the same time won't blow up the heap).
updating with code
messageBodyPart = new MimeBodyPart();
FileDataSource fileDataSource =new FileDataSource("locationTo.big.file");
messageBodyPart.setDataHandler(new DataHandler(fileDataSource));
messageBodyPart.setFileName("big.file");
multipart.addBodyPart(messageBodyPart);
<rinse..repeat>
message.setContent(multipart);
Transport.send(msg);
If I attach 5 10mb attachments, 50mb won't be eaten up by the heap all at once?

Why not use an Executor, with a thread pool growing/shrinking within reason. Each task submitted is a Runnable or Callable. The Task sends via JavaMail, which DOES not take much memory if you implement your own DataSource implementations for the attachments and/or message body. (I am assuming you have have InputStream acccess to the attachments)
Adding code as sample (note this code was written many years ago, and is pretty bad for many reasons. But it shows the concept)
public static void sendMailAndThrowException(SMTPParams sparams,String subject, DataSource msgTextSource,DataSource[] fids,boolean debug) throws MessagingException {
Session session=getMailSession(sparams);
PrintStream f = null;
if (debug) {
f= getPrintStream();
}
// null is System.out by javamail api
session.setDebug(debug);
session.setDebugOut(f);
try
{
// create a message
MimeMessage msg = new MimeMessage(session);
msg.setFrom(new InternetAddress(sparams.getFrom()));
// Recipients are comma delimitted
String to_list[] = sparams.getRecipients().split(",");
InternetAddress[] address = new InternetAddress[to_list.length];
for( int i=0; i< to_list.length; i++)
{
// MJB: remove extraneous spaces, sanity check
String temp = to_list[i].trim();
if (temp.length()>0) {
address[i] = new InternetAddress(to_list[i].trim());
}
}
// Addresses are always TO, never CC or BCC in this library
msg.setRecipients(Message.RecipientType.TO, address);
if ((msg.getAllRecipients() == null) || (msg.getAllRecipients().length==0)) {
throw new MessagingException("No valid recipients");
}
// Set the subject
msg.setSubject(subject,"UTF-8");
// create the Multipart and add its parts to it
Multipart mp = new MimeMultipart();
if (msgTextSource != null) {
// create and fill the first message part
MimeBodyPart mbp1 = new MimeBodyPart();
mbp1.setDataHandler(new DataHandler(msgTextSource));
mp.addBodyPart(mbp1);
}
if( fids != null)
{
for (int i=0;i<fids.length;i++) {
// create the second message part
if (fids[i]==null) continue;
MimeBodyPart mbp2 = new MimeBodyPart();
// attach the file to the message
mbp2.setDataHandler(new DataHandler(fids[i]));
mbp2.setFileName(fids[i].getName());
mp.addBodyPart(mbp2);
}
}
// add the Multipart to the message
msg.setContent(mp);
// set the Date: header
msg.setSentDate(new java.util.Date());
// Connect to SMTP server
smtpSend(session, msg, sparams);
}
catch (MessagingException mex)
{
throw mex;
} finally {
closeDebug(f);
}
}

JavaMail allows easily for sending mails with large attachments if given enough RAM to hold the various pieces.
Any particular reason you cannot use that?