Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I am using the following code to download messages from Gmail.
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.Date;
import java.util.Properties;
import javax.mail.Address;
import javax.mail.BodyPart;
import javax.mail.FetchProfile;
import javax.mail.Flags;
import javax.mail.Folder;
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Multipart;
import javax.mail.Part;
import javax.mail.Session;
import javax.mail.Store;
import javax.mail.search.FlagTerm;
import org.openqa.selenium.WebDriver;
public class MailReader {
WebDriver driver;
Folder inbox;
String m;
String gmailID = "xyz#gmail.com";
String gmailPass = "xyz";
String storeMessage;
public MailReader()
{
}
public String readMail() {
System.out.println("Inside readMail()...");
final String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
/* Set the mail properties */
Properties props = System.getProperties();
// Set manual Properties
props.setProperty("mail.pop3.socketFactory.class", SSL_FACTORY);
props.setProperty("mail.pop3.socketFactory.fallback", "false");
props.setProperty("mail.pop3.port", "995");
props.setProperty("mail.pop3.socketFactory.port", "995");
props.put("mail.pop3.host", "pop.gmail.com");
try
{
/* Create the session and get the store for read the mail. */
Session session = Session.getDefaultInstance(
System.getProperties(), null);
Store store = session.getStore("pop3");
store.connect("pop.gmail.com", 995, gmailID,
gmailPass);
/* Mention the folder name which you want to read. */
// inbox = store.getDefaultFolder();
// inbox = inbox.getFolder("INBOX");
inbox = store.getFolder("INBOX");
/* Open the inbox using store. */
inbox.open(Folder.READ_ONLY);
/* Get the messages which is unread in the Inbox */
Message messages[] = inbox.search(new FlagTerm(new Flags(
Flags.Flag.SEEN), false));
System.out.println("No. of Unread Messages : " + messages.length);
/* Use a suitable FetchProfile */
FetchProfile fp = new FetchProfile();
fp.add(FetchProfile.Item.ENVELOPE);
fp.add(FetchProfile.Item.CONTENT_INFO);
inbox.fetch(messages, fp);
try
{
m = printAllMessages(messages);
inbox.close(true);
store.close();
}
catch (Exception ex)
{
System.out.println("Exception arise at the time of read mail");
ex.printStackTrace();
}
}
catch (MessagingException e)
{
System.out.println("Exception while connecting to server: "
+ e.getLocalizedMessage());
e.printStackTrace();
System.exit(2);
}
return m;
}
public String printAllMessages(Message[] msgs) throws Exception
{
String s = null;
for (int i = 0; i < msgs.length; i++)
{
//System.out.println("MESSAGE #" + (i + 1) + ":");
s = printEnvelope(msgs[i]);
}
return s;
}
public String printEnvelope(Message message) throws Exception
{
Address[] a;
// FROM
if ((a = message.getFrom()) != null) {
for (int j = 0; j < a.length; j++) {
System.out.println("FROM: " + a[j].toString());
}
}
// TO
if ((a = message.getRecipients(Message.RecipientType.TO)) != null) {
for (int j = 0; j < a.length; j++) {
System.out.println("TO: " + a[j].toString());
}
}
String subject = message.getSubject();
Date receivedDate = message.getReceivedDate();
Date sentDate = message.getSentDate(); // receivedDate is returning
// null. So used getSentDate()
String content = message.getContent().toString();
System.out.println("Subject : " + subject);
if (receivedDate != null) {
System.out.println("Received Date : " + receivedDate.toString());
}
System.out.println("Sent Date : " + sentDate.toString());
System.out.println("Content : " + content);
return(getContent(message));
}
public String getContent(Message msg)
{
try {
String contentType = msg.getContentType();
System.out.println("Content Type : " + contentType);
Multipart mp = (Multipart) msg.getContent();
BodyPart bp = mp.getBodyPart(0);
int count = mp.getCount();
for (int i = 0; i < count; i++) {
String s = getText(mp.getBodyPart(i));
if(i == 1) {
return s;
}
//dumpPart(mp.getBodyPar((i));
}
} catch (Exception ex) {
System.out.println("Exception arise at get Content");
ex.printStackTrace();
}
return m;
}
/*
public void dumpPart(Part p) throws Exception {
// Dump input stream ..
InputStream is = p.getInputStream();
// If "is" is not already buffered, wrap a BufferedInputStream
// around it.
if (!(is instanceof BufferedInputStream)) {
is = new BufferedInputStream(is);
}
int c;
System.out.println("Message : ");
while ((c = is.read()) != -1) {
System.out.write(c);
}
}*/
boolean textIsHtml = false;
/**
* Return the primary text content of the message.
*/
public String getText(Part p) throws
MessagingException, IOException {
if (p.isMimeType("text/*")) {
String s = (String)p.getContent();
textIsHtml = p.isMimeType("text/html");
return s;
}
if (p.isMimeType("multipart/alternative")) {
// prefer html text over plain text
Multipart mp = (Multipart)p.getContent();
String text = null;
for (int i = 0; i < mp.getCount(); i++) {
Part bp = mp.getBodyPart(i);
if (bp.isMimeType("text/plain")) {
if (text == null)
text = getText(bp);
continue;
} else if (bp.isMimeType("text/html")) {
String s = getText(bp);
if (s != null)
return s;
} else {
return getText(bp);
}
}
return text;
} else if (p.isMimeType("multipart/*")) {
Multipart mp = (Multipart)p.getContent();
for (int i = 0; i < mp.getCount(); i++) {
String s = getText(mp.getBodyPart(i));
if (s != null)
return s;
}
}
return null;
}
}
I want to modify the code so that:
It only downloads the most recent 10 unread messages
It only downloads messages which are sent from a specific address (For example, only messages sent from myemail#example.com
How can this be done?
Thanks!
The 10 most recent messages are the last 10 messages in your INBOX.
But some of them could be read, and some of them could be deleted. To find the 10 most recent unread messages you'll need to use a FlagTerm to search for messages where the SEEN flag is false. You might want to use an AndTerm to find messages where the DELETED flag is also false.
Note that Folder.search doesn't download any of the messages, it just tells you which messages match. You can then look at the last 10 of those messages and do whatever you need to do to "download" them.
Hopefully that's enough of a hint to get you started. If you still can't get it to work, show us what code you're using and what results you're getting.
Related
I am having an issue with downloading csv file attachments from Gmail. All the settings/preferences seem to work fine. The plugin will download most of the file without issue, in fact the program doesn't produce any errors but it is cutting off the last line of the file and partially from the second to last line.
Can anyone see an issue in my code that would point to why this is happening?
Edit: Updated code to reflect for loop changes.
Edit2: Is there an overflow or buffer size I need to be worried about? The size of the files I will be downloading aren't very big. They are csv files with at least 140 lines in them.
public void searchEmailByID (int emailID) {
Properties properties = new Properties();
//server setting
properties.put("mail.imap.host", GmailHost);
properties.put("mail.imap.port", GmailPort);
//SSL setting
properties.setProperty("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
properties.setProperty("mail.imap.socketFactory.fallback", "false");
properties.setProperty("mail.imap.socketFactory.port", String.valueOf(GmailPort));
Session session = Session.getDefaultInstance(properties);
try {
//connects to message store
Store store = session.getStore("imap");
store.connect(GmailUser, GmailPassword);
//opens the inbox folder
Folder folderInbox = store.getFolder("Inbox");
folderInbox.open(Folder.READ_ONLY);
//creates the search criteria
SearchTerm searchCondition = new SearchTerm() {
#Override
public boolean match(Message message) {
try {
if (message.getMessageNumber() == emailID) {
return true;
}
} catch (NullPointerException ex) {
System.out.println("Null exception at line 50, may be due to variation in timing");
//ex.printStackTrace();
}
return false;
}
};
//performs search through the folder
Message[] foundMessages = folderInbox.search(searchCondition);
setRecentlySearchMsg(foundMessages[0]);
for (int i = 0; i < foundMessages.length; i++) {
Message message = foundMessages[i];
String subject = message.getSubject(); //subject in the target email
Date date = message.getReceivedDate(); //date the email was received
System.out.println("Found message #" + i + ": " + subject + ", Date Received: " + date.toString());
String contentType = message.getContentType();
System.out.println("Content type: " + contentType);
Multipart multipart = (Multipart) message.getContent();
int count = 0; //count object to ensure overflows aren't triggered
for (int j = 0; j < multipart.getCount(); j++) {
MimeBodyPart part = (MimeBodyPart) multipart.getBodyPart(j); //using the i loop to establish connection to message
if(part.getContentType().contains("TEXT/PLAIN")) {
continue;
} else if(part.getContentType().contains("TEXT/HTML")) {
break;
}
if (Part.ATTACHMENT.equalsIgnoreCase(part.getDisposition())) {
String dateToday = new SimpleDateFormat("MM-dd-yyyy").format(new Date()); //Retrieving today's date
String path = "/file/location/here/" + dateToday + "_" + part.getFileName();
if(part.getFileName().contains("User")) {
userFileLocation = path;
} else if(part.getFileName().contains("Teams")) {
teamFileLocation = path;
}
part.saveFile(path);
System.out.println("File successfully saved in folder with name: " + path);
System.out.println("File is located at: " + path);
count++;
}
if (count == 2){
break;
}
}
}
//disconnect
folderInbox.close(false);
store.close();
} catch (NoSuchProviderException ex) {
System.out.println("No provider.");
ex.printStackTrace();
} catch (MessagingException ex) {
System.out.println("Could not connect to the message store.");
ex.printStackTrace();
} catch (IOException ex) {
System.out.println("Error parsing message attachment");
ex.printStackTrace();
}
}
I am able to get mail subject and body contents using Java Api.
But in my application i received a template in an email which contains a url behind an image. I need to get that url, I found that the url was displayed when i view the source for that email manually.
Remember that i am not downloading the email instead i am connecting to mail server and then reading the mail data for any specific user
Is there a way that i can view the source of email like i am getting the subject of mail.
Here is the code:
import org.jsoup.Jsoup;
import javax.mail.*;
import javax.mail.internet.MimeMessage;
import java.io.IOException;
import java.util.Properties;
public class VerifyEmails {
public Message message;
public int i, n;
public String result;
public void check(String host, String user, String password) throws IOException, MessagingException {
Properties properties = new Properties();
properties.put("mail.imap.host", host);
properties.put("mail.imap.user", user);
properties.put("mail.imap.port", "143");
properties.put("mail.imap.starttls.enable", "false");
Session emailSession = Session.getDefaultInstance(properties);
Store store = emailSession.getStore("imap");
System.out.println("test1 " + store);
store.connect(host, user, password);
Folder emailFolder = store.getFolder("INBOX");
emailFolder.open(Folder.READ_ONLY);
Message[] messages = emailFolder.getMessages();
System.out.println("messages.length---" + messages.length);
for (i = 0, n = messages.length; i < n; i++) {
message = messages[i];
System.out.println("---------------------------------");
System.out.println("Email Number " + (i + 1));
System.out.println("Subject: " + message.getSubject());
System.out.println("From: " + message.getFrom()[0]);
System.out.println("Email contents are :" + message.getContentType());
System.out.println("Email headers are :" + message.getContent());
}
if (message instanceof MimeMessage) {
MimeMessage m = (MimeMessage) message;
Object contentObject = message.getContent();
if (contentObject instanceof Multipart) {
BodyPart clearTextPart = null;
BodyPart htmlTextPart = null;
Multipart content = (Multipart) contentObject;
int count = content.getCount();
for (int i = 0; i < count; i++) {
BodyPart part = content.getBodyPart(i);
if (part.isMimeType("text/plain")) {
clearTextPart = part;
String test = String.valueOf(clearTextPart.getAllHeaders());
System.out.println("check1" + test);
break;
} else if (part.isMimeType("text/html")) {
htmlTextPart = part;
}
}
if (clearTextPart != null) {
result = (String) clearTextPart.getContent();
String test = String.valueOf(clearTextPart.getAllHeaders());
System.out.println("check2" + test);
System.out.println("plain text: " + result);
} else if (htmlTextPart != null) {
String html = (String) htmlTextPart.getContent();
result = Jsoup.parse(html).text();
System.out.println("html text: " + result);
}
} else if (contentObject instanceof String) // a simple text message
{
result = (String) contentObject;
System.out.println("String text: " + result);
} else // not a mime message
{
result = null;
System.out.println("null : " + result);
}
emailFolder.close(false);
store.close();
}
}
}
I am using the following code to successfully retrieve messages from my Gmail account.
// Import Statements
public class ConfirmEmail {
WebDriver driver;
Folder inbox;
String gmailID = "xxxxxxxxxxx#gmail.com";
String gmailPass = "xxxxxxxx";
String storeMessage;
public ConfirmEmail()
{
}
public void MailReader() {
System.out.println("Inside MailReader()...");
final String SSL_FACTORY = "javax.net.ssl.SSLSocketFactory";
/* Set the mail properties */
Properties props = System.getProperties();
// Set manual Properties
props.setProperty("mail.pop3.socketFactory.class", SSL_FACTORY);
props.setProperty("mail.pop3.socketFactory.fallback", "false");
props.setProperty("mail.pop3.port", "995");
props.setProperty("mail.pop3.socketFactory.port", "995");
props.put("mail.pop3.host", "pop.gmail.com");
try
{
/* Create the session and get the store for read the mail. */
Session session = Session.getDefaultInstance(
System.getProperties(), null);
Store store = session.getStore("pop3");
store.connect("pop.gmail.com", 995, gmailID,
gmailPass);
/* Mention the folder name which you want to read. */
// inbox = store.getDefaultFolder();
// inbox = inbox.getFolder("INBOX");
inbox = store.getFolder("INBOX");
/* Open the inbox using store. */
inbox.open(Folder.READ_ONLY);
/* Get the messages which is unread in the Inbox */
Message messages[] = inbox.search(new FlagTerm(new Flags(
Flags.Flag.SEEN), false));
System.out.println("No. of Unread Messages : " + messages.length);
/* Use a suitable FetchProfile */
FetchProfile fp = new FetchProfile();
fp.add(FetchProfile.Item.ENVELOPE);
fp.add(FetchProfile.Item.CONTENT_INFO);
inbox.fetch(messages, fp);
try
{
printAllMessages(messages);
inbox.close(true);
store.close();
}
catch (Exception ex)
{
System.out.println("Exception arise at the time of read mail");
ex.printStackTrace();
}
}
catch (MessagingException e)
{
System.out.println("Exception while connecting to server: "
+ e.getLocalizedMessage());
e.printStackTrace();
System.exit(2);
}
}
public void printAllMessages(Message[] msgs) throws Exception
{
for (int i = 0; i < msgs.length; i++)
{
System.out.println("MESSAGE #" + (i + 1) + ":");
printEnvelope(msgs[i]);
}
}
public void printEnvelope(Message message) throws Exception
{
Address[] a;
// FROM
if ((a = message.getFrom()) != null) {
for (int j = 0; j < a.length; j++) {
System.out.println("FROM: " + a[j].toString());
}
}
// TO
if ((a = message.getRecipients(Message.RecipientType.TO)) != null) {
for (int j = 0; j < a.length; j++) {
System.out.println("TO: " + a[j].toString());
}
}
String subject = message.getSubject();
Date receivedDate = message.getReceivedDate();
Date sentDate = message.getSentDate(); // receivedDate is returning
// null. So used getSentDate()
String content = message.getContent().toString();
System.out.println("Subject : " + subject);
if (receivedDate != null) {
System.out.println("Received Date : " + receivedDate.toString());
}
System.out.println("Sent Date : " + sentDate.toString());
System.out.println("Content : " + content);
getContent(message);
}
public void getContent(Message msg)
{
try {
String contentType = msg.getContentType();
System.out.println("Content Type : " + contentType);
Multipart mp = (Multipart) msg.getContent();
int count = mp.getCount();
for (int i = 0; i < count; i++) {
dumpPart(mp.getBodyPart(i));
}
} catch (Exception ex) {
System.out.println("Exception arise at get Content");
ex.printStackTrace();
}
}
public void dumpPart(Part p) throws Exception {
// Dump input stream ..
InputStream is = p.getInputStream();
// If "is" is not already buffered, wrap a BufferedInputStream
// around it.
if (!(is instanceof BufferedInputStream)) {
is = new BufferedInputStream(is);
}
int c;
System.out.println("Message : ");
while ((c = is.read()) != -1) {
System.out.write(c);
}
}
}
With this code I am successfully able to print messages to console. Works flawlessly 100% of the time.
However, I need to store the "bodyPart" (i.e, the actual message or body of message) in a String so I could search the String using Regex. I need to extract links begining with http.
How can I convert the message to a string?
Thanks
I'm not quite sure what you are asking (because you said you already print out your Messages ... so when you print them, why can't you store them in a String?)
if you realy just want the bodyPart stored in a String variable:
Multipart mp = (Multipart) msg.getContent();
BodyPart bp = mp.getBodyPart(0);
String content = bp.getContent().toString();
I've been working with Kafka for two months, and I used this code to consume messages locally. I recently decided to distribute Zookeeper and Kafka and everything seems to work just fine. My issue started when I tried to use the consumer's code from a remote IP; Once I change seeds.add("127.0.0.1"); to seeds.add("104.131.40.xxx"); I get this error message:
run:
Error communicating with Broker [104.131.40.xxx] to find Leader for [temperature, 0] Reason:
java.net.ConnectException: Connection refused Can't find metadata for Topic and Partition. Exiting
BUILD SUCCESSFUL (total time: 21 seconds)r code here
this is the code that I currently use:
/*
Kafka API consumer reads 10 readings from the "temperature" topic
*/
package simpleexample;
import kafka.api.FetchRequest;
import kafka.api.FetchRequestBuilder;
import kafka.api.PartitionOffsetRequestInfo;
import kafka.common.ErrorMapping;
import kafka.common.TopicAndPartition;
import kafka.javaapi.*;
import kafka.javaapi.consumer.SimpleConsumer;
import kafka.message.MessageAndOffset;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class SimpleExample {
public static void main(String args[]) {
SimpleExample example = new SimpleExample();
//long maxReads = Long.parseLong(args[0]);
long maxReads = 10;
//String topic = args[1];
String topic = "temperature";
//int partition = Integer.parseInt(args[2]);
int partition =0;
List<String> seeds = new ArrayList<String>();
//seeds.add(args[3]);
seeds.add("104.131.40.xxx");
//int port = Integer.parseInt(args[4]);
int port =9092;
try {
example.run(maxReads, topic, partition, seeds, port);
} catch (Exception e) {
System.out.println("Oops:" + e);
e.printStackTrace();
}
}
private List<String> m_replicaBrokers = new ArrayList<String>();
public SimpleExample() {
m_replicaBrokers = new ArrayList<String>();
}
public void run(long a_maxReads, String a_topic, int a_partition, List<String> a_seedBrokers, int a_port) throws Exception {
// find the meta data about the topic and partition we are interested in
//
PartitionMetadata metadata = findLeader(a_seedBrokers, a_port, a_topic, a_partition);
if (metadata == null) {
System.out.println("Can't find metadata for Topic and Partition. Exiting");
return;
}
if (metadata.leader() == null) {
System.out.println("Can't find Leader for Topic and Partition. Exiting");
return;
}
String leadBroker = metadata.leader().host();
String clientName = "Client_" + a_topic + "_" + a_partition;
SimpleConsumer consumer = new SimpleConsumer(leadBroker, a_port, 100000, 64 * 1024, clientName);
long readOffset = getLastOffset(consumer,a_topic, a_partition, kafka.api.OffsetRequest.EarliestTime(), clientName);
int numErrors = 0;
while (a_maxReads > 0) {
if (consumer == null) {
consumer = new SimpleConsumer(leadBroker, a_port, 100000, 64 * 1024, clientName);
}
FetchRequest req = new FetchRequestBuilder()
.clientId(clientName)
.addFetch(a_topic, a_partition, readOffset, 100000) // Note: this fetchSize of 100000 might need to be increased if large batches are written to Kafka
.build();
FetchResponse fetchResponse = consumer.fetch(req);
if (fetchResponse.hasError()) {
numErrors++;
// Something went wrong!
short code = fetchResponse.errorCode(a_topic, a_partition);
System.out.println("Error fetching data from the Broker:" + leadBroker + " Reason: " + code);
if (numErrors > 5) break;
if (code == ErrorMapping.OffsetOutOfRangeCode()) {
// We asked for an invalid offset. For simple case ask for the last element to reset
readOffset = getLastOffset(consumer,a_topic, a_partition, kafka.api.OffsetRequest.LatestTime(), clientName);
continue;
}
consumer.close();
consumer = null;
leadBroker = findNewLeader(leadBroker, a_topic, a_partition, a_port);
continue;
}
numErrors = 0;
long numRead = 0;
for (MessageAndOffset messageAndOffset : fetchResponse.messageSet(a_topic, a_partition)) {
long currentOffset = messageAndOffset.offset();
if (currentOffset < readOffset) {
System.out.println("Found an old offset: " + currentOffset + " Expecting: " + readOffset);
continue;
}
readOffset = messageAndOffset.nextOffset();
ByteBuffer payload = messageAndOffset.message().payload();
byte[] bytes = new byte[payload.limit()];
payload.get(bytes);
System.out.println(String.valueOf(messageAndOffset.offset()) + ": " + new String(bytes, "UTF-8"));
numRead++;
a_maxReads--;
}
if (numRead == 0) {
try {
Thread.sleep(1000);
} catch (InterruptedException ie) {
}
}
}
if (consumer != null) consumer.close();
}
public static long getLastOffset(SimpleConsumer consumer, String topic, int partition,
long whichTime, String clientName) {
TopicAndPartition topicAndPartition = new TopicAndPartition(topic, partition);
Map<TopicAndPartition, PartitionOffsetRequestInfo> requestInfo = new HashMap<TopicAndPartition, PartitionOffsetRequestInfo>();
requestInfo.put(topicAndPartition, new PartitionOffsetRequestInfo(whichTime, 1));
kafka.javaapi.OffsetRequest request = new kafka.javaapi.OffsetRequest(
requestInfo, kafka.api.OffsetRequest.CurrentVersion(), clientName);
OffsetResponse response = consumer.getOffsetsBefore(request);
if (response.hasError()) {
System.out.println("Error fetching data Offset Data the Broker. Reason: " + response.errorCode(topic, partition) );
return 0;
}
long[] offsets = response.offsets(topic, partition);
return offsets[0];
}
private String findNewLeader(String a_oldLeader, String a_topic, int a_partition, int a_port) throws Exception {
for (int i = 0; i < 3; i++) {
boolean goToSleep = false;
PartitionMetadata metadata = findLeader(m_replicaBrokers, a_port, a_topic, a_partition);
if (metadata == null) {
goToSleep = true;
} else if (metadata.leader() == null) {
goToSleep = true;
} else if (a_oldLeader.equalsIgnoreCase(metadata.leader().host()) && i == 0) {
// first time through if the leader hasn't changed give ZooKeeper a second to recover
// second time, assume the broker did recover before failover, or it was a non-Broker issue
//
goToSleep = true;
} else {
return metadata.leader().host();
}
if (goToSleep) {
try {
Thread.sleep(1000);
} catch (InterruptedException ie) {
}
}
}
System.out.println("Unable to find new leader after Broker failure. Exiting");
throw new Exception("Unable to find new leader after Broker failure. Exiting");
}
private PartitionMetadata findLeader(List<String> a_seedBrokers, int a_port, String a_topic, int a_partition) {
PartitionMetadata returnMetaData = null;
loop:
for (String seed : a_seedBrokers) {
SimpleConsumer consumer = null;
try {
consumer = new SimpleConsumer(seed, a_port, 100000, 64 * 1024, "leaderLookup");
List<String> topics = Collections.singletonList(a_topic);
TopicMetadataRequest req = new TopicMetadataRequest(topics);
kafka.javaapi.TopicMetadataResponse resp = consumer.send(req);
List<TopicMetadata> metaData = resp.topicsMetadata();
for (TopicMetadata item : metaData) {
for (PartitionMetadata part : item.partitionsMetadata()) {
if (part.partitionId() == a_partition) {
returnMetaData = part;
break loop;
}
}
}
} catch (Exception e) {
System.out.println("Error communicating with Broker [" + seed + "] to find Leader for [" + a_topic
+ ", " + a_partition + "] Reason: " + e);
} finally {
if (consumer != null) consumer.close();
}
}
if (returnMetaData != null) {
m_replicaBrokers.clear();
for (kafka.cluster.Broker replica : returnMetaData.replicas()) {
m_replicaBrokers.add(replica.host());
}
}
return returnMetaData;
}
}
You need to set the advertised.host.name instead of host.name in the kafka server.properties configuration file.
i'm reading data from serial port for average interval say 1 second,and at the same time writing read data to textArea and textfile,problem is i'm not getting correct data at some time,may be because i'm doing all three process in a single program,how to do writing to text area and text file by separate thred?
this is my code:
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStream;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Iterator;
import java.util.TooManyListenersException;
import java.util.TreeMap;
import javax.comm.CommPortIdentifier;
import javax.comm.SerialPort;
import javax.comm.SerialPortEvent;
import javax.comm.SerialPortEventListener;
import javax.comm.UnsupportedCommOperationException;
import javax.swing.JOptionPane;
import com.pressure.constants.Constants;
import com.pressure.online.OnlineStartWindow;
public class SerailReader implements SerialPortEventListener {
// DECLARES INPUT STREAM TO READ SRIAL PORT
private InputStream inputStream;
// DECLARES PORT
private CommPortIdentifier port;
private SerialPort serialPort;
// DATE TO CREATE FILE NAME
private static final SimpleDateFormat SDF = new SimpleDateFormat("dd-MM-yy");
private int index = 0;
private File file;
private OnlineStartWindow onlineStartwindow;
private int[] tempIntArray = new int[233];
private int newData = 0;
private String outFolder;
private String filename = "0";
private FileWriter fileWriter;
private BufferedWriter buffOut;
private StringBuffer line;
private String packetFilename;
TreeMap<Integer, Float> channelMap = new TreeMap<Integer, Float>();
ThreadPrintsAndWrites p;
public FileWriter getFileWriter() {
return fileWriter;
}
public void setFileWriter(FileWriter fileWriter) {
this.fileWriter = fileWriter;
}
public BufferedWriter getBuffOut() {
return buffOut;
}
public void setBuffOut(BufferedWriter buffOut) {
this.buffOut = buffOut;
}
// SETTER GETTER TO OnlineStartwindow OBJECT
public OnlineStartWindow getOnlineStartwindow() {
return onlineStartwindow;
}
public void setOnlineStartwindow(OnlineStartWindow onlineStartwindow) {
this.onlineStartwindow = onlineStartwindow;
}
// SETTER GETTER TO SERIALPORT
public SerialPort getSerialPort() {
return serialPort;
}
public void setSerialPort(SerialPort serialPort) {
this.serialPort = serialPort;
}
// ********* connects to serial port ***********//
public void SerialReadmethod(OnlineStartWindow onlineStartwindow,
String outFolderPath) throws Exception {
setOnlineStartwindow(onlineStartwindow);
outFolder = outFolderPath;
// SELECTS PORT NAME SELECTED
port = CommPortIdentifier.getPortIdentifier(getOnlineStartwindow()
.getComPort());
System.out.println("port name " + port);
// CHEAK WETHER SELECTED PORT AVAILABLE OR NOT
if (port.getPortType() == CommPortIdentifier.PORT_SERIAL) {
if (port.getName().equals(getOnlineStartwindow().getComPort())) {
JOptionPane.showMessageDialog(null, "Successpully opened port",
"Online Dump", JOptionPane.INFORMATION_MESSAGE);
}
}
// OPENS SERAIL PORT
serialPort = (SerialPort) port.open("SimpleReadApp1111", 1000);
// OPENS SERIAL PORT INPUT STREAM TO READ DATA
try {
inputStream = serialPort.getInputStream();
} catch (IOException e) {
System.out.println("IO Exception");
}
// ADDS LISTNER TO SERIALPORT
try {
serialPort.addEventListener(this);
} catch (TooManyListenersException e) {
System.out.println("Tooo many Listener exception");
}
// EVENT GENERATED WHEN DATA WILL BE AVAILABELE ON SERIALPORT
// INPUTSTREAM
serialPort.notifyOnDataAvailable(true);
try {
// SETS SELECTED BAUDRATE
int BAUDRATE = Integer.parseInt((getOnlineStartwindow()
.getBaudRate()).trim());
// SETS SELECTED DATA BITS
int DATABITS = Integer.parseInt(getOnlineStartwindow()
.getDataBits().trim());
if (DATABITS == 8) {
DATABITS = SerialPort.DATABITS_8;
} else if (DATABITS == 7) {
DATABITS = SerialPort.DATABITS_7;
} else if (DATABITS == 6) {
DATABITS = SerialPort.DATABITS_6;
} else if (DATABITS == 5) {
DATABITS = SerialPort.DATABITS_5;
}
// SETS SELECTED STOPBITS
int STOPBITS = 0;
if (getOnlineStartwindow().getStopBits() == "1") {
STOPBITS = SerialPort.STOPBITS_1;
} else if (getOnlineStartwindow().getStopBits() == "1.5") {
STOPBITS = SerialPort.STOPBITS_1_5;
} else if (getOnlineStartwindow().getStopBits() == "2") {
STOPBITS = SerialPort.STOPBITS_2;
}
// SETS SELECTED PARITY
int PARITY = 0;
if (getOnlineStartwindow().getParity() == "NONE") {
PARITY = SerialPort.PARITY_NONE;
} else if (getOnlineStartwindow().getParity() == "EVEN") {
PARITY = SerialPort.PARITY_EVEN;
} else if (getOnlineStartwindow().getParity() == "ODD") {
PARITY = SerialPort.PARITY_ODD;
}
// SETS SELECTED FLOW CONTROL
int FLOWCONTROL = 0;
if (getOnlineStartwindow().getFlowControl() == "NONE") {
FLOWCONTROL = SerialPort.FLOWCONTROL_NONE;
} else if (getOnlineStartwindow().getFlowControl() == "XON/XOFF") {
FLOWCONTROL = SerialPort.FLOWCONTROL_XONXOFF_IN;
}
serialPort
.setSerialPortParams(BAUDRATE, DATABITS, STOPBITS, PARITY);
// no handshaking or other flow control
serialPort.setFlowControlMode(FLOWCONTROL);
} catch (UnsupportedCommOperationException e) {
System.out.println("UnSupported comm operation");
}
}
// *********this method will automaticaly calls when u get data on port and
// arranges packet from start frame to end frame *************//
public void serialEvent(SerialPortEvent event) {
// switch (event.getEventType()) {
//
// case SerialPortEvent.DATA_AVAILABLE:
if(event.getEventType()==SerialPortEvent.DATA_AVAILABLE){
//dataAvailabel = inputStream.available();
// READING DATA CHARECTER BY CHARECTER
while (newData != -1) {
try {
newData = inputStream.read();
if (newData == -1) {
break;
}
if (Constants.SF == (char) newData) {
index = 0;
// System.out.println("start frame");
}
tempIntArray[index] = newData;
if (Constants.EF == (char) newData) {
selectToDispalyAndWrite(tempIntArray);
// disp(tempIntArray);
}
index++;
} catch (IOException ex) {
System.err.println(ex);
// return;
}
}
// ///////////////// completes
}
}
// DISPLYS PACKET TO TEXT AREA AND CREATES .PSI FILE
public void selectToDispalyAndWrite(int[] readBufferArray) {
if (getOnlineStartwindow().getDump().isSelected()) {
packetFilename = Integer.toString(readBufferArray[1])
+ Integer.toString(readBufferArray[2])
+ Integer.toString(readBufferArray[3]);
try {
if (getOnlineStartwindow().getFileTypeSelection() == "text") {
displayAndWriteToTextFile(readBufferArray);
} else {
displayAndWriteToExcelFile(readBufferArray);
}
} catch (IOException e) {
e.printStackTrace();
}
} else {
printToTextArea(readBufferArray);
}
}
public void printToTextArea(int[] readBufferArray) {
int i = 0;
int portname = 0;
Float portval = 0.0f;
int len = 0;
i = 0;
writeloop: while (len != readBufferArray.length) {
// while ((char) readBufferArray[i] != Constants.EF) {
if ((char) readBufferArray[i] == Constants.SF) {
// WRITES DASH LINE TO TEXT AREA
getOnlineStartwindow()
.getTextArea()
.append(
"\r\n\r\n-----------------------------------------------------------------------\r\n");
getOnlineStartwindow().getTextArea().append(
"Time :" + readBufferArray[i + 4] + " Min "
+ readBufferArray[i + 5] + " Sec.");
getOnlineStartwindow()
.getTextArea()
.append(
"\r\n-----------------------------------------------------------------------\r\n");
}
if ((char) readBufferArray[i] == Constants.EF) {
for (Iterator<Integer> iterator = channelMap.keySet()
.iterator(); iterator.hasNext();) {
int key = iterator.next();
Float value = channelMap.get(key);
getOnlineStartwindow().getTextArea().append(
"Port_" + key + " " + value + "\r\n");
}
channelMap.clear();
break writeloop;
}
i++;
}
}
public void displayAndWriteToTextFile(int[] readBufferArray)
throws IOException {
int i = 0;
int portname = 0;
Float portval = 0.0f;
if (!(filename.equalsIgnoreCase(packetFilename))) {
filename = packetFilename;
if (buffOut != null && fileWriter != null) {
drawLine('*');
buffOut.close();
fileWriter.close();
}
// GET CURRENT DATE
Date date = new Date();
file = new File(outFolder + "\\" + SDF.format(date) + "-"
+ packetFilename + ".txt");
if (!file.exists()) {
fileWriter = new FileWriter(file, true);
buffOut = new BufferedWriter(fileWriter);
drawLine('*');
drawLine('*');
} else {
fileWriter = new FileWriter(file, true);
buffOut = new BufferedWriter(fileWriter);
}
}
// LOOP TO DISPLY ALL PORT NAME AND PRESSURE VALUES
int len = 0;
i = 0;
writeloop: while (len != readBufferArray.length) {
// while ((char) readBufferArray[i] != Constants.EF) {
if ((char) readBufferArray[i] == Constants.SF) {
// WRITES DASH LINE TO TEXT AREA
getOnlineStartwindow()
.getTextArea()
.append(
"\r\n\r\n-----------------------------------------------------------------------\r\n");
getOnlineStartwindow().getTextArea().append(
"Time :" + readBufferArray[i + 4] + " Min "
+ readBufferArray[i + 5] + " Sec.");
getOnlineStartwindow()
.getTextArea()
.append(
"\r\n-----------------------------------------------------------------------\r\n");
drawLine('-');
buffOut.write("TIME: " + readBufferArray[i + 4] + " Min "
+ readBufferArray[i + 5] + " Sec." + "\r\n\r\n");
}
if ((char) readBufferArray[i] == Constants.ST) {
portname = readBufferArray[i + 1];
portval = getFloatValue(i);
channelMap.put(portname, portval);
}
if ((char) readBufferArray[i] == Constants.EF) {
for (Iterator<Integer> iterator = channelMap.keySet()
.iterator(); iterator.hasNext();) {
int key = iterator.next();
Float value = channelMap.get(key);
getOnlineStartwindow().getTextArea().append(
"Port_" + key + " " + value + "\r\n");
}
channelMap.clear();
break writeloop;
}
i++;
}
}
}
Thanks in advance
Here an approach from a design standpoint.
Create a class type for the data. This will be a container (inherit the appropriate queue). And inside of this class as you set and get the data make sure you lock.
Create a new thread in main to start the serial port and read the messages. Pass the thread one object (your list class object). The serial port will chuck data into this list. This will be sets in the object.
You main thread (the window) will have a timer that fires and reads the data from queue. You may have to play with the timer some. You don't want it firing too often and eating all your processing time in the main window. However if it is too slow you may not be updating enough and getting messages out of the queue when they are available. This timer should get all of the data out the queue and display it.
The queue and the lock are the important part here. The queue will keep messages in order. The lock will make sure you don't try to write new data to your queue while you are in the middle of reading data.
I was under the impression that Java does not support serial port for Windows, it only does it for Solaris or Linux.
your class should implement the Runnable Interface. and create a thread that reads from the serial port in its run method.
The Main thread could do the writing part to the TextArea.
You may need to do some synchronization between the Main Thread and the Runnable so that writing to the TextArea/File only takes place after appropriate data has been read from the port.