Storm bolt executes more than its parent - java

I have a Topology which contains a KafkaSpout and 2 bolts.
BoltParseJsonInput and its execute method:
public void execute(Tuple input) {
// TODO Auto-generated method stub
String data = input.getString(4);
js = new JSONObject(data);
String userId = js.getString("userId");
String timestamp = js.getString("timestamp");
counter++;
System.out.println(counter);
collector.emit(input, new Values(userId, timestamp));
collector.ack(input);
}
BoltInsertRedis and its execute method
public void execute(Tuple input) {
// TODO Auto-generated method stub
String userId = input.getStringByField("userId");
int timestamp = 0;
try {
timestamp = convertTimestampToEpoch(input.getStringByField("timestamp"));
} catch (ParseException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
String timestep = this.prefix + timestamp/10;
String curTimestamp = jedis.hget(timestep, userId);
if(curTimestamp == null || Integer.parseInt(curTimestamp) < timestamp) {
jedis.hset(timestep, userId, Integer.toString(timestamp));
}
collector.ack(input);
}
BoltInsertRedis get the input from BoltParseJsonInput
builder.setBolt("ParseJsonInput-Bolt", new BoltParseJsonInput()).shuffleGrouping("Kafka-Spout");
builder.setBolt("BoltRedisUserLastActive-Bolt", new BoltRedisUserLastActive()).shuffleGrouping("ParseJsonInput-Bolt");
But when I submit this topology into Storm, BoltInsertRedis execute more than BoltParseJsonInput
Can you explain to me what is the problem here?

I found that my ParseJsonBolt had made an exception at message 25700 and it keeps replaying execution at that point. When I made a try catch, it works well

Related

Retrieve results as single string each time a method is called in java

I want to retrieve the results of a method in a single string each time the function is called. I have a method which returns different results every time. I want to put all the results as a single string.
Tried to use append() method of java but the results are getting replaced every time as the function is called each time. but i need to retrieve the previous results as well.
my code is as follows.
public void createPanel2()
{
panel2 = new JPanel();
panel2.setLayout( new FlowLayout() );
query = new JLabel("query");
textbox =new JTextField(10);
submit = new JButton("submit");
panel2.add(query);
panel2.add(textbox);
panel2.add(submit);
submit.addActionListener(new ActionListener(){
#Override
public void actionPerformed(ActionEvent e){
String str =textbox.getText();
String serverUrl = "http://localhost:8983/solr/collection1";
SolrServer solr = new HttpSolrServer(serverUrl);
try {
for (SolrDocument next : simpleSolrQuery(solr, str +
"")) {
prettyPrint(System.out, next);
}
} catch (SolrServerException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
initFilterAndButton();
}
SolrDocumentList simpleSolrQuery(SolrServer solr,
String query) throws SolrServerException {
SolrQuery solrQuery = new SolrQuery(query);
//SolrQuery query = new SolrQuery(searchTerm);
//query.setStart((pageNum - 1) * numItemsPerPage);
//query.setRows(numItemsPerPage);
//solrQuery.setRows(Integer.MAX_VALUE);
QueryResponse resp = solr.query(solrQuery);
//System.out.println("resp"+resp);
final SolrDocumentList hits = resp.getResults();
/*for (SolrDocument d : hits) {
for (Iterator<Map.Entry<String, Object>> i = d.iterator(); i
.hasNext();) {
Map.Entry<String, Object> e2 = i.next();
System.out.println(e2.getKey() + "\t" + e2.getValue());
}
System.out.println("------------------------");
}*/
System.out.println("hits"+resp.getElapsedTime());
System.out.println("size"+hits.size());
System.out.println("num found"+hits.getNumFound());
//String str ="hello";
//createPanel1(hits);
return hits;
}
void prettyPrint(PrintStream out, SolrDocument doc) {
List<String> sortedFieldNames =
new ArrayList<String>(doc.getFieldNames());
Collections.sort(sortedFieldNames);
out.println();
// StringBuilder contentstring=new StringBuilder();
// ArrayList<String> contents=new ArrayList<>();
for (String field : sortedFieldNames) {
if(field.equals("content")){
textarea.append(String.format("%s: %s",
field,doc.getFieldValue(field)+"\n"));
out.println(String.format("\t%s: %s",
field, doc.getFieldValue(field)));
contentsmethod(doc.getFieldValue(field).toString());
// contents.add(doc.getFieldValue(field).toString());
// System.out.println("conetnts"+contentstring);
}
}
// String test=contentstring.toString();
out.println();
}
public void contentsmethod(String fieldsvalues) {
// TODO Auto-generated method stub
StringBuilder contentstring=new StringBuilder();
contentstring.append(fieldsvalues);
try {
Desktop.getDesktop().browse(new URL(serverQuery+URLEncoder.encode(contentstring.toString())).toURI());
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (URISyntaxException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
});
}
The code above is calling the prettyPrint method every time and the results of that method should be retrieved to a single string which should hold the previous called results as well.
here i want to retrieve the results of contentstring.append(fieldsvalues);
But contentstring is returning only the current results and not appending the previous results.
this is obvious because the method is called everytime. Is there any work around to retrieve the previous results along with the current ones as well.
You need to move this StringBuilder contentstring=new StringBuilder(); outside of your method. Every time your method call is made, you create a new String. This is why you only get the current value.
You can make a List outside of your method and add the resulting String in that list. Otherwhise, you can create the string outside and append the results to it without instantiating a new one at every method call.
You have to do something like this:
StringBuilder contentstring = new StringBuilder();
public void contentsmethod(String fieldsvalues) {
// TODO Auto-generated method stub
contentstring.append(fieldsvalues);
try {
Desktop.getDesktop().browse(new URL(serverQuery+URLEncoder.encode(contentstring.toString())).toURI());
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (URISyntaxException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

Mismatch of return datatype

i am facing a problem regrading specifying the return data type. I have the FOComp class which implements callabale, the call() method of the 'FOComp' returns data type List<ArrayList<Mat>> as shown in the code of 'FOComp' class below.
and the method 'getResults()' returns data of type ArrayList<Mat> as shown in the code below. and currently, at run time, when I execute the code, I receive the folowing error:
Multiple markers at this line
The return type is incompatible with Callable<ArrayList<Mat>>.call()
The return type is incompatible with Callable<List<Mat>>.call()
kindly please let me know how to fix it.
'FOComp' class:
static class FOComp implements Callable<List<Mat>> {//should return list contains 4 mats(0,45,90,135)
private ArrayList<Mat> gaussianMatList = null;
private List<ArrayList<Mat>> results_4OrientAngles_List = null;
public FOComp(ArrayList<Mat> gaussianMatList) {
// TODO Auto-generated constructor stub
this.gaussianMatList = gaussianMatList;
this.results_4OrientAngles_List = new ArrayList<ArrayList<Mat>>();
}
public List<ArrayList<Mat>> call() throws Exception {
// TODO Auto-generated method stub
try {
featOrient = new FeatOrientation(this.gaussianMatList);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
featOrient.start();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
this.results_4OrientAngles_List.add(featOrient.getResults());
return results_4OrientAngles_List;
}
}
'getResults':
public ArrayList<Mat> getResults() {
if (this.crossAddOrientMapsList != null) {
if (!this.crossAddOrientMapsList.isEmpty()) {
if (this.crossAddOrientMapsList.size() == 4) {
double[] theta = new double[4];
theta[0] = 0;
theta[1] = 45;
theta[2] = 90;
theta[3] = 135;
for (int i = 0; i < this.crossAddOrientMapsList.size(); i++) {
MatFactory.writeMat(FilePathUtils.newOutputPath("FinalCrossAdd_" + theta[i]+"_degs"), this.crossAddOrientMapsList.get(i));
//ImageUtils.showMat(this.crossAddOrientMapsList.get(i), "OrientMap_" + theta[i] + " degs");
}
return this.crossAddOrientMapsList;
} else {
Log.WTF(TAG, "getResults", "crossAddOrientMapsList != 4 !!");
return null;
}
} else {
Log.E(TAG, "getResults", "crossAddOrientMapsList is empty.");
return null;
}
} else {
Log.E(TAG, "getResults", "crossAddOrientMapsList is null");
return null;
}
}
class FOComp implements Callable<List<Mat>>
and
public List<ArrayList<Mat>> call()
aren't really compatible... Your call() method should be
#Override public List<Mat> call()
Also, it is good practice to avoid implementation classes in method signatures, use the interfaces instead (in this case, use List rather than ArrayList). That will also fix your problem with one of the "multiple markers" :-)
Cheers,
You class declaration says that you are going to return a List of Mat (FOComp implements Callable<List<Mat>>), but your call method signature says you are going to return a List of ArrayList of Mat (List<ArrayList<Mat>>).
You will need to make them consistent.

How to Grab records from an object

I am having a very difficult time on figuring out how to get records from an object that i have passed in the path to the file.
import java.util.ArrayList;
import edu.trident.cpt237.recordreaper.CabRecordReaper;
public class CabOrginazer implements CabInfo
{
private final String FARE = "";
private final String GAS = "";
private final String SERVICE = "";
private final String MILES = "";
private final double VALUE = 0.0;
CabRecordReaper reaper = new CabRecordReaper("C:/CabRecords/September.txt" );
public void cabOrginazer()
{
}
#Override
public Records getType() {
// TODO Auto-generated method stub
return null;
}
#Override
public String getDate() {
// TODO Auto-generated method stub
return null;
}
#Override
public String CabId() {
// TODO Auto-generated method stub
return null;
}
#Override
public double getValue() {
// TODO Auto-generated method stub
return 0;
}
#Override
public double getPerGallonCost() {
// TODO Auto-generated method stub
return 0;
}
public static void main(String[] args) {
// TODO Auto-generated method stub
}
}
I have try'd using a Scanner but does not work so i took it out. I can't do any of my other methods until i can grab what is in side of that reaper. The only methods inside of the CabRecordReaper is hasMoreRecord() which determines if there is more lines in the file.
I'm not quite understanding your question but I think you are having difficulties with reading from a .txt file?
try (BufferedReader reader = Files.newBufferedReader(Paths.get("path here")) {
String line = null;
while ((line = reader.readLine()) != null) {
// Add your code here.
System.out.println(true);
}
} catch (IOException e) {
e.printStackTrace();
}

Checking if a message contains a string

I have have a class that check id a phrase is contained in a message, I tried to do it with Matcher and Pattern and with String.contains(), but the results returned are odd.
Here is the class:
public class MotsClesFilter implements EmailFilter {
final String NAME = "Filtrage par mots cles";
/*private Pattern chaineSpam;
private Matcher chaineCourriel;*/
private int nbOccMotSpam;
private byte confidenceLevel;
#Override
public String getFilterName() {
return this.NAME;
}
#Override
public byte checkSpam(MimeMessage message) {
analyze(message);
if(this.nbOccMotSpam==0)
this.confidenceLevel = 1;
else if (this.nbOccMotSpam>0 && this.nbOccMotSpam<2)
this.confidenceLevel = CANT_SAY;
else if (this.nbOccMotSpam>1 && this.nbOccMotSpam<3)
this.confidenceLevel = 50;
else if (this.nbOccMotSpam>3 && this.nbOccMotSpam<4)
this.confidenceLevel = 65;
else if (this.nbOccMotSpam>4 && this.nbOccMotSpam<5)
this.confidenceLevel = 85;
else this.confidenceLevel = 90;
return (getConfidenceLevel());
}
public void analyze(MimeMessage message){
try {
List<String> listeChaines = new ArrayList<String>();
BufferedReader bis = new BufferedReader(new InputStreamReader(new FileInputStream(new File("SpamWords.txt"))));
while(bis.ready()){
String ligne = bis.readLine();
listeChaines.add(ligne);
}
String mail = ((String.valueOf(message.getContent())));
//System.out.println(mail);
for (int j =0; j<listeChaines.size();j++){
//System.out.println(listeChaines.get(j));
Pattern chaineSpam = Pattern.compile(listeChaines.get(j),Pattern.CASE_INSENSITIVE);
Matcher chaineCourriel = chaineSpam.matcher(mail);
if (chaineCourriel.matches())
this.nbOccMotSpam++;
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (MessagingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#Override
public byte getConfidenceLevel() {
// TODO Auto-generated method stub
return this.confidenceLevel;
}
#Override
public boolean enabled() {
// TODO Auto-generated method stub
return true;
}
}
The results returned by checkSpam are always 1 if use matches and 90 if I use find, it also returns 90 when I use mail.contains(listeChaines.get(j)).
That means that the message doesn't match any of the strings in the file, but that there are at least 5 strings in the file that can be found inside the message.
matches() checks if the whole string matches the pattern. Not if some substring matches it.

Getting SVN Log of a particular Date Range in Java

I am trying to get the log from a SVN repo using SVNKit.
public static void svnLogTest() {
final SvnOperationFactory svnOperationFactory = new SvnOperationFactory();
final SvnLog log = svnOperationFactory.createLog();
SVNURL url = null;
try {
url = SVNURL
.parseURIEncoded("https://svn-repo");
} catch (SVNException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
log.setSingleTarget(SvnTarget.fromURL(url));
log.addRange(SvnRevisionRange.create(SVNRevision.create(111),
SVNRevision.create(222)));
log.getRevisionRanges();
SVNLogEntry logEntry = null;
try {
logEntry = log.run();
System.out.println(logEntry.getAuthor() + " " + logEntry.getRevision() + " "
+ logEntry.getDate());
} catch (SVNException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
But it will give me only the first revision, How should I iterate to print the log for a particular date range ?
This is because
log.run();
always returns only one log entry (the same is true for other SvnOperation#run methods). To get all entries use receiver:
log.setReceiver(new ISvnObjectReceiver<SVNLogEntry>() {
#Override
public void receive(SvnTarget target, SVNLogEntry logEntry) throws SVNException {
//process logEntry here
}
});
log.run();

Categories

Resources