I would just have the id, the content and the hashtag and time of every tweet in a text file, and I dont know how to store infomration in lists of tweet,
I created a tweet class as follows:
public class Tweet {
private String type;
private String origin;
private String tweetText;
private String url;
private String tweetID;
private String tweetDate;
private int retCount;
private String favourit;
private String mEntities;
private String hashtags;
public Tweet(String tweetID,String origin) {
this.tweetID = tweetID;
this.origin = origin;
}
public Tweet(String type, String origin, String tweetText, String url, String tweetID, String tweetDate, int retCount, String favourit, String mEntities, String hashtags) {
this.type = type;
this.origin = origin;
this.tweetText = tweetText;
this.url = url;
this.tweetID = tweetID;
this.tweetDate = tweetDate;
this.retCount = retCount;
this.favourit = favourit;
this.mEntities = mEntities;
this.hashtags = hashtags;
}
public String getType() {
return type;
}
public void setType(String type) {
this.type = type;
}
public String getOrigin() {
return origin;
}
public void setOrigin(String origin) {
this.origin = origin;
}
public String getTweetText() {
return tweetText;
}
public void setTweetText(String tweetText) {
this.tweetText = tweetText;
}
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public String getTweetID() {
return tweetID;
}
public void setTweetID(String tweetID) {
this.tweetID = tweetID;
}
public String getTweetDate() {
return tweetDate;
}
public void setTweetDate(String tweetDate) {
this.tweetDate = tweetDate;
}
public int getRetCount() {
return retCount;
}
public void setRetCount(int retCount) {
this.retCount = retCount;
}
public String getFavourit() {
return favourit;
}
public void setFavourit(String favourit) {
this.favourit = favourit;
}
public String getmEntities() {
return mEntities;
}
public void setmEntities(String mEntities) {
this.mEntities = mEntities;
}
public String getHashtags() {
return hashtags;
}
public void setHashtags(String hashtags) {
this.hashtags = hashtags;
}
and my data file has the following format:
***
***
Type:status
Origin: Here's link to listen live to our discussion of #debtceiling #politics :
Text: Here's link to listen live to our discussion of :
URL:
ID: 96944336150867968
Time: Fri Jul 29 09:05:05 CDT 2011
RetCount: 0
Favorite: false
MentionedEntities:
Hashtags: debtceiling politics
***
***
Type:status
Origin: Now we're talking #debtceiling w/ Dick Polman #NewsWorksWHYY #PhillyInquirer & Bill Galston #BrookingsInst #NoLabelsOrg
Text: Now we're talking w/ Dick Polman & Bill Galston
URL:
ID: 96943803600089088
Time: Fri Jul 29 09:02:58 CDT 2011
RetCount: 1
Favorite: false
MentionedEntities: 136337303 151106990 14495726 15161791
Hashtags: debtceiling
***
***
I want to read this file and stock information into lists, i begin with this code, but i dont know how to solve that
public static List<String> readTweets(File file) throws IOException {
List<String> tweets = new ArrayList<String>();
//logger.info("Read tweets from {}", file.getAbsolutePath());
BufferedReader reader = new BufferedReader(new FileReader(file));
String line;
String[] fields;
while ((line = reader.readLine()) != null) {
fields = line.split(",");
if (fields.length > 1)
tweets.add(fields[1]);
}
return tweets;
}
By the looks of the code you are experimenting with, this is what I would do:
public static List<String> readTweets(File file) throws IOException {
List<String> tweets = new ArrayList<String>();
List<String> lines = Files.readAllLines(file.toPath());
for(int i = 0; i < lines.length(); i++){
String line = lines.get(i);
String[] part = line.split(",");
if(part.length < 1) tweets.add(part[i]);
}
}
But, if I wrote an application that was purely for printing the contents of tweets into the console, here's how I'd do it:
TweetReader.java
package Testers;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.List;
public class TweetReader {
public static List<Tweet> readTweets(File file) throws IOException {
boolean processEnd = false;
String type = "";
String origin = "";
String tweetText = "";
String url = "";
String tweetID = "";
String tweetDate = "";
int retCount = 0;
String favourite = "";
String mEntities = "";
String hashTags = "";
List<Tweet> tweets = new ArrayList<Tweet>();
List<String> lines = Files.readAllLines(file.toPath());
for(int i = 0; i < lines.size(); i++){
String line = lines.get(i);
line = line.trim();
if(line.equals("***")){
if(processEnd){
Tweet tweet = new Tweet(type, origin, tweetText, url, tweetID, tweetDate, retCount, favourite, mEntities, hashTags);
tweets.add(tweet);
processEnd = false;
}else processEnd = true;
}else{
if(line.contains(":")){
String header = line.substring(0, line.indexOf(":"));
//System.out.println(header); //You can uncomment this for troubleshooting
if(header.equals("Type")) type = line.substring(line.length() > 5 ? 5 : line.length());
else if(header.equals("Origin")) origin = line.substring(line.length() > 8 ? 8 : line.length());
else if(header.equals("Text")) tweetText = line.substring(line.length() > 6 ? 6 : line.length());
else if(header.equals("URL")) url = line.substring(line.length() > 5 ? 5 : line.length());
else if(header.equals("ID")) tweetID = line.substring(line.length() > 4 ? 4 : line.length());
else if(header.equals("Time")) tweetDate = line.substring(line.length() > 6 ? 6 : line.length());
else if(header.equals("RetCount")) retCount = Integer.parseInt(line.substring(line.length() > 10 ? 10 : line.length()));
else if(header.equals("Favorite")) favourite = line.substring(line.length() > 11 ? 11 : line.length());
else if(header.equals("MentionedEntities")) mEntities = line.substring(line.length() > 19 ? 19 : line.length());
else if(header.equals("Hashtags")) hashTags = line.substring(line.length() > 10 ? 10 : line.length());
else throw new IOException("Line cannot be identified as part of a tweet:" + line);
}else throw new IOException("Line cannot be processed:" + line);
}
}
return tweets;
}
public static void main(String[] args){
File log = new File("log.txt");
List<Tweet> tweets = new ArrayList<Tweet>();
try {
File f = new File(".").getAbsoluteFile();
File[] array = f.listFiles();
for(int i = 0; i < array.length; i++){
File tweet = array[i];
if(tweet.isFile() && !tweet.getName().contains("log.txt") && !tweet.getName().contains(".jar")){
log("Reading file: " + tweet.getAbsolutePath(), log);
List<Tweet> tweetlist = readTweets(tweet);
tweets.addAll(tweetlist);
}
}
System.out.println("Reading tweets now");
for(int i = 0; i < tweets.size(); i++){
Tweet t = tweets.get(i);
log("Type = " + t.getType(), log);
log("Origin = " + t.getOrigin(), log);
log("Text = " + t.getTweetText(), log);
log("URL = " + t.getURL(), log);
log("ID = " + t.getTweetID(), log);
log("Date = " + t.getTweetDate(), log);
log("Ret count = " + t.getRetCount(), log);
log("Favourite = " + t.getFavourite(), log);
log("Mentioned entities = " + t.getMentionedEntities(), log);
log("Hashtags = " + t.getHashtags(), log);
log("Tweet finished", log);
}
} catch (IOException e) {
log(e, log);
}
log("Finished reading tweets.", log);
}
private static void log(IOException e, File log) {
log(e.getMessage(), log);
StackTraceElement[] array = e.getStackTrace();
for(int i = 0; i < array.length; i++){
log(" " + array[i], log);
}
}
private static void log(String string, File log) {
try {
BufferedWriter writer = new BufferedWriter(new FileWriter(log, true));
writer.write(string);
writer.newLine();
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Tweet.java
package Testers;
public class Tweet {
private String type;
private String origin;
private String tweetText;
private String url;
private String tweetID;
private String tweetDate;
private int retCount;
private String favourit;
private String mEntities;
private String hashtags;
public Tweet(String tweetID,String origin) {
this.tweetID = tweetID;
this.origin = origin;
}
public Tweet(String type, String origin, String tweetText, String url, String tweetID, String tweetDate, int retCount, String favourit, String mEntities, String hashtags) {
this.type = type;
this.origin = origin;
this.tweetText = tweetText;
this.url = url;
this.tweetID = tweetID;
this.tweetDate = tweetDate;
this.retCount = retCount;
this.favourit = favourit;
this.mEntities = mEntities;
this.hashtags = hashtags;
}
public String getType() {
return type;
}
public String getOrigin(){
return origin;
}
public String getTweetText(){
return tweetText;
}
public String getURL(){
return url;
}
public String getTweetID(){
return tweetID;
}
public String getTweetDate(){
return tweetDate;
}
public int getRetCount(){
return retCount;
}
public String getFavourite(){
return favourit;
}
public String getMentionedEntities(){
return mEntities;
}
public String getHashtags(){
return hashtags;
}
}
//global attribute
List<Tweet> tweetList = new ArrayList<>();
String line = "";
String[] fields;
while (line != null) {
line = reader.readLine();
line = reader.readLine();
//these two are for ***
for(int i = 0;i<10;i++){
line = reader.readLine();
tweets.add(line);
// these are for the other data
}
Tweet tweet = createTweetFromList(tweets);
tweetList.add(tweet);
}
Related
Super simple program. Loop asking for student data (name, address, gpa) which is stored in a linked list. Output the data to a regular text file that can be opened in notepad.
I'm so close, but cannot figure out how to get my data written into the txt file. To start, here's my classes:
public class Student {
private String name;
private String address;
private double gpa;
public Student(String name, String address, double gpa) {
this.name = name;
this.address = address;
this.gpa = gpa;
}
public String getName() {
return this.name;
}
public String getAddress() {
return this.address;
}
public double getGpa() {
return this.gpa;
}
public String toString() {
return this.name + ", " + this.address + ", " + this.gpa;
}
public class WriteFile {
private String path;
private boolean append_to_file = false;
public WriteFile(String file_path) {
path = file_path;
}
public WriteFile(String file_path, boolean append_value) {
path = file_path;
append_to_file = append_value;
}
public void writeToFile(String textLine)throws IOException {
FileWriter write = new FileWriter(path, append_to_file);
PrintWriter print_line = new PrintWriter(write);
print_line.printf("%s" + "%n", textLine);
print_line.close();
}
}
public class ReadFile {
private String path;
public ReadFile(String file_path) {
path = file_path;
}
public String[] OpenFile() throws IOException {
FileReader fr = new FileReader(path);
BufferedReader textReader = new BufferedReader(fr);
int numberOfLines = 3;
String[] textData = new String[numberOfLines];
int i;
for (i=0; i < numberOfLines; i++) {
textData[i] = textReader.readLine();
}
textReader.close();
return textData;
}
int readLines() throws IOException {
FileReader file_to_read = new FileReader(path);
BufferedReader bf = new BufferedReader(file_to_read);
String aLine;
int numberOfLines = 0;
while ((aLine = bf.readLine()) != null) {
numberOfLines++;
}
bf.close();
return numberOfLines;
}
}
In main I can get it to write the "Text to be written to file" into my .txt file, but I still can't figure out how to write studentData into that file.
public class Main {
public static void main(String[] args) throws IOException {
LinkedList < Student > studentData = new LinkedList < > ();
Scanner scanner = new Scanner(System.in);
scanner.useDelimiter(System.lineSeparator());
while (true) {
System.out.println("Student name: ");
String name = scanner.next();
if (name.isEmpty()) {
break;
}
System.out.println("Student address: ");
String address = scanner.next();
System.out.println("Student GPA: ");
double gpa = scanner.nextDouble();
studentData.add(new Student(name, address, gpa));
}
studentData.sort(Comparator.comparing(Student::getName));
for (Student info: studentData) {
System.out.println(info.getName() + "," + info.getAddress() + "," + info.getGpa());
}
String file_name = "C:\\Users\\hidden\\OneDrive - hidden\\Desktop\\Portfolio_Project.txt";
try {
ReadFile file = new ReadFile(file_name);
String[] aryLines = file.OpenFile();
int i;
for (i = 0; i < aryLines.length; i++) {
System.out.println(aryLines[1]);
}
} catch (IOException e) {
System.out.println("Error. Please try again.");
}
// Write to a file
WriteFile data = new WriteFile(file_name);
data.writeToFile("Text to be written to file");
System.out.println("Text File Written To Portfolio_Project.txt on Desktop");
}
}
This is my first post and english is not my native language, so please don't be cruel.
As i explained in the title I have a program that poll my mail with Apache Camel, and if there's a csv file in the attachments the program take it and process it. When i take the attachments from the email the method exchange.getIn().getAttachments() return an empty container, but I am pretty sure that the email is beeing readed. I am almost sure that I am not doing something regargins SSL but I don't really know what to do.
I am using Java OSGI with annotations and this bundles included:
org.apache.camel.camel-core 2.15.6
org.apache.camel.camel-mail 2.15.6
com.sun.mail.imap 1.6.0
com.sun.mail.javax.mail 1.6.0
javax.mail.api 1.6.0
The source code is on GitHub. Interesting classes are listed below:
ConfigReaderFactory take the email informations via ConfigAdmin and build the url.
#org.osgi.service.component.annotations.Component(name = "factory", property = "service.pid=it.sensorsystem.core.config.factory", configurationPolicy = ConfigurationPolicy.IGNORE)
public class ConfigReaderFactory implements ManagedServiceFactory {
private static final String DELETE_CONDITIONS = "readLock=changed&idempotent=false&noop=true&delete=true";
private static final String NON_DELETE_CONDITIONS = "noop=true";
private volatile DependencyManager dependencyManager;
private final Map<String, Component> components = new HashMap<>();
private String url;
private String name;
private int confLine;
private String endpointType;
private String ip;
private String username;
private String password;
private String folder;
private boolean delete;
private CamelService camel;
private TypeConverter converter;
private EventPublisher publisher;
private Double latitude;
private Double longitude;
private String email;
#Reference(service = TypeConverter.class)
public void setTypeConverter(TypeConverter converter) {
this.converter = converter;
}
public void unsetTypeConverter(TypeConverter converter) {
this.converter = null;
}
#Reference(service = EventPublisher.class)
public void setEventPublisher(EventPublisher publisher) {
this.publisher = publisher;
}
public void unsetEventPublisher(EventPublisher publisher) {
this.publisher = null;
}
#Reference(service = CamelService.class)
public void setCamelService(CamelService camel) {
this.camel = camel;
}
public void unsetCamelService(CamelService camel) {
this.camel = null;
}
#Override
public String getName() {
return this.getClass().getName();
}
#Override
public void updated(String pid, #SuppressWarnings("rawtypes") Dictionary props) throws ConfigurationException {
if (components.containsKey(pid)) {
return;
}
if (props != null) {
List<String> attributes = new ArrayList<>();
List<String> csvTypes = new ArrayList<>();
int count = 1;
String configurationLine;
while ((configurationLine = (String) props.get(Integer.toString(count++))) != null) {
List<String> values = Utils.getValuesFromLine(configurationLine);
attributes.add(values.size() >= 1 ? values.get(0) : TypeConverter.NAMELESS);
csvTypes.add(values.size() >= 2 ? values.get(1) : TypeConverter.NAMELESS);
}
confLine = Integer.parseInt((String) props.get(Config.CONFIG_LINE));
name = (String) props.get(Config.NAME);
initConfigParameters(props);
buildURL();
System.out.println("[URL] " + url);
try {
Map<String, Object> params = new HashMap<>();
params.put(Constants.ATTRIBUTES, attributes);
params.put(Constants.TYPES, csvTypes);
params.put(Constants.CONF_LINE, confLine);
params.put(Constants.SOURCE, name);
params.put(Constants.PUBLISHER, publisher);
params.put(Constants.CONVERTER, converter);
params.put(Constants.LATITUDE, latitude);
params.put(Constants.LONGITUDE, longitude);
params.put(Constants.ENDPOINT_TYPE, endpointType);
params.put(Constants.EMAIL, email);
Processor processor = new CSVProcessor(params);
camel.start(FrameworkUtil.getBundle(this.getClass()).getBundleContext(), processor, url);
} catch (Exception e) {
e.printStackTrace();
}
}
}
#Override
public void deleted(String pid) {
Component component = components.remove(pid);
dependencyManager.remove(component);
component.stop();
}
private void buildURL() {
url = "";
switch(endpointType) {
case Constants.FTP:
url += "ftp://" + username + "#" + ip + "/" + folder + "?";
if(!password.equals("")) {
url += "password=" + password + "&";
}
break;
case Constants.FILE:
url += "file://" + folder + "?";
break;
case Constants.EMAIL:
url += "imaps://imap.gmail.com?username="+email+"&password="+password
+"&delete=false&unseen=false&consumer.delay=100";
}
if(endpointType.equals(Constants.FTP) || endpointType.equals(Constants.FILE)) {
if (delete) {
url += DELETE_CONDITIONS;
} else {
url += NON_DELETE_CONDITIONS;
}
}
}
private void initConfigParameters(#SuppressWarnings("rawtypes") Dictionary props) {
confLine = Integer.parseInt((String) props.get(Config.CONFIG_LINE));
name = (String) props.get(Config.NAME);
endpointType = (String) props.get(Config.ENDPOINT_TYPE);
ip = (String) props.get(Config.IP_ADDRESS);
username = (String) props.get(Config.USERNAME);
password = (String) props.get(Config.PASSWORD);
folder = (String) props.get(Config.FOLDER);
email = (String) props.get(Config.EMAIL);
delete = ((String) props.get(Config.DELETE)).equals("true");
if((String) props.get(Config.LATITUDE)!=null) {
latitude = Double.parseDouble((String) props.get(Config.LATITUDE));
}
if((String) props.get(Config.LONGITUDE)!=null) {
longitude = Double.parseDouble((String) props.get(Config.LONGITUDE));
}
printParameters();
}
private void printParameters() {
System.out.println("\nStarting camel with parameters:");
System.out.println("[sensor_name]::" + name);
System.out.println("[latitude]::" + latitude);
System.out.println("[longitude]::" + longitude);
System.out.println("[endpoint_type]::" + endpointType);
if(endpointType.equals("ftp")) {
System.out.println("[ip_address]::" + ip);
System.out.println("[folder]::" + folder);
System.out.println("[user]::" + username);
System.out.println("[password]::" + password);
} else if(endpointType.equals("file")) {
System.out.println("[folder]::" + folder);
} else if(endpointType.equals("email")) {
System.out.println("[email]::" + email);
System.out.println("[password]::" + password);
}
System.out.println("[delete]::" + delete);
}
}
CSVProcessor take the attachments from the email and process the .csv files.
public class CSVProcessor implements Processor{
private List<String> attributes;
private List<String> csvTypes;
private int confLine;
private String source;
private String endpointType;
private TypeConverter converter;
private EventPublisher publisher;
private Long timestamp;
private Double latitude;
private Double longitude;
#SuppressWarnings("unchecked")
public CSVProcessor(Map<String, Object> params) {
this.attributes = (List<String>) params.get(Constants.ATTRIBUTES);
this.csvTypes = (List<String>) params.get(Constants.TYPES);
this.confLine = (int) params.get(Constants.CONF_LINE);
this.source = (String) params.get(Constants.SOURCE);
this.publisher = (EventPublisher) params.get(Constants.PUBLISHER);
this.converter = (TypeConverter) params.get(Constants.CONVERTER);
this.latitude = (Double) params.get(Constants.LATITUDE);
this.longitude = (Double) params.get(Constants.LONGITUDE);
this.endpointType = (String) params.get(Constants.ENDPOINT_TYPE);
}
#Override
public void process(Exchange exchange) throws Exception {
if(endpointType.equals(Constants.EMAIL)) {
Map<String, DataHandler> attachments = exchange.getIn().getAttachments();
System.out.println("Detected email [attachments: "+exchange.getIn().getAttachments().size()+"]");
System.out.println("[BODY]\n"+exchange.getIn().getBody(String.class));
if(exchange.getIn().hasAttachments()) {
System.out.println("Detected email attachments: "+attachments);
for(String name : attachments.keySet()) {
DataHandler dh = attachments.get(name);
String filename = dh.getName();
System.out.println("Detected email attachment with name: "+filename);
if(filename.contains(".csv")) {
processBody(exchange.getContext().getTypeConverter()
.convertTo(String.class, dh.getInputStream()));
}
}
}
} else {
processBody(exchange.getIn().getBody(String.class));
}
}
private void processBody(String body) {
int count = 1;
for(String line : Utils.getLines(body)){
if(count++>confLine){
for(IOTEvent event: processLine(line)){
publisher.publish(event);
}
}
}
}
private boolean processTimeAndPosition(String line) {
Iterator<String> valueIterator = Utils.getValuesFromLine(line).iterator();
Iterator<String> attributeIterator = attributes.iterator();
Iterator<String> typeIterator = csvTypes.iterator();
boolean systemTimestamp = true;
while(attributeIterator.hasNext()){
String attribute = attributeIterator.next();
String type = typeIterator.next();
String value = valueIterator.next();
switch(attribute.toLowerCase()){
case Constants.TIMESTAMP:
systemTimestamp = false;
timestamp = (Long) converter.convert(type, value);
break;
case Constants.LATITUDE:
latitude = (Double) converter.convert(type, value);
break;
case Constants.LONGITUDE:
longitude = (Double) converter.convert(type, value);
break;
}
}
return systemTimestamp;
}
private List<IOTEvent> processLine(String line){
Iterator<String> valueIterator = Utils.getValuesFromLine(line).iterator();
Iterator<String> attributeIterator = attributes.iterator();
Iterator<String> typeIterator = csvTypes.iterator();
List<IOTEvent> IOTEvents = new ArrayList<>();
boolean systemTimestamp = processTimeAndPosition(line);
// Resetting iterators
valueIterator = Utils.getValuesFromLine(line).iterator();
attributeIterator = attributes.iterator();
typeIterator = csvTypes.iterator();
while(attributeIterator.hasNext()){
String attribute = attributeIterator.next();
String type = typeIterator.next();
String value = valueIterator.next();
switch(attribute.toLowerCase()){
case Constants.TIMESTAMP:
case Constants.LATITUDE:
case Constants.LONGITUDE:
continue;
}
IOTEvent iotEvent = new IOTEvent();
iotEvent.setSource(source);
iotEvent.setValue(new Value(type, converter.convert(type, value)));
if(systemTimestamp) {
iotEvent.setCdate(new Date());
} else {
iotEvent.setCdate(new Date(timestamp));
}
iotEvent.setType(attribute);
iotEvent.setSystemCDate(systemTimestamp);
if(latitude!=null && longitude!=null )
iotEvent.setPoint(new Point(latitude, longitude));
IOTEvents.add(iotEvent);
}
return IOTEvents;
}
}
DefaultCamelService starts camel:
#Component(immediate=true)
public class DefaultCamelService implements CamelService{
private CamelContext camelContext=null;
#Override
public void start(BundleContext context, Processor processor, String url) throws Exception {
if(camelContext==null) {
camelContext = new OsgiDefaultCamelContext(context);
}
camelContext.addRoutes(new RouteBuilder() {
public void configure() {
from(url).process(processor);
}
});
try {
camelContext.start();
} catch(Exception e) {
System.out.println("Error connecting to endpoint:"+url);
}
}
#Override
public void stop() throws Exception {
camelContext.stop();
}
}
This is the result on console that regards email:
Connecting to MailStore: imaps://imap.gmail.com:993 (SSL enabled), folder=INBOX
Getting folder INBOX
Polling mailbox folder: imaps://imap.gmail.com:993 (SSL enabled), folder=INBOX
Configured property: peek on bean: com.sun.mail.imap.IMAPMessage#6dd74603 with value: true
Fetching 1 messages. Total 1 messages.
Processing message: messageNumber=[1], from=[Simone Colaci <sim.colaci#gmail.com>], to=[sensorsystem.csv#gmail.com], subject=[csv testing], sentDate=[Sep 14, 2017 1:15:24 PM], receivedDate=[Sep 14, 2017 1:15:45 PM]
Detected email [attachments: 0]
Close mailbox folder INBOX from imaps://imap.gmail.com:993 (SSL enabled), folder=INBOX
Please help me, remember that this is my first post and english is not my native language. I checked everything on internet.
Thank you.
Edit 1:
This is the body of the email sent by my Gmail account to sensorsystem.csv#gmail.com:
[BODY]
--94eb2c19b976cd06bd055924656c
Content-Type: multipart/alternative; boundary="94eb2c19b976cd06ba055924656a"
--94eb2c19b976cd06ba055924656a
Content-Type: text/plain; charset="UTF-8"
Hi,
this is a csv file
--94eb2c19b976cd06ba055924656a
Content-Type: text/html; charset="UTF-8"
<div dir="ltr"><div>Hi,<br></div>this is a csv file<br><div><div><div class="gmail_quote"><br><div dir="ltr"><br></div>
</div><br></div></div></div>
--94eb2c19b976cd06ba055924656a--
--94eb2c19b976cd06bd055924656c
Content-Type: text/csv; charset="US-ASCII"; name="esrlNcepRe_LonPM180_e063_990c_26e6.csv"
Content-Disposition: attachment;
filename="esrlNcepRe_LonPM180_e063_990c_26e6.csv"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_j7kc6dcl0
dGltZSxsYXRpdHVkZSxsb25naXR1ZGUsYWlyLHByZXMKVVRDLGRlZ3JlZXNfbm9ydGgsZGVncmVl
c19lYXN0LGRlZ0ssUGFzY2FscwoyMDAyLTA3LTAxVDAwOjAwOjAwWiw0Mi41LC0xNDcuNSwyODcu
MSwxMDIyNzAuMAo=
--94eb2c19b976cd06bd055924656c--
Edit 2:
Tried with latest Camel libraries listed below but result is the same:
org.apache.camel.camel-core 2.19.3
org.apache.camel.camel-mail 2.19.3
Edit 3:
Debug also give this warnings from Java Mail:
Sep 18, 2017 10:20:59 AM javax.mail.Session loadResource
WARNING: expected resource not found: /META-INF/javamail.default.providers
Sep 18, 2017 10:20:59 AM javax.mail.Session loadResource
WARNING: expected resource not found: /META-INF/javamail.default.address.map
I am trying to run my code and I keep getting error. The json file I'm reading in is completely fine with the keys and values. And the one I am trying to write out is just a json file with "{ }" and that's it. Please help me. It's the only error I'm getting.
MAIN CLASS
import java.io.Serializable;
class LibraryOfMovieDescriptions implements Serializable {
public static void main(String[] args){
MovieLibrary movieLibrary;
movieLibrary = new MovieLibrary("movies.json");
movieLibrary.toJsonFile("movies2.json");
}
}
MovieDescription Class
import org.json.JSONException;
import org.json.JSONObject;
import org.json.JSONArray;
import java.io.Serializable;
public class MovieDescription implements Serializable {
private String title;
private String rating;
private String release;
private String runtime;
private String plot;
private String filename;
private String genre;
private String actors;
public JSONObject toJSONObject() throws JSONException {
JSONObject obj = new JSONObject();
obj.put("Title", title);
obj.put("Rated", rating);
obj.put("Released", release);
obj.put("Runtime", runtime);
obj.put("Plot", plot);
obj.put("Filename", filename);
JSONArray a = new JSONArray();
String[] sArray = this.actors.split(" , ");
for(int i = 0; i < sArray.length; i++){
a.put(i);
}
obj.put("Actors", a);
JSONArray g = new JSONArray();
String[] gArray = this.genre.split(" , ");
for(int i = 0; i < gArray.length; i++){
g.put(i);
}
obj.put("Genre", g);
return obj;
}
public MovieDescription(JSONObject jsonObj) throws JSONException{
this.title = jsonObj.getString("Title");
this.rating = jsonObj.getString("Rated");
this.release = jsonObj.getString("Released");
this.plot = jsonObj.getString("Plot");
this.runtime = jsonObj.getString("Runtime");
this.filename = jsonObj.getString("Filename");
JSONArray g = jsonObj.getJSONArray("Genre");
for(int i = 0; i < g.length(); i++){
this.genre += g.get(i) + ", ";
}
JSONArray a = jsonObj.getJSONArray("Actors");
for(int i = 0; i < a.length(); i++){
this.actors += a.get(i) + ", ";
}
}
public MovieDescription(){
title = " ";
rating = " ";
release = " ";
runtime = " ";
plot = " ";
filename = " ";
genre = " ";
actors = " ";
}
public MovieDescription(String title, String rating, String release, String runtime, String plot, String filename,
String genre, String actors){
this.title = title;
this.rating = rating;
this.release = release;
this.runtime = runtime;
this.plot = plot;
this.filename = filename;
this.genre = genre;
this.actors = actors;
}
public void setTitle(String title){
this.title = title;
}
public String getTitle(){
return title;
}
public void setRating(String rating){
this.rating = rating;
}
public String getRating(){
return rating;
}
public void setRelease(String release){
this.release = release;
}
public String getRelease(){
return this.release;
}
public void setRuntime(String runtime){
this.runtime = runtime;
}
public String getRuntime(){
return runtime;
}
public void setPlot(String plot){
this.plot = plot;
}
public String getPlot(){
return plot;
}
public void setFilename(String filename){
this.filename = filename;
}
public String getFilename(){
return filename;
}
public void setGenre(String genre){
this.genre = genre;
}
public String getGenre(){
return genre;
}
public void setActors(String actors){
this.actors = actors;
}
public String getActors(){
return actors;
}
public String toString(){
String string = ("Title: " + title + "\n" + "Rating: " + rating + "\n" + "Released: " + release + "\n" +
"Runtime: " + runtime + "\n" + "Plot: " + plot + "\n" + "Filename: " + filename + "\n" + "Genre: " + genre
+ "\n" + "Actors: " + actors + "\n");
return string;
}
}
MovieLibrary Class
import org.json.JSONObject;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import org.json.JSONTokener;
import java.io.Serializable;
import java.io.ObjectOutputStream;
public class MovieLibrary implements Serializable {
private List<MovieDescription> movieLib = new ArrayList<MovieDescription>();
private int arraySize;
public MovieLibrary(){
arraySize = 0;
}
public boolean isEmpty(){
return arraySize == 0;
}
public MovieDescription get(String aTitle){
int i = indexOf(aTitle);
if(i == -1){
return null;
}
return movieLib.get(i);
}
public boolean add(MovieDescription aClip){
movieLib.add(aClip);
arraySize++;
return true;
}
public boolean remove(String aTitle){
int i = indexOf(aTitle);
if(i != -1){
movieLib.remove(i);
arraySize--;
}
return true;
}
public String[] getTitles(){
String[] s = new String[movieLib.size()];
for(int i = 0; i < movieLib.size(); i++){
s[i] = movieLib.get(i).getTitle();
}
return s;
}
private int indexOf(String aTitle){
for(int i = 0; i < movieLib.size(); i++)
if(((movieLib.get(i)).getTitle()).equals(aTitle)){
return i;
}
return -1;
}
public MovieLibrary(String jsonFile){
FileInputStream in = null;
JSONObject jsonObj = null;
String[] titles;
try{
in = new FileInputStream(jsonFile);
jsonObj = new JSONObject(new JSONTokener(in));
titles = JSONObject.getNames(jsonObj);
System.out.println("Adding Movies...");
for(int i = 0; i < titles.length; i++){
JSONObject temp = jsonObj.getJSONObject(titles[i]);
MovieDescription movies = new MovieDescription(temp);
movieLib.add(movies);
}
System.out.println(this.getTitles());
in.close();
}
catch(Exception e){
e.printStackTrace();
if(in != null){
try{
in.close();
}
catch(IOException z){
z.printStackTrace();
System.exit(0);;
}
}
}
}
public void toJsonFile(String jsonFileName){
JSONObject jsonObj = new JSONObject();
ObjectOutputStream out = null;
try{
for(int i = 0; i < movieLib.size(); i++)
jsonObj.put(movieLib.get(i).getTitle(),movieLib.get(i).toJSONObject());
out = new ObjectOutputStream(new FileOutputStream(jsonFileName));
out.writeObject(jsonObj.toString());
out.close();
}
catch(Exception e){
e.printStackTrace();
if(out != null){
try{
out.close();
}
catch(IOException z){
z.printStackTrace();
System.exit(0);
}
}
}
}
}
And here is the error I am getting:
Adding Movies...
[Ljava.lang.String;#4554617c
Also, after I run the program and check my "movies2.json" (the one I'm trying to write to) the file turns to this.
weird stuff:
The first message is because System.out.println is passed an array of strings (this.getTitles()). To display this array of string, one must convert it to a single string Arrays.toString(this.getTitles()), or use a for loop iterating on the array to display one title per line.
The wrong output is because ObjectOutputStream is an output stream for serializing Java objects (even though in this case it is a String object) in binary. FileWriter should be a better option to consider, as its API allows you to write characters to a file.
I am trying to read from a text file of the type (Artist, Title, Genre, Recordcompany, Release year, Number of songs, Playtime):
The Beatles, Abbey Road, Rock, Apple Records, 1969, 17, 47.16
Sia, 1000 Forms of Fear, Pop, Intertia, 2014, 12, 48.41
Taylor Swift, Speak Now
I have created a CD class:
package q;
public class CD {
//
private String artist;
private String titel;
private String genre;
private String recordcompany;
private int year; //
private int songs; //
private double playtime; //
public CD() { //
}
public CD(String newArtist, String newTitel) {
artist = newArtist;
titel = newTitel;
}
public CD(String newArtist, String newTitel, String newGenre, String newRecordcompany, int newYear, int newSongs, double newPlaytime) {
artist = newArtist;
titel = newTitel;
genre = newGenre;
recordcompany = newRecordcompany;
year = newYear;
songs = newSongs;
playtime = newPlaytime;
}
public String getArtist() { //
return artist;
}
public String getTitel() {
return titel;
}
public String getGenre() {
return genre;
}
public String getRecordcompany() {
return recordcompany;
}
public int getYear() {
return year;
}
public int getsong() {
return songs;
}
public double getPlaytime() {
return playtime;
}
public void setArtist(String newArtist) { //
artist = newArtist;
}
public void setTitel(String newTitel) {
titel = newTitel;
}
public void setGenre(String newGenre) {
genre = newGenre;
}
public void setRecordcompany(String newRecordcompany) {
recordcompany = newRecordcompany;
}
public void setYear(int newYear) {
year = newYear;
}
public void setSongs(int newSongs) {
songs = newSongs;
}
public void setplaytime(double newPlaytime) {
playtime = newPlaytime;
}
#
Override public String toString() { //
return ("Artist " + artist + System.lineSeparator() + "Titel: " + titel + System.lineSeparator() + "Genre: " + genre + System.lineSeparator() + "Recordcompany: " + recordcompany + System.lineSeparator() + "Year: " + year + System.lineSeparator() + "Songs: " + songs + System.lineSeparator() + "Playtime: " + playtime + System.lineSeparator());
}
}
I am trying to read from the text file and then convert it into a arraylist. I know I have overcomplicated it by first conerting the text file to a string array and then converting it to an arraylist. I would like to use set and get methods when I create the array, if it is possible. I wonder if any of you have any tips for me for how I can make the code less complicated and include set and get methods if possible, thank you.
package q;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
public class Q {
public static void main(String[] args) throws FileNotFoundException {
String artist = "";
String titel = "";
String genre = "";
String recordcompany = "";
String year = "";
String songs = "";
String playtime = "";
try {
BufferedReader br = new BufferedReader(new FileReader("nej.txt"));
String line = null;
while ((line = br.readLine()) != null) {
String tmp[] = line.split(",");
artist = tmp[0];
titel = tmp[1];
if (tmp.length > 2) {
genre = tmp[2];
recordcompany = tmp[3];
year = (tmp[4]);
songs = (tmp[5]);
playtime = (tmp[6]);
} else {
genre = "";
recordcompany = "";
year = "";
songs = "";
playtime = "";
}
List < String > unsorted = Arrays.asList(tmp);
for (String e: unsorted) {
System.out.println(e);
}
}
br.close();
} catch (IOException e) {
System.out.println(e);
}
}
}
EDITED
package q;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
public class Q {
public static void main(String[] args) throws FileNotFoundException {
String artist = "";
String titel = "";
String genre = "";
String recordcompany = "";
int year = 0;
int songs = 0;
double playtime = 0.0;
try {
BufferedReader br = new BufferedReader(new FileReader("nej.txt"));
String line = null;
List < CD > cdsList = new ArrayList < > ();
while ((line = br.readLine()) != null) {
CD cd = new CD();
String tmp[] = line.split(",");
cd.setArtist(tmp[0]);
cd.setTitel(tmp[1]);
if (tmp.length > 2) {
cd.setGenre(tmp[2]);
cd.setRecordcompany(tmp[3]);
cd.setYear(Integer.parseInt(tmp[4].trim()));
cd.setSongs(Integer.parseInt(tmp[5].trim()));
cd.setplaytime(Double.parseDouble(tmp[6].trim()));
}
System.out.println(cdsList);
}
br.close();
} catch (IOException e) {
System.out.println(e);
}
}
}
Something like below.
public static void main(String[] args) {
try {
BufferedReader br = new BufferedReader(new FileReader("nej.txt"));
String line = null;
List < CD > cdsList = new ArrayList < > ();
while ((line = br.readLine()) != null) {
CD cd = new CD();
String tmp[] = line.split(",");
cd.setArtist(tmp[0]);
cd.setTitel(tmp[1]);
if (tmp.length > 2) {
cd.setGenre(tmp[2]);
cd.setRecordcompany(tmp[3]);
cd.setYear(Integer.parseInt(tmp[4].trim()));
cd.setSongs(Integer.parseInt(tmp[5].trim()));
cd.setplaytime(Double.parseDouble(tmp[6].trim()));
}
cdsList.add(cd);
}
System.out.println(cdsList);
br.close();
} catch (IOException e) {
System.out.println(e);
}
}
I have tool code use Google search API.
My code:
import com.google.gson.Gson;
class GoogleResults {
private ResponseData responseData;
public ResponseData getResponseData() {
return responseData;
}
public void setResponseData(ResponseData responseData) {
this.responseData = responseData;
}
public String toString() {
return "ResponseData[" + responseData + "]";
}
static class ResponseData {
private List<Result> results;
public List<Result> getResults() {
return results;
}
public void setResults(List<Result> results) {
this.results = results;
}
public String toString() {
return "Results[" + results + "]";
}
}
static class Result {
private String url;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public String toString() {
return "Result[url:" + url + " ]";
}
}
}
public class CrawData {
public static void main(String[] args) throws IOException, InterruptedException {
String query;
int n;
int k=0;
String site;
String resultset;
Scanner st = new Scanner(System.in);
System.out.print(" Input key search: ");
query = st.nextLine();
System.out.print("Input site: ");
site = st.nextLine();
System.out.print("Input number of result: ");
n = st.nextInt();
resultset = query + " site:" + site;
for (int j = 0; j < n; j = j + 1) {
Thread.sleep(4000);
String address = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&start="+j+"&q=";
String charset = "UTF-8";
URL url = new URL(address + URLEncoder.encode(resultset, charset));
Reader reader = new InputStreamReader(url.openStream(), charset);
GoogleResults results = new Gson().fromJson(reader,
GoogleResults.class);
int total = results.getResponseData().getResults().size();
// Show title and URL of each results
for (int i = 0; i <= total - 1; i++) {
String Url = results.getResponseData().getResults().get(i)
.getUrl();
k = k+1;
System.out.println("URL: " +Url+ " " + k);
}
}
}
}
when i run it, i have trouble about result return of code.
My system return list url of website.. but it not stable.
Some picture:
my error
Have error: Exception in thread "main" java.lang.NullPointerException
at CrawData.main(CrawData.java:107)
help me...
Sorry my english is too bad.. :(
My guess is that on this line:
String Url = results.getResponseData().getResults().get(i)
.getUrl();
Either get(i) is returning null or getUrl() is returning null. You should add some error handling logic:
if (results.getResponseData().getResults().get(i) != null &&
results.getResponseData().getResults().get(i).getUrl() !=null) {
String Url = results.getResponseData().getResults().get(i)
.getUrl();
k = k+1;
System.out.println("URL: " +Url+ " " + k);
} else {
// Print some type of error here. Try to figure out why the result or the
// url is null
}