Develop Reference

How do I load and save an arrayList<String> into a text class in java?

I am trying to implement a save and load function in my program that saves an arrayList to a textfile and then can later load all of the past lists I have saved and print them out. I am currently using these two methods:
public static void save(Serializable data, String fileName) throws Exception {
try (ObjectOutputStream oos = new ObjectOutputStream((Files.newOutputStream(Paths.get(fileName))))) {
oos.writeObject(data);
}
}
public static Object load(String fileName) throws Exception {
try (ObjectInputStream oos = new ObjectInputStream((Files.newInputStream(Paths.get(fileName))))) {
return oos.readObject();
}
}
As well as a class that represents a list as serializable data.
The problem with this is that it won't save the data after I terminate the program, and when it loads data it prints it with a great deal of extra text besides the list I want it to return. Is there a better or easier way of doing this?

I once had a same problem. So I will show you how to do it ...
Be the below Level class is what is to be saved:
public class Level implements Serializable {
private int level = 1;
private int star;
private int point;
// Constructor
public Level() {
}
public void setLevel(int level) {
this.level = level;
}
public int getLevel() {
return level;
}
public void setStar(int stars) {
this.star = stars;
}
public int getStar() {
return star;
}
public void setPoint(int points) {
this.point = points;
}
public int getPoint() {
return point;
}
#Override
public String toString() {
return "Level-" + level + " " +
(star > 1 ? star + " stars": star + " star") + " " +
(point > 1 ? point + " points" : point + " point") + "\n";
}
}
We will save the list into this file:
private static final String FILENAME = "data.level";
This is the List of our objects:
List<Level> mLevels;
Call this method to save the list into the file:
private void save() {
if(mLevels.isEmpty()) {
return;
}
try
{
ObjectOutputStream oos = new ObjectOutputStream(mContext.openFileOutput(FILENAME, Context.MODE_PRIVATE));
oos.writeObject(mLevels);
}
catch (IOException e)
{
//Toast.makeText(mContext,e.getMessage(),Toast.LENGTH_SHORT).show();
}
}
Use this method to load the list from saved file. Notice, we cast the list of our object with this (List<Level>) ois.readObject(); in the method:
private void load() {
try
{
ObjectInputStream ois = new ObjectInputStream(mContext.openFileInput(FILENAME));
mLevels = (List<Level>) ois.readObject();
}
catch (IOException e)
{
//Toast.makeText(mContext,e.getMessage(),Toast.LENGTH_SHORT).show();
}
catch (ClassNotFoundException e)
{
//Toast.makeText(mContext,e.getMessage(),Toast.LENGTH_SHORT).show();
}
if(mLevels == null) {
mLevels = new ArrayList<>();
//Toast.makeText(mContext,"List created",Toast.LENGTH_SHORT).show();
}
}
By now, you should get the idea of how to save your own list.

Logical problems causing multiple instances of a note to be created

Here is the (without most of the functions) definition of a class called note.
public class Note
{
private String text;
String fileName = "";
NoteManager noteManager = null;
List<String> hyperlinks = new ArrayList<String>();
public static final int BUFFER_SIZE = 512;
public Note(NoteManager noteManager) {
this.noteManager = noteManager;
this.text = "";
}
public Note(NoteManager noteManager, String content) {
this(noteManager);
if (content == null)
setText("");
else
setText(content);
}
public Note(NoteManager noteManager, CharSequence content) {
this(noteManager, content.toString());
}
....some functions....
public static Note newFromFile(NoteManager noteManager, Context context,
String filename) throws IOException
{
FileInputStream inputFileStream = context.openFileInput(filename);
StringBuilder stringBuilder = new StringBuilder();
byte[] buffer = new byte[BUFFER_SIZE];
int len;
while ((len = inputFileStream.read(buffer)) > 0)
{
String line = new String(buffer, 0, len);
stringBuilder.append(line);
buffer = new byte[Note.BUFFER_SIZE];
}
Note n = new Note(noteManager, stringBuilder.toString().trim());
n.fileName = filename;
inputFileStream.close();
return n;
}
.... some functions attributed to this class
}
These notes are managed by a class called NoteManager.java, which I have abbreviated below:
public class NoteManager
{
Context context=null;
ArrayList<Note> notes = new ArrayList<Note>();
..... some functions...
public void addNote(Note note)
{
if (note == null || note.noteManager != this || notes.contains(note)) return;
note.noteManager = this;
notes.add(note);
try
{
note.saveToFile(context);
} catch (IOException e)
{
e.printStackTrace();
}
}
....some functions....
public void loadNotes()
{
String[] files = context.fileList();
notes.clear();
for (String fname:files)
{
try
{
notes.add(Note.newFromFile(this, context, fname));
} catch (IOException e)
{
e.printStackTrace();
}
}
}
}
public void addNote(Note note)
{
if (note == null || notes.contains(note)) return;
note.noteManager = this;
notes.add(note);
try
{
note.saveToFile(context);
} catch (IOException e)
{
e.printStackTrace();
}
}
I am trying to work out why this notepad app creates random new notes when the app is fully shutdown and then reopened, however I just cannot see what the problem is. I have cut out all the functions which didnt seem to relate to the problem, so the logical error must be here somewhere.
How does one go about finding what I am guessing to be some kind of circular reference or lack of checks?

Android typically uses UTF-8, with multi-byte characters. Creating a new String on a arbitrary byte sub-array can have issues at begin and end, if you deviate from ASCII.
public static Note newFromFile(NoteManager noteManager, Context context,
String filename) throws IOException
{
Path path = Paths.get(filename);
byte[] bytes = Files.readAllBytes(path);
String content = new String(bytes, "UTF-8");
Note n = new Note(noteManager, content.trim());
n.fileName = filename;
noteManager.add(n); // One registration?
return n;
}
The problem of having multiple instances of a node might need the addition within newFromFile or maybe an extra check:
public void addNote(Note note)
{
if (note == null || note.noteManager != this || notes.contains(note)) {
return;
}
note.noteManager = this;
notes.add(note);
And finally a Note must be well defined.
public class Note extends Comparable<Note> {
private NoteManager noteManager:
private final String content; // Immutable.
public NoteManager(NoteManager noteManager, String content) {
this.noteManager = noteManager;
this.content = content;
}
... compare on the immutable content
... hashCode on content
Not being to be able to change the content, and comparing on the string content, means notes cannot be doubled, change in the set, mixing up the set ordering.

PIG Custom loader's getNext() is being called again and again

I have started working with Apache Pig for one of our projects. I have to create a custom input format to load our data files. For this, I followed this example Hadoop:Custom Input format. I also created my custom RecordReader implementation to read the data (we get our data in binary format from some other application) and parse that to proper JSON format.
The problem occurs when I use my custom loader in Pig script. As soon as my loader's getNext() method is invoked, it calls my custom RecordReader's nextKeyValue() method, which works fine. It reads the data properly, passes it back to my loader which parses the data and returns a Tuple. So far so good.
The problem arises when my loader's getNext() method is called again and again. It gets called, works fine, and returns the proper output (I debugged it till return statement). But then, instead of letting the execution go further, my loader gets called again. I tried to see the number of times my loader is called, and I could see the number go till 20K!
Can somebody please help me understand the problem in my code?
Loader
public class SimpleTextLoaderCustomFormat extends LoadFunc {
protected RecordReader in = null;
private byte fieldDel = '\t';
private ArrayList<Object> mProtoTuple = null;
private TupleFactory mTupleFactory = TupleFactory.getInstance();
#Override
public Tuple getNext() throws IOException {
Tuple t = null;
try {
boolean notDone = in.nextKeyValue();
if (!notDone) {
return null;
}
String value = (String) in.getCurrentValue();
byte[] buf = value.getBytes();
int len = value.length();
int start = 0;
for (int i = 0; i < len; i++) {
if (buf[i] == fieldDel) {
readField(buf, start, i);
start = i + 1;
}
}
// pick up the last field
readField(buf, start, len);
t = mTupleFactory.newTupleNoCopy(mProtoTuple);
mProtoTuple = null;
} catch (InterruptedException e) {
int errCode = 6018;
String errMsg = "Error while reading input";
e.printStackTrace();
throw new ExecException(errMsg, errCode,
PigException.REMOTE_ENVIRONMENT, e);
}
return t;
}
private void readField(byte[] buf, int start, int end) {
if (mProtoTuple == null) {
mProtoTuple = new ArrayList<Object>();
}
if (start == end) {
// NULL value
mProtoTuple.add(null);
} else {
mProtoTuple.add(new DataByteArray(buf, start, end));
}
}
#Override
public InputFormat getInputFormat() throws IOException {
//return new TextInputFormat();
return new CustomStringInputFormat();
}
#Override
public void setLocation(String location, Job job) throws IOException {
FileInputFormat.setInputPaths(job, location);
}
#Override
public void prepareToRead(RecordReader reader, PigSplit split)
throws IOException {
in = reader;
}
Custom InputFormat
public class CustomStringInputFormat extends FileInputFormat<String, String> {
#Override
public RecordReader<String, String> createRecordReader(InputSplit arg0,
TaskAttemptContext arg1) throws IOException, InterruptedException {
return new CustomStringInputRecordReader();
}
}
Custom RecordReader
public class CustomStringInputRecordReader extends RecordReader<String, String> {
private String fileName = null;
private String data = null;
private Path file = null;
private Configuration jc = null;
private static int count = 0;
#Override
public void close() throws IOException {
// jc = null;
// file = null;
}
#Override
public String getCurrentKey() throws IOException, InterruptedException {
return fileName;
}
#Override
public String getCurrentValue() throws IOException, InterruptedException {
return data;
}
#Override
public float getProgress() throws IOException, InterruptedException {
return 0;
}
#Override
public void initialize(InputSplit genericSplit, TaskAttemptContext context)
throws IOException, InterruptedException {
FileSplit split = (FileSplit) genericSplit;
file = split.getPath();
jc = context.getConfiguration();
}
#Override
public boolean nextKeyValue() throws IOException, InterruptedException {
InputStream is = FileSystem.get(jc).open(file);
StringWriter writer = new StringWriter();
IOUtils.copy(is, writer, "UTF-8");
data = writer.toString();
fileName = file.getName();
writer.close();
is.close();
System.out.println("Count : " + ++count);
return true;
}
}

Try this in Loader
//....
boolean notDone = ((CustomStringInputFormat)in).nextKeyValue();
//...
Text value = new Text(((CustomStringInputFormat))in.getCurrentValue().toString())

I'm getting a Null pointer exception in my code because of the hashmap not being filled or maybe not

I was given this exercise:
Implement the following class that loads and prints a set of data values.
import java.util.Iterator;
public class MyLoader {
public void loadAndPrintValues(Iterator<String> keysToLoad, Data data, Printer printer) {
// Load data values like this:
// String value = data.loadValue(key);
// Print loaded data value like this:
// printer.printEntry(key, value);
}
}
However when I did the exercise I got a NullPointerException, probably from the while (keysToLoad.hasNext()) or from the key = keysToLoad.next();. I assume I got the exception because "data" was not getting filled but I can't figure out how to do it. Here is my code and error message
public interface Data{
//I made this method
public void makeEntry(String key, String value);
//given
public String loadValue(String key);
}
public interface Printer {
public void displayEntry(String key, String value);
}
import java.util.Iterator;
import java.util.HashMap;
public class MyLoader implements Data, Printer {
Data data; // = new MyLoader();
Printer printer; // = new MyLoader();
Iterator<String> iter; // = new MyLoader();
String key = "";
String value = "";
HashMap<String, String> ht = new HashMap<String, String>();
public MyLoader(){
// this.database = null;
// this.key = null;
// this.value = null;
// this.ht = null;
// this.iter = null;
System.out.println("now in the constructor");
}
public MyLoader(Iterator<String> iter, Data data, Printer printer){
this.data = data;
this.printer = printer;
this.iter = iter;
}
public void loadAndPrintValues(Iterator<String> keysToLoad, Data data, Printer printer) {
try {
if (ht.isEmpty()){
System.out.println("ht is empty");
throw new NullPointerException("Database is empty.");
}else {
this.data = data;
}
while (keysToLoad.hasNext()){
// Load data values like this:
key = keysToLoad.next();
value = data.loadValue(key);
// Print loaded data value like this:
printer.printEntry(key, value);
}
}catch (NullPointerException npe){
System.out.println("caught null pointer ");
System.out.println(npe.getMessage());
}
}
#Override
public void makeEntry(String key, String value){
ht.put(key, value);
}
#Override
public void printEntry(String key, String value) {
System.out.println("[" + key + " : " + value + "]");
}
#Override
public String loadValue(String key) {
System.out.println("loadValue:" + key);
if(this.ht.containsKey(key))
return this.ht.get(key);
else {
System.out.println("No key in database.");
throw new NullPointerException("No key in data.");
}
}
}
public class test {
public static void main(String[] args) {
try {
MyLoader mdbl = new MyLoader();
mdbl.makeEntry("0", "zero");
mdbl.makeEntry("1", "One");
mdbl.makeEntry("2", "Two");
mdbl.makeEntry("3", "Three");
mdbl.loadAndPrintValues(mdbl.iter, mdbl.data, mdbl.printer);
}
catch(NullPointerException e){
System.out.println(e.getMessage());
}
}
}

mdbl.iter is never assigned a value, i.e. is null (by default, since you are using the MyLoader constructor without arguments in which iter is not assigned). So when you pass it to a method that tries to do operations on it, you naturally get a NullPointerException (not an error).
You should not have
try{ /* ... */ } catch (NullPointerException npe){ /* ... */ }
blocks because any decent IDE will allow you to get to the line where the exception was thrown with one click, which is not the case if you catch the exception and simply print a message to System.out.

You are passing null values for Iterator, Data and Printer parameter of the method loadAndPrintValues() which will result in NullPointerException.

in this line
mdbl.loadAndPrintValues(mdbl.iter, mdbl.data, mdbl.printer);
you didn't initialize the mdbl.data and mdbl.printer

Never ever catch a NullPointerException. Remove the catch block with NullPointerException, run it and then post the stack trace.

using dbpedia spotlight in java or scala

Does anyone know where to find a little how to on using dbpedia spotlight in java or scala? Or could anyone explain how it's done? I can't find any information on this...

The DBpedia Spotlight wiki pages would be a good place to start.
And I believe the installation page has listed the most popular ways (using a jar, or set up a web service) to use the application.
It includes instructions on using the Java/Scala API with your own installation, or calling the Web Service.
There are some additional data needed to be downloaded to run your own server for full service, good time to make a coffee for yourself.

you need download dbpedia spotlight (jar file) after that u can use next two classes ( author pablomendes ) i only make some change .
public class db extends AnnotationClient {
//private final static String API_URL = "http://jodaiber.dyndns.org:2222/";
private static String API_URL = "http://spotlight.dbpedia.org:80/";
private static double CONFIDENCE = 0.0;
private static int SUPPORT = 0;
private static String powered_by ="non";
private static String spotter ="CoOccurrenceBasedSelector";//"LingPipeSpotter"=Annotate all spots
//AtLeastOneNounSelector"=No verbs and adjs.
//"CoOccurrenceBasedSelector" =No 'common words'
//"NESpotter"=Only Per.,Org.,Loc.
private static String disambiguator ="Default";//Default ;Occurrences=Occurrence-centric;Document=Document-centric
private static String showScores ="yes";
#SuppressWarnings("static-access")
public void configiration(double CONFIDENCE,int SUPPORT,
String powered_by,String spotter,String disambiguator,String showScores){
this.CONFIDENCE=CONFIDENCE;
this.SUPPORT=SUPPORT;
this.powered_by=powered_by;
this.spotter=spotter;
this.disambiguator=disambiguator;
this.showScores=showScores;
}
public List<DBpediaResource> extract(Text text) throws AnnotationException {
LOG.info("Querying API.");
String spotlightResponse;
try {
String Query=API_URL + "rest/annotate/?" +
"confidence=" + CONFIDENCE
+ "&support=" + SUPPORT
+ "&spotter=" + spotter
+ "&disambiguator=" + disambiguator
+ "&showScores=" + showScores
+ "&powered_by=" + powered_by
+ "&text=" + URLEncoder.encode(text.text(), "utf-8");
LOG.info(Query);
GetMethod getMethod = new GetMethod(Query);
getMethod.addRequestHeader(new Header("Accept", "application/json"));
spotlightResponse = request(getMethod);
} catch (UnsupportedEncodingException e) {
throw new AnnotationException("Could not encode text.", e);
}
assert spotlightResponse != null;
JSONObject resultJSON = null;
JSONArray entities = null;
try {
resultJSON = new JSONObject(spotlightResponse);
entities = resultJSON.getJSONArray("Resources");
} catch (JSONException e) {
//throw new AnnotationException("Received invalid response from DBpedia Spotlight API.");
}
LinkedList<DBpediaResource> resources = new LinkedList<DBpediaResource>();
if(entities!=null)
for(int i = 0; i < entities.length(); i++) {
try {
JSONObject entity = entities.getJSONObject(i);
resources.add(
new DBpediaResource(entity.getString("#URI"),
Integer.parseInt(entity.getString("#support"))));
} catch (JSONException e) {
LOG.error("JSON exception "+e);
}
}
return resources;
}
}
second class
/**
* #author pablomendes
*/
public abstract class AnnotationClient {
public Logger LOG = Logger.getLogger(this.getClass());
private List<String> RES = new ArrayList<String>();
// Create an instance of HttpClient.
private static HttpClient client = new HttpClient();
public List<String> getResu(){
return RES;
}
public String request(HttpMethod method) throws AnnotationException {
String response = null;
// Provide custom retry handler is necessary
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false));
try {
// Execute the method.
int statusCode = client.executeMethod(method);
if (statusCode != HttpStatus.SC_OK) {
LOG.error("Method failed: " + method.getStatusLine());
}
// Read the response body.
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
} catch (HttpException e) {
LOG.error("Fatal protocol violation: " + e.getMessage());
throw new AnnotationException("Protocol error executing HTTP request.",e);
} catch (IOException e) {
LOG.error("Fatal transport error: " + e.getMessage());
LOG.error(method.getQueryString());
throw new AnnotationException("Transport error executing HTTP request.",e);
} finally {
// Release the connection.
method.releaseConnection();
}
return response;
}
protected static String readFileAsString(String filePath) throws java.io.IOException{
return readFileAsString(new File(filePath));
}
protected static String readFileAsString(File file) throws IOException {
byte[] buffer = new byte[(int) file.length()];
#SuppressWarnings("resource")
BufferedInputStream f = new BufferedInputStream(new FileInputStream(file));
f.read(buffer);
return new String(buffer);
}
static abstract class LineParser {
public abstract String parse(String s) throws ParseException;
static class ManualDatasetLineParser extends LineParser {
public String parse(String s) throws ParseException {
return s.trim();
}
}
static class OccTSVLineParser extends LineParser {
public String parse(String s) throws ParseException {
String result = s;
try {
result = s.trim().split("\t")[3];
} catch (ArrayIndexOutOfBoundsException e) {
throw new ParseException(e.getMessage(), 3);
}
return result;
}
}
}
public void saveExtractedEntitiesSet(String Question, LineParser parser, int restartFrom) throws Exception {
String text = Question;
int i=0;
//int correct =0 ; int error = 0;int sum = 0;
for (String snippet: text.split("\n")) {
String s = parser.parse(snippet);
if (s!= null && !s.equals("")) {
i++;
if (i<restartFrom) continue;
List<DBpediaResource> entities = new ArrayList<DBpediaResource>();
try {
entities = extract(new Text(snippet.replaceAll("\\s+"," ")));
System.out.println(entities.get(0).getFullUri());
} catch (AnnotationException e) {
// error++;
LOG.error(e);
e.printStackTrace();
}
for (DBpediaResource e: entities) {
RES.add(e.uri());
}
}
}
}
public abstract List<DBpediaResource> extract(Text text) throws AnnotationException;
public void evaluate(String Question) throws Exception {
evaluateManual(Question,0);
}
public void evaluateManual(String Question, int restartFrom) throws Exception {
saveExtractedEntitiesSet(Question,new LineParser.ManualDatasetLineParser(), restartFrom);
}
}
main()
public static void main(String[] args) throws Exception {
String Question ="Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
System.out.println("resource : "+c.getResu());
}

I just add one little fix for your answer.
Your code is running, if you add the evaluate method call:
public static void main(String[] args) throws Exception {
String question = "Is the Amazon river longer than the Nile River?";
db c = new db ();
c.configiration(0.0, 0, "non", "CoOccurrenceBasedSelector", "Default", "yes");
c.evaluate(question);
System.out.println("resource : "+c.getResu());
}
Lamine

In the request method of the second class (AnnotationClient) in Adel's answer, the author Pablo Mendes hasn't finished
TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
which is an annoying warning that needs to be removed by replacing
byte[] responseBody = method.getResponseBody(); //TODO Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended.
// Deal with the response.
// Use caution: ensure correct character encoding and is not binary data
response = new String(responseBody);
with
Reader in = new InputStreamReader(method.getResponseBodyAsStream(), "UTF-8");
StringWriter writer = new StringWriter();
org.apache.commons.io.IOUtils.copy(in, writer);
response = writer.toString();