Develop Reference

Java how to remove duplicates from ArrayList

I have a CSV file which contains rules and ruleversions. The CSV file looks like this:
CSV FILE:
#RULENAME, RULEVERSION
RULE,01-02-01
RULE,01-02-02
RULE,01-02-34
OTHER_RULE,01-02-04
THIRDRULE, 01-02-04
THIRDRULE, 01-02-04
As you can see, 1 rule can have 1 or more rule versions. What I need to do is read this CSV file and put them in an array. I am currently doing that with the following script:
private static List<String[]> getRulesFromFile() {
String csvFile = "rulesets.csv";
BufferedReader br = null;
String line = "";
String delimiter = ",";
List<String[]> input = new ArrayList<String[]>();
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
if (!line.startsWith("#")) {
String[] rulesetEntry = line.split(delimiter);
input.add(rulesetEntry);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return input;
}
But I need to adapt the script so that it saves the information in the following format:
ARRAY (
=> RULE => 01-02-01, 01-02-02, 01-02-04
=> OTHER_RULE => 01-02-34
=> THIRDRULE => 01-02-01, 01-02-02
)
What is the best way to do this? Multidimensional array? And how do I make sure it doesn't save the rulename more than once?

You should use a different data structure, for example an HashMap, like this.
HashMap<String, List<String>> myMap = new HashMap<>();
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
if (!line.startsWith("#")) {
String[] parts = string.split(delimiter);
String key = parts[0];
String value = parts[1];
if (myMap.containsKey(key)) {
myMap.get(key).add(value);
} else {
List<String> values = new ArrayList<String>();
values.add(value);
myMap.put(key, values);
}
}
}
This should work!

See using an ArrayList is not a good data structure of choice here.
I would personally suggest you to use a HashMap> for this particular purpose.
The rules will be your keys and rule versions will be your values which will be a list of strings.
While traversing your original file, just check if the rule (key) is present, then add the value to the list of rule versions (values) already present, otherwise add a new key and add the value to it.

For instance like this:
public List<String> removeDuplicates(List<String> myList) {
Hashtable<String, String> hashtable=new Hashtable<String, String>();
for(String s:myList) {
hashtable.put(s, s);
}
return new ArrayList<String>(hashtable.values());
}

This is exactly what key - value pairs can be used for. Just take a look at the Map Interface. There you can define a unique key containing various elements as value, perfectly for your issue.

Code:
// This collection will take String type as a Key
// and Prevent duplicates in its associated values
Map<String, HashSet<String>> map = new HashMap<String,HashSet<String>>();
// Check if collection contains the Key you are about to enter
// !REPLACE! -> "rule" with the Key you want to enter into your collection
// !REPLACE! -> "whatever" with the Value you want to associate with the key
if(!map.containsKey("rule")){
map.put("rule", new HashSet<String>());
}
else{
map.get("rule").add("whatever");
}
Reference:
Set
Map

Java - Write hashmap to a csv file

I have a hashmap with a String key and String value. It contains a large number of keys and their respective values.
For example:
key | value
abc | aabbcc
def | ddeeff
I would like to write this hashmap to a csv file such that my csv file contains rows as below:
abc,aabbcc
def,ddeeff
I tried the following example here using the supercsv library: http://javafascination.blogspot.com/2009/07/csv-write-using-java.html. However, in this example, you have to create a hashmap for each row that you want to add to your csv file. I have a large number of key value pairs which means that several hashmaps, with each containing data for one row need to be created. I would like to know if there is a more optimized approach that can be used for this use case.

Using the Jackson API, Map or List of Map could be written in CSV file. See complete example here
/**
* #param listOfMap
* #param writer
* #throws IOException
*/
public static void csvWriter(List<HashMap<String, String>> listOfMap, Writer writer) throws IOException {
CsvSchema schema = null;
CsvSchema.Builder schemaBuilder = CsvSchema.builder();
if (listOfMap != null && !listOfMap.isEmpty()) {
for (String col : listOfMap.get(0).keySet()) {
schemaBuilder.addColumn(col);
}
schema = schemaBuilder.build().withLineSeparator(System.lineSeparator()).withHeader();
}
CsvMapper mapper = new CsvMapper();
mapper.writer(schema).writeValues(writer).writeAll(listOfMap);
writer.flush();
}

Something like this should do the trick:
String eol = System.getProperty("line.separator");
try (Writer writer = new FileWriter("somefile.csv")) {
for (Map.Entry<String, String> entry : myHashMap.entrySet()) {
writer.append(entry.getKey())
.append(',')
.append(entry.getValue())
.append(eol);
}
} catch (IOException ex) {
ex.printStackTrace(System.err);
}

As your question is asking how to do this using Super CSV, I thought I'd chime in (as a maintainer of the project).
I initially thought you could just iterate over the map's entry set using CsvBeanWriter and a name mapping array of "key", "value", but this doesn't work because HashMap's internal implementation doesn't allow reflection to get the key/value.
So your only option is to use CsvListWriter as follows. At least this way you don't have to worry about escaping CSV (every other example here just joins with commas...aaarrggh!):
#Test
public void writeHashMapToCsv() throws Exception {
Map<String, String> map = new HashMap<>();
map.put("abc", "aabbcc");
map.put("def", "ddeeff");
StringWriter output = new StringWriter();
try (ICsvListWriter listWriter = new CsvListWriter(output,
CsvPreference.STANDARD_PREFERENCE)){
for (Map.Entry<String, String> entry : map.entrySet()){
listWriter.write(entry.getKey(), entry.getValue());
}
}
System.out.println(output);
}
Output:
abc,aabbcc
def,ddeeff

Map<String, String> csvMap = new TreeMap<>();
csvMap.put("Hotel Name", hotelDetails.getHotelName());
csvMap.put("Hotel Classification", hotelDetails.getClassOfHotel());
csvMap.put("Number of Rooms", hotelDetails.getNumberOfRooms());
csvMap.put("Hotel Address", hotelDetails.getAddress());
// specified by filepath
File file = new File(fileLocation + hotelDetails.getHotelName() + ".csv");
// create FileWriter object with file as parameter
FileWriter outputfile = new FileWriter(file);
String[] header = csvMap.keySet().toArray(new String[csvMap.size()]);
String[] dataSet = csvMap.values().toArray(new String[csvMap.size()]);
// create CSVWriter object filewriter object as parameter
CSVWriter writer = new CSVWriter(outputfile);
// adding data to csv
writer.writeNext(header);
writer.writeNext(dataSet);
// closing writer connection
writer.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

If you have a single hashmap it is just a few lines of code. Something like this:
Map<String,String> myMap = new HashMap<>();
myMap.put("foo", "bar");
myMap.put("baz", "foobar");
StringBuilder builder = new StringBuilder();
for (Map.Entry<String, String> kvp : myMap.entrySet()) {
builder.append(kvp.getKey());
builder.append(",");
builder.append(kvp.getValue());
builder.append("\r\n");
}
String content = builder.toString().trim();
System.out.println(content);
//use your prefered method to write content to a file - for example Apache FileUtils.writeStringToFile(...) instead of syso.
result would be
foo,bar
baz,foobar

My Java is a little limited but couldn't you just loop over the HashMap and add each entry to a string?
// m = your HashMap
StringBuilder builder = new StringBuilder();
for(Entry<String, String> e : m.entrySet())
{
String key = e.getKey();
String value = e.getValue();
builder.append(key);
builder.append(',');
builder.append(value);
builder.append(System.getProperty("line.separator"));
}
string result = builder.toString();

Updating a file in java having values in form of key-value

I have a requirement to update a file (just one particular row) which contains value in form of key value.
app.num_hosts=4
app.resourceid=broker0
I was planning to read all the file in a map, then modify the particular field and rewrite the file. Is this a good way to do update a file? Which API could i use to write a map into a file?
By searching through the existing questions i couldn't find a way to do update just single row without rewriting the entire file.

Sounds like you essentially want to use java.util.properties library.
public static void main(String[] args) {
Properties prop = new Properties();
OutputStream output = null;
try {
//load the file into properties object
FileInputStream input = new FileInputStream("config.properties");
prop.load(input);
input.close();
// set the properties value
output = new FileOutputStream("config.properties");
prop.setProperty("app.num_hosts", "4");
prop.setProperty("app.resourceid", "broker0");
prop.store(output, null);
} catch (IOException io) {
io.printStackTrace();
} finally {
if (output != null) {
try {
output.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
This blog post outlines it further, but what your going to want to do is first read the properties file, make your updates and then write it back out.

One option with Java Properties
Other wise, alternative options would be, you can use IO Api and manually update it like below:
1) Create a map, which has key and value that you want to going to update in file.
HashMap<String, String> replaceValesMap = new HashMap<String, String>();
2) Read a file from path as it gives you real path i.e. war/fileName.layout
String filepath = getServletContext().getRealPath("fileName.layout");
3) Create a method, which read file and replace value, return modified strings.
public static String getreportPdfString(HashMap<String, String> replaceValesMap,String fileppath){
String generatedString = "";
File file = new File(fileppath);
StringBuffer strContent = new StringBuffer("");
FileInputStream fin = null;
try {
fin = new FileInputStream(file);
int ch;
while ((ch = fin.read()) != -1)
strContent.append((char) ch);
fin.close();
} catch (Exception e) {
System.out.println(e);
}
String fileString= strContent.toString();
for (Map.Entry<String, String> entry : replaceValesMap.entrySet()) {
fileString = StringUtils.replace(fileString, entry.getKey(),entry.getValue());
}
return fileString;
}
4) Finally write into file:
try (PrintStream out = new PrintStream(new FileOutputStream("fileName.layout"))) {
out.print(text);
}

Search text file for a specific line

I want to search for specific lines of text in a text file. If the piece of text I am looking for is on a specific line, I would like to read further on that line for more input.
So far I have 3 tags I am looking for.
#public
#private
#virtual
If I find any of these on a line, I would like to read what comes next so for example I could have a line like this:
#public double getHeight();
If I determine that the tag I found is #public then I have to take the following part after the white-space until I reach the semicolon. The problem is, that I can't really think of an efficient way to do this without excessive use of charAt(..) which neither looks pretty but probably isn't good either in the long run for a large file, or for multiple files in a row.
I would like help to solve this efficiently as I currently can't comprehend how I would do it. The code itself is used to parse comments in a C++ file, to later generate a Header file. The Pseudo Code part is where I am stuck. Some people suggest BufferedReader, others say Scanner. I went with Scanner as that seems to be the replacement for BufferedReader.
public void run() {
Scanner scanner = null;
String filename, path;
StringBuilder puBuilder, prBuilder, viBuilder;
puBuilder = new StringBuilder();
prBuilder = new StringBuilder();
viBuilder = new StringBuilder();
for(File f : files) {
try {
filename = f.getName();
path = f.getCanonicalPath();
scanner = new Scanner(new FileReader(f));
} catch (FileNotFoundException ex) {
System.out.println("FileNotFoundException: " + ex.getMessage());
} catch (IOException ex) {
System.out.println("IOException: " + ex.getMessage());
}
String line;
while((line = scanner.nextLine()) != null) {
/**
* Pseudo Code
* if #public then
* puBuilder.append(line.substring(after white space)
* + line.substring(until and including the semicolon);
*/
}
}
}

I may be misunderstanding you.. but are you just looking for String.contains()?
if(line.contains("#public")){}

String tag = "";
if(line.startsWith("#public")){
tag = "#public";
}else if{....other tags....}
line = line.substring(tag.length(), line.indexOf(";")).trim();
This gives you a string that goes from the end of the tag (which in this case is public), and then to the character preceding the semi-colon, and then trims off the whitespace on the ends.

if (line.startsWith("#public")) {
...
}

if you are allow to use open source libraries i suggest using the apache common-io and common-lang libraries. these are widely use java librariues that will make you life a lot more simpler.
String text = null;
InputStream in = null;
List<String> lines = null;
for(File f : files) {
try{
in = new FileInputStream(f);
lines = IOUtils.readLines(in);
for (String line: lines){
if (line.contains("#public"){
text = StringUtils.substringBetween("#public", ";");
...
}
}
}
catch (Exception e){
...
}
finally{
// alway remember to close the resource
IOUtils.closeQuietly(in);
}
}

MAP Problem with JAVA and Sqlite database

I have another question for you:
The code below does the following
For each file in a folder
Open the file and read its contents
Take and divide each line into tokens
Save each token (word) in a hasMap
Prepare a database query (Select form words ...)
For each match found between the tokens and the words contained in the database Write 1.0, if true, otherwise 0.0;
The problem arises at this point:
try{
while (rs_mail.next()) {
if(result_m.contains(rs_mail.getString("voc_w").toString())) //HERE I GET THE ERROR! java.lang.NullPointerException
out_final.print("1.0;");
else
out_final.print("0.0;");
}//Close While
} //Close TRY
finally{
rs_mail.close();
//result_m.clear();
mail.clear(); //Clear MAP
}
Below the complete code:
String path ="C:/Users/.../file";
File currentDIR = new File("C:/Users/.../file");
File files_mail[]=currentDIR.listFiles();
String tmp_mail="";
// prepares the file tmpTraning.txt to receive value 1.0, 0.0 obtained by comparison with database
PrintWriter out_final=null;
File ff=new File("C:/Users/.../tmpTraning.txt");
//Seach for File in DIR
for( File fX : files_mail ){
String name_Filex = fX.getName();
FileReader fr = null;
BufferedReader fINx = null;
String sx;
//Create MAP
Map<String, Set<String>> mail = new HashMap<String, Set<String>>();
//Open File
try{
Set<String> sq = new HashSet<String>();
fr = new FileReader(path+"/"+name_Filex);
fINx = new BufferedReader(fr);
sx = fINx.readLine();
//scroll the file
while(sx != null) {
StringTokenizer stq = new StringTokenizer(sx);
while(stq.hasMoreTokens()) { //Extract form line the single word
tmp_mail = stq.nextToken();
sq.add(tmp_mail.toString().toLowerCase()); //add the word to sq -> HashMap
mail.put(nome_Filex, sq);
}// Close st.hasMoreTokens()
sx = fINx.readLine();
} //Close while for scroll File
fr.close(); //Close fileReader
sq.clear(); //Clear HasSet
} //Close il TRAY
catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
Set<String> result_m = mail.get(name_Filex);
ResultSet rs_mail = stmt.executeQuery("SELECT DISTINCT voc.words as voc_w FROM voc_words as voc");
//Prepare for writing on the file " tmpTraning.txt "
OutputStreamWriter fout_f = new OutputStreamWriter(new FileOutputStream(ff,true));
out_final = new PrintWriter(fout_f);
try{
while (rs_mail.next()) {
//If the word extract from the database is in MAP (name_Filex) then print 1.0; on the file tmpTraning.txt
if(result_m.contains(rs_mail.getString("voc_w").toString())) //HERE I GET THE ERROR! java.lang.NullPointerException
out_final.print("1.0;");
else
//else print 0.0;
out_final.print("0.0;");
}
} //Close TRY
finally{
rs_mail.close();
//result_m.clear();
mail.clear(); //Clear MAP
}
out_final.println(""); //Send CR char ASCII to set the coursor for the next file on the new line
out_final.close();
out_final.flush();
} // End SCAN DIR
Thanks for any advice!
Code changes - print the contents of result_m:
String path ="...";
File currentDIR = new File("...");
File files_mail[]=currentDIR.listFiles();
String tmp_mail="";
// prepares the file tmpTraning.txt to receive value 1.0, 0.0 obtained by comparison with database
PrintWriter out_final=null;
File ff=new File("...");
//Seach for File in DIR
for( File fX : currentDIR.listFiles() ){
String name_Filex = fX.getName();
String sx;
//Create MAP
Map<String, Set<String>> mail = new HashMap<String, Set<String>>();
//Open File
try{
Set<String> sq = new HashSet<String>();
BufferedReader fINx = new BufferedReader(new FileReader(fX));
sx = fINx.readLine();
//scroll the file
while(sx != null) {
StringTokenizer stq = new StringTokenizer(sx);
while(stq.hasMoreTokens()) { //Extract form line the single word
tmp_mail = stq.nextToken();
sq.add(tmp_mail.toString().toLowerCase()); //add the word to sq -> HashMap
mail.put(name_Filex, sq);
}// Close st.hasMoreTokens()
sx = fINx.readLine();
} //Close while for scroll File
fr.close(); //Close fileReader
sq.clear(); //Clear HasSet
} //Close il TRAY
catch (FileNotFoundException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
/*
* print the contents of result_m
*/
System.out.println("----- START FILE -----");
Set<String> result_m = mail.get(name_Filex);
Object[] toArray_m = mail.get(name_Filex).toArray();
for (int ncc=0; ncc<result_m.size();ncc++){
System.out.println(toArray_m[ncc]);
}
System.out.println("----- END FILE -----");
} // End SCAN DIR
if the file read by the program contains blank lines (no char, no string), it saves a null value

Is there a good reason that you are calling toString() on a string? If getString("voc_w") is null, then it will cause your exception.
As manji pointed out in the comment, it's either the call I mentioned, or the result_m Set that are null.

There are a great many issues with your code, unfortunately. Here's one that springs to my eye immediately.
You are using string manipulation to create the filename that you are passing in to the FileReader constructor, despite the fact that as far as I can see the file being opened is one returned from a listing of the directory. That's just asking for trouble. Instead, the code would be better off written more like this...
// Lots of things are omitted; this is just a sketch...
String path = "...";
File currentDIR = new File(path);
for (File fX : currentDIR.listFiles()) {
try {
BufferedReader fINx = new BufferedReader(new FileReader(fX));
// ...
} catch (IOException e) {
e.printErrorStack();
}
}
However, you have more problems than that. For example, the use of sq.clear() has bad code smell. You've just stowed a reference to that Set in the Map; why are you deleting its contents again? The variable is falling out of scope; you can simply leave that code out. The down-stream consequences of that clear() are that result_m later on will be an empty set, so its contains test will always return false. I can't tell offhand whether that's the cause of your rogue null, but it's got to be wrong on the basis of what you're claiming to want to do.
Try refactoring that code into several smaller pieces that are easier to verify correct. I suggest as a first cut: a private method to get the Set of words from a File (supplied as argument), a private method to compare a Set of words against the database, and a method that combines those two with some looping to achieve your overall goal.