I am trying to read a text file in that contains numbers separated by commas. the file is large and may contain up to a few thousand numbers. i need to add these numbers to a list
List<Integer> listIntegers = new ArrayList<Integer>();
what would be the best approach to take? I am currently reading in the file like this;
StringBuilder sb = new StringBuilder();
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader("D:\\generated30-1.cav"));
String line = null;
while ((line = br.readLine()) != null)
{
sb.append(line.replaceAll(",",""));
if (sb.length() > 0)
{
sb.append("\n");
}
}
}
catch (IOException e)
{
e.printStackTrace();
}
finally
{
try
{
if (br != null)
{
br.close();
}
}
catch (IOException ex)
{
ex.printStackTrace();
}
}
String contents = sb.toString();
If the numbers are separated by commas, you should not be removing the commas. I would use a Scanner. I would use try-with-resources instead of an explicit close(). And I would split each line on comma (\\s* globs optional whitespace). Like,
List<Integer> listIntegers = new ArrayList<>();
File f = new File("D:\\generated30-1.cav");
try (Scanner sc = new Scanner(f)) {
while (sc.hasNextLine()) {
String line = sc.nextLine();
String[] tokens = line.split("\\s*,\\s*");
for (String token : tokens) {
listIntegers.add(Integer.parseInt(token));
}
}
} catch (Exception e) {
e.printStackTrace();
}
return listIntegers;
Since your input file does not contain spaces after the commas, then you should not be replacing the commas, as if you have multi digit numbers, you will not be able to differentiate between where the number starts and ends. Instead just append line:
sb.append(line);
And then you can do:
List<Integer> list = Arrays.stream(contents.split(","))
.map(Integer::valueOf)
.collect(toList());
Which will create a stream from the Array created by split and then map them to int's and then collect them to a List
I'd definitely try and leverage the Files.lines as of JDK8:
List<Integer> result;
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
stream.map(s -> /* perform your mapping operation here */)
.collect(Collectors.toList());
} catch (IOException e) { e.printStackTrace(); }
reading:
Introduction to Java 8
Streams
Java 8 Map, Filter, and Collect Examples
The map method documentation
Related
I'm currently trying to make a program in java to sort a content of a file in alphabetical order, the file contains :
camera10:192.168.112.43
camera12:192.168.2.112
camera1:127.0.0.1
camera2:133.192.31.42
camera3:145.154.42.58
camera8:192.168.12.205
camera3:192.168.2.122
camera5:192.168.112.54
camera123:192.168.2.112
camera4:192.168.112.1
camera6:192.168.112.234
camera7:192.168.112.20
camera9:192.168.2.112
And I would like to sort them and write that back into the file (which in this case is "daftarCamera.txt"). But somehow my algorithm sort them in the wrong way, which the result is :
camera10:192.168.112.43
camera123:192.168.2.112
camera12:192.168.2.112
camera1:127.0.0.1
camera2:133.192.31.42
camera3:145.154.42.58
camera3:192.168.2.122
camera4:192.168.112.1
camera5:192.168.112.54
camera6:192.168.112.234
camera7:192.168.112.20
camera8:192.168.12.205
camera9:192.168.2.112
while the result I want is :
camera1:127.0.0.1
camera2:133.192.31.42
camera3:145.154.42.58
camera3:192.168.2.122
camera4:192.168.112.1
camera5:192.168.112.54
camera6:192.168.112.234
camera7:192.168.112.20
camera8:192.168.12.205
camera9:192.168.2.112
camera10:192.168.112.43
camera12:192.168.2.112
camera123:192.168.2.112
Here's the code I use :
public void sortCamera() {
BufferedReader reader = null;
BufferedWriter writer = null;
ArrayList <String> lines = new ArrayList<String>();
try {
reader = new BufferedReader(new FileReader (log));
String currentLine = reader.readLine();
while (currentLine != null){
lines.add(currentLine);
currentLine = reader.readLine();
}
Collections.sort(lines);
writer = new BufferedWriter(new FileWriter(log));
for (String line : lines) {
writer.write(line);
writer.newLine();
}
} catch(IOException e) {
e.printStackTrace();
} finally {
try {
if (reader != null) {
reader.close();
}
if(writer != null){
writer.close();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}
You are currently performing a lexical sort, to perform a numerical sort you would need a Comparator that sorts the lines numerically on the value between "camera" and : in each line. You could split by :, and then use a regular expression to grab the digits, parse them and then compare. Like,
Collections.sort(lines, (String a, String b) -> {
String[] leftTokens = a.split(":"), rightTokens = b.split(":");
Pattern p = Pattern.compile("camera(\\d+)");
int left = Integer.parseInt(p.matcher(leftTokens[0]).replaceAll("$1"));
int right = Integer.parseInt(p.matcher(rightTokens[0]).replaceAll("$1"));
return Integer.compare(left, right);
});
Making a fully reproducible example
List<String> lines = new ArrayList<>(Arrays.asList( //
"camera10:192.168.112.43", //
"camera12:192.168.2.112", //
"camera1:127.0.0.1", //
"camera2:133.192.31.42", //
"camera3:145.154.42.58", //
"camera8:192.168.12.205", //
"camera3:192.168.2.122", //
"camera5:192.168.112.54", //
"camera123:192.168.2.112", //
"camera4:192.168.112.1", //
"camera6:192.168.112.234", //
"camera7:192.168.112.20", //
"camera9:192.168.2.112"));
Collections.sort(lines, (String a, String b) -> {
String[] leftTokens = a.split(":"), rightTokens = b.split(":");
Pattern p = Pattern.compile("camera(\\d+)");
int left = Integer.parseInt(p.matcher(leftTokens[0]).replaceAll("$1"));
int right = Integer.parseInt(p.matcher(rightTokens[0]).replaceAll("$1"));
return Integer.compare(left, right);
});
System.out.println(lines);
And that outputs
[camera1:127.0.0.1, camera2:133.192.31.42, camera3:145.154.42.58, camera3:192.168.2.122, camera4:192.168.112.1, camera5:192.168.112.54, camera6:192.168.112.234, camera7:192.168.112.20, camera8:192.168.12.205, camera9:192.168.2.112, camera10:192.168.112.43, camera12:192.168.2.112, camera123:192.168.2.112]
Your sorting algorithm assumes the ordinary collating sequence, i.e. sorts the strings by alphabetical order, as if the digits were letters. Hence the shorter string come first.
You need to specify an ad-hoc comparison function that splits the string and extracts the numerical value of the suffix.
I have a CSV file which contains rules and ruleversions. The CSV file looks like this:
CSV FILE:
#RULENAME, RULEVERSION
RULE,01-02-01
RULE,01-02-02
RULE,01-02-34
OTHER_RULE,01-02-04
THIRDRULE, 01-02-04
THIRDRULE, 01-02-04
As you can see, 1 rule can have 1 or more rule versions. What I need to do is read this CSV file and put them in an array. I am currently doing that with the following script:
private static List<String[]> getRulesFromFile() {
String csvFile = "rulesets.csv";
BufferedReader br = null;
String line = "";
String delimiter = ",";
List<String[]> input = new ArrayList<String[]>();
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
if (!line.startsWith("#")) {
String[] rulesetEntry = line.split(delimiter);
input.add(rulesetEntry);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return input;
}
But I need to adapt the script so that it saves the information in the following format:
ARRAY (
=> RULE => 01-02-01, 01-02-02, 01-02-04
=> OTHER_RULE => 01-02-34
=> THIRDRULE => 01-02-01, 01-02-02
)
What is the best way to do this? Multidimensional array? And how do I make sure it doesn't save the rulename more than once?
You should use a different data structure, for example an HashMap, like this.
HashMap<String, List<String>> myMap = new HashMap<>();
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
if (!line.startsWith("#")) {
String[] parts = string.split(delimiter);
String key = parts[0];
String value = parts[1];
if (myMap.containsKey(key)) {
myMap.get(key).add(value);
} else {
List<String> values = new ArrayList<String>();
values.add(value);
myMap.put(key, values);
}
}
}
This should work!
See using an ArrayList is not a good data structure of choice here.
I would personally suggest you to use a HashMap> for this particular purpose.
The rules will be your keys and rule versions will be your values which will be a list of strings.
While traversing your original file, just check if the rule (key) is present, then add the value to the list of rule versions (values) already present, otherwise add a new key and add the value to it.
For instance like this:
public List<String> removeDuplicates(List<String> myList) {
Hashtable<String, String> hashtable=new Hashtable<String, String>();
for(String s:myList) {
hashtable.put(s, s);
}
return new ArrayList<String>(hashtable.values());
}
This is exactly what key - value pairs can be used for. Just take a look at the Map Interface. There you can define a unique key containing various elements as value, perfectly for your issue.
Code:
// This collection will take String type as a Key
// and Prevent duplicates in its associated values
Map<String, HashSet<String>> map = new HashMap<String,HashSet<String>>();
// Check if collection contains the Key you are about to enter
// !REPLACE! -> "rule" with the Key you want to enter into your collection
// !REPLACE! -> "whatever" with the Value you want to associate with the key
if(!map.containsKey("rule")){
map.put("rule", new HashSet<String>());
}
else{
map.get("rule").add("whatever");
}
Reference:
Set
Map
I am trying to replace ? with - in my text document but just the ArrayList<String> is being written in the new file without all lines of the old one. How can I fix that?
File file = new File("D:\\hl_sv\\L09MF.txt");
ArrayList<String> lns = new ArrayList<String>();
Scanner scanner;
try {
scanner = new Scanner(file);
int lineNum = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
lineNum++;
if (line.contains("?")) {
line = line.replace("?", "-");
lns.add(line);
// System.out.println("I found it on line " + lineNum);
}
}
lines.clear();
lines = lns;
System.out.println("Test: " + lines);
FileWriter writer;
try {
writer = new FileWriter("D:\\hl_sv\\L09MF2.txt");
for (String str : lines) {
writer.write(str);
}
writer.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
I don't understand why you're storing the lines in a List to begin with. I would perform the transform and print while I read. You don't need to test for the presence of the ? (replace won't alter anything if it isn't present). And, I would also use a try-with-resources. Something like
File file = new File("D:\\hl_sv\\L09MF.txt");
try (PrintWriter writer = new PrintWriter("D:\\hl_sv\\L09MF2.txt");
Scanner scanner = new Scanner(file)) {
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
writer.println(line.replace('?', '-'));
}
} catch (Exception e) {
e.printStackTrace();
}
Examine this code:
if (line.contains("?")) {
line = line.replace("?", "-");
lns.add(line);
}
You are only adding the current line (with the replacement) if it had a ? in it, ignoring other lines. Restructure it to always add the existing line.
if (line.contains("?")) {
line = line.replace("?", "-");
}
lns.add(line);
Additionally, the part
if (line.contains("?"))
scans line to look for a ?, and then the code
line.replace("?", "-");
does the same thing, but this time also replacing any ? with -. You may as well scan line just once:
lns.add(line.replace("?", "-"));
Note that creating an ArrayList just to hold the new lines wastes a fair amount of memory if the file is large. A better pattern would be to write each line, modified if necessary, right after you read in the corresponding line.
Within your while loop you have an if statement checking the line which adds the altered line to the array. You also need to add the unaltered lines to the array.
This should fix your issue:
int lineNum = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
lineNum++;
if (line.contains("?")) {
line = line.replace("?", "-");
lns.add(line);
// System.out.println("I found it on line " + lineNum);
}
else{
lns.add(line);
}
Previously, you were only adding the line to your ArrayList if it contained a "?" character. You need to add the line to the ArrayList whether or not it contains "?"
I would use a different approach if I'm trying to work on the functionality you want to implement, please check this approach and tell me if this helps you :)
public void saveReplacedFile() {
//1. Given a file in your system
File file = new File("D:\\hl_sv\\L09MF.txt");
try {
//2. I will read it, not necessarily with Scanner, but use a BufferedReader instead
BufferedReader bufferedReader = new BufferedReader(new FileReader(file));
//3. Define a variable that will hold the value of each line
String line = null;
//and also the information of your file
StringBuilder contentHolder = new StringBuilder();
//why not get your line separator based on your O.S?
String lineSeparator = System.getProperty("line.separator");
//4. Check your file line by line
while ((line = bufferedReader.readLine()) != null) {
contentHolder.append(line);
contentHolder.append(lineSeparator);
}
//5. By this point, your contentHolder will contain all the data of your text
//But it is still a StringBuilder type object, why not convert it to a String?
String contentAsString = contentHolder.toString();
//6. Now we can replace your "?" with "-"
String replacedString = contentAsString.replace("?", "-");
//7. Now, let's save it in a new file using BufferedWriter :)
File fileToBeSaved = new File("D:\\hl_sv\\L09MF2.txt");
BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter(fileToBeSaved));
bufferedWriter.write(replacedString);
//Done :)
} catch (FileNotFoundException e) {
// Exception thrown if the file does not exist in your system
e.printStackTrace();
} catch (IOException e) {
// Exception thrown due to an issue with IO
e.printStackTrace();
}
}
Hope this is helpful. Happy coding :)
If you can use Java 8 then your code can be simplified to
try (PrintStream ps = new PrintStream("D:\\hl_sv\\L09MF2.txt");
Stream<String> stream = Files.lines(Paths.get("D:\\hl_sv\\L09MF.txt"))) {
stream.map(line -> line.replace('?', '-')).forEach(ps::println);
} catch (IOException e) {
e.printStackTrace();
}
I know there is already a question related to this: How to keep quotes when parsing csv file? (But it's for C#)
Let's say I have a csv with values e.g:
12312414-DEF_234, "34-DE, 234-EG, 36354-EJ", 23
...
When I parse it with OpenCSV, it doesn't keep the quotes.
CSVReader reader = new CSVReader(new FileReader("../path.csv"), ',', '\"');
List<String[]> list = reader.readAll();
String[][] csvArray = new String[list.size()][];
csvArray = list.toArray(csvArray);
So, after I store all of the values into an array, when I try to print out the values (for checking), the quotes are not there.
...
System.out.println(csvArray[i][j]);
// output below
// 34-DE, 234-EG, 36354-EJ
How can I keep the quotes? The reason is because I am going to be changing some values, and need to re-output it back into a csv.
The CSVReader has to parse and remove the quotes, otherwise you wouldn't get one value 34-DE, 234-EG, 36354-EJ, but three values "34-DE, 234-EG and 36354-EJ". So it's OK that the quotes are being removed.
The CSVWriter should add them again for every value that needs quoting.
Have you tried to write the array back into a CSV? The value 34-DE, 234-EG, 36354-EJ - actually any value that contains a comma - should be quoted.
public static void readCSV(){
String csvFile = "input.csv";
BufferedReader br = null;
String line = "";
String splitter = ",";
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
// use comma as separator
String[] words = line.split(splitter);
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
So far, I have 2 arrays: one with stock codes and one with a list of file names. What I want to do is input the .txt files from each of the file names from the second array and then split this input into: 1. Arrays for each file 2. Arrays for each part with each file.
I have this:
ImportFiles f1 = new ImportFiles("File");
for (String file : FileArray.filearray) {
if (debug) {
System.out.println(file);
}
try {
String line;
String fileext = "C:\\ASCIIpdbSKJ\\"+file+".txt";
importstart = new BufferedReader(new FileReader(fileext));
for (line = importstart.readLine(); line != null; line = importstart.readLine()) {
importarray.add (line);
if (debug){
System.out.println(importarray.size());
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
importarray.add ("End")
This approach works to create a large array of all the files, will it be easier to change the input method to split it as it is coming in or split the large array I have?
At this point, the stock code array is irrelevant. Once I have split the arrays down I know where I will go from there.
Thanks.
Edit: I am aware that this code is incomplete in terms of { } but it is only printstreams and debugging missed off.
If you want to get a map with a filename and all its lines from all the files, here are relevant code parts:
Map<String, List<String>> fileLines = new HashMap<String, List<String>>();
for (String file : FileArray.filearray)
BufferedReader reader = new BufferedReader(new FileReader(fileext));
List<String> lines = new ArrayList<String>();
while ((line = reader.readLine()) != null){
lines.add(line);
}
fileLines.put(file, lines);
}