Develop Reference

Split lines with "," and do a trim on every element [duplicate]

This is some code that I found to help with reading in a 2D Array, but the problem I am having is this will only work when reading a list of number structured like:
73
56
30
75
80
ect..
What I want is to be able to read multiple lines that are structured like this:
1,0,1,1,0,1,0,1,0,1
1,0,0,1,0,0,0,1,0,1
1,1,0,1,0,1,0,1,1,1
I just want to essentially import each line as an array, while structuring them like an array in the text file.
Everything I have read says to use scan.usedelimiter(","); but everywhere I try to use it the program throws straight to the catch that replies "Error converting number". If anyone can help I would greatly appreciate it. I also saw some information about using split for the buffered reader, but I don't know which would be better to use/why/how.
String filename = "res/test.txt"; // Finds the file you want to test.
try{
FileReader ConnectionToFile = new FileReader(filename);
BufferedReader read = new BufferedReader(ConnectionToFile);
Scanner scan = new Scanner(read);
int[][] Spaces = new int[10][10];
int counter = 0;
try{
while(scan.hasNext() && counter < 10)
{
for(int i = 0; i < 10; i++)
{
counter = counter + 1;
for(int m = 0; m < 10; m++)
{
Spaces[i][m] = scan.nextInt();
}
}
}
for(int i = 0; i < 10; i++)
{
//Prints out Arrays to the Console, (not needed in final)
System.out.println("Array" + (i + 1) + " is: " + Spaces[i][0] + ", " + Spaces[i][1] + ", " + Spaces[i][2] + ", " + Spaces[i][3] + ", " + Spaces[i][4] + ", " + Spaces[i][5] + ", " + Spaces[i][6]+ ", " + Spaces[i][7]+ ", " + Spaces[i][8]+ ", " + Spaces[i][9]);
}
}
catch(InputMismatchException e)
{
System.out.println("Error converting number");
}
scan.close();
read.close();
}
catch (IOException e)
{
System.out.println("IO-Error open/close of file" + filename);
}
}

I provide my code here.
public static int[][] readArray(String path) throws IOException {
//1,0,1,1,0,1,0,1,0,1
int[][] result = new int[3][10];
BufferedReader reader = new BufferedReader(new FileReader(path));
String line = null;
Scanner scanner = null;
line = reader.readLine();
if(line == null) {
return result;
}
String pattern = createPattern(line);
int lineNumber = 0;
MatchResult temp = null;
while(line != null) {
scanner = new Scanner(line);
scanner.findInLine(pattern);
temp = scanner.match();
int count = temp.groupCount();
for(int i=1;i<=count;i++) {
result[lineNumber][i-1] = Integer.parseInt(temp.group(i));
}
lineNumber++;
scanner.close();
line = reader.readLine();
}
return result;
}
public static String createPattern(String line) {
char[] chars = line.toCharArray();
StringBuilder pattern = new StringBuilder();;
for(char c : chars) {
if(',' == c) {
pattern.append(',');
} else {
pattern.append("(\\d+)");
}
}
return pattern.toString();
}

The following piece of code snippet might be helpful. The basic idea is to read each line and parse out CSV. Please be advised that CSV parsing is generally hard and mostly requires specialized library (such as CSVReader). However, the issue in hand is relatively straightforward.
try {
String line = "";
int rowNumber = 0;
while(scan.hasNextLine()) {
line = scan.nextLine();
String[] elements = line.split(',');
int elementCount = 0;
for(String element : elements) {
int elementValue = Integer.parseInt(element);
spaces[rowNumber][elementCount] = elementValue;
elementCount++;
}
rowNumber++;
}
} // you know what goes afterwards

Since it is a file which is read line by line, read each line using a delimiter ",".
So Here you just create a new scanner object passing each line using delimter ","
Code looks like this, in first for loop
for(int i = 0; i < 10; i++)
{
Scanner newScan=new Scanner(scan.nextLine()).useDelimiter(",");
counter = counter + 1;
for(int m = 0; m < 10; m++)
{
Spaces[i][m] = newScan.nextInt();
}
}

Use the useDelimiter method in Scanner to set the delimiter to "," instead of the default space character.
As per the sample input given, if the next row in a 2D array begins in a new line, instead of using a ",", multiple delimiters have to be specified.
Example:
scan.useDelimiter(",|\\r\\n");
This sets the delimiter to both "," and carriage return + new line characters.

Why use a scanner for a file? You already have a BufferedReader:
FileReader fileReader = new FileReader(filename);
BufferedReader reader = new BufferedReader(fileReader);
Now you can read the file line by line. The tricky bit is you want an array of int
int[][] spaces = new int[10][10];
String line = null;
int row = 0;
while ((line = reader.readLine()) != null)
{
String[] array = line.split(",");
for (int i = 0; i < array.length; i++)
{
spaces[row][i] = Integer.parseInt(array[i]);
}
row++;
}
The other approach is using a Scanner for the individual lines:
while ((line = reader.readLine()) != null)
{
Scanner s = new Scanner(line).useDelimiter(',');
int col = 0;
while (s.hasNextInt())
{
spaces[row][col] = s.nextInt();
col++;
}
row++;
}
The other thing worth noting is that you're using an int[10][10]; this requires you to know the length of the file in advance. A List<int[]> would remove this requirement.

How to ignore duplicate strings when using RegEx to match string?

EDIT: editted for clarity as to what I'm having trouble with. I'm not getting the right responses as its counting dupes. I HAVE to use RegEx, can use tokenizer however but I did not.
What I am trying to do here is, there is 5 input files. I need to calculate how many "USER DEFINED VARIABLES" there are. Please ignore the messy code, I'm just learning Java.
I replaced: everything within ( and ), all non-word characters, any statements such as int, main etc, any digit with a space infront of it, and any blank space with a new line then trim it.
This leaves me with a list that has a variety of strings which I will match with my RegEx. However, at this point, how make my count only include unique identifiers?
EXAMPLE:
For example, in the input file I have attached beneath the code, I am receiving
"distinct/unique identifiers: 10" in my output file, when it should be "distinct/unique identifiers: 3"
And for example, in the 5th input file I have attached, I should have "distinct/unique identifiers: 3" instead I currently have "distinct/unique identifiers: 6"
I cannot use Set, Map etc.
Any help is great! Thanks.
import java.util.*
import java.util.regex.*;
import java.io.*;
public class A1_123456789 {
public static void main(String[] args) throws IOException {
if (args.length < 1) {
System.out.println("Wrong number of arguments");
System.exit(1);
}
for (int i = 0; i < args.length; i++) {
FileReader jk = new FileReader(args[i]);
BufferedReader ij = new BufferedReader(jk);
FileWriter fw = null;
BufferedWriter bw = null;
String regex = "\\b(\\w+)(\\s+\\1\\b)+";
Pattern p = Pattern.compile("[_a-zA-Z][_a-zA-Z0-9]{0,30}");
String line;
int count = 0;
while ((line = ij.readLine()) != null) {
line = line.replaceAll("\\(([^\\)]+)\\)", " " );
line = line.replaceAll("[^\\w]", " ");
line = line.replaceAll("\\bint\\b|\\breturn\\b|\\bmain\\b|\\bprintf\\b|\\bif\\b|\\belse\\b|\\bwhile\\b", " ");
line = line.replaceAll(" \\d", "");
line = line.replaceAll(" ", "\n");
line = line.trim();
Matcher m = p.matcher(line);
while (m.find()) {
count++;
}
}
try {
String s1 = args[i];
String s2 = s1.replaceAll("input","output");
fw = new FileWriter(s2);
bw = new BufferedWriter(fw);
bw.write("distinct/unique identifiers: " + count);
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
if (bw != null) {
bw.close();
}
if (fw != null) {
bw.close();
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
}
//This is the 3rd input file below.
int celTofah(int cel)
{
int fah;
fah = 1.8*cel+32;
return fah;
}
int main()
{
int cel, fah;
cel = 25;
fah = celTofah(cel);
printf("Fah: %d", fah);
return 0;
}
//This is the 5th input file below.
int func2(int i)
{
while(i<10)
{
printf("%d\t%d\n", i, i*i);
i++;
}
}
int func1()
{
int i = 0;
func2(i);
}
int main()
{
func1();
return 0;
}

Try this
LinkedList dtaa = new LinkedList();
String[] parts =line.split(" ");
for(int ii =0;ii<parts.length;ii++){
if(ii == 0)
dtaa.add(parts[ii]);
else{
if(dtaa.contains(parts[ii]))
continue;
else
dtaa.add(parts[ii]);
}
}
count = dtaa.size();
instead of
Matcher m = p.matcher(line);
while (m.find()) {
count++;
}

Amal Dev has suggested a correct implementation, but given the OP wants to keep Matcher, we have:
// Previous code to here
// Linked list of unique entries
LinkedList uniqueMatches = new LinkedList();
// Existing code
while ((line = ij.readLine()) != null) {
line = line.replaceAll("\\(([^\\)]+)\\)", " " );
line = line.replaceAll("[^\\w]", " ");
line = line.replaceAll("\\bint\\b|\\breturn\\b|\\bmain\\b|\\bprintf\\b|\\bif\\b|\\belse\\b|\\bwhile\\b", " ");
line = line.replaceAll(" \\d", "");
line = line.replaceAll(" ", "\n");
line = line.trim();
Matcher m = p.matcher(line);
while (m.find()) {
// New code - get this match
String thisMatch = m.group();
// If we haven't seen this string before, add it to the list
if(!uniqueMatches.contains(thisMatch))
uniqueMatches.add(thisMatch);
}
}
// Now see how many unique strings we have collected
count = uniqueMatches.size();
Note I haven't compiled this, but hopefully it works as is...

Trying to save a clone of an array in a txt file but returns NULL

I am making an app that keeps username and scores from a game in a txt file. The concept is that when it writes a new username and score to the txt file it should open the .txt file, read it and then make a clone of it adding a new uername and score entry in the txt file.
I am thinking of making this with 2 object arrays. The first is the one that is read in and the new will be the one is writen which will have one more entry.
So if player[i] is readen player[i+1] should be writen with new entry.
I am giving u the code below!
private Player[] myplayer=null;
private Player[] mynewplayer=null;
//open Players.txt
int i;
int n;
String filename="players.txt";
try
{
FileReader fp=new FileReader(filename);
BufferedReader bf=new BufferedReader(fp);
n=Integer.parseInt(bf.readLine());
myplayer=new Player[n];
int x=n+1;
mynewplayer=new Player[x];
for(i=0;i<n;i++)
{
String s=bf.readLine();
String user="",score="";
user=s.substring(0,s.indexOf(","));
s=s.substring(s.indexOf(",")+1);
score=s;
myplayer[i]=new Player(user,Double.parseDouble(score));
for(i=0;i<n;i++)
{
mynewplayer[i]= myplayer[i];
}
mynewplayer[x]=new Player(Username,Double.parseDouble(score));
}
bf.close();
fp.close();
}catch(IOException e)
{
System.out.println("Exception was "+e.getMessage());
}
//----------------------------------WRITE mytxt!-------------
n=myplayer.length;
try
{
filename="players.txt";
FileWriter fp=new FileWriter(filename);
fp.write(""+n+"\n");
for(i=0;i<n+1;i++)
fp.write(""+mynewplayer[i]+"\n");
fp.close();
}catch(IOException e)
{
System.out.println("Exception was "+e.getMessage());
}
//----------------------------------WRITE mytxt!-----------
//Get on Message
String s="";
for(i=0;i<mynewplayer.length;i++)
s=s+mynewplayer[i]+"\n";
JOptionPane.showMessageDialog(null,"Players are \n "+s);
Problem is that when it's written, it returns null for mynewplayer.
I suppose the mynewplayer doesnt really take the entries of the "myplayer" but neither writes the new username.
Compile doesnt show any errors. Just writes NULL to the textfile.
Ask me if u want further info on the code writen!
Thanks in advance!

Here is an edited version of your code, with some improvements and there should be a comment around code that I changed, explaining what I did.
Player[] myPlayer = null; // first word uncapitalized, every
Player[] myNewPlayer = null; // other word begins with a capital
//open Players.txt
int i, n; // combine the variables into 1 line
String filename = "players.txt";
try {
FileReader fp = new FileReader(filename);
BufferedReader bf = new BufferedReader(fp);
n = Integer.parseInt(bf.readLine());
// not needed
//myPlayer = new Player[n];
// NOT NEEDED int x = n + 1;
myNewPlayer = new Player[n + 1];
for (i = 0; i < n; i++) {
String s = bf.readLine();
String user, score; // combine variables, doesnt need to initalize them
String[] items = s.split(","); // Splits the line into array elements on every delimiter -> ,
//user = s.substring(0, s.indexOf(","));
//s = s.substring(s.indexOf(",") + 1);
//score = s;
user = items[0];
score = items[1];
// this line below isnt actually needed
//myPlayer[i] = new Player(user, Double.parseDouble(score));
// Create a new player clone, dont copy the previous one
myNewPlayer[i] = new Player(user, Double.parseDouble(score));
}
// We've read all the variables from the text file, now we create the last one
// Since myNewPlayer is (n+1) size, the range of the array is
// 0 to n
// the last index will be n New Score Variable
myNewPlayer[n] = new Player("Username variable", Double.parseDouble("22"));
bf.close();
fp.close();
} catch (IOException e) {
System.out.println("Exception was " + e.getMessage());
}
//----------------------------------WRITE mytxt!-------------
// This is called a ternary operator
// it is a 1 line if statement
// the format is like so
// booleanLogic ? trueAnswer Execution : falseAnswer Execution;
// if () { true }else { false }
n = myNewPlayer != null ? myNewPlayer.length : 0;
// CHANGED HERE - was using the first array rather than second
// dont need the 1st array
try {
filename = "players.txt";
FileWriter fp = new FileWriter(filename);
// Dont need "" before the items
fp.write(n + "\n");
for (i = 0; i < n; i++) {
fp.write(myNewPlayer[i] + "\n");
}
fp.close();
} catch (IOException e) {
System.out.println("Exception was " + e.getMessage());
}
//----------------------------------WRITE mytxt!-----------
//Get on Message
String s = "";
for (i = 0; i < myNewPlayer.length; i++) {
// s += ""; is like doing s = s + "";
s += myNewPlayer[i] + "\n";
}
JOptionPane.showMessageDialog(null, "Players are \n " + s);

I believe that your problem is this:
for(i=0;i<n;i++)
{
String s=bf.readLine();
String user="",score="";
user=s.substring(0,s.indexOf(","));
s=s.substring(s.indexOf(",")+1);
score=s;
myplayer[i]=new Player(user,Double.parseDouble(score));
for(i=0;i<n;i++)
{
mynewplayer[i]= myplayer[i];
}
mynewplayer[x]=new Player(Username,Double.parseDouble(score));
}
You have nested loops, which is fine, but they use the same counter (the variable i ).
So what is happening is the first line of the file is read, and then added to myplayer[0]. However, instead of just also adding it to mynewplayer[0], you start another loop on i. This loop:
for(i=0;i<n;i++)
{
mynewplayer[i]= myplayer[i];
}
is going to copy the first player into mynewplayer[0]...and then null into every other entry (since myplayer only has the firsdt element filled.
The problem is that after that loop completes, i will equal n, so when you get back to the top of the outer loop, the check $i
Perhaps what you should do is this:
for(i=0;i<n;i++)
{
String s=bf.readLine();
String user="",score="";
user=s.substring(0,s.indexOf(","));
s=s.substring(s.indexOf(",")+1);
score=s;
myplayer[i]=new Player(user,Double.parseDouble(score));
mynewplayer[i]= new Player(user,Double.parseDouble(score));
}
mynewplayer[x]=new Player(<the new username>,Double.parseDouble(<the new score>));

Fastest way for data look up from a .txt file which contains 11million+ lines of info

So I have been trying to work this out for a while but unable to come to a "rapid" solution. I do have a solution in place but it takes literaly 3 days for it to complete but unfortunatly that is far too long.
What i am trying to do:
So I have a text file (call this 1.txt) that contains unique time stamps, and have a secondary text file (call this 2.txt) that contains mixed data, and the intention is to read the first time stamp from 1.txt and find the match in 2.txt and output it in a new file, and continually do this. There are approximately 100,000 time stamps in 1.txt and over 11million lines in 2.txt.
What i have achieved:
So far what I got is it gets the first time stamp, and have a nested loop where by it loops through the 11 million lines to find a match. Once match is found, itll store that in a variable, up until it moves onto the next timestamp, where it writes out that data. Solution below:
public class fedOrganiser5 {
private static String directory = "C:\\Users\\xxx\\Desktop\\Files\\";
private static String file = "combined.txt";
private static Integer fileNo = 1;
public static void main(String[] args) throws IOException {
String sCurrentLine = "";
int i = 1;
String mapperValue = "";
String outputFirst = "";
String outputSecond = "";
String outputThird = "";
long timer;
int counter = 0;
int test = 0;
timer = System.currentTimeMillis();
try {
BufferedReader reader = new BufferedReader(new FileReader(directory + "newfile" + fileNo + ".txt"));
BufferedWriter writer = new BufferedWriter(new FileWriter(directory + "final_" + fileNo + ".txt"));
BufferedReader mapper = new BufferedReader(new FileReader(directory + file));
for (sCurrentLine = reader.readLine(); sCurrentLine != null; sCurrentLine = reader.readLine()) {
if (!sCurrentLine.trim().isEmpty() && sCurrentLine.trim().length() > 2) {
sCurrentLine = sCurrentLine.replace(" ", "").replace(", ", "").replace(",", "").replace("[", "");
try {
if (counter>0) {
writer.write(outputFirst + outputSecond + outputThird);
outputFirst = "";
outputSecond = "";
outputThird = "";
counter = 0;
test=0;
i++;
mapper.close();
mapper = new BufferedReader(new FileReader(directory + file));
System.out.println("Writing out details for " + sCurrentLine);
}
for (mapperValue = mapper.readLine(); mapperValue != null; mapperValue = mapper.readLine()) {
test++;
System.out.println("Find match " + i + " - " + test);
if (mapperValue.contains(sCurrentLine)) {
System.out.println("Match found - Mapping " + sCurrentLine + i);
if (mapperValue.contains("[EVENT=agentStateEvent]")) {
outputFirst += mapperValue.trim() + "\r\n";
counter++;
} else if (mapperValue.contains("[EVENT=TerminalConnectionCreated]")) {
outputSecond += mapperValue.trim() + "\r\n";
counter++;
} else {
outputThird += mapperValue.trim() + "\r\n";
counter++;
}
}
}
}
catch (Exception e)
{
System.err.println("Error: "+sCurrentLine + " " + mapperValue);
}
}
}
System.out.println("writing final record out");
writer.write(outputFirst + outputSecond + outputThird);
writer.close();
System.out.println("complete!");
System.out.print("Time taken: " +
((TimeUnit.MILLISECONDS.toMinutes(System.currentTimeMillis())-TimeUnit.MILLISECONDS.toMinutes(timer)))
+ " minutes");
}
catch (Exception e)
{
System.err.println("Error: Target File Cannot Be Read");
}
}
}
The problem?
I have tried looking through other solutions on google and forums but unable to seek a suitable or a faster approach to do this (or its something thats beyond my depth of knowledge). Looping through 11million lines for every time stamp takes approximately 10 minutes, and with 10,000 timestamps, you can imagine how long the process will take. Can someone provide me some friendly advice of where to look or any APIs that can speed this process up?

Want to thank everyone for their suggestions. Will certainly try the database method proposed by Roman as it may be the quickest for the type of work I am trying to do, but if no success will try the other solutions proposed :)

Print specific lines(Words/numbers) from csv file

Lets say I have CSV file like this:
Football Contest blabla bla,,,,,,,
Team number1,Team number2,Points team1,Points team2,Red cards
Sweden,France,1,2,"
Sweden,Brazil,3,5,2
Sweden,Germany,2,2,3
Sweden,Spain,3,5,"
And in this file I only want to print out the matches that got red cards. So in this example I would like to print:
Sweden - Brazil = 2 Sweden - Germany = 3
This is my current code, and Im stuck how to move on.
try {
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
String lines = br.readLine();
String result[] = lines.split(",");
do{
System.out.println();
}while((lines = br.readLine()) != null);
//String result[] = lines.split(",");
//System.out.println(result[1]);
br.close();
}catch (FileNotFoundException e){
System.out.println("File not found : "+ file.toString());
}catch (IOException e ){
System.out.println("Unable to read file: "+ file.toString());
}
EDIT I got helped with:
while (line != null) {
String result[] = line.split(",");
if (result.length == 5) { //red cards present?
System.out.println(result[0] + " - " + result[1] + " " + result[4]);
}
line = br.readLine(); //read next
}
But the problem I have now is that it still prints all because of the " in the csv file. Why cant I do something like this?
if (result[4] == "1,2,3,4,5" ){
System.out.println(result[0] + " - " + result[1] + " " + result[4]);
}

If you the index of red card and it looks like Integer, Then see for that index is integer or not if Yes the print 0,1 and 4
index[0]=Team number1
index[1]=Team number2
index[4]=red cards

Your try-catch block should look like this:
try {
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
String line = br.readLine();
while (line != null) {
String result[] = line.split(",");
if (result.length == 5 && result[4].matches("[0-9]")) { //red cards present?
System.out.println(result[0] + " - " + result[1] + " " + result[4]);
}
line = br.readLine(); //read next
}
//close readers
br.close();
fr.close();
} catch (FileNotFoundException e) {
System.out.println("File not found : " + file.toString());
} catch (IOException e) {
System.out.println("Unable to read file: " + file.toString());
}
If there are trailing whitespaces in your file, you have to trim the String first: result[4].trim().matches("[0-9]") or use another regex: result[4].matches("\\d\\s")

Why cant I do something like this?
if (result[4] == "1,2,3,4,5" ){
The problem with this is == tests for identity: it will compare if the reference in result[4] is the same reference as the constant in your source code. That expression will always be false. You need to check for equality, not identity:
if (Objects.equals(result[4], "1,2,3,4,5")) {
or
if (result[4] != null && result[4].equals("1,2,3,4,5")) {
or
if ("1,2,3,4,5".equals(result[4])) {
Note that Objects.equals() was (finally) added to the Java standard library in Java 8. Without it, you must guard against a NullPointerException before you call the .equals() method on an object. Traditionally I have preferred the last version because invoking the method on the string literal means I can be assured it is never null and will just work.

You can try like this
public class HelloWorld{
public static void main(String []args){
String str1= "Sweden,Brazil,3,5,4";
String str2="Sweden,Germany,2,2,3";
String str3="Football Contest blabla bla,,,,,,,";
String result1[]=str1.split(",");
String result2[]=str2.split(",");
String result3[]=str3.split(",");
if(result1.length>=5){
System.out.print(result1[0]+"-"+result1[1]+"="+result1[4]);
System.out.println();
}
if(result2.length>=5){
System.out.print(result2[0]+"-"+result2[1]+"="+result2[4]);
System.out.println();
}
if(result3.length>=5){
System.out.print(result3[0]+"-"+result3[1]+"="+result3[4]);
System.out.println();
}
}
}

try this:
do {
if (result[4] instanceof Integer) {
System.out.print(result[0]+"="+result[1]+"="+result[4])
}
} while ((lines = br.readLine()) != null);