Develop Reference

How to get scanner to read pipe delimited txt file in java

I've been trying to get a scanner to read and sort a pipe delimited text file using delimiter \|but i cant seem to get it to sort or display the reformatted file. When I open the filechooser it also throws and error when i try to import the scanner. The data in the txt file will be a mixture of strings and ints
Scanner scanner = new Scanner("/n");
scanner.useDelimiter(Pattern.compile("[|\n]"));
JTable JT = new Jtable();
List<Integer> List = new ArrayList<>();
System.out.println(scanner.nextInt());
System.out.println(scanner.nextInt());
File readFile; //creates file variable
JFileChooser jFileChooser_Open = new JFileChooser();
int option = jFileChooser_Open.showOpenDialog(this);
readFile = jFileChooser_Open.getSelectedFile();//instantiates file and
gets selected file from jfilechooser
String line;
try (FileReader inStream = new FileReader(readFile); BufferedReader
inBuffer = new BufferedReader(inStream)) {
//create variable line
while ((line = inBuffer.readLine()) != -1 ) {
while (scanner.hasNext()) {
List.add(line);
List.add(scanner.nextInt());
}
System.out.println(List);
}
} catch (IOException ex) {
Logger.getLogger(NewJFrame.class.getName()).log(Level.SEVERE,null, ex);
}
Also I want to get the newly formatted string to be displayed in my table or a text area. But if you guys could help me get it to print to the console, then I would greatly appreciate it.

Optimising CSV parsing to be faster

I'm working on this "program" that reads data from 2 large csv files (line by line), compares an Array element from the files and, when a match is found, it writes my necessary data into a 3rd file. The only problem I have is that it is very slow. It reads 1-2 lines per second, which is extremely slow, considering I have millions of records. Any ideas on how could I make it faster? Here's my code:
public class ReadWriteCsv {
public static void main(String[] args) throws IOException {
FileInputStream inputStream = null;
FileInputStream inputStream2 = null;
Scanner sc = null;
Scanner sc2 = null;
String csvSeparator = ",";
String line;
String line2;
String path = "D:/test1.csv";
String path2 = "D:/test2.csv";
String path3 = "D:/newResults.csv";
String[] columns;
String[] columns2;
Boolean matchFound = false;
int count = 0;
StringBuilder builder = new StringBuilder();
FileWriter writer = new FileWriter(path3);
try {
// specifies where to take the files from
inputStream = new FileInputStream(path);
inputStream2 = new FileInputStream(path2);
// creating scanners for files
sc = new Scanner(inputStream, "UTF-8");
// while there is another line available do:
while (sc.hasNextLine()) {
count++;
// storing the current line in the temporary variable "line"
line = sc.nextLine();
System.out.println("Number of lines read so far: " + count);
// defines the columns[] as the line being split by ","
columns = line.split(",");
inputStream2 = new FileInputStream(path2);
sc2 = new Scanner(inputStream2, "UTF-8");
// checks if there is a line available in File2 and goes in the
// while loop, reading file2
while (!matchFound && sc2.hasNextLine()) {
line2 = sc2.nextLine();
columns2 = line2.split(",");
if (columns[3].equals(columns2[1])) {
matchFound = true;
builder.append(columns[3]).append(csvSeparator);
builder.append(columns[1]).append(csvSeparator);
builder.append(columns2[2]).append(csvSeparator);
builder.append(columns2[3]).append("\n");
String result = builder.toString();
writer.write(result);
}
}
builder.setLength(0);
sc2.close();
matchFound = false;
}
if (sc.ioException() != null) {
throw sc.ioException();
}
} finally {
//then I close my inputStreams, scanners and writer

Use an existing CSV library rather than rolling your own. It will be far more robust than what you have now.
However, your problem is not CSV parsing speed, it that your algorithm is O(n^2), for each line in the first file, you need to scan the second file. This kind of algorithm explodes very quickly with the size of data, when you have millions of rows, you'll run into problems. You need a better algorithm.
The other problem is you are re-parsing the second file for every scan. You should at least read it into an memory as an ArrayList or something first at the start of the program so you only need to load and parse it once.

Use univocity-parsers' CSV parser as it won't take much longer than a couple of seconds to process two files with 1 million rows each:
public void diff(File leftInput, File rightInput) {
CsvParserSettings settings = new CsvParserSettings(); //many config options here, check the tutorial
CsvParser leftParser = new CsvParser(settings);
CsvParser rightParser = new CsvParser(settings);
leftParser.beginParsing(leftInput);
rightParser.beginParsing(rightInput);
String[] left;
String[] right;
int row = 0;
while ((left = leftParser.parseNext()) != null && (right = rightParser.parseNext()) != null) {
row++;
if (!Arrays.equals(left, right)) {
System.out.println(row + ":\t" + Arrays.toString(left) + " != " + Arrays.toString(right));
}
}
leftParser.stopParsing();
rightParser.stopParsing();
}
Disclosure: I am the author of this library. It's open-source and free (Apache V2.0 license).

Creating a constructor to read a txt file

I am creating a program that will produces the statistics of a baseball team
i am trying to create a constructor to read the file into the teamName instance variable and the battingAverages array.
the txt file contains the one word name of the team followed by 20 batting averages.
"Tars 0.592 0.427 0.194 0.445 0.127 0.483 0.352 0.190 0.335 0.207 0.116 0.387 0.243 0.225 0.401 0.382 0.556 0.319 0.475 0.279 "
I am struggling to find how to go about this and get it started?

I ran this and this might be close to what you want. Instead of making a confusing constructor, make a private method that the constructor will call to read in the file into the array.
import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Scanner;
public class Baseball {
private File textFile;
private Scanner input;
private String teamName;
//this will only work if you know there will be 20 entries everytime
//otherwise I recommend loading the data into an ArrayList
private double []battingAvgs = new double[20];
public Baseball(String file){
textFile = new File(file);
readInFile(textFile);
}
//private method that reads in the file into an array
private void readInFile(File textFile){
try {
input = new Scanner(textFile);
//read first string into variable teamName
teamName = input.next();
int i=0;
//iterate through rest of file adding it to an ArrayList
while(input.hasNext()){
battingAvgs[i] = input.nextDouble();
i++;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
//print out array
public void printArray(){
for(Double a: battingAvgs){
System.out.println(a);
}
}
}

Well, if these are all on one line in a specific file then what you could do is construct a bufferedreader to read the first line of your file, split the line based on spaces, and then parse the teamName and batting averages out.
BufferedReader br = new BufferedReader(new FileReader("myfile.txt"));
String[] line = br.readLine().split(" ");
br.close();
teamName = line[0];
battingAverages = new int[20];
for(int i = 0; i < 20; i++)
battingAverages[i] = Integer.parseInt(line[i+1]);
These might throw IOExceptions, which you will need to catch. I think Java 7 has a method to automatically handle these kinds of errors (not sure about this), but as I am new to Java 7's added functionality, I would just manually check for those exceptions.

You need to use the BufferedReader, FileInputStream, and InputStreamReader. Your file.txt should have the batting averages on every line, as shown below.
0.592
0.427
0.194
Here is an example of a class that when created, it will read a text file line by line and add each line to the array list:
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.util.*;
public class Class {
ArrayList<Double> averages;
public Class() {
averages = new ArrayList<Double>();
try {
FileInputStream in = new FileInputStream("inputFile.txt"); //your file path/name
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine;
while((strLine = br.readLine())!= null) averages.add(Double.parseDouble(strLine));
}catch(Exception e){
System.out.println(e);
}
}
}
Hope this helps

Try using the Scanner class.
File file=new File("TestFile.txt"); //Create a new file
Scanner scan=new Scanner(file);//Create a Scanner object (Throws FileNotFoundException)
if(scan.hasNext()) //Check to make sure that there is actually something in the file.
{
String line=scan.nextLine(); //Read the line of data
String[] array=line.split(" "); //Split line into the different parts
teamName=array[0]; //The team name is located in the first index of the array
battingAverages=new double[array.length-1];//Create a new array to hold the batting average values
for(int i=0;i<battingAverages.length;i++) //Loop through all of the averages
{
double average=Double.parseDouble(array[i+1]);//Convert the string object into a double
battingAverages[i]=average; //Add the converted average to the array
}
System.out.print(teamName+" "+Arrays.toString(battingAverages)); //[Optional] Print out the resulting values
}

Java Program read input from a text file and modify it accordingly

I am writing a Java program that inputs a test file, performs some modifications to the data, then writes it to a new file output.
The input text file looks like this...
url = http://184.154.145.114:8013/wlraac name = wlr samplerate = 44100 channels =2 format = S16le~
url = http://newstalk.fmstreams.com:8080 name = newstalk samplerate = 22050 channels = 1 format = S16le
The program needs to be able to change the samplerate to 44100, and the channels to 1, if they don't already have these values. I would also remove the url and name pieces completely. After these changes, the new line needs to be written out to a different output text file.
So far, all my program can do is select a file and display the contents of the file to the user. Could someone please point me in the right direction for how my program should work
to achieve my required outcome.
As somebody asked here is what I have so far
import java.io.BufferedReader;
import java.io.File;
import java.io.FileReader;
import javax.swing.JFileChooser;
import javax.swing.JFrame;
public class reader2 {
public reader2() {
}
public static void main(String[] args) {
reader(args);
}
public static void reader(String[] args) {
JFileChooser chooser = new JFileChooser();
chooser.setCurrentDirectory(new File("."));
chooser.setFileFilter(new javax.swing.filechooser.FileFilter() {
public boolean accept(File f) {
return f.getName().toLowerCase().endsWith(".txt")
|| f.isDirectory();
}
public String getDescription() {
return "Text Documents (.txt)";
}
});
int r = chooser.showOpenDialog(new JFrame());
if (r == JFileChooser.APPROVE_OPTION) {
String name = chooser.getSelectedFile().getName();
String pathToFIle = chooser.getSelectedFile().getPath();
System.out.println(name);
try{
BufferedReader reader = new BufferedReader( new FileReader( pathToFIle ) ); //Setup the reader
while (reader.ready()) { //While there are content left to read
String line = reader.readLine(); //Read the next line from the file
String[] tokens = line.split( "url = " ); //Split the string at every # character. Place the results in an array.
for (String token : tokens){ //Iterate through all of the found results
//System.out.println(token);
System.out.println(token);
}
}
reader.close(); //Stop using the resource
}catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
}
}
}

You will need to do something like this ...
Read the contents of the file, one line at a time
Split the line up into the individual components, such as splitting it on the 'space' character
Change the sample rate and channel values according to your question
Write the line out to a file, and start again from step 1.
If you give this a try, post some code on StackExchange with any problems and we'll try to assist.

can you try
File file = new File( fileName );
File tempFile = File.createTempFile("buffer", ".tmp");
FileWriter fw = new FileWriter(tempFile);
Reader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
while(br.ready()) {
String line = br.readLine();
String newLine = line.replaceAll( "samplerate =\\s*\\d+", "samplerate = 44100");
newLine = newLine.replaceAll( "channels =\\s*\\d+", "channels = 1");
fw.write(newLine + "\n");
}
fw.close();
br.close();
fr.close();
// Finally replace the original file.
tempFile.renameTo(file);
Ref: Files java replacing characters

file input from the user

I was trying to take the input of the filename from the user and then proceed to doing all the calculations. but it keeps returning me an error. the file exists in the same directory.
import java.io.*;
import java.util.*;
public class test{
public static void main(String args[]) throws FileNotFoundException {
//File fin = new File ("matrix1.txt");
Scanner scanner = new Scanner(System.in);
scanner.nextLine(); // removes the first line in the input file
String rowLine = scanner.nextLine();
String[] arr = rowLine.split("=");
int rows = Integer.parseInt(arr[1].trim());
String colLine = scanner.nextLine();
String[] arr2 = colLine.split("=");
int cols = Integer.parseInt(arr2[1].trim());
double [][]matrix = new double [rows][cols];
for (int i=0; i<rows;i++){
for (int j=0; j<cols;j++) {
matrix[i][j]= scanner.nextDouble();
}
}
System.out.println(rows);
System.out.println(cols);
for (int i=0; i<rows; i++)
{ for (int j=0;j<cols;j++) {
System.out.println(matrix[i][j]);
}
}
}
}

There is one issue with the code. The scanner will just give you the name of the file as string from command line. So, you need to first get the command line argument and then create one more scanner using the constructor which takes file object. e.g.
Scanner scanner = new Scanner(System.in);
Scanner fileScanner = new Scanner(new File(scanner.nextLine()));
String rowLine = fileScanner.nextLine();
System.out.println(rowLine);
String[] arr = rowLine.split("=");
int rows = Integer.parseInt(arr[1].trim())

You realize that you are only using a Scanner of type System.in, right? This means that you aren't even looking at a file, you are looking at user input only. This is regardless of whether you have the first line commented out or not. To use a file, you could use a FileInputStream or a couple other File handling classes.
FileInputStream fs = new FileInputStream(new File("matrix1.txt"));
//do stuff with the stream
Heres the java docs for FileInputStream: http://download.oracle.com/javase/1.4.2/docs/api/java/io/FileInputStream.html
Edit: After seeing your comment on what the actual error was, I realize there are more problems with the code than just the way you are handling input. Your error is almost certainly happening at one of the first 2 array accessors, the arr1.trim() calls. That means the user input has nothing on the right side of the "=" sign, or there is no "=" sign in the user input.