CSV reader perils - Java - java

I'm trying to read a CSV file and split each line into 4 different integer values via a two-dimensional array in java.
I'm using openCSV 3.8.
For the sake of simplicity, say this is the contents of the CSV file (the full file contains 306 lines just like these):
76,67,0,1
77,65,3,1
78,65,1,2
83,58,2,2
I can read the file just fine, and I can use System.out.println to output each single value to the console, like this:
76
67
0
1
77
65
3
1
78
65
1
2
85
58
2
2
Unfortunately with my code below, designed to enter each value into a separate array element only saves the 4 values in the last line of the file.
And here is my java code (don't mind the size of the iaData array, it's sized for the full CSV file):
public static void main(String[] args) {
//String outputStr = "";
int[][] iaData = new int[306][4];
int i = 0;
int x = 0;
try
{
//Get the CSVReader instance with specifying the delimiter to be used
CSVReader reader = new CSVReader(new FileReader("haberman.data"),',');
String [] nextLine = new String[1250];
//Read one line at a time
while ((nextLine = reader.readNext()) != null)
{
for (i = 0; i <= 305; i++)
{
for (x = 0; x <= 3; x++)
{
iaData[i][x] = Integer.parseInt(nextLine[x]);
}
}
}
for (int z = 0; z <= 3; z++)
{
System.out.println(iaData[0][z] + "\n");
}
reader.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
With this code, I would expect my System.out.println(iaData[0][z] + "\n"); to output the following to the console (the values in the first line of the file):
76
67
0
1
Unfortunately it's not the case, it actually outputs the following (the 4 values in the last line of the file):
83
58
2
2
What is wrong with my code such that iaData[0][0/1/2/3] actually outputs what I would expect to be held in iaData[**3**][0/1/2/3]?

For every line, you start writing with first index i=0.
So for every line you override all information from the line before:
while ((nextLine = reader.readNext()) != null)
{
for (i = 0; i <= 305; i++)
{
for (x = 0; x <= 3; x++)
{
iaData[i][x] = Integer.parseInt(nextLine[x]);
}
}
}
This should solve your problem:
int i = 0;
while ((nextLine = reader.readNext()) != null) {
for (x = 0; x <= 3; x++) {
iaData[i][x] = Integer.parseInt(nextLine[x]);
}
i++;
}

Your current problem is that for each line you are iterating over the entire 2D array, both rowwise and columnwise. This has the effect that only the last row will reflect the currently read line.
Instead, the row counter i should only be incremented after another line has been read from the CSV file. Currently, you are incrementing i for the same line over and over again.
// Read one line at a time
int i = 0;
while ((nextLine = reader.readNext()) != null) {
for (x = 0; x <= 3; x++) {
iaData[i][x] = Integer.parseInt(nextLine[x]);
}
// increment i once, after having processed a single line from the file
++i;
}

There are various things in your code that simply don't make sense.
First, you are init'ing nextLine to probably hold 1250 lines.
String [] nextLine = new String[1250];
But then, you are pushing a single line into that:
while ((nextLine = reader.readNext()) != null)
Either you don't need the first statement; or something is wrong about the second one.
For the actual bug you are observing: it doesn't help that you keep writing the same lines 395 times, too. And that is what your code is doing; because for each line that you read, you do
for (i = 0; i <= 305; i++)
{
for (x = 0; x <= 3; x++)
{
hiaData[i][x] = Integer.parseInt(nextLine[x]);
So you keep overwriting your data, instead of
line1
line2
...
you keep writing
line i
line i...
Thus, in the end, you only got your last line in there.
My recommendation: don't do all that manual size-dependent iterating yourself. Simply use
List<String[]> allLines = csvreader.readAll();
4 values, 395 lines ... that is nothing. Just push all of that into memory with that one call.
Meaning: the library offers you a nice abstraction that requires you one line of code to read all content. You choose to write 20, 30 lines of code instead. And that is the thing: the more code you write, the higher are chances to create bugs.
And finally: do not use such numbers as 305, 4, 1205 directly in your source code. If at all, use constants there. But hard-coding those values in your loops ... very bad practice - you have to change each and any place that deals with your input when for example 5 columns come in, instead of 4.

If the program is going to read the whole file into an array ... is it not enough to read it into memory, and process it later?
Like so:
import au.com.bytecode.opencsv.CSVReader;
import java.io.IOException;
import java.io.StringReader;
import java.util.List;
public class Csv {
public static void main(String[] args) throws IOException {
String string = "76,67,0,1\n" +
"77,65,3,1\n" +
"78,65,1,2\n" +
"83,58,2,2";
CSVReader reader = new CSVReader(new StringReader(string),',');
List<String[]> lines = reader.readAll();
for (String[] entries : lines) {
for (String entry: entries) {
System.out.println(Integer.parseInt(entry));
}
}
}
}
Beware of memory issues and set an upper bound or process a line a time by changing the loop like so:
String[] entries;
while((entries = reader.readNext()) != null) {
for (String entry : entries) {
System.out.println(Integer.parseInt(entry));
}
}

Problems:
1.
for (i = 0; i <= 305; i++)
This cycle is not needed, as reader.readNext() will gather all the lines without the need of further help. So, this is certainly an improvement:
int i = 0;
while ((nextLine = reader.readNext()) != null)
{
//for (i = 0; i <= 305; i++)
//{
for (x = 0; x <= 3; x++)
{
iaData[i++][x] = Integer.parseInt(nextLine[x]);
}
//}
}
2.
The other problem you have is that you iterate only the elements of the first element:
for (int z = 0; z <= 3; z++)
{
System.out.println(iaData[0][z] + "\n");
}
Instead:
for (i = 0; i < iaData.length; i++)
{
for (int z = 0; z <= 3; z++)
{
System.out.println(iaData[i][z] + "\n");
}
System.out.println("");
}

Related

turning an array of strings into a 2d array of chars

my code reads input from a file that has a maze written inside, inside the file looks like this.
7 7
GOOOOXO
XXOXOOX
OXOOOXX
XXXOOXO
XXXXOXX
SOOOOOX
XXXXXXX (these are each on a separate line not sure how to show that here)
the numbers are the dimensions and G is goal S is start, that doesn't really matter for now. I read the file into an array of Strings line by line. So my array looks like this {"GOOOOXO",...,"XXXXXXX"} now I want to convert the strings to a 2d array of chars where the characters are stored the same way as the maze is displayed above.
The first nested for loop just reads the files input and saves each line as a string in the array, the second array is meant to convert the strings into characters to save them into the 2d array and still manage to keep the shape of the maze. I get an array index out of bounds exception. But to be honest with you after hours of being stuck on this my brain is fried, I can't figure out my issue.
int dimensionsX = input.nextInt();
int dimensionsY = input.nextInt();
String[] lines = new String[dimensionsY];
char[][] maze2 = new char[dimensionsX][dimensionsY];
for (int j = 0; j < dimensionsY; ++j) {
if (input.hasNextLine()) {
lines[j] = input.nextLine();
System.out.println(lines[j]);
//System.out.println(j);
}
}
for (int i = dimensionsX; i > 0; --i) {
for (int j = dimensionsY; j > 0; --i) {
maze2[i][j] = lines[i].charAt(j);
System.out.print(maze2[i][j]);
if (i == lines.length) {
System.out.println();
}
}
}
For example:
int dimensionsX = input.nextInt();
int dimensionsY = input.nextInt();
String[] lines = new String[dimensionsY];
char[][] maze2 = new char[dimensionsY][];
for (int j = 0; j < dimensionsY; ++j) {
if (input.hasNextLine()) {
lines[j] = input.nextLine();
if(lines[j].length()<dimensionsX){
System.err.println("we are doomed. Input malformed in line j: it has less then" + dimensionsX + "characters");
System.exit(1);
}
maze2[j] = lines[j].toCharArray();
System.out.println(lines[j]);
//System.out.println(j);
}else{
System.err.println("we are doomed. Input has too few lines");
System.exit(1);
}
}
//taken as is, but makes me wonder if it is intended to have dimensionsX as the Y coordinate but well...
for (int i = dimensionsX; i > 0; --i) {
for (int j = dimensionsY; j > 0; --j) {
System.out.print(maze2[j-1][i-1]);
}
System.out.println();
}
Of course you want to do something different than exiting the program if the input is malformed e.g. throw an exception, or do nothing at all if your input is always correct(well is it?).

Count no of lines between two lines

How to achieve the following. Here is a sample text file..
AH
1
2
3
BH
21
BT03
CH
CT02
AT10
This is a sample text file. I need read each line and find the count.. For example, there are 10 Lines within A record.. And 3 lines within B record and two lines within C Record.
How do I get this count. H-Header and T-Trailer. You may say, read the T record to find the count, but that count might be wrong. Hence I am trying to find the count and update the trailer records correctly.
I cannot upload the Java code I have written as I'm on mobile now.. Any suggestions is highly appreciated
Thanks
I'm not sure if I understand your question, you just want to count lines in a file? Try something like this:
Path path = Paths.get("yourFile.txt");
long lineCount = Files.lines(path).count();
use below code and read all file in array.
String[] stringArray = new String[10];
int length = stringArray.length;
for (int i = 0; i < length; i++) {
String stringI = stringArray[i];
if (stringI.contains("H")) {
// Head rec
String headRecName = stringI.replace("H", "");
int count = 1;
for (int j = i + 1; j < length; j++) {
count++;
String stringJ = stringArray[j];
if (stringJ.contains("T") && headRecName.equals(stringJ.replace("T", ""))) {
stringArray[j] = stringArray[j] + count;
}
}
}
}
}

Processing range of lines in a 2d array or file

I'm very new to programming. I'm studying on my own, and the examples from the book I'm using are too easy so I'm trying to do something harder. Lotteries are one of my hobbies and I think the problem I chose will make it easier for me to learn Java.
This program calculates frequencies (how many times each number from 1 t0 70 appears in my txt file) in Keno, a type of lottery(In Keno, a draw consists of 20 numbers out of 70, instead of the widespread 6 out of 49).
I want to calculate the frequencies not for the entire txt file, but just for a section of it, for example if the file has x lines, I want just the lines between x-5 and x-10, something like this.I don't know the number of lines in my file, perhaps thousands, but it always has 20 columns.
The program works fine for the entire file, but I run into trouble when trying to work just on a section of it. I think that I should read the file into a 2d array and then I could process the lines I want. I'm having hard times transferring every line into a matrix. I've read every post regarding reading a file into a 2d array but couldn't make it work.
Below is one of the many attempts I made over more than a week
public static void main(String args[]) {
int[][] matricea = new int [30][40];
int x=0, y=0;
try {
BufferedReader reader = new BufferedReader(
new FileReader("C:\\keno.txt")
);
int[] numbers = new int[72]; //each keno draw has 70 numbers
for (int i = 0; i < 71; i++ ){
numbers[i] = 0;
}
int k=0; // k counts the lines
String draw;
while ( (draw = reader.readLine()) != null ) {
String[] pieces = draw.split(" +");
k++;
for (String str : pieces) {
int str_int = Integer.parseInt(str);
matricea[x][y] = str_int;
System.out.print(matricea [x][y] + " ");
y = y + 1;
}
x = x + 1;
System.out.println(" ");
}
for (int j = 1; j <= 20; j++) {
int drawnNumber = Integer.parseInt(pieces[j]);
numbers[drawnNumber]++;
}
System.out.println(" nr. of lines is " + k);
reader.close();
for (int i = 0; i < 71; i++) {
System.out.println(i + ": " + numbers[i]);
}
} catch (Exception e) {
System.out.println("There was a problem opening and processing the file.");
}
}
Formatting and improving current code
I'll show how to solve your problem, but first I would like to point out a few things in your code that could be improved by some formatting, or that perhaps are unnecessary.
Just to help you improve your coding skills, readability is very important! :)
And don't forget, consistency is key! If you like one style better than the more common style, or the preferred style, that's fine as long as you use it throughout your coding. Don't switch between two styles.
If you don't care to read these comments, you can find the solution at the bottom of my answer. Just note that your original code will be different in my solution because I have formatted it to be most readable for me.
Spacing in variable declarations
Original code
int[][] matricea = new int [30][40];
int x=0, y=0;
Spacing modified
int[][] matricea = new int[30][40];
int x = 0, y = 0;
Notice the space removed between int and [30][40], and the space added between the variables and the initialization, i.e. - x=0 => x = 0.
Initializing an int array to contain all 0's
Original code
int[] numbers = new int[72]; //each keno draw has 70 numbers
for (int i = 0; i < 71; i++ ){
numbers[i] = 0;
}
Same as
int[] numbers = new int[72]; //each keno draw has 70 numbers
You don't have to set each value to 0, Java will do that for you. In fact, Java has default values, or null values, for all types. Thanks to Debosmit Ray!
I won't go into the exceptions to this case, or when or why or how, you can read about that in this post, and pay close attention to Aniket Thakur's answer.
But why do you have an array of size 72, if there are only 70 possibilities?
Choosing variable names
Original code
int k=0; // k counts the lines
Same as
int numLines = 0;
You should always make your variables names something meaningful to their purpose. If you ever have to put a comment like k counts the lines to describe a variable's purpose, consider if a better name would work instead.
Functionalizing code
Original code
while ( (draw = reader.readLine()) != null ) {
String[] pieces = draw.split(" +");
k++;
for (String str : pieces) {
int str_int = Integer.parseInt(str);
matricea[x][y] = str_int;
System.out.print(matricea [x][y] + " ");
y = y + 1;
}
x = x + 1;
System.out.println(" ");
}
Same as
while ( (draw = reader.readLine()) != null ) {
processLine(draw);
}
Of course, you'll have to make the method processLine(String line), but that won't be hard. It's just taking what you have and moving it to a separate method.
The original code's while loop is very busy and messy, but with the latter option makes the purpose clear, and the code clean.
Of course each situation is different, and you might find that only removing part of the code into a method would be a better solution. Just play around and see what makes sense.
Error!
Original code
for (int j = 1; j <= 20; j++) {
int drawnNumber = Integer.parseInt(pieces[j]);
numbers[drawnNumber]++;
}
This code should not work, since pieces is declared in the while loop above it, and is local to that above loop. This for loop is outside the scope of where pieces exists.
I'll tell you how to fix it, but I'm not sure what the code is supposed to be doing. Just let me know what its purpose is, and I'll provide you with a solution.
After formatting!
This is what the code may look like after applying my above comments. I have added comments to parts that I have changed.
public static void main(String args[]) {
try {
doKenoStuff();
} catch(IOException e) {
System.out.println("There was a problem opening and processing the file.");
}
}
public static void doKenoStuff() throws IOException {
BufferedReader reader = new BufferedReader(
new FileReader("C:\\keno.txt")
);
int[][] matricea = new int[30][40];
int[] numbers = new int[72]; //each keno draw has 70 numbers
// We can clean up our loop condition by removing
// the assignment (draw = reader.readLine) from it.
// Just make sure draw doesn't begin as null.
String draw = "";
int row;
for(row = 0; draw != null; row++) {
draw = reader.readLine();
// We read a line from the file, then send it
// to extractLineData which will collect the info
// from each column, and update matricea and numbers
extractLineData(draw, row, matricea, numbers);
}
System.out.println("Number of lines: " + row);
System.out.println("Each number's drawing stats:");
for (int i = 0; i < 71; i++) {
System.out.println(i + ": " + numbers[i]);
}
reader.close();
}
public static void extractLineData(String line, int row, int[][] matrix, int[] numbers) {
String linePieces = line.split(" +");
for(int column = 0; column < linePieces.length; column++) {
int number = Integer.parseInt(linePieces[column]);
matrix[row][column] = number;
numbers[number]++;
}
}
Solution
Note: I am not perfect, and my code is not either. I'm not trying to say that what I have suggested is in any way the only way to do this. It could definitely be improved, but it is a start. You should take my solution and see how you can improve it yourself.
What can you find that you could code in a cleaner, or faster, or better way?
So, how do we fix the problem?
We have a method that reads from the start of a file, to the end of it, and it logs the data it finds inside matricea.
A quick and easy solution is to simply make that method take in two parameters, a starting line number, and an ending line number.
public static void doKenoStuff(int start, int end) throws IOException {
Then we simply make a loop to skip over the starting lines! It's that easy!!!
for(int i = 0; i < start - 1; i++) {
reader.readLine();
}
Don't forget that we may not need the big 30 row matricea to be 30 rows anymore. We can shrink that down to end - start + 1. That way, if a user wants to read from line 45 to line 45, we only need 45 - 45 + 1 = 1 row in matricea.
int[][] matricea = new int[end - start + 1][40];
And the very last thing we need to add is a condition in our line reading loop, that prevents us from going past the ending line.
for(row = 0; draw != null, row <= end; row++) {
And there you have it. Simple as that!
Complete solution
public static void main(String args[]) {
int start = 7, end = 18;
try {
doKenoStuff(start, end);
} catch(IOException e) {
System.out.println("There was a problem opening and processing the file.");
}
}
public static void doKenoStuff(int start, int end) throws IOException {
BufferedReader reader = new BufferedReader(
new FileReader("C:\\keno.txt")
);
int[][] matricea = new int[end - start + 1][40];
int[] numbers = new int[72]; //each keno draw has 70 numbers
for(int i = 0; i < start - 1; i++) {
reader.readLine();
}
String draw = "";
int row;
for(row = 0; draw != null, row <= end; row++) {
draw = reader.readLine();
extractLineData(draw, row, matricea, numbers);
}
System.out.println("Number of lines: " + row);
System.out.println("Each number's drawing stats:");
for (int i = 0; i < 71; i++) {
System.out.println(i + ": " + numbers[i]);
}
reader.close();
}
public static void extractLineData(String line, int row, int[][] matrix, int[] numbers) {
String linePieces = line.split(" +");
for(int column = 0; column < linePieces.length; column++) {
try {
int number = Integer.parseInt(linePieces[column]);
matrix[row][column] = number;
numbers[number]++;
} catch (NumberFormatException) {
// You don't have to do anything in this block, but
// you can print out what input gave the exception if you want.
System.out.println("Bad input: \"" + linePieces[column] + "\"");
}
}
}

How to transform Strings from a txt file to different substrings in java

So I have this method loadLevel in my class LevelLoader and I want to read the file and convert it into a tilemap. That's basically an 2d array of Tiles. A tile has 2 parameters: id and damage
The file looks like this:
00.05;00.05;00.04;02.03;
00.05;01.00;01.00;02.03;
00.05;01.00;01.00;02.04;
00.05;00.05;03.00;02.01;
The first number is the id and the second value is the damage. This level has 4 rows and 4 columns.
Currently, my method looks like this:
public TileMap loadLevel(String path){
Tile[][] tiles = new Tile[12][12];
try {
BufferedReader br = new BufferedReader(new FileReader(path));
String line = null;
int i = 0;
while ((line = br.readLine()) != null) {
for(int j = 0; j < line.length() / 6; j++){
int id = valueOf(line.substring(j*6, 2);
int damage = valueOf(line.substring(j*6 + 3, 2);
tiles[i][j].setTile(id, damage);
//this is where the error comes from
}
i++;
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
return new TileMap(tiles);
}
I use 12 as max width and height.
When I try to run this, I get a NullPointerException in my for-loop. I've tried different methods to load this file, but I cannot find a one that works.
Can Anyone explain why this happens and possibly say what I should do?
Thanks for reading this.
You haven't initialized your Tile array elements, only the array.
For that, you need to use the following idiom:
tiles[outer index][inner index] = new Tile(); // assuming default no-args constructor
Default value for Objects is null, upon which you invoke setTile, hence throwing NullPointerException.
You have created your array, but not its elements.
Tile[][] tiles = new Tile[12][12];
creates an array of Tile, but each entry is null. You need to explicitly create them, something like:
for (int i = 0; i < 12; ++i) {
for (int j = 0; j < 12; ++j) {
tiles[i][j] = new Tile();
}
}

Java: how to reverse the order of every N elements in an ArrayList

I have an input text file of thousands of lines of words, where I would like write to the output file, where I reverse the order of every 10 lines. I have iterated over the entire text file, and stored it in an ArrayList<String> array and I am now trying to figure out how I can reverse the order of every 10 lines in the entire ArrayList.
so for example the output should be like this: Line : 10, 9, 8, 7 ...... 1, 20, 19, 18, 17.....11, 30, 29, 28, 27.....21 and so on until I have done this for the entire text file (Stored in the ArrayList). Below is the portion of code I have been using to try and reverse the lines as stated.
for(int i = array.size()-1; i >= array.size()-10; i--){
array.add(array.get(i));
}
for (String text : array) {
w.println(text);
}
}
What I have so far, reads and reverses only the last 10 lines of the input file and writes it to the output file. I have been having trouble figuring out a way to iteratively achieve this type of pattern throughout the entire data set, making sure that I do not hit an index out of bounds error.
Use a simple streaming approach. This is basically the current solution applied every ten lines instead of once at the end.
Read ten lines in.
Reverse these lines1.
Write the ten reversed lines out.
Repeat until the entire file is processed.
The only edge case is doing something appropriate at the end when the file isn't a multiple of 10 lines.
This same streaming approach can be used to create a new each-10 reversed list. It only 'becomes complicated' when trying to mutating the original list.
1 Steps 2 and 3 can be combined by iterating the list of ten lines backward when writing to the output.
Two approaches:
If it is already in memory, in an ArrayList, simply update that list.
If not already in memory, process 10 lines at a time.
This allows infinitely large data to be processed without running out of memory.
Option 1.
List<String> list = new ArrayList<>();
// code filling list
for (int i = 0; i < list.size(); i += 10) {
int endOfBlock = Math.min(i + 10, list.size());
for (int j = i, k = endOfBlock - 1; j < k; j++, k--) {
String temp = list.get(j);
list.set(j, list.get(k));
list.set(k, temp);
}
}
Option 2.
try (BufferedReader in = new BufferedReader(new FileReader(inFile)),
PrintWriter out = new PrintWriter(new FileWriter(outFile))) {
String[] buf = new String[10];
int len = 0;
for (String line; (line = in.readLine()) != null; ) {
buf[len++] = line;
if (len == 10) {
while (len > 0)
out.println(buf[--len]);
}
}
while (len > 0)
out.println(buf[--len]);
}
Try this
for (int i = 0, size = array.size(); i < size; i += 10)
for (int from = i, to = Math.min(i + 10, size); from < to;)
Collections.swap(array, from++, --to);
Use a secondary counter variable and an if statement to check for bounds, such as:
while(counter<array.size()+10)
int counter = 9; //since index 9 is the 10th line
for(int i=counter; i>counter-10; i--){
if(i<array.size()){
array.add(array.get(i));
}
}
counter+=10;
}
for (String text : array){
w.println(text);
}
My only concern here is that you appear to just continue adding to the existing array rather than reordering it or adding the elements to a new array?

Categories

Resources