I'm using BufferedReader to read a .csv file. I have no problem reading the file and extracting the data. However, the problem that I do have is that I have to hard-code my array declaration. For example:
String[][] numbers=new String[5258][16];
The .csv file I was using had 5258 rows and 16 columns. I'd like to be able to do something like this though:
String[][] numbers=new String[rowsInFile][16];
In other words, I want the variable 'rowsInFile' to be equivalent to the amount of rows in the file (I don't want to count the columns, because every .csv file I will be running through this program has 16 columns).
Here's the code I have so far:
int row = 0;
int col = 0;
String fileInput = JOptionPane.showInputDialog(null,
"Please enter the path of the CSV file to read:");
File file = new File(fileInput);
BufferedReader bufRdr;
bufRdr = new BufferedReader(new FileReader(file));
String line = null;
//get rows in the file
int rowsInFile = 0;
while(bufRdr.readLine() != null) {
rowsInFile++;
row++;
}
String[][] numbers=new String[rowsInFile][16];
//read each line of text file
row = 0;
while((line = bufRdr.readLine()) != null) {
StringTokenizer st = new StringTokenizer(line,",");
col=0;
while (st.hasMoreTokens()) {
//get next token and store it in the array
numbers[row][col] = st.nextToken();
col++;
}
row++;
}
However, I'm getting a null pointer exception. Any ideas of what I should do?
P.S. Yes, this code is surrounded by a try/catch statement.
The problem is, once you go through the BufferedReader once, you can't go back through it again. In other words, you have to use a new BufferedReader.
bufRdr = new BufferedReader(new FileReader(file));
row = 0;
while((line = bufRdr.readLine()) != null) {
Alternatively, you could use a dynamic array structure like an ArrayList<String[]> or a LinkedList<String[]> to store the rows.
LinkedList<String[]> numbers = new LinkedList<String[]>();
while( (line = bufRdr.readLine()) != null ) {
numbers.add(line.split(","));
}
Then instead of doing numbers[i][j], you use numbers.get(i)[j].
Instead of an array use something dynamic like a List. For example:
List<String[]> data = new ArrayList<String[]>();
Also using String's split() method will simplify the loading of the row.
Your problem is that BufferedReaders work by reading until the end of a file, and then they get stuck there. Your code requires reading through the file twice, but because you already reached an EOF, the BufferedReader is stuck returning null. I tend to solve this by stuffing the lines into an ArrayList, and using the size() method to get the number of lines. The source code looks something like this:
int rowsInFile=0;
ArrayList<String> lines = new ArrayList<String>();
String tmp = "";
while(tmp=bugRdr.readLine())
{
lines.add(tmp);
}
rowsInFile = lines.size();
String[][] numbers = new String[rowsInFile][16];
int row = 0;
for(String line : lines)
{
StringTokenizer st = new StringTokenizer(line,",");
col=0;
while (st.hasMoreTokens()) {
//get next token and store it in the array
numbers[row][col] = st.nextToken();
col++;
}
row++;
}
Related
i'm doing some java coding and i have to import a file into a Jtable that has 4 columns while my file has 3 (separated by whitespaces, i need the first column of each line to be auto increment, here is my code:
try {
FileReader files = new FileReader(file);
BufferedReader buf = new BufferedReader(files);
String line = null;
String tokens[] = null;
while ((line = buf.readLine()) != null) {
tokens = line.split("\\p{javaWhitespace}+");
//System.out.println( Arrays.toString( tokens ));
model.addRow(tokens);
}
}
and this is what I'm getting :
and this is my file :
Simply add an additional token to the front of your data. This is easiest using a Vector and not an array. The first item in the vector is your row index, the next itmes are filled from your tokens array. For example:
try {
FileReader files = new FileReader(file);
BufferedReader buf = new BufferedReader(files);
String line = null;
String tokens[] = null;
int count = 0;
while ((line = buf.readLine()) != null) {
tokens = line.split("\\p{javaWhitespace}+");
Vector<Object> row = new Vector<>();
row.add(count);
count++;
for (String text: tokens) {
row.add(text);
}
model.addRow(row); // add the Vector, not the tokens array
}
}
There are other ways, including extending the table model such that it automatically does this, and these may need to be done, depending on your needs -- for example, are the rows to renumber if one row is deleted or added during the running of the program? If so the logic needs to be within the table model.
I have a requirement like:
My input will be a csv file where I will be having values like below:
action, userId, firstName, email, lastName
1,2,sample,abc#gmail.com,test
2,3,justtest,def#gmail.com,test
I have to read this csv file based on headers. Say for ex: if action =1 and email is null, then I have to change from action to 2 or email to null or something like this.
I have no idea on how to read and parse the values based on the headers. This is my code what I tried:
String csvFile = "C:\\Test.csv";
// create BufferedReader to read csv file
BufferedReader br;
String line = "";
br = new BufferedReader(new FileReader(csvFile));
br.readLine();
// Read the file line by line starting from the second line
while ((line = br.readLine()) != null) {
// Get all tokens available in line
String[] tokens = line.split(",");
if (tokens.length > 0) {
for (int i = 0; i < tokens.length; i++) {
System.out.println("All Rows ------->" + tokens[i]);
}
}
}
This is just printing all the values in new line like below:
All Rows ------->1411184866
All Rows ------->category
All Rows ------->123456
All Rows ------->Test
All Rows ------->TestFullName
All Rows ------->rsap#gmail.com
All Rows ------->3423131
Please help me completing this code. Thanks in advance.
For parsing the CSV file into a post-processable format, I would first create an enum that models the columns of the file, in correct order:
enum Column {
ACTION, USERID, FIRSTNAME, EMAIL, LASTNAME;
public static final Column[] VALUES = values();
}
Then, I would read the file into a mapping of columns into lists of column values (this data structure is also called a "multimap"):
Map<Column, List<String>> columns =
new LinkedHashMap<>();
// initialize the map
for (Column c : Column.VALUES) {
columns.put(c, new ArrayList<>());
}
String csvFile = "C:\\Test.csv";
String line = "";
// create BufferedReader to read csv file
BufferedReader br = new BufferedReader(new FileReader(csvFile));
br.readLine();
// Read the file line by line starting from the second line
while ((line = br.readLine()) != null) {
// Get all tokens available in line
String[] tokens = line.split(",");
if (tokens.length > 0) {
for (int i = 0; i < tokens.length; i++) {
columns.get(Column.VALUES[i]).add(tokens[i].trim());
}
}
}
Now that you have the data ordered, you can start post-processing the values based on your business rules. Of course you can also apply the rules while reading the file, but this might hurt readability of the code. Accessing individual cells in the file is easy; e.g. the email address on the second row can be retrieved using columns.get(Column.EMAIL).get(1).
Running
System.out.println(columns);
with your example file outputs this:
{ACTION=[1, 2], USERID=[2, 3], FIRSTNAME=[sample, justtest],
EMAIL=[abc#gmail.com, def#gmail.com], LASTNAME=[test, test]}
Use Apache Common CSV libraries (or any other suitable libraries) to read the CSV files and then apply the business logic in the program.
https://commons.apache.org/proper/commons-csv/
It will give you the list of rows in the form of CSVRecord and then you just need to apply business logic based on the values by iterarting over the list. The first element in the list will be your header.
Reader in = new FileReader("File Name");
CSVParser parser = new CSVParser(in, CSVFormat.EXCEL);
List<CSVRecord> csvRecords = parser.getRecords();
I have a text file with 300 lines or so. And the format is like:
Name Amount Unit CountOfOrder
A 1 ml 5000
B 1 mgm 4500
C 4 gm 4200
// more data
I need to read the text file line by line because each line of data should be together for further processing.
Now I just use string array for each line and access the data by index.
for each line in file:
array[0] = {data from the 'Name' column}
array[1] = {data from the 'Amount' column}
array[2] = {data from the 'Unit' column}
array[3] = {data from the 'CountOfOrder' column}
....
someOtherMethods(array);
....
However, I realized that if the text file changes its format (e.g. switch two columns, or insert another column), it would break my program (accessing through index might be wrong or even cause exception).
So I would like to use the title as reference to access each column. Maybe HashMap is a good option, but since I have to keep each line of data together, if I build a HashMap for each line, that would be too expensive.
Does anyone have any thought on this? Please help!
you only need a single hash map to map your column names to the proper column index. you fill the arrays by indexing with integers as you did before, to retrieve a column by name you'd use array[hashmap.get("Amount")].
You can read the file using opencsv.
CSVReader reader = new CSVReader(new FileReader("yourfile.txt"), '\t');
List<String[]> lines = reader.readAll();
The fist line contains the headers.
you can read each line of the file and assuming that the first line of the file has the column header you can parse that line to get all the names of the columns.
String[] column_headers = firstline.split("\t");
This will give you the name of all the columns now you just read through splitting on tabs and they will all line up.
You could do something like this:
BufferedReader in = new BufferedReader(new InputStreamReader(
new FileInputStream(FILE)));
String line = null;
String[] headers = null;
String[] data = null;
Map<String, List<String>> contents = new HashMap<String, List<String>>();
if ((line = in.readLine()) != null) {
headers = line.split("\t");
}
for(String h : headers){
contents.put(h, new ArrayList<String>());
}
while ((line = in.readLine()) != null) {
data = line.split("\t");
if(data.length != headers.length){
throw new Exception();
}
for(int i = 0; i < data.length; i++){
contents.get(headers[i]).add(data[i]);
}
}
It would give you flexibility, and would only require making the map once. You can then get the data lists from the map, so it should be a convenient data structure for the rest of your program to use.
This will give you individual list of columns.
public static void main(String args[]) throws FileNotFoundException, IOException {
List<String> headerList = new ArrayList<String>();
List<String> column1 = new ArrayList<String>();
List<String> column2 = new ArrayList<String>();
List<String> column3 = new ArrayList<String>();
List<String> column4 = new ArrayList<String>();
int lineCount=0;
BufferedReader br = new BufferedReader(new FileReader("file.txt"));
try {
StringBuilder sb = new StringBuilder();
String line = br.readLine();
String tokens[];
while (line != null) {
tokens = line.split("\t");
if(lineCount != 0)
{
int count = 0;
column1.add(tokens[count]); ++count;
column2.add(tokens[count]); ++count;
column3.add(tokens[count]); ++count;
column4.add(tokens[count]); ++count;
continue;
}
if(lineCount==0){
for(int count=0; count<tokens.length; count++){
headerList.add(tokens[count]);
lineCount++;
}
}
}
} catch (IOException e) {
} finally {
br.close();
}
}
using standard java.util.Scanner
String aa = " asd 9 1 3 \n d -1 4 2";
Scanner ss = new Scanner(aa);
ss.useDelimiter("\n");
while ( ss.hasNext()){
String line = ss.next();
Scanner fs = new Scanner(line);
System.out.println( "1>"+ fs.next()+" " +fs.nextInt() +" " +fs.nextLong()+" " +fs.nextBigDecimal());
}
using a bunch of hashmap's is ok...i won't be afraid ;)
if you need to process a lot of data...then try to translate your problem into a dataprocessing transformation
for example:
read all of you data into a hashmap's, but store them in a database using some JPA implementation....then you can go round'a'round your data ;)\
I have OpenCSV reader in a Java project and its reading data from CSV files for me, but right now Im hardcoding the number of colums in the loop that reads it.
I can recall there was some method that could get the number of columns but I dont remember what it was called and how to use it.
Here is the code:
public ArrayList<ArrayList<String>> getCSVContent(String filepath) throws Exception {
CSVReader reader = new CSVReader(new FileReader(FILE_PATH));
ArrayList<ArrayList<String>> array = new ArrayList<ArrayList<String>>();
String[] nextLine;
while ((nextLine = reader.readNext()) != null) {
ArrayList<String> list = new ArrayList<String>();
for (int i = 0; i < 5; i++) { //5 is the number of columns in the file
list.add(nextLine[i]);
}
array.add(list);
}
reader.close();
return array;
}
Just count the items in the array with nextLine.length.
for (int i = 0; i < nextLine.length; i++) {
list.add(nextLine[i]);
}
Or use a for-each loop:
for (String col : nextLine) {
list.add(col);
}
Well a easy way is that you read the first line with a Scanner of BufferedReader and count the ";" or what you use to split the columns.
You are able to count it if you use
string.toCharArray();
and ++ a Integer if it is ';'.
A second way is the look at the methods of CSV Reader. I don't know them but you can type anywhere in eclipse(don't know how it works in netbeans) "reader." and press control + space. If you have luck there is one.
Try this one to get the column count: reader.getColumnCount()
I have a multidimensional array built from Strings that is initially created with the size [50][50], this is too big and now the array is full of null values, I am currently trying to remove these said null values, I have managed to resize the array to [requiredSize][50] but cannot shrink it any further, could anyone help me with this? I have scoured the internet for such an answer but cannot find it.
Here is my complete code too (I realise there may be some very unclean parts in my code, I am yet to clean anything up)
import java.io.*;
import java.util.*;
public class FooBar
{
public static String[][] loadCSV()
{
FileInputStream inStream;
InputStreamReader inFile;
BufferedReader br;
String line;
int lineNum, tokNum, ii, jj;
String [][] CSV, TempArray, TempArray2;
lineNum = tokNum = ii = jj = 0;
TempArray = new String[50][50];
try
{
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Please enter the file path of the CSV");
String fileName = in.readLine();
inStream = new FileInputStream(fileName);
inFile = new InputStreamReader(inStream);
br = new BufferedReader(inFile);
StringTokenizer tok,tok2;
lineNum = 0;
line = br.readLine();
tokNum = 0;
tok = new StringTokenizer(line, ",");
while( tok.hasMoreTokens())
{
TempArray[tokNum][0] = tok.nextToken();
tokNum++;
}
tokNum = 0;
lineNum++;
while( line != null)
{
line = br.readLine();
if (line != null)
{
tokNum = 0;
tok2 = new StringTokenizer(line, ",");
while(tok2.hasMoreTokens())
{
TempArray[tokNum][lineNum] = tok2.nextToken();
tokNum++;
}
}
lineNum++;
}
}
catch(IOException e)
{
System.out.println("Error file may not be accessible, check the path and try again");
}
CSV = new String[tokNum][50];
for (ii=0; ii<tokNum-1 ;ii++)
{
System.arraycopy(TempArray[ii],0,CSV[ii],0,TempArray[ii].length);
}
return CSV;
}
public static void main (String args[])
{
String [][] CSV;
CSV = loadCSV();
System.out.println(Arrays.deepToString(CSV));
}
}
The CSV file looks as follows
Height,Weight,Age,TER,Salary
163.9,46.8,37,72.6,53010.68
191.3,91.4,32,92.2,66068.51
166.5,51.1,27,77.6,42724.34
156.3,55.7,21,81.1,50531.91
It can take any size obviously but this is just a sample file.
I just need to resize the array so that it will not contain any null values.
I also understand a list would be a better option here but it is not possible due to outside constraints. It can only be an multi dimensional array.
I think you need 3 changes to your program
After your while loop lineNum will be 1 more than the number of lines in the file so instead of declaring CSV to String[tokNum][50] declare it as CSV = new String[tokNum][lineNum-1];
tokNum will be the number of fields in a row so your for loop condition should be ii<tokNum rather than ii<tokNum-1
The last parameter for your arraycopy should be lineNum-1
i.e. the modified code to build your CSV array is:
CSV = new String[tokNum][lineNum-1];
for (ii=0; ii<tokNum ;ii++)
{
System.arraycopy(TempArray[ii],0,CSV[ii],0,lineNum-1);
}
and the output will then be:
[[Height, 163.9, 191.3, 166.5, 156.3], [Weight, 46.8, 91.4, 51.1, 55.7],
[Age, 37, 32, 27, 21], [TER, 72.6, 92.2, 77.6, 81.1],
[Salary, 53010.68, 66068.51, 42724.34, 50531.91]]
Notice that you don't really need to handle the first line of the file separately from the others but that is something you can cover as part of your cleanup.
10 to 1 this is a homework assignment. However, it looks like you've put somethought into it.
Don't make the TempArray variable. Make a "List of List of Strings". Something like:
List<List<String>> rows = new ArrayList<ArrayList<String>>();
while(file.hasMoreRows()) { //not valid syntax...but you get the jist
String rowIText = file.nextRow(); //not valid syntax...but you get the jist
List<String> rowI = new ArrayList<String>();
//parse rowIText to build rowI --> this is your homework
rows.add(rowI);
}
//now build String[][] using fully constructed rows variable
Here's an observation and a suggestion.
Observation: Working with (multidimensional) arrays is difficult in Java.
Suggestion: Don't use arrays to represent complex data types in Java.
Create classes for your data. Create a List of people:
class Person {
String height; //should eventually be changed to a double probably
String weight; // "
//...
public Person( String height, String weight /*, ... */ ) {
this.height = height;
this.weight = weight;
//...
}
}
List<Person> people = new ArrayList<Person>();
String line;
while ( (line = reader.nextLine()) != null ) {
String[] records = line.split(",");
people.add(new Person (records[0], records[1] /*, ... */));
}