Develop Reference

java class to read csv files instead of text files

The task is to create a java program that reads information from three .csv files and output a list of transcripts, ordered in descending order of aggregate mark, to a file in the current directory called "RankedList.txt". The program should show whether students have passed their year at university and what grade they achieved. The students took two modules, IR101 and IR102. This data is stored in two .csv files, IR101.csv and IR102.csv. Their names and registration numbers are stored in students.csv.
The rules of assessment stipulate the following:
Students must pass both modules in order to proceed to Stage 2. The pass mark for a module is 40.
Students who do not pass both modules will be deemed to have failed.
Students who fail only one of the two modules will be allowed a resit attempt.
Students who fail both modules will be required to repeat the year.
Students who pass both modules will be awarded a class based on their aggregate mark using the following scale:
70 – 100 = 1st
60 – 69.9 = 2.1
50 – 59.9 = 2.2
40 – 49.9 = 3rd
I have been able to complete this task however one problem I have faced is that my code only works for .txt files. If someone could show me how to change my code to work with .csv files I would be most grateful. My program so far is as follows:
package assignment;
import java.io.*;
import java.util.*;
public class StudentsMarks {
public static void main(String[] args) throws FileNotFoundException,IOException {
String currDir = "C:\\Users\\phili_000.Philip.001\\workspace\\ce152\\src\\ass\\StudentsMarks.java";
Scanner sc = new Scanner(new File(currDir+"IRStudents.csv"));
HashMap<Integer, String> students = new HashMap<Integer, String>();
while (sc.hasNext()) {
String line = sc.nextLine();
students.put(sc.nextInt(), sc.next());
String[] parts = line.split(",");
}
sc = new Scanner(new File(currDir+"IR101.csv"));
HashMap<Integer, Double> ir1 = new HashMap<Integer, Double>();
while (sc.hasNext()) {
String line = sc.nextLine();
ir1.put(sc.nextInt(), sc.nextDouble());
String[] parts = line.split(",");
}
sc = new Scanner(new File(currDir+"IR102.csv"));
HashMap<Integer, Double> ir2 = new HashMap<Integer, Double>();
while (sc.hasNext()) {
String line = sc.nextLine();
ir2.put(sc.nextInt(), sc.nextDouble());
String[] parts = line.split(",");
}
File output=new File(currDir+"RankedList.txt");
BufferedWriter b=new BufferedWriter(new FileWriter(output));
Iterator<Integer> ids = students.keySet().iterator();
while (ids.hasNext()) {
Integer id=ids.next();
b.write(id+" "+students.get(id));
b.newLine();
Double marks1=ir1.get(id);
Double marks2=ir2.get(id);
Double aggregate=(marks1+marks2)/2;
b.write("IR101\t "+marks1+"\t IR102\t "+marks2+"\t Aggregate "+aggregate);
b.newLine();
String classStd;
if(aggregate>=70){
classStd="1st";
}else if(aggregate>=60){
classStd="2.1";
}else if(aggregate>=50){
classStd="2.2";
}else if(aggregate>=40){
classStd="3rd";
}else{
classStd="failed";
}
String outcome;
if(marks1<40 && marks2<40){
outcome="Repeat the year";
}else if(marks1<40){
outcome="Resit IR101";
}else if(marks2<40){
outcome="Resit IR102";
}else{
outcome="Proceed to Stage 2";
}
b.write("Class:\t " + classStd + "\t Outcome: " + outcome);
b.newLine();
b.write("----------------------------------------------------");
b.newLine();
}
b.flush();
b.close();
}
}

String csvFile = "path.csv";
BufferedReader br = null;
String line = "";
String cvsSplitBy = ",";
try {
br = new BufferedReader(new FileReader(csvFile));
while ((line = br.readLine()) != null) {
// use comma as separator
String[] parts = line.split(cvsSplitBy);
}
} catch (Exception e) {
e.printStackTrace();
}
when reading a csv you should read the file line by line at the same time you should split the string in the line by using split method then you will
get an array of strings.

need to find remove duplicates from a text file comparing 1st and 5th string from every line

As part of a project I'm working on, I'd like to clean up a file I generate of duplicate line entries. These duplicates often won't occur near each other, however. I came up with a method of doing so in Java (which basically find a duplicates in the file, I stored two strings in two arrayLists and iterating but it was not working because of nested for loops i am getting into the condition manyways.
I need an integrated solution for this, however. Preferably in Java. Any ideas?
List item
public class duplicates {
static BufferedReader reader = null;
static BufferedWriter writer = null;
static String currentLine;
public static void main(String[] args) throws IOException {
int count=0,linecount=0;;
String fe = null,fie = null,pe=null;
File file = new File("E:\\Book.txt");
ArrayList<String> list1=new ArrayList<String>();
ArrayList<String> list2=new ArrayList<String>();
reader = new BufferedReader(new FileReader(file));
while((currentLine = reader.readLine()) != null)
{
StringTokenizer st = new StringTokenizer(currentLine,"/"); //splits data into strings
while (st.hasMoreElements()) {
count++;
fe=(String) st.nextElement();
//System.out.print(fe+"/// ");
//System.out.println("count="+count);
if(count==1){ //stores 1st string
pe=fe;
// System.out.println("first element "+fe);
}
else if(count==5){
fie=fe; //stores 5th string
// System.out.println("fifth element "+fie);
}
}
count=0;
if(linecount>0){
for(String s1:list1)
{
for(String s2:list2){
if(pe.equals(s1)&&fie.equals(s2)){ //checking condition
System.out.println("duplicate found");
//System.out.println(s1+ " "+s2);
}
}
}
}
list1.add(pe);
list2.add(fie);
linecount++;
}
}
}
i/p:
/book1/_cwc/B737/customer/Special_Reports/
/Airbook/_cwc/A330-200/customer/02_Watchlists/
/book1/_cwc/B737/customer/Special_Reports/
/jangeer/_cwc/Crj_200/customer/plots/
/Airbook/_cwc/A330-200/customer/02_Watchlists/
/jangeer/_cwc/Crj_200/customer/06_Performance_Summaries/
/jangeer/_cwc/Crj_200/customer/02_Watchlists/
/jangeer/_cwc/Crj_200/customer/01_Highlights/
/jangeer/_cwc/ERJ170/customer/01_Highlights/
o/p:
/book1/_cwc/B737/customer/Special_Reports/
/Airbook/_cwc/A330-200/customer/02_Watchlists/
/jangeer/_cwc/Crj_200/customer/plots/
/jangeer/_cwc/Crj_200/customer/06_Performance_Summaries/
/jangeer/_cwc/Crj_200/customer/02_Watchlists/
/jangeer/_cwc/Crj_200/customer/01_Highlights/

Use a Set<String> instead of Arraylist<String>.
Duplicates aren't allowed in a Set, so if you just add everyline to it, then get them back out, you'll have all distinct strings.
Performance-wise it's also quicker than your nested for-loop.

public static void removeDups() {
String[] input = new String[] { //Lets say you read whole file in this string array
"/book1/_cwc/B737/customer/Special_Reports/",
"/Airbook/_cwc/A330-200/customer/02_Watchlists/",
"/book1/_cwc/B737/customer/Special_Reports/",
"/jangeer/_cwc/Crj_200/customer/plots/",
"/Airbook/_cwc/A330-200/customer/02_Watchlists/",
"/jangeer/_cwc/Crj_200/customer/06_Performance_Summaries/",
"/jangeer/_cwc/Crj_200/customer/02_Watchlists/",
"/jangeer/_cwc/Crj_200/customer/01_Highlights/",
"/jangeer/_cwc/ERJ170/customer/01_Highlights/"
};
ArrayList<String> outPut = new ArrayList<>(); //The array list for storing output i.e. distincts.
Arrays.stream(input).distinct().forEach(x -> outPut.add(x)); //using java 8 and stream you get distinct from input
outPut.forEach(System.out::println); //I will write back to the file, just for example I am printing out everything but you can write back the output to file using your own implementation.
}
The output when I ran this method was
/book1/_cwc/B737/customer/Special_Reports/
/Airbook/_cwc/A330-200/customer/02_Watchlists/
/jangeer/_cwc/Crj_200/customer/plots/
/jangeer/_cwc/Crj_200/customer/06_Performance_Summaries/
/jangeer/_cwc/Crj_200/customer/02_Watchlists/
/jangeer/_cwc/Crj_200/customer/01_Highlights/
/jangeer/_cwc/ERJ170/customer/01_Highlights/
EDIT
Non Java 8 answer
public static void removeDups() {
String[] input = new String[] {
"/book1/_cwc/B737/customer/Special_Reports/",
"/Airbook/_cwc/A330-200/customer/02_Watchlists/",
"/book1/_cwc/B737/customer/Special_Reports/",
"/jangeer/_cwc/Crj_200/customer/plots/",
"/Airbook/_cwc/A330-200/customer/02_Watchlists/",
"/jangeer/_cwc/Crj_200/customer/06_Performance_Summaries/",
"/jangeer/_cwc/Crj_200/customer/02_Watchlists/",
"/jangeer/_cwc/Crj_200/customer/01_Highlights/",
"/jangeer/_cwc/ERJ170/customer/01_Highlights/"
};
LinkedHashSet<String> output = new LinkedHashSet<String>(Arrays.asList(input)); //output is your set of unique strings in preserved order
}

Creating objects via txt file into an array in Java

I am trying to complete a little program.
I've got a text file (.txt) to store different data on objects that i've got.
The structure of the file is the next (exemples data.txt) :
Sedane
2005
195000
Diesel
Blue
SUV
2013
34000
Fuel
Black
Each object is made true a class that i've build called Cars.
So the 1 line is the type of car, the 2nd the year of built, the 3rd line is the milage, the 4th is the type of fuel, and the 5th line is the color of the car.
So basicly i need to open the file, and load the data into the memory when i execute my program into an array with object in it.
I'm ok to open the file but i'm blocked when it comes to reading the data and putting it in an array.
The array size is 2 for this exemple, but if i have more entries in the file it's going to adapt it's size when loading at the startup of the program.
Here's what i've got unti now (for my code ...)
public static void loadCars () {
FileReader fopen;
BufferedReader opened;
String line;
try {
fEntree = new FileReader( "data.txt" );
opened = new BufferedReader( fopen );
while ( opened.ready() ) {
line = opened.readLine();
// Don't know what to do here ????
}
opened.close();
} catch ( IOException e ) {
System.out.println( "File doesn't exist !" );
}
}

Someting like this will do the trick. I'm adding the file contents line by line to an Arraylist instead of an array though. This way you don't have to know how big your array needs to be before hand. Plus you can always change it to an array later.
public ArrayList<String> readFileToMemory(String filepath)
{
in = new BufferedReader(new FileReader( "data.txt" ));
String currentLine = null;
ArrayList<String> fileContents = new ArrayList<String>();
try
{
while((currentLine = in.readLine()) != null)
{
fileContents.add(currentLine);
}
}
catch(IOException e)
{
e.printStackTrace();
}
finally
{
try
{
in.close();
}
catch(IOException e)
{
e.printStackTrace();
}
}
return fileContents;
}

LineNumberReader lnr = new LineNumberReader(new FileReader(new File("File1")));
lnr.skip(Long.MAX_VALUE);
long length = lnr.getLineNumber();
lnr.close();
in = new BufferedReader(new FileReader( "data.txt" ));
Car[] cars= new Car[length/5];
String currentLine;
int i=0;
for(int i=0;i<length/5;i+=5) {
String name = in.readLine();
String year = in.readLine();
String miles = in.readLine();
String gas = in.readLine();
String color = in.readLine();
cars[i] = new Car(name,year,miles,gas,color);
}
You'll have to handle exceptions too, surround stuff in try catch structures.

You can look at my solution here below (I also corrected/simplified some problems with the variables for reading the file, anyway this was not the main topic):
public static void loadCars() {
FileReader fopen;
BufferedReader opened;
String line;
ArrayList<Car> carList = new ArrayList<Car>();
try {
fopen = new FileReader("data.txt");
opened = new BufferedReader(fopen);
int nFields = 5; // we have 5 fields in the Car class
String[] fields = new String[nFields]; // to temporary store fields values read line by line
int lineCounter = 0;
while ((line = opened.readLine()) != null) {
fields[lineCounter] = line;
lineCounter++;
if ((lineCounter) % nFields == 0) { //it means we have all 5 fields values for a car
carList.add(new Car(fields)); //therefore we create a new car and we add it to the list of cars
}
}
opened.close();
} catch (IOException e) {
System.out.println("File doesn't exist !");
}
}
Basically we use an ArrayList to store all the cars, and we read the file, waiting to have all the fields values in order to create the Car object. I store the fields values in an array of Strings: I don't know how you implemented the Car class, but maybe it is useful to create a constructor that takes as parameter an array of strings, so it can set the fields, for instance:
class Car {
private String type;
private String year;
private String milage;
private String fuel;
private String color;
public Car(String[] fields) {
type=fields[0];
year=fields[0];
milage=fields[0];
fuel=fields[0];
type=fields[0];
}
}
But I've to say that probably this is a little 'too static'.
For simplicity I assumed that all your fields are of String type, but probably fields like 'year' or 'milage' might be of int type. In this case you can use array of Object[] (instead of String[]), and then cast the value with the right type.
I hope this may help you.

Reading from file and splitting the data in Java

I'm trying to read data from a .txt file. The format looks like this:
ABC, John, 123
DEF, Mark, 456
GHI, Mary, 789
I am trying to get rid of the commas and put the data into an array or structure (structure most likely).
This is the code I used to to extract each item:
package prerequisiteChecker;
import java.util.*;
import java.io.*;
public class TestUnit {
public static void main(String[]args){
try {
FileInputStream fstream = new FileInputStream("courses.txt");
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String strLine;
while ((strLine = br.readLine()) != null) {
String[] splitOut = strLine.split(", ");
for (String token : splitOut)
System.out.println(token);
}
in.close();
} catch (Exception e){
System.err.println("Error: " + e.getMessage());
}
}
}
At one point I had a print line in the "while" loop to see if the items would be split. They were. Now I'm just at a loss on what to do next. I'm trying to place each grouping into one structure. For example: ID - ABC. First Name - John. Room - 123.
I have a few books on Java at home and tried looking around the web. There is so much out there, and none of it seemed to lead me in the right direction.
Thanks.
Michael

create a class that looks something like this:
class structure {
public String data1;
public String data2;
public String data3;
}
This will form your basic data structure that you can use to hold the kind of data you have mentioned in your question. Now, you might want to follow proper object oriented methods like declaring all your fields as private, and writting getters and setters. you can find more on there here ... http://java.dzone.com/articles/getter-setter-use-or-not-use-0
Now, just outside your while loop, create an ArrayList like this: ArrayList<structure> list = new ArrayList<structure>(); This will be used to hold all the different rows of data that you will parse.
Now, in your while loop do something like this:
structure item = new structure();//create a new instance for each row in the text file.
item.data1 = splitOut[0];
item.data2 = splitOut[1];
item.data3 = splitOut[2];
list.add(item);
this will basically take the data that you parse in each row, put in the data structure that you declared by creating a new instance of it for each new row that is parsed. this finally followed by inserting that data item in the ArrayList using the list.add(item) in the code as shown above.

I would create a nice structure to store your information. I'm not sure if how you want to access the data, but here's a nice example. I'll go off of what you previously put. Please note that I only made the variables public because they're final. They cannot change once you make the Course. If you want the course mutable, create getters and setters and change the instance variables to private. After, you can use the list to retrieve any course you'd like.
package prerequisiteChecker;
import java.util.*;
import java.io.*;
public class TestUnit {
public static void main(String[] args) {
try {
FileInputStream fstream = new FileInputStream("courses.txt");
// use DataInputStream to read binary NOT text
// DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(fstream));
String strLine;
List<Course> courses = new LinkedList<Course>();
while ((strLine = br.readLine()) != null) {
String[] splitOut = strLine.split(", ");
if (splitOut.length == 3) {
courses.add(new Course(splitOut[0], splitOut[1],
splitOut[2]));
} else {
System.out.println("Invalid class: " + strLine);
}
}
in.close();
} catch (Exception e) {
System.err.println("Error: " + e.getMessage());
}
}
public static class Course {
public final String _id;
public final String _name;
public final String _room;
public Course(String id, String name, String room) {
_id = id;
_name = name;
_room = room;
}
}
}

public class File_ReaderWriter {
private static class Structure{
public String data;
}
public static void main(String[] args) throws IOException{
String allDataString;
FileInputStream fileReader = new FileInputStream ("read_data_file.txt");
DataInputStream in = new DataInputStream(fileReader);
BufferedReader bufferReader = new BufferedReader(new InputStreamReader(in));
String[] arrayString = {"ID - ", " NAME - ", " ROOM - "};
int recordNumber = 0;
Structure[] structure = new Structure[10];
for (int i = 0; i < 10; i++)
structure[i] = new Structure();
while((allDataString = bufferReader.readLine()) != null){
String[] splitOut = allDataString.split(", ");
structure[recordNumber].data = "";
for (int i = 0; i < arrayString.length; i++){
structure[recordNumber].data += arrayString[i] + splitOut[i];
}
recordNumber++;
}
bufferReader.close();
for (int i = 0; i < recordNumber; i++){
System.out.println(structure[i].data);
}
}
}
I modify your given code. It works. Try it and if any query then ask.

Java : Resizing a multidimensional array

I have a multidimensional array built from Strings that is initially created with the size [50][50], this is too big and now the array is full of null values, I am currently trying to remove these said null values, I have managed to resize the array to [requiredSize][50] but cannot shrink it any further, could anyone help me with this? I have scoured the internet for such an answer but cannot find it.
Here is my complete code too (I realise there may be some very unclean parts in my code, I am yet to clean anything up)
import java.io.*;
import java.util.*;
public class FooBar
{
public static String[][] loadCSV()
{
FileInputStream inStream;
InputStreamReader inFile;
BufferedReader br;
String line;
int lineNum, tokNum, ii, jj;
String [][] CSV, TempArray, TempArray2;
lineNum = tokNum = ii = jj = 0;
TempArray = new String[50][50];
try
{
BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
System.out.println("Please enter the file path of the CSV");
String fileName = in.readLine();
inStream = new FileInputStream(fileName);
inFile = new InputStreamReader(inStream);
br = new BufferedReader(inFile);
StringTokenizer tok,tok2;
lineNum = 0;
line = br.readLine();
tokNum = 0;
tok = new StringTokenizer(line, ",");
while( tok.hasMoreTokens())
{
TempArray[tokNum][0] = tok.nextToken();
tokNum++;
}
tokNum = 0;
lineNum++;
while( line != null)
{
line = br.readLine();
if (line != null)
{
tokNum = 0;
tok2 = new StringTokenizer(line, ",");
while(tok2.hasMoreTokens())
{
TempArray[tokNum][lineNum] = tok2.nextToken();
tokNum++;
}
}
lineNum++;
}
}
catch(IOException e)
{
System.out.println("Error file may not be accessible, check the path and try again");
}
CSV = new String[tokNum][50];
for (ii=0; ii<tokNum-1 ;ii++)
{
System.arraycopy(TempArray[ii],0,CSV[ii],0,TempArray[ii].length);
}
return CSV;
}
public static void main (String args[])
{
String [][] CSV;
CSV = loadCSV();
System.out.println(Arrays.deepToString(CSV));
}
}
The CSV file looks as follows
Height,Weight,Age,TER,Salary
163.9,46.8,37,72.6,53010.68
191.3,91.4,32,92.2,66068.51
166.5,51.1,27,77.6,42724.34
156.3,55.7,21,81.1,50531.91
It can take any size obviously but this is just a sample file.
I just need to resize the array so that it will not contain any null values.
I also understand a list would be a better option here but it is not possible due to outside constraints. It can only be an multi dimensional array.

I think you need 3 changes to your program
After your while loop lineNum will be 1 more than the number of lines in the file so instead of declaring CSV to String[tokNum][50] declare it as CSV = new String[tokNum][lineNum-1];
tokNum will be the number of fields in a row so your for loop condition should be ii<tokNum rather than ii<tokNum-1
The last parameter for your arraycopy should be lineNum-1
i.e. the modified code to build your CSV array is:
CSV = new String[tokNum][lineNum-1];
for (ii=0; ii<tokNum ;ii++)
{
System.arraycopy(TempArray[ii],0,CSV[ii],0,lineNum-1);
}
and the output will then be:
[[Height, 163.9, 191.3, 166.5, 156.3], [Weight, 46.8, 91.4, 51.1, 55.7],
[Age, 37, 32, 27, 21], [TER, 72.6, 92.2, 77.6, 81.1],
[Salary, 53010.68, 66068.51, 42724.34, 50531.91]]
Notice that you don't really need to handle the first line of the file separately from the others but that is something you can cover as part of your cleanup.

10 to 1 this is a homework assignment. However, it looks like you've put somethought into it.
Don't make the TempArray variable. Make a "List of List of Strings". Something like:
List<List<String>> rows = new ArrayList<ArrayList<String>>();
while(file.hasMoreRows()) { //not valid syntax...but you get the jist
String rowIText = file.nextRow(); //not valid syntax...but you get the jist
List<String> rowI = new ArrayList<String>();
//parse rowIText to build rowI --> this is your homework
rows.add(rowI);
}
//now build String[][] using fully constructed rows variable

Here's an observation and a suggestion.
Observation: Working with (multidimensional) arrays is difficult in Java.
Suggestion: Don't use arrays to represent complex data types in Java.
Create classes for your data. Create a List of people:
class Person {
String height; //should eventually be changed to a double probably
String weight; // "
//...
public Person( String height, String weight /*, ... */ ) {
this.height = height;
this.weight = weight;
//...
}
}
List<Person> people = new ArrayList<Person>();
String line;
while ( (line = reader.nextLine()) != null ) {
String[] records = line.split(",");
people.add(new Person (records[0], records[1] /*, ... */));
}