Develop Reference

How to remove row which contains blank cell from csv file in Java

I'm trying to do data cleaning on dataset. by data cleaning i meant removing the row which containes NaN or duplicates values or empty cell. here is my code
dataset look like this:
Sno Country noofDeaths
1 32432
2 Pakistan NaN
3 USA 3332
3 USA 3332
excel file image:
public class data_reader {
String filePath="src\\abc.csv";
public void readData() {
BufferedReader br = null;
String line = "";
HashSet<String> lines = new HashSet<>();
try {
br = new BufferedReader(new FileReader(filePath));
while ((line = br.readLine()) != null) {
if(!line.contains("NaN") || !line.contains("")) {
if (lines.add(line)) {
System.out.println(line);
}
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
it is working fine for NaN values and duplicates rows but not for empty cell, please help how to do this.
!line.contains("")
this is not working.

Condition !line.contains("") - doesn't make sence because every string contains empty string.
General suggestions:
don't hard code file-path, code must be reusable;
use try with resources;
camel-case names.
public class DataReader {
public static void main(String[] args) {
new DataReader().readData("src\\abc.csv");
}
public void readData(String filePath) {
try(BufferedReader br = new BufferedReader(new FileReader(filePath))) {
HashSet<String> lines = new HashSet<>();
String line = null;
while ((line = br.readLine()) != null) {
if(!line.contains("NaN")) {
for (String cell: line.split(",")) {
if (!cell.isBlank()&&lines.add(cell)) {
System.out.print(cell + " ");
}
}
}
System.out.println();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}

Seems to me this is a pretty easy problem to solve. Given a CSV file with an empty row
foo,bar,baz
1,One,123
,,
2,Two,456
3,Three,789
You can read the lines and define an empty line as one which contains empty strings separated by commas. You could read the contents of the file, store the populated lines into a string buffer, and then save the contents of the buffer once the empty lines are extracted out. The code below accomplishes this:
public static void main(String[] args) throws IOException {
String file ="test.csv";
BufferedReader reader = new BufferedReader(new FileReader(file));
String line = null;
StringBuilder sbuff = new StringBuilder();
while ((line = reader.readLine()) != null) {
String[] tokens = line.split(",");
if (containsText(tokens)) {
sbuff.append(line + "\n");
}
}
reader.close();
System.out.println(sbuff.toString());
// save file here
}
public static boolean containsText(String[] tokens) {
for (String token: tokens) {
if (token.length() > 0)
return true;
}
return false;
}
After running the code, the output is:
foo,bar,baz
1,One,123
2,Two,456
3,Three,789
This same code can be used to determine if a cell is empty with a simple method:
public static boolean isCellEmpty(String[] tokens) {
for (String token: tokens) {
if (token.isBlank())
return true;
}
return false;
}

Pass in file text into hashmap

I am having a bit of an issues trying to pass in a file read by my program and sorted accordantly. I am not used to working with files, and i ran out of ideas as to how this could be achieved.
/////////////////////////////////////// class reads file ///////////////////////////////////
import java.io.*;
public class InFileReader {
private BufferedReader inputStream = null;
private String fileLine;
private StringBuilder sb;
public String getFile(File fileRead) throws FileNotFoundException,
IOException {
inputStream = new BufferedReader(new FileReader(fileRead)); //reads files
sb = new StringBuilder();
while((fileLine = inputStream.readLine()) != null){//keep reading lines in file till there is none
sb.append(fileLine).append("\n");
}
return sb.toString(); //returns StringBuffer read values in String form
}
}
///////////////////////////////////////////////////// end of read file class ///////////////////////
public void getFile(File fileRead) throws FileNotFoundException,
IOException {
try {
String input = fileReader.getFile(fileRead.getAbsoluteFile());
HashMap<Integer, Thing.Ship> hashmap = new HashMap<>();
while (!input.isEmpty()) { // as long as there is data in the file keep looping
Scanner sc = new Scanner(input); // scan file
if (!input.startsWith("//")) { // take out "//" from directory
String type = "";
if (sc.hasNext()) { // if there are character lines get next line
type = sc.next();
}
if (type.equalsIgnoreCase("port")) { // looks for "port"
world.assignPort(new Thing.SeaPort(sc)); // assigns value to Seaport
} else if (type.equalsIgnoreCase("dock")) {
world.assignDock(new Thing.Dock(sc));
} else if (type.equalsIgnoreCase("ship")) {
Thing.Ship s = new Thing.Ship(sc);
hashmap.put(s.getIndex(), s);
world.assignShip(s);
} else if (type.equalsIgnoreCase("pship")) {
Thing.Ship s = new Thing.PassengerShip(sc);
hashmap.put(s.getIndex(), s);
world.assignShip(s);
} else if (type.equalsIgnoreCase("cship")) {
Thing.Ship s = new Thing.CargoShip(sc);
hashmap.put(s.getIndex(), s);
world.assignShip(s);
} else if (type.equalsIgnoreCase("person")) {
world.assignPerson(new Thing.Person(sc));
}
}
}
//inputOut.setText(type);
inputOut.setText(world.toString());
} catch (Exception e) {
System.out.println(e + "-----");
}
}
Here fileRead knows where to find the file to be read "C:\Users\abe\IdeaProjects\CreateSeaPortDataFile\src\text.txt"
public void getFile(File fileRead) throws FileNotFoundException,
IOException {
this is where things just fall apart:
String input = fileReader.getFile(fileRead.getAbsoluteFile());
My intent here is to pass the location of the file so that the getFile class can read it and then be sorted into the hashmap.
again i am not familiar with how to work with file, any suggestion or comment would be greatly appreciated.
thank you in advanced.

If you get a FileNotFoundException then the file was not found.
You say the filename was "C:\Users\abe\IdeaProjects\CreateSeaPortDataFile\src\text.txt".
If you type that name in the code you must escape the backslash:
"C:\\Users\\abe\\IdeaProjects\\CreateSeaPortDataFile\\src\\text.txt".

Need help regarding duplicates records in JAVA

Here, I wrote my code with my output. I do not know about how to work with duplicate records.
class Price{
int productID,totalSold=0,totalPrice=0;
double price;
public Price(int productID,double price) {
this.price= price;
this.productID=productID;
}
}
class Slips{
int personID, productID, numberOfSold,product1Sold,product2Sold,product3Sold,product4Sold,performance;
public Slips(int personID,int productID,int numberOfSold)
{
this.personID=personID;
this.productID=productID;
this.numberOfSold=numberOfSold;
}
}
class PriceComparator implements Comparator<Price>{
public int compare(Price p1,Price p2)
{
return Integer.compare(p2.totalPrice, p1.totalPrice);
}
}
public class PTestDec2009 {
static ArrayList<Price> ALPrice=new ArrayList<>();
static ArrayList<Slips> ALSlips=new ArrayList<>();
public static void main(String[] args) throws FileNotFoundException, IOException {
getPriceInfo("C:\\Users\\Mayank Patel\\Documents\\NetBeansProjects\\PTestDec2009\\src\\ptestdec2009\\price.txt");
getSlipsInfo("C:\\Users\\Mayank Patel\\Documents\\NetBeansProjects\\PTestDec2009\\src\\ptestdec2009\\slips.txt");
updateSalability();
updateSalesPersonPerformance();
}
private static void getPriceInfo(String fileName) throws FileNotFoundException, IOException {
String line=null;
BufferedReader bufferReader=new BufferedReader(new FileReader(fileName));
while((line=bufferReader.readLine())!=null)
{
StringTokenizer tokenizer=new StringTokenizer(line);
while(tokenizer.hasMoreTokens())
{
ALPrice.add(new Price(Integer.parseInt(tokenizer.nextToken()),Double.parseDouble(tokenizer.nextToken())));
}
}
}
private static void getSlipsInfo(String fileName) throws FileNotFoundException, IOException {
String line=null;
BufferedReader bufferReader= new BufferedReader(new FileReader(fileName));
while((line=bufferReader.readLine())!=null)
{
StringTokenizer tokenizer=new StringTokenizer(line);
while(tokenizer.hasMoreTokens())
{
ALSlips.add(new Slips(Integer.parseInt(tokenizer.nextToken()),Integer.parseInt(tokenizer.nextToken()), Integer.parseInt(tokenizer.nextToken())));
//updateSalesPersonPerformance(Integer.parseInt(tokenizer.nextToken()),Integer.parseInt(tokenizer.nextToken()), Integer.parseInt(tokenizer.nextToken()));
}
}
}
private static void updateSalesPersonPerformance()
{
for(int plooper=0;plooper<ALPrice.size();plooper++)
{
for(int slooper=0;slooper<ALSlips.size();slooper++)
{
if(ALPrice.get(plooper).productID == ALSlips.get(slooper).productID)
{
switch (ALSlips.get(slooper).productID) {
case 1:
ALSlips.get(slooper).product1Sold += ALSlips.get(slooper).numberOfSold;
break;
case 2:
ALSlips.get(slooper).product2Sold += ALSlips.get(slooper).numberOfSold;
break;
case 3:
ALSlips.get(slooper).product3Sold += ALSlips.get(slooper).numberOfSold;
break;
case 4:
ALSlips.get(slooper).product4Sold += ALSlips.get(slooper).numberOfSold;
break;
default:
break;
}
ALSlips.get(slooper).performance += ALPrice.get(plooper).price* ALSlips.get(slooper).numberOfSold;
}
}
}
System.out.println("\nPERSONID"+"\t"+"PRODUCT1"+"\t"+"PRODUCT2"+"\t"+"PRODUCT3"+"\t"+"PRODUCT4"+"\t"+"PERFORMANCE");
for(int slooper=0;slooper<ALSlips.size();slooper++)
{
System.out.println(ALSlips.get(slooper).personID+"\t\t"+ALSlips.get(slooper).product1Sold+"\t\t"+ALSlips.get(slooper).product2Sold+"\t\t"+ALSlips.get(slooper).product3Sold+"\t\t"+ALSlips.get(slooper).product4Sold+"\t\t"+ALSlips.get(slooper).performance);
}
}
private static void updateSalability() {
for(int plooper=0;plooper<ALPrice.size();plooper++)
{
for(int slooper=0;slooper<ALSlips.size();slooper++)
{
if(ALPrice.get(plooper).productID == ALSlips.get(slooper).productID)
{
ALPrice.get(plooper).totalSold += ALSlips.get(slooper).numberOfSold;
ALPrice.get(plooper).totalPrice += ALPrice.get(plooper).price * ALSlips.get(slooper).numberOfSold;
}
}
}
Collections.sort(ALPrice,new PriceComparator());
System.out.println("PRODUCTID"+"\t"+"TotalSold"+"\t"+"TotalPrice");
for(int plooper=0;plooper<ALPrice.size();plooper++)
{
System.out.println(ALPrice.get(plooper).productID+"\t\t"+ALPrice.get(plooper).totalSold+"\t\t"+ALPrice.get(plooper).totalPrice);
}
}
}
so please help me to get my expected output. I attached image file with my output and expected output.
Thanks in advance.

just use this:
public static void removeDuplicates(){
ArrayList<Slips> newRay = new ArrayList<Slips>();
for(Slips s : ALSlips){
boolean hasAlready = false;
for(Slips d: newRay){
if(d.personID == s.personID){
d.product1Sold += s.product1Sold;
d.product2Sold += s.product2Sold;
d.product3Sold += s.product3Sold;
d.product4Sold += s.product4Sold;
d.numberOfSold += s.numberOfSold;
d.performance += s.performance;
hasAlready = true;
}
}
if(!hasAlready){
newRay.add(s);
}
}
ALSlips = newRay;
}
I beg of you though, please rewrite this, or comment it, or something!! unless you are doing this for your job. then leave it how it is. they won't be able to fire you.

If you want to eliminate duplicates you can use either set or hashset instead of arraylist but if your stuffing an object into a collection you need to override equals method so that you hashset treats object as duplicate based on some attribute rather than its address for example overriden equals for your problem can be
#Override
public String eqauls(Slips s)
{
if(this.personId==s.personId)
return true;
else
return false;
}
this should be placed in your slips class.
I am not completely clear about your requirement but this is how you can eliminate duplicates

Rather than use an ArrayList use a HashMap or a HashSet
The key to this Map would be the String PRODUCTID
presently you are adding to an ArrayList, but if you put to your HashMap, you can try to get it first. If it exists, you can update the value in it.
An example would be something like
HashMap myMap <String, Price val> = new HashMap <> ();
Price newPrice = ....; // like your existing code
String prodid = newPrice.getProductId ();
Price oldVal = myMap.get (prodid);
if (oldVal != null) {
// update newPrice with sum of val
}
myMap.put (prodid, newPrice); // will add or replace

Scanning File with String and int Java

So i have a .dat file that is a list. Each line starts with a letter as a command for my alarm clock program and some are followed by numbers in time format (XX:XX). I need to scan the letter arr[0] so the program knows which method to implement and then separate the integers from the colon to have arr[1] and arr[2]. I can't seem to figure out how to do both.
Example :T 25:00
while (scan.hasNextLine()){
String[] data = scan.nextLine().split(" ");
if (data[0] == "D"){
myClock.getTime();
}
else if (data[0] == "A"){
myClock.getAlarmTime();
}
else if (data[0] == "T"){
myClock.setTime(data[1], data[2]);
}
else if (data[0] == "S"){
myClock.setAlarmTime(data[1], data[2]);
}
else if (data[0] == "O"){
myClock.setAlarmOn(true);
}
else if (data[0] == "F"){
myClock.setAlarmOn(false);
}
else{
myClock.quit();
tells me I have incompatible types and that .String cannot be converted to int. So how do I convert or scan the two separately?

Instead of
String str[] = scan.nextLine().split(" ");
place
String str[] = scan.nextLine().split(" |:");
...............
if (str[0].equals("T"))
myClock.setTime(Integer.parseInt(str[1], Integer.parseInt(str[2]);
One more issue is if (data[0] == "T" ) means compaing strings
In java, we should use equals() for checking strings equality whether both are same or not.
Instead of
if (data[0] == "T" )
use
if (data[0].equals( "T" ))

If your error is at this line myClock.setAlarmTime(data[1], data[2]);, try to convert it to int.
Mby your functions for setAlarmTime and setTime need ints and you are passing Strings, compiler normally advise this.
Try this:
myClock.setAlarmTime(Integer.valueOf(data[1]), Integer.valueOf(data[2]));
Also, you want to separate this T 25:00to this: { "T","25", "00" }
And with this line String[] data = scan.nextLine().split(" "); you will get: { "T","25:00" }
You need to split again your data[1] before do setAlarmTime and setTime with:
String[] timeSplit = data[1].split(":");
myClock.setAlarmTime(Integer.valueOf(timeSplit[0]), Integer.valueOf(timeSplit[1]));

loosely based on #Boris the Spider comment
public class EnumCommand {
public static void main(String[] args) {
BufferedReader br = null;
try {
br = new BufferedReader(new FileReader("csv.dat"));
String s;
while ((s = br.readLine()) != null) {
StringTokenizer st = new StringTokenizer(s);
MyCommand.valueOf(st.nextToken()).execute(s);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (br != null) {
try {
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
private enum MyCommand {
D {
#Override
public void execute(String predicate) {
//getTime
}
},
A {
#Override
public void execute(String predicate) {
//getAlarm Time
}
},
T {
#Override
public void execute(String predicate) {
//setTime .. Parse here for time
System.out.println(predicate);
}
},
S {
#Override
public void execute(String predicate) {
//set alarm Time Parse here for time
System.out.println(predicate);
}
},
O {
#Override
public void execute(String predicate) {
//On alarm
}
},
F {
#Override
public void execute(String predicate) {
//Off alarm
}
};
public abstract void execute(String predicate);
}
}

Java parse csv class

I am making a class in java to parse in CSV's. It will read in the file line by line and parse out each field according to the regex pattern into an array, and then print that array. I put all this together in a main driver below. I looked over everything and it seems to be functional but for some reason whenever I run it, it just gets stuck in an infinite loop and will not cease. I have looked over this many times and can just not find where this would happen. Any help with this issue would be greatly appreciated!
import java.io.FileInputStream;
import java.io.IOException;
import java.util.regex.Pattern;
/**
*
*/
public class Csv {
private FileInputStream fin;
private String line;
private String[] parsedFields;
public boolean isEOL(char n) {
boolean eol;
if (n == '\n' || n == '\r') {
eol = true;
}
else
eol=false;
return eol;
}
public String getLine()
{
try
{
char c;
c= (char) fin.read();
while(!isEOL(c))
{
line+=c;
}
}
catch (IOException e) {
System.out.println("Input Error.");
}
return line;
}
public void parseFields(String s)
{
Pattern CSVLine=Pattern.compile("\"([^\"]*)\"|(?<=,|^)([^,]*)(?:,|$)");
parsedFields=CSVLine.split(s);
}
public void execute()
{
String field=getLine();
parseFields(field);
}
public void setFin(FileInputStream usrFin)
{
fin=usrFin;
}
public void outputFields()
{
for(int i=0; i<parsedFields.length;i++)
{
System.out.println(parsedFields[i]);
}
}
public static void main(String args[])
{
try {
FileInputStream fis;
fis = new FileInputStream("test.txt");
Csv test= new Csv();
test.setFin(fis);
test.execute();
test.outputFields();
}
catch (IOException e) {
e.printStackTrace();
}
}
}

c= (char) fin.read();
while(!isEOL(c))
{
line+=c;
}
In this part, you loop, adding c, but you never read again. c never changes during the loop, and probably is stuck there. You need to have the c = fin.read(); inside the loop too.

public static List<String> readLine(String filePath){
List<String> listStr = new ArrayList<String>();
try {
BufferedReader br = new BufferedReader(new FileReader(filePath));
String line = null;
Pattern pat = Pattern.compile(LINE_PATTERN_REGEXP);
while((line=br.readLine())!=null){
Matcher matcher = pat.matcher(line);
if(!matcher.find()){
listStr.add(line);
}
}
br.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
return listStr;
}
}