I'm writing a simple program that take a csv file and produce a csv with a new column. My problem is: the program quote the old columns and all the columns.
Below my code
public class CSVmodifier {
public static void modify(String Path,String Escape) throws IOException {
int i=0;
String filename = new File(Path).getName().toString();
//setName(string) modify the original filename
String fileoutname = setName(filename);
File file= new File(fileoutname);
try {
FileWriter out = new FileWriter(file);
Reader reader =
Files.newBufferedReader(Paths.get(Path));
CSVReader csvReader = new CSVReader(reader);
CSVWriter csvWriter = new CSVWriter(out);
String[] nextRecord;
while ((nextRecord = csvReader.readNext()) != null) {
int dimension = nextRecord.length;
String[] newline = new String[dimension+1];
int y = 0;
//formatNumber create a string number with 9
//zero in the front-> 1 > "000000001"
newline[0]=formatNumber(i+1);
while(y<dimension) {
newline[y+1] = nextRecord[y];
y++;
}
i+=1;
csvWriter.writeNext(newline);
}
csvWriter.close();
} finally {
}
}
public static String formatNumber(int i) {
String formatted = String.format("%09d", i);
return formatted;
}
}
my sample is :
"John","Doe","120 jefferson st.","Riverside", "NJ", "08075"
the wrong output is :
"000000001","""John""",""Doe"",""120 jefferson st."",""Riverside"", ""NJ"", ""08075"""
I cannot upload the file, but i'll show you a sample file (input line) that give the same problem:
'1231';'02512710795';'+142142';'2019/12/12';'statale';'blablabla';'iradsasad';'-123131';'+414214141';
'003';'08206810965';'+000000001492106';'2019/06/23';'Scuola statale elemetare';'Ola!'
There Output line:
'000000001';"'1231';'02512710795';'+142142';'2019/12/12';'statale';'blablabla';'iradsasad';'-123131';'+414214141'; "
'000000002';"'003';'08206810965';'+000000001492106';'2019/06/23';'Scuola statale'; "
I assume your CSVWriter class is com.opencsv.CSVWriter.
You are using csvWriter.writeNext(String[]), which is a shortcut for writeNext(nextLine, true); (JavaDoc).
The second parameter, which is always true in this case, is applyQuotesToAll:
True if all values are to be quoted. False applies quotes only to values which contain the separator, escape, quote, or new line characters.
So all you need to do, is changing this line:
csvWriter.writeNext(newline)
to this:
csvWriter.writeNext(newline, false);
This is working for me, check and let me know. Just change the logic in while block it replaces multiple double quotes to a single quote.
public class CSVmodifier {
public static void modify(String Path, String Escape) throws IOException {
int i = 0;
String filename = new File(Path).getName().toString();
//setName(string) modify the original filename
String fileoutname = setName(filename);
File file = new File(fileoutname);
try {
FileWriter out = new FileWriter(file);
Reader reader
= Files.newBufferedReader(Paths.get(Path));
CSVReader csvReader = new CSVReader(reader);
CSVWriter csvWriter = new CSVWriter(out);
String[] nextRecord;
while ((nextRecord = csvReader.readNext()) != null) {
int dimension = nextRecord.length;
String[] newline = new String[dimension + 1];
int y = 0;
//formatNumber create a string number with 9
//zero in the front-> 1 > "000000001"
newline[0] = formatNumber(i + 1);
while (y < dimension) {
newline[y + 1] = nextRecord[y].replace("\"\"", "\"");
if (newline[y + 1].startsWith("\"") && newline[y + 1].endsWith("\"")) {
newline[y + 1] = newline[y + 1].substring(1, newline[y + 1].length() - 1);
}
y++;
}
i += 1;
csvWriter.writeNext(newline);
}
csvWriter.close();
} finally {
}
}
public static String formatNumber(int i) {
String formatted = String.format("%09d", i);
return formatted;
}
}
Related
I have a text file from which i am trying to search for a String which has multiple lines. A single string i am able to search but i need multi line string to be searched.
I have tried to search for single line which is working fine.
public static void main(String[] args) throws IOException
{
File f1=new File("D:\\Test\\test.txt");
String[] words=null;
FileReader fr = new FileReader(f1);
BufferedReader br = new BufferedReader(fr);
String s;
String input="line one";
// here i want to search for multilines as single string like
// String input ="line one"+
// "line two";
int count=0;
while((s=br.readLine())!=null)
{
words=s.split("\n");
for (String word : words)
{
if (word.equals(input))
{
count++;
}
}
}
if(count!=0)
{
System.out.println("The given String "+input+ " is present for "+count+ " times ");
}
else
{
System.out.println("The given word is not present in the file");
}
fr.close();
}
And below are the file contents.
line one
line two
line three
line four
Use the StringBuilder for that, read every line from file and append them to StringBuilder with lineSeparator
StringBuilder lineInFile = new StringBuilder();
while((s=br.readLine()) != null){
lineInFile.append(s).append(System.lineSeparator());
}
Now check the searchString in lineInFile by using contains
StringBuilder searchString = new StringBuilder();
builder1.append("line one");
builder1.append(System.lineSeparator());
builder1.append("line two");
System.out.println(lineInFile.toString().contains(searchString));
More complicated solution from default C (code is based on code from book «The C programming language» )
final String searchFor = "Ich reiß der Puppe den Kopf ab\n" +
"Ja, ich reiß' ich der Puppe den Kopf ab";
int found = 0;
try {
String fileContent = new String(Files.readAllBytes(
new File("puppe-text").toPath()
));
int i, j, k;
for (i = 0; i < fileContent.length(); i++) {
for (k = i, j = 0; (fileContent.charAt(k++) == searchFor.charAt(j++)) && (j < searchFor.length());) {
// nothig
}
if (j == searchFor.length()) {
++found;
}
}
} catch (IOException ignore) {}
System.out.println(found);
Why don't you just normalize all the lines in the file to one string variable and then just count the number of occurrences of the input in the file. I have used Regex to count the occurrences but can be done in any custom way you find suitable.
public static void main(String[] args) throws IOException
{
File f1=new File("test.txt");
String[] words=null;
FileReader fr = new FileReader(f1);
BufferedReader br = new BufferedReader(fr);
String s;
String input="line one line two";
// here i want to search for multilines as single string like
// String input ="line one"+
// "line two";
int count=0;
String fileStr = "";
while((s=br.readLine())!=null)
{
// Normalizing the whole file to be stored in one single variable
fileStr += s + " ";
}
// Now count the occurences
Pattern p = Pattern.compile(input);
Matcher m = p.matcher(fileStr);
while (m.find()) {
count++;
}
System.out.println(count);
fr.close();
}
Use StringBuilder class for efficient string concatenation.
Try with Scanner.findWithinHorizon()
String pathToFile = "/home/user/lines.txt";
String s1 = "line two";
String s2 = "line three";
String pattern = String.join(System.lineSeparator(), s1, s2);
int count = 0;
try (Scanner scanner = new Scanner(new FileInputStream(pathToFile))) {
while (scanner.hasNext()) {
String withinHorizon = scanner.findWithinHorizon(pattern, pattern.length());
if (withinHorizon != null) {
count++;
} else {
scanner.nextLine();
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
System.out.println(count);
Try This,
public static void main(String[] args) throws IOException {
File f1 = new File("./src/test/test.txt");
FileReader fr = new FileReader(f1);
BufferedReader br = new BufferedReader(fr);
String input = "line one";
int count = 0;
String line;
while ((line = br.readLine()) != null) {
if (line.contains(input)) {
count++;
}
}
if (count != 0) {
System.out.println("The given String " + input + " is present for " + count + " times ");
} else {
System.out.println("The given word is not present in the file");
}
fr.close();
}
My problem is instead of the fixed value of iValueNext, I want the next value on the excel sheet to run, which is 125,152,...
import java.util.*;
import java.io.*;
public class ConvertingData
{
public static void main (String [] args)
{
int i=1;
int j;
int iValue;
int iValueNext;
try
{
Scanner ifsInput = new Scanner(new File("input.csv"));
PrintStream ifsOutput = new PrintStream(new File("output.csv"));
while(ifsInput.hasNextLine())
{
String tokens[] = ifsInput.nextLine().split(",");
String Repeat = tokens[tokens.length - 1];
String Value = tokens[tokens.length - 3];
iValue = Integer.parseInt( Value );
for (i=iValue;i<=iValueNext;i++)
{
System.out.println(i+","+Repeat);
ifsOutput.println(i+","+Repeat);
}
}
ifsInput.close();
ifsOutput.close();
}
catch (FileNotFoundException sMsg)
{
System.out.println("File not found");
}
}
}
Here is part of the csv file:
89,31,31
125,1,32
152,-12,20
155,1,21
181,6,27
287,1,28
290,1,29
308,-8,21
If you need to "peek" at the next line while processing the current line, first read all the lines in:
List<String> lines = new ArrayList<String>();
while(ifsInput.hasNextLine()) {
lines.add(ifsInput.nextLine());
}
ifsInput.close();
then process the lines one by one with access to the next line:
for (int i = 0; i < lines.size(); i++) {
String line = lines.get(i);
String nextLine = i < lines.size() - 1 ? null : lines.get(i + 1);
String tokens[] = line.split(",");
String nextTokens[] = nextLine.split(",");
// whatever logic you need
ifsOutput.close();
}
I'm trying to understand a piece of code. I didn't write it, I'm just trying to make it work.
It's meant to transform a .csv file.
The code is this:
import java.io.*;
import au.com.bytecode.opencsv.*;
public class StockParser
{
public static void main(String[] args) throws FileNotFoundException, IOException
{
CSVReader reader = new CSVReader(new FileReader("/home/cloudera/Desktop/training.csv"));
String [] nextLine;
String [] previousLine;
String [] headernew = new String [reader.readNext().length +1];
CSVWriter writer = new CSVWriter(new FileWriter("/home/cloudera/Desktop/final.csv"), ',');
nextLine = reader.readNext();
for (int i = 0; i < nextLine.length;i++)
{
headernew[i] = nextLine[i];
}
headernew[headernew.length-1] = "action";
writer.writeNext(headernew);
previousLine = reader.readNext();
while ((nextLine = reader.readNext()) != null)
{
// nextLine[] is an array of values from the line
System.out.println(nextLine[0] + nextLine[1] + " etc...");
headernew = new String [nextLine.length + 1];
for (int i = 0; i < headernew.length-1;i++)
{
headernew[i] = nextLine[i];
}
if (Double.parseDouble(previousLine[4]) < Double.parseDouble(nextLine[4]))
{
headernew[headernew.length-1] = "SELL";
}
else
{
headernew[headernew.length-1] = "BUY";
}
writer.writeNext(headernew);
previousLine = nextLine;
}
reader.close();
writer.close();
}
}
It works generally, but there's a problem: the input file's first line is
Open,High,Low,Close,Volume,Adj Close
followed by lines like
59.30,60.05,58.88,59.41,3373800,59.41
The output file should have the same first line, + action, and then similar lines, + BUY or SELL, but when I run this code, it somehow loses the
Open,High,Low,Close,Volume,Adj Close,action
line,
and the next lines look like
"59.64","60.26","58.88","59.83","3069100","59.83","BUY"
Where did the quotes come from, and what should I do to get rid of them?
This is an answer to:
Where did the quotes come from, and what should I do to get rid of
them?
The constructor described in the documentation of CSVWriter allows specifying a quote-character. Try the following:
CSVWriter writer = new CSVWriter(new FileWriter("/home/cloudera/Desktop/final.csv"), ',', CSVWriter.NO_QUOTE_CHARACTER);
The last parameter should suppress all quoting characters.
To answer you simply, use
CSVWriter writer = new CSVWriter(new FileWriter(
"/home/cloudera/Desktop/final.csv"), ',',CSVWriter.NO_QUOTE_CHARACTER);
But there are few more issues with your code which I tried to correct making my assumptions.
public static void main(String[] args) throws FileNotFoundException,
IOException {
CSVReader reader = new CSVReader(new FileReader(
"training.csv"));
String[] nextLine;
String[] previousLine;
nextLine = reader.readNext();
String[] headernew = new String[nextLine.length + 1];
CSVWriter writer = new CSVWriter(new FileWriter(
"final.csv"), ',', CSVWriter.NO_QUOTE_CHARACTER);
for (int i = 0; i < nextLine.length; i++) {
headernew[i] = nextLine[i];
}
headernew[headernew.length - 1] = "action";
writer.writeNext(headernew);
previousLine = reader.readNext();
while ((nextLine = reader.readNext()) != null) {
// nextLine[] is an array of values from the line
System.out.println(nextLine[0] + nextLine[1] + " etc...");
headernew = new String[nextLine.length + 1];
for (int i = 0; i < headernew.length - 1; i++) {
headernew[i] = nextLine[i];
}
if (Double.parseDouble(previousLine[4]) < Double
.parseDouble(nextLine[4])) {
headernew[headernew.length - 1] = "SELL";
} else {
headernew[headernew.length - 1] = "BUY";
}
writer.writeNext(headernew);
previousLine = nextLine;
}
reader.close();
writer.close();
}
Ok, EddyG suggested the right direction for finding the problem. The opencsv jar has Class CSVWriter, and it has a different constructor variants, among which is public CSVWriter(Writer writer,char separator,char quotechar). Fiddling with quotechar improved things.
Source: opencsv documentation
So i have this code and i'm having trouble retrieving data from my csv file and putting them into an array.
This is what i have on my CSV file
D001,55,Lab,Butch
D002,22,Husky,Ben
D003,12,Maltese,John
D004,34,GermanSheperd,James
D005,76,Rot,Smith
public static void CSVInputFile() throws IOException {
FileReader inFileReader;
BufferedReader in;
String inStr;
File myFile;
String dogID;
int size;
String breed;
String name;
myFile = new File("DogFile.csv");
inFileReader = new FileReader(myFile);
in = new BufferedReader(inFileReader);
inStr = in.readLine();
Dog[] NewReadDog = new Dog[5];
int i = 0;
while (inStr != null) {
StringTokenizer dogTok = new StringTokenizer(inStr, ",");
while (dogTok.hasMoreTokens()) {
dogID = dogTok.nextToken();
size = new Integer(dogTok.nextToken());
breed = dogTok.nextToken();
name = dogTok.nextToken();
NewReadDog[i] = new Dog(dogID, size, breed, name);
i++;
System.out.println("dog " + i + " is stored");
}
}
System.out.println("\nOutput Dogs from CSV FILE: ");
for (int count = 0; count < NewReadDog.length; count++) {
System.out.println(NewReadDog[count]);
}
in.close();
}
I'm just starting to learn coding so please bear with me.
thanks
You have to read the next line when finished tokenizing the current one:
while (inStr != null) {
StringTokenizer dogTok = new StringTokenizer(inStr, ",");
while (dogTok.hasMoreTokens()) {
[...]
}
System.out.println("dog " + i + " is stored");
inStr = in.readLine();
i++; //replaced here
}
I have this code to sort a text file using arrays in java, but it always discard the first line of the text while sorting.
Here is my code:
import java.io.*;
public class Main {
public static int count(String filename) throws IOException {
InputStream is = new BufferedInputStream(new FileInputStream(filename));
try {
byte[] c = new byte[1024];
int count = 0;
int readChars = 0;
while ((readChars = is.read(c)) != -1) {
for (int i = 0; i < readChars; ++i) {
if (c[i] == '\n') {
++count;
}
}
}
return count;
} finally {
is.close();
}
}
public static String[] getContents(File aFile) throws IOException {
String[] words = new String[count(aFile.getName()) + 1];
BufferedReader input = new BufferedReader(new FileReader(aFile));
String line = null; //not declared within while loop
int i = 0;
while ((line = input.readLine()) != null) {
words[i] = line;
i++;
}
java.util.Arrays.sort(words);
for (int k = 0; k < words.length; k++) {
System.out.println(words[k]);
}
return words;
}
public static void main(String[] args) throws IOException {
File testFile = new File("try.txt");
getContents(testFile);
}
}
Here is the text file try.txt:
Daisy
Jane
Amanda
Barbara
Alexandra
Ezabile
the output is:
Alexandra
Amanda
Barbara
Ezabile
Jane
Daisy
To solve this problem I have to insert an empty line in the beginning of the text file, is there a way not to do that? I don't know what goes wrong?
I compiled your code (on a Mac) and it works for me. Try opening the file in a hexeditor and see if there is some special character at the beginning of your file. That might be causing the sorting to be incorrect for the first line.
You probably have a BOM (Byte Order Marker) at the beginning at the file. By definition they will be interpreted as zero-width non-breaking-space.
So if you have
String textA = new String(new byte[] { (byte)0xef, (byte)0xbb, (byte) 0xbf, 65}, "UTF-8");
String textB = new String(new byte[] { 66}, "UTF-8");
System.err.println(textA + " < " + textB + " = " + (textA.compareTo(textB) < 0));
The character should show up in your length of the strings, so try printing the length of each line.
System.out.println(words[k] + " " + words[k].length());
And use a list or some other structure so you don't have to read the file twice.
Try something simpler, like this:
public static String[] getContents(File aFile) throws IOException {
List<String> words = new ArrayList<String>();
BufferedReader input = new BufferedReader(new FileReader(aFile));
String line;
while ((line = input.readLine()) != null)
words.add(line);
Collections.sort(words);
return words.toArray(new String[words.size()]);
}
public static void main(String[] args) throws IOException {
File testFile = new File("try.txt");
String[] contents = getContents(testFile);
for (int k = 0; k < contents.length; k++) {
System.out.println(contents[k]);
}
}
Notice that you don't have to iterate over the file to determine how many lines it has, instead I'm adding the lines to an ArrayList, and at the end, converting it to an array.
Use List and the add() method to read your file contents.
Then use Collections.sort() to sort the List.