Incomplete OpenCSV import - java

I tried importing the records from a CSV file using OpenCSV. I observed that OpenCSV was actually missing out some of the entries in the last row of my file. The code is as below. I know that I can do a writeAll operation. But i need individual rows for some other operations. The number of records are around 56000 in my file.
CSVReader reader = new CSVReader(new BufferedReader(new InputStreamReader(new FileInputStream(inputFile))));
CSVWriter writer = new CSVWriter(new BufferedWriter(new OutputStreamWriter(new FileOutputStream(new File(path2+File.separator+fileName)))), ',');
List<String[]> fileContents = new ArrayList<String[]>();
fileContents = reader.readAll();
for(int i=0; i<fileContents.size(); i++){
String[] row = fileContents.get(i);
writer.writeNext(row);

I figured out the error. I wasnt closing the writer.
writer.close();
And the file was completely written. Don't know why it doesnt complete writing the file otherwise.

Had a similar problem as described by Abhiroop when using StatefulBeanToCsv<T> beanWriter. Thanks for documenting the error with writer.close!
Since a Writer implements Closeable which extends AutoCloseable, it should be wrapped in a try-with-resources as shown below.
For me, the advantage of StatefulBeanToCsv is that it can be used with a MappingStrategy<T>.
public void exportToCsv(List<T> beans) {
try (PrintWriter printWriter = new PrintWriter(FILE_PATH_NAME)) {
final StatefulBeanToCsv<T> beanWriter = new StatefulBeanToCsvBuilder<T>(printWriter).build();
beanWriter.write(beans);
} catch (Exception e) {
throw new IllegalStateException("export failed", e);
}
}

please check if prior to the missing values, one of the values has a comma inside (i.e. "this is a csv value, with a comma inside it").

Related

Create a new CSV file with java to add a new column

I am looking at the performance of a particular stock over a period of 30 days. I downloaded the data from Yahoo finance which appears as a CSV file. If I would like to create a new column in my dataset to show the daily percentage change between open and close using java refer to column H as where the output should appear, how should I do so?
Thanks in advance!
You can just edit your file line-by-line and add a separator character using the concat() function.
Open the file with a FileReader or BufferedReader and then start parsing.
Official Javadoc: https://docs.oracle.com/javase/7/docs/api/java/lang/String.html#concat(java.lang.String)
Please see also this.
With OpenCSV: http://opencsv.sourceforge.net/
public class OpenCSVTest {
public static void main(String[] args) {
StringWriter target = new StringWriter();
try(CSVReader reader = new CSVReader(new StringReader("my,test"));
CSVWriter writer = new CSVWriter(target);) {
for(String[] line : reader) {
String[] added = Arrays.copyOf(line, line.length + 1);
added[added.length-1] = "addition";
writer.writeNext(added);
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(target.toString());
}
}
You can replace the StringReader and StringWriter with your input and output. The reader will then iterate through each line in your csv for you. You can make a copy of the original line and add your new column and write it out into the target.
Breakdown of the code:
try(CSVReader reader = new CSVReader(new StringReader("my,test"));
CSVWriter writer = new CSVWriter(target);) { ... }
This is a try-with-resources block. It makes sure that the reader and writer are closed after they are done.
for(String[] line : reader) {
String[] added = Arrays.copyOf(line, line.length + 1);
added[added.length-1] = "addition";
writer.writeNext(added);
}
This is a standard for loop. The CSVReader implements the Iterable interface, which enables us to use it in the for loop directly.
Arrays.copyOf(line, line.length + 1); is a function that creates a copy of the array passed to it, but with a new size. Because you want to add a column, we make a copy of the original array my,test and add 1 more space at the end of it, where we can then assign the new value to it by doing: added[added.length-1] = "addition";
Finally, we just pass that to the writer which will then correctly write the values into the target, in this case a StringWriter, in your case likely a file.

Write csv file from byte[]

I am getting a file in byte[] which have comma separated values in quotes, I want to save it as CSV by using OpenCSV. I am using this to save it in CSV format.
Following is my code to convert byte[] to array and then store it in file
byte[] bytes = myByteStream.getFile();
String decoded = new String(bytes, "UTF-8");
//String lines[] = decoded.split("\\r?\\n");
FileOutputStream fos = new FileOutputStream("/home/myPC/Desktop/test.csv");
OutputStreamWriter osw = new OutputStreamWriter(fos, "UTF-8");
CSVWriter writer = new CSVWriter(osw);
String[] row = {decoded};
writer.writeNext(row);
writer.close();
osw.close();
But this above code puts extra quotes around and also merge all lines in one line.
Any help on how to do this properly ?
You can prevent adding quotes to the cell values within the constructor of CSVWriter, for example:
CSVWriter writer = new CSVWriter(osw, CSVWriter.DEFAULT_SEPARATOR, CSVWriter.NO_QUOTE_CHARACTER);
Regarding the whole byte array persisted as a single row. Are you sure there are newlines within the original file.
If so you might get away by doing the following:
BufferedReader reader = new BufferedReader(new InputStreamReader(stream, "ASCII"));
String line;
while ((line = reader.readLine()) != null) {
// Handle the line, ideally in a separate method
}
Got this from splitting a byte array on a particular byte
I think Apache Commons CSV is a better CSV library. From the pasted code it is not clear if you want to somehow change the contents of the file or just duplicate it: in the latter case you don't even need to parse the file as CSV records - just copy it byte-for-byte. In the former case, ie if you need to modify the content of the file, you need something like (Commons CSV used here)
CSVParser parser = CSVFormat.newFormat(',').parse(
new InputStreamReader(new ByteArrayInputStream(bytes), "UTF8"));
CSVPrinter printer = CSVFormat.newFormat(',').print(out);
for (CSVRecord record : parser) {
try {
printer.printRecord(record);
} catch (Exception e) {
throw new RuntimeException("Error at line "
+ parser.getCurrentLineNumber(), e);
}
}
parser.close();
printer.close();
Look at the Javadoc for CSVFormat

Export array values to csv file java

I just need help exporting array elements to a csv file. I don't know what's wrong with my code. Any help is gladly appreciated. Thanks.
for (int index = 0; index < cols.length; index++)
{
FileWriter fw = new FileWriter(newFileName);
if (index == cols.length - 1)
{
fw.append(cols[index]);
}
else
{
fw.append(cols[index]);
fw.append(",");
}
}
When I run this. Nothing happens to my csv file. Infact, it wipes everything of. Please help.
Assuming you're storing String in your array, I post an alternative solution:
BufferedWriter br = new BufferedWriter(new FileWriter("myfile.csv"));
StringBuilder sb = new StringBuilder();
// Append strings from array
for (String element : array) {
sb.append(element);
sb.append(",");
}
br.write(sb.toString());
br.close();
Best regards.
you need to flush the written data, close your FileWriter.
finally {
fw.close(); // close will automatically flush the data
}
Also, use BufferedWriter or PrintWriter instead as they are highly efficient and evolved than FileWriter.
Btw, declare the FileWriter outta your for loop. Currently it will overwrite the column for each iteration.
If you use the more effective BufferedWriter combined with try-with-resources the writer is automatically closed. (In Java 7)
try {BufferedWriter br = new BufferedWriter(new FileWriter(newFileName)) {
// your code
}
You have to use
FileWriter fw = new FileWriter(newFileName,true);
to append content, or the file will be overwritten.
The name of the append method can be misleading, in fact it append to an internal buffer, not to the file content.
The best way would be to use a CSVWriter. If you have a maven project add this as a dependency:
<dependency>
<groupId>com.opencsv</groupId>
<artifactId>opencsv</artifactId>
<version>4.4</version>
</dependency>
And then you can use it like this:
try {
//the list that contains the String arrays
List<String[]> whatever = new ArrayList<>();
CSVWriter writer = new CSVWriter(new FileWriter(csvFilename));
String[] header = "Name,Age,Height,etc..".split(",");
writer.writeNext(header);
writer.writeAll(whatever, true); //And the second argument is boolean which represents whether you want to write header columns (table column names) to file or not.
writer.close();
System.out.println("CSV file created succesfully.");
} catch (Exception e) {
System.out.println("exception :" + e.getMessage());
}

Read and write to file, but overwrites the file

I'm trying to figure out why this doesn't work. All the files which are shown actually exist. The 'logging.toString()' is a .txt file and it reads all the text in the logging and writes it back with the String which I want to be added. Although when I do this, it overwrites it. But I dont want that. Help?
try{
FileWriter fstream = new FileWriter(logging.toString());
BufferedWriter out = new BufferedWriter(fstream);
FileInputStream fstreams = new FileInputStream(logging);
BufferedReader br = new BufferedReader(new InputStreamReader(fstreams));
String strLine;
while ((strLine = br.readLine()) != null){
htmlTextArea = htmlTextArea + strLine + "\n";
}
out.write(htmlTextArea + logto);
out.close();
} catch (Exception ex){}
Why wouldn't it? You don't pass an append flag:
FileWriter(String filename, boolean append)
The API docs are your friend; they're often helpful for understanding behavior.
Change this line
FileWriter fstream = new FileWriter(logging.toString());
to
FileWriter fstream = new FileWriter(logging.toString(), true);
That way, you tell Java you wish to APPEND the file. There's more in the Javadocs for FileWriter.
Because that's the way FileWriter is implemented.
If you want to append, you should use a different constructor: new FileWriter( logging.toString(), true );
If you want to add something to file but not overwrite it use
FileWriter fstream = new FileWriter(logging.toString(),true);

How to write an ArrayList of Strings into a text file?

I want to write an ArrayList<String> into a text file.
The ArrayList is created with the code:
ArrayList arr = new ArrayList();
StringTokenizer st = new StringTokenizer(
line, ":Mode set - Out of Service In Service");
while(st.hasMoreTokens()){
arr.add(st.nextToken());
}
import java.io.FileWriter;
...
FileWriter writer = new FileWriter("output.txt");
for(String str: arr) {
writer.write(str + System.lineSeparator());
}
writer.close();
You can do that with a single line of code nowadays.
Create the arrayList and the Path object representing the file where you want to write into:
Path out = Paths.get("output.txt");
List<String> arrayList = new ArrayList<> ( Arrays.asList ( "a" , "b" , "c" ) );
Create the actual file, and fill it with the text in the ArrayList:
Files.write(out,arrayList,Charset.defaultCharset());
I would suggest using FileUtils from Apache Commons IO library.It will create the parent folders of the output file,if they don't exist.while Files.write(out,arrayList,Charset.defaultCharset()); will not do this,throwing exception if the parent directories don't exist.
FileUtils.writeLines(new File("output.txt"), encoding, list);
If you need to create each ArrayList item in a single line then you can use this code
private void createFile(String file, ArrayList<String> arrData)
throws IOException {
FileWriter writer = new FileWriter(file + ".txt");
int size = arrData.size();
for (int i=0;i<size;i++) {
String str = arrData.get(i).toString();
writer.write(str);
if(i < size-1)**//This prevent creating a blank like at the end of the file**
writer.write("\n");
}
writer.close();
}
If you want to serialize the ArrayList object to a file so you can read it back in again later use ObjectOuputStream/ObjectInputStream writeObject()/readObject() since ArrayList implements Serializable. It's not clear to me from your question if you want to do this or just write each individual item. If so then Andrey's answer will do that.
You might use ArrayList overloaded method toString()
String tmp=arr.toString();
PrintWriter pw=new PrintWriter(new FileOutputStream(file));
pw.println(tmp.substring(1,tmp.length()-1));
I think you can also use BufferedWriter :
BufferedWriter writer = new BufferedWriter(new FileWriter(new File("note.txt")));
String stuffToWrite = info;
writer.write(stuffToWrite);
writer.close();
and before that remember too add
import java.io.BufferedWriter;
Write a array list to text file using JAVA
public void writeFile(List<String> listToWrite,String filePath) {
try {
FileWriter myWriter = new FileWriter(filePath);
for (String string : listToWrite) {
myWriter.write(string);
myWriter.write("\r\n");
}
myWriter.close();
System.out.println("Successfully wrote to the file.");
} catch (IOException e) {
System.out.println("An error occurred.");
e.printStackTrace();
}
}

Categories

Resources