Develop Reference

How do I read a .csv file in Java with some cells containing multiple lines?

I am trying to read a .csv file in a Java program. The file has some cells which contain multiple lines.
I am on a linux OS, so I tried removing the line breaks with the following:
awk -v RS="" '{gsub (/\n/,"")}1' cleanPaperAuthor.csv > cleanPaperAuthor1.csv
That DID result in the multi-line data in the cell being displayed all on one line. But when I attempted to read in the file in java, the reader still thought that it had encountered the end of the line in the middle of the cell data.
So I tried
awk -v RS="" '{gsub (/\r/,"")}1' cleanPaperAuthor1.csv > cleanPaperAuthor2.csv
That resulted in ALL data in the .csv file being put on one line.
So then I tried
awk -v RS="" '{gsub (/\r\n/,"")}1' cleanPaperAuthor.csv > cleanPaperAuthor3.csv.
I'm not sure yet if that worked - I am still in the process of opening the file.
I know there is a CSVReader class out there, but I would really like to figure out what I can do without having to deal with getting that set up and changing my code. Anyone out there have any ideas? I'm completely befuddled at this point.

Using a CSV parser is extremely easy; both the setup and the API. And, in addition to handling the values that span multiple lines it can take care of things like commas in quoted elements and parsing just the values inside the quotes "" etc. for you. Plus, you can use the library to serialize your text back to CSV as well.
Here's an example with OpenCSV to read a line of csv values.
String input = "value1, \"value2\", \"value3, 1234\", \"value4\n"
+ "value5\n"
+ "value6\"";
try (CSVReader reader = new CSVReader(new StringReader(input))) {
String [] tokens;
while ((tokens = reader.readNext()) != null) {
System.out.println(Arrays.toString(tokens));
}
} catch (IOException e) {
e.printStackTrace();
}
Output : ("value3, 1234" is one value.)
[value1, value2, value3, 1234, value4
value5
value6]
Just make sure to add Apache Commons Lang 3.x jar to your classpath.

String UPLOADED_FOLDER = "/home/Rahul/Developement/Rahul/personal/uploadedfile/";
try {
// ** get the file and store at to that location **
byte[] bytes = file.getBytes();
Path path = Paths.get(UPLOADED_FOLDER + file.getOriginalFilename());
Files.write(path, bytes);
redirectAttributes.addFlashAttribute("You successfully uploaded '" + file.getOriginalFilename() + "'");
} catch (IOException e) {
e.printStackTrace();
}
try {
String fileName = file.getOriginalFilename();
System.out.println("/home/Rahul/Developement/Rahul/personal/uploadedfile/" + fileName);
String filePath = new File("/home/Rahul/Developement/Rahul/personal/uploadedfile/")
.getAbsolutePath();
boolean check = true;
File file1 = new File("/home/Rahul/Developement/Rahul/personal/uploadedfile/" + fileName);
System.out.println(file1.exists());
// TO CHECK FILE IS CSV OR NOT
if (fileName.endsWith(".csv")) {
check = true;
System.out.println("extension");
if (!fileName.isEmpty()) {
// *** to read the file from the location
// **("/home/Rahul/Developement/Rahul/personal/uploadedfile/")**
BufferedReader br = new BufferedReader(new FileReader(
"/home/Rahul/Developement/Rahul/personal/uploadedfile/" + fileName));
InputStream is = new FileInputStream(
"/home/Rahul/Developement/Rahul/personal/uploadedfile/" + fileName);
}

Split strings with an "Enter"

I have code that splits a string into 3 strings, then prints them. I want each one to be separated by the equivalent of an "Enter". Here's the code:
String accval = text;
try {
PrintWriter writer = new PrintWriter(new BufferedWriter(
new FileWriter("sdcard/YS Data/Accelerometer.html",
true)));
String[] tempArr = accval.split("\\s+");
String x = tempArr[0] + "_"; //I want the enter to be where the underlines are:
String y = tempArr[1] + "_";
String z = tempArr[2] + "_";
for (String a : tempArr) {
writer.println("<h3 style=padding-left:20px;>" + x + y
+ z + "</h3><br>");
}
writer.flush();
writer.close();
} catch (IOException e) {
// put notification here later!!!
e.printStackTrace();
}
This outputs:
x=-0.125_y=0.9375_z=0.375
x=-0.125_y=0.9375_z=0.375
with the strings separated by underscores.
However, I want it to look like this:
x=-0.125
y=0.9375
z=0.375
x=-0.125
y=0.9375
z=0.375
Thanks for any help.
EDIT:
I've implemented the answer of #Julius in the following code that prints how I wanted it:
Code:
String accval = text;
try {
PrintWriter writer = new PrintWriter(new BufferedWriter(
new FileWriter("sdcard/YS Data/Accelerometer.html",
true)));
String[] tempArr = accval.split("\\s+");
String x = tempArr[0];
String y = tempArr[1];
String z = tempArr[2];
for (String a : tempArr) {
writer.println("<h3 style=padding-left:20px;>" + x + "<br>" + y + <br>
+ z + "</h3><br>");
}
writer.flush();
writer.close();
} catch (IOException e) {
// put notification here later!!!
e.printStackTrace();
}
Which prints:
x=0.25
y=125
z=1.23
x=0.125
y=725
z=0.935

if you want the line returns to be displayed in the browser, this is the way to go:
writer.println("<h3 style=padding-left:20px;>" + x + "<br/>" + y + "<br/>" + z + "<br/></h3>");

You can use
System.getProperty("line.separator")
to get a line separator.

You could explicitly write CR+LF as in the other answers here. You can also use the default line break by just using println separately for each item, e.g.:
for (String a : tempArr) {
writer.println("<h3 style=padding-left:20px;>");
writer.println(x);
writer.println(y);
writer.println(z);
writer.println("</h3><br>");
}
This is slightly more verbose but won't run into inconsistent line-break issues on systems where CRLF is not the default line ending.
Note, however, that the linebreaks in the HTML probably won't be rendered, unless your CSS specifies that they should be. You probably want to just write a "<br>" tag after each element instead of an actual line break, e.g.:
writer.println("<h3 style=padding-left:20px;>");
writer.println(x + "<br>");
writer.println(y + "<br>");
writer.println(z + "<br>");
writer.println("</h3><br>");
You wouldn't have to use println for this (you could use print or just concatenate as you were doing before), but it does make the generated source a bit more readable.
Off topic, your use of linebreaks in an "<h3>" header tag isn't really semantically appropriate. A "<div>" with appropriate styling would be a more accurate representation, unless these are actually serving as section headers.

"\r\n" is the correct string for a line break on Windows systems; "\n" is correct on Linux and other Unix-based systems. This will work on either type of system:
String newline = String.format("%n");
which will set newline to "\r\n" or "\n" as appropriate (or perhaps some other sequence on some OS's from a different planet :)
EDIT:%n by definition inserts System.getProperty("line.separator") as noted in another answer. So these are equivalent.

How to write new line character to a file in Java

I have a string that contains new lines. I send this string to a function to write the String to a text file as:
public static void writeResult(String writeFileName, String text)
{
try
{
FileWriter fileWriter = new FileWriter(writeFileName);
BufferedWriter bufferedWriter = new BufferedWriter(fileWriter);
bufferedWriter.write(text);
// Always close files.
bufferedWriter.close();
}
catch(IOException ex) {
System.out.println("Error writing to file '"+ writeFileName + "'");}
} //end writeResult function
But when I open the file, I find it without any new lines.
When I display the text in the console screen, it is displayed with new lines. How can I write the new line character in the text file.
EDIT:
Assume this is the argument text that I sent to the function above:
I returned from the city about three o'clock on that
may afternoon pretty well disgusted with life.
I had been three months in the old country, and was
How to write this string as it is (with new lines) in the text file. My function write the string in one line. Can you provide me with a way to write the text to the file including new lines ?
EDIT 2:
The text is originally in a .txt file. I read the text using:
while((line = bufferedReader.readLine()) != null)
{
sb.append(line); //append the lines to the string
sb.append('\n'); //append new line
} //end while
where sb is a StringBuffer

In EDIT 2:
while((line = bufferedReader.readLine()) != null)
{
sb.append(line); //append the lines to the string
sb.append('\n'); //append new line
} //end while
you are reading the text file, and appending a newline to it. Don't append newline, which will not show a newline in some simple-minded Windows editors like Notepad. Instead append the OS-specific line separator string using:
sb.append(System.lineSeparator()); (for Java 1.7 and 1.8)
or
sb.append(System.getProperty("line.separator")); (Java 1.6 and below)
Alternatively, later you can use String.replaceAll() to replace "\n" in the string built in the StringBuffer with the OS-specific newline character:
String updatedText = text.replaceAll("\n", System.lineSeparator())
but it would be more efficient to append it while you are building the string, than append '\n' and replace it later.
Finally, as a developer, if you are using notepad for viewing or editing files, you should drop it, as there are far more capable tools like Notepad++, or your favorite Java IDE.

SIMPLE SOLUTION
File file = new File("F:/ABC.TXT");
FileWriter fileWriter = new FileWriter(file,true);
filewriter.write("\r\n");

The BufferedWriter class offers a newLine() method. Using this will ensure platform independence.

bufferedWriter.write(text + "\n"); This method can work, but the new line character can be different between platforms, so alternatively, you can use this method:
bufferedWriter.write(text);
bufferedWriter.newline();

Split the string in to string array and write using above method (I assume your text contains \n to get new line)
String[] test = test.split("\n");
and the inside a loop
bufferedWriter.write(test[i]);
bufferedWriter.newline();

This approach always works for me:
String newLine = System.getProperty("line.separator");
String textInNewLine = "this is my first line " + newLine + "this is my second
line ";

Put this code wherever you want to insert a new line:
bufferedWriter.newLine();

PrintWriter out = null; // for writting in file
String newLine = System.getProperty("line.separator"); // taking new line
out.print("1st Line"+newLine); // print with new line
out.print("2n Line"+newLine); // print with new line
out.close();

Here is a snippet that gets the default newline character for the current platform.
Use
System.getProperty("os.name") and
System.getProperty("os.version").
Example:
public static String getSystemNewline(){
String eol = null;
String os = System.getProperty("os.name").toLowerCase();
if(os.contains("mac"){
int v = Integer.parseInt(System.getProperty("os.version"));
eol = (v <= 9 ? "\r" : "\n");
}
if(os.contains("nix"))
eol = "\n";
if(os.contains("win"))
eol = "\r\n";
return eol;
}
Where eol is the newline

StringEscapeUtils.unescapeHtml doesn't work on strings read from files

I'm trying to read in a file that contains unicode characters, convert those characters to their corresponding symbols and then print the resulting text to a new file. I'm trying to use StringEscapeUtils.unescapeHtml to do this but the lines are just being printed as is, with the unicode points still intact. I did a practice run by copying a single line from the file, making a string from that and then calling StringEscapeUtils.unescapeHtml on that, which works perfectly. My code is below:
class FileWrite
{
public static void main(String args[])
{
try{
String testString = " \"text\":\"Dude With Knit Hat At Party Calls Beer \u2018Libations\u2019 http://t.co/rop8NSnRFu\" ";
FileReader instream = new FileReader("Home Timeline.txt");
BufferedReader b = new BufferedReader(instream);
FileWriter fstream = new FileWriter("out.txt");
BufferedWriter out = new BufferedWriter(fstream);
out.write(StringEscapeUtils.unescapeHtml3(testString) + "\n");//This gives the desired output,
//with unicode points converted
String line = b.readLine().toString();
while(line != null){
out.write(StringEscapeUtils.unescapeHtml3(line) + "\n");
line = b.readLine();
}
//Close the output streams
b.close();
out.close();
}
catch (Exception e){//Catch exception if any
System.err.println("Error: " + e.getMessage());
}
}
}

//This gives the desired output,
//with unicode points converted
out.write(StringEscapeUtils.unescapeHtml3(testString) + "\n");
You are mistaken. Java unescapes String literals of this form at compile time when it builds them into the class file:
"\u2018Libations\u2019"
There are no HTML 3 escapes in this code. The method you have chosen is designed to unescape escape sequences of the form ‘.
You probably want the unescapeJava method.

You're strings are being both read and written using your platforms default encoding. You want to explicitly specify the character set to use as 'UTF-8':
Input stream:
BufferedReader b = new BufferedReader(new InputStreamReader(
new FileInputStream("Home Timeline.txt"),
Charset.forName("UTF-8")));
Output stream:
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream("out.txt"),
Charset.forName("UTF-8")));

Error reading UTF-8 file in Java

I am trying to read in some sentences from a file that contains unicode characters. It does print out a string but for some reason it messes up the unicode characters
This is the code I have:
public static String readSentence(String resourceName) {
String sentence = null;
try {
InputStream refStream = ClassLoader
.getSystemResourceAsStream(resourceName);
BufferedReader br = new BufferedReader(new InputStreamReader(
refStream, Charset.forName("UTF-8")));
sentence = br.readLine();
} catch (IOException e) {
throw new RuntimeException("Cannot read sentence: " + resourceName);
}
return sentence.trim();
}

The problem is probably in the way that the string is being output.
I suggest that you confirm that you are correctly reading the Unicode characters by doing something like this:
for (char c : sentence.toCharArray()) {
System.err.println("char '" + ch + "' is unicode codepoint " + ((int) ch)));
}
and see if the Unicode codepoints are correct for the characters that are being messed up. If they are correct, then the problem is output side: if not, then input side.

First, you could create the InputStreamReader as
new InputStreamReader(refStream, "UTF-8")
Also, you should verify if the resource really contains UTF-8 content.

One of the most annoying reason could be... your IDE settings.
If your IDE default console encoding is something like latin1 then you'll be struggling very long with different variations of java code but nothing help untill you correctly set some IDE options.