Develop Reference

Java - write string to file line by line vs one-liner / cannot convert String to String[]

Relatively new to programming. I want to read a URL, modify the text string, then write it to a line-separated csv textfile.
The read & modify parts run. Also, outputting the string to terminal (using Eclipse) looks fine (csv, line by line), like this;
data_a,data_b,data_c,...
data_a1,data_b1,datac1...
data_a2,data_b2,datac2...
.
.
.
But I'm unable to write the same string to file - it just becomes a one-liner (see my below for-loops, attempts no. 1 & 2);
data_a,data_b,data_c,data_a1,data_b1,datac1,data_a2,data_b2,datac2...
I guess I'm looking for a way to, in the FileWriter or BufferedWriter loops, convert the string finalDataA to array string (i.e. include the string suffix "[0]") but I have not yet found such an approach that would not give errors of the type "Cannot convert String to String[]". Any suggestions?
String data = "";
String dataHelper = "";
try {
URL myURL = new URL(url);
HttpURLConnection myConnection = (HttpURLConnection) myURL.openConnection();
if (myConnection.getResponseCode() == URLStatus.HTTP_OK.getStatusCode()) {
BufferedReader in = new BufferedReader(new InputStreamReader(myConnection.getInputStream()));
while ((data = in.readLine()) != null) {
dataHelper = dataHelper + "\n" + data;
}
in.close();
String trimmedData = dataHelper.trim().replaceAll(" +", ",");
String parts[] = trimmedData.split(Pattern.quote(")"));// ,1.,");
String dataA = parts[1];
String finalDataA[] = dataA.split("</PRE>");
// parts 2&3 removed in this example
// Console output for testing purpose - This prints out many many lines of csv-data
System.out.println(finalDataA[0]);
//This returns the value 1
System.out.println(finalDataA.length);
// Attempt no. 1 to write to file - writes a oneliner
for(int i = 0; i < finalDataA.length; i++) {
try (BufferedWriter bw = new BufferedWriter(new FileWriter(pathA, true))) {
String s;
s = finalDataA[i];
bw.write(s);
bw.newLine();
bw.flush();
}
}
// Attempt no. 2 to write to file - writes a oneliner
FileWriter fw = new FileWriter(pathA);
for (int i = 0; i < finalDataA.length; i++) {
fw.write(finalDataA[i] + "\n");
}
fw.close();
}
} catch (Exception e) {
System.out.println("Exception" +e);
}

Create the BufferedWriter and the FileWriter ahead of the for loop, not every time around it.

From your code comments, finalDataA has one element, so the for-loop will be executed only once. Try splitting finalDataA[0] into rows.
Something like this:
String endOfLineToken = "..."; //your variant
String[] lines = finalDataA[0].split(endOfLineToken)
BufferdWriter bw = new BufferedWriter(new FileWriter(pathA, true));
try
{
for (String line: lines)
{
bw.write(line);
bw.write(endOfLineToken);//to put back line endings
bw.newLine();
bw.flush();
}
}
catch (Exception e) {}

Why does my method return 66

So I have this method here that should return the amount of lines in a csv file. Pretty simple right? Thing is instead of returning the amount of lines in the csv file(in this case 15) it returns 66. I honestly have know Idea why this would happen. I checked the csv file and verified that it is indeed 15 lines long with no empty lines. Also does anyone know why my Jpanes wont display without those three lines commented lines, my ide says the variables aren't in use anywhere.
public static int getLineCount(){
int line=0;
try {
Scanner inputStream = new Scanner(file);
while (inputStream.hasNext()) {
String data =inputStream.next();//this line is useless but the program doesn't display with out it
String[] values = data.split(",");//this line is useless but the program doesn't display with out it
i++;//this line is useless but the program doesn't display with out it
line++;
}
}catch(FileNotFoundException e){
e.printStackTrace();
}
return line;
}

Use BufferedReader instead of Scanner:
BufferedReader reader = new BufferedReader(new InputStreamReader(file));
while(reader.readLine() != null){
line++;
}

public static int getLineCount(){
String csvFilePath = "C:\\Users\\uzochi\\desktop\\txt.csv";
String line = "";
int numberOfLines=0;
try {
BufferedReader br = new BufferedReader(new FileReader(csvFilePath));
while (( line = br.readLine()) != null) {
numberOfLines++;
}
}catch(FileNotFoundException e){
//
}catch (IOException ex) {
//
}
return numberOfLines;
}

How can I read from the next line of a text file, and pause, allowing me to read from the line after that later?

I wrote a program that generates random numbers into two text files and random letters into a third according the two constant files. Now I need to read from each text file, line by line, and put them together. The program is that the suggestion found here doesn't really help my situation. When I try that approach it just reads all lines until it's done without allowing me the option to pause it, go to a different file, etc.
Ideally I would like to find some way to read just the next line, and then later go to the line after that. Like maybe some kind of variable to hold my place in reading or something.
public static void mergeProductCodesToFile(String prefixFile,
String inlineFile,
String suffixFile,
String productFile) throws IOException
{
try (BufferedReader br = new BufferedReader(new FileReader(prefixFile)))
{
String line;
while ((line = br.readLine()) != null)
{
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(productFile, true))))
{
out.print(line); //This will print the next digit to the right
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
}
}
}
}
EDIT: The digits being created according to the following. Basically, constants tell it how many digits to create in each line and how many lines to create. Now I need to combine these together without deleting anything from either text file.
public static void writeRandomCodesToFile(String codeFile,
char fromChar, char toChar,
int numberOfCharactersPerCode,
int numberOfCodesToGenerate) throws IOException
{
for (int i = 1; i <= PRODUCT_COUNT; i++)
{
int I = 0;
if (codeFile == "inline.txt")
{
for (I = 1; I <= CHARACTERS_PER_CODE; I++)
{
int digit = (int)(Math.random() * 10);
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(codeFile, true))))
{
out.print(digit); //This will print the next digit to the right
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
System.exit(1);
}
}
}
if ((codeFile == "prefix.txt") || (codeFile == "suffix.txt"))
{
for (I = 1; I <= CHARACTERS_PER_CODE; I++)
{
Random r = new Random();
char digit = (char)(r.nextInt(26) + 'a');
digit = Character.toUpperCase(digit);
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(codeFile, true))))
{
out.print(digit);
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
System.exit(1);
}
}
}
//This will take the text file to the next line
if (I >= CHARACTERS_PER_CODE)
{
{
Random r = new Random();
char digit = (char)(r.nextInt(26) + 'a');
try (PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(codeFile, true))))
{
out.println(""); //This will return a new line for the next loop
}
catch (FileNotFoundException e)
{
System.err.println("File error: " + e.getMessage());
System.exit(1);
}
}
}
}
System.out.println(codeFile + " was successfully created.");
}// end writeRandomCodesToFile()

Being respectfull with your code, it will be something like this:
public static void mergeProductCodesToFile(String prefixFile, String inlineFile, String suffixFile, String productFile) throws IOException {
try (BufferedReader prefixReader = new BufferedReader(new FileReader(prefixFile));
BufferedReader inlineReader = new BufferedReader(new FileReader(inlineFile));
BufferedReader suffixReader = new BufferedReader(new FileReader(suffixFile))) {
StringBuilder line = new StringBuilder();
String prefix, inline, suffix;
while ((prefix = prefixReader.readLine()) != null) {
//assuming that nothing fails and the files are equals in # of lines.
inline = inlineReader.readLine();
suffix = suffixReader.readLine();
line.append(prefix).append(inline).append(suffix).append("\r\n");
// write it
...
}
} finally {/*close writers*/}
}
Some exceptions may be thrown.
I hope you don't implement it in one single method.
You can make use of iterators too, or a very simple reader class (method).
I wouldn't use List to load the data at least I guarantee that the files will be low sized and that I can spare the memory usage.

My approach as we discussed by storing the data and interleaving it. Like Sergio said in his answer, make sure memory isn't a problem in terms of the size of the file and how much memory the data structures will use.
//the main method we're working on
public static void mergeProductCodesToFile(String prefixFile,
String inlineFile,
String suffixFile,
String productFile) throws IOException
{
try {
List<String> prefix = read(prefixFile);
List<String> inline = read(inlineFile);
List<String> suffix = read(productFile);
String fileText = interleave(prefix, inline, suffix);
//write the single string to file however you want
} catch (...) {...}//do your error handling...
}
//helper methods and some static variables
private static Scanner reader;//I just prefer scanner. Use whatever you want.
private static StringBuilder sb;
private static List<String> read(String filename) throws IOException
{
List<String> list = new ArrayList<String>;
try (reader = new Scanner(new File(filename)))
{
while(reader.hasNext())
{ list.add(reader.nextLine()); }
} catch (...) {...}//catch errors...
}
//I'm going to build the whole file in one string, but you could also have this method return one line at a time (something like an iterator) and output it to the file to avoid creating the massive string
private static String interleave(List<String> one, List<String> two, List<String> three)
{
sb = new StringBuilder();
for (int i = 0; i < one.size(); i++)//notice no checking on size equality of words or the lists. you might want this
{
sb.append(one.get(i)).append(two.get(i)).append(three.get(i)).append("\n");
}
return sb.toString()
}
Obviously there is still some to be desired in terms of memory and performance; additionally there are ways to make this slightly more extensible to other situations, but it's a good starting point. With c#, I could more easily make use of the iterator to make interleave give you one line at a time, potentially saving memory. Just a different idea!

Read file, replace string and create a new one with all content

I am trying to replace ? with - in my text document but just the ArrayList<String> is being written in the new file without all lines of the old one. How can I fix that?
File file = new File("D:\\hl_sv\\L09MF.txt");
ArrayList<String> lns = new ArrayList<String>();
Scanner scanner;
try {
scanner = new Scanner(file);
int lineNum = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
lineNum++;
if (line.contains("?")) {
line = line.replace("?", "-");
lns.add(line);
// System.out.println("I found it on line " + lineNum);
}
}
lines.clear();
lines = lns;
System.out.println("Test: " + lines);
FileWriter writer;
try {
writer = new FileWriter("D:\\hl_sv\\L09MF2.txt");
for (String str : lines) {
writer.write(str);
}
writer.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}

I don't understand why you're storing the lines in a List to begin with. I would perform the transform and print while I read. You don't need to test for the presence of the ? (replace won't alter anything if it isn't present). And, I would also use a try-with-resources. Something like
File file = new File("D:\\hl_sv\\L09MF.txt");
try (PrintWriter writer = new PrintWriter("D:\\hl_sv\\L09MF2.txt");
Scanner scanner = new Scanner(file)) {
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
writer.println(line.replace('?', '-'));
}
} catch (Exception e) {
e.printStackTrace();
}

Examine this code:
if (line.contains("?")) {
line = line.replace("?", "-");
lns.add(line);
}
You are only adding the current line (with the replacement) if it had a ? in it, ignoring other lines. Restructure it to always add the existing line.
if (line.contains("?")) {
line = line.replace("?", "-");
}
lns.add(line);
Additionally, the part
if (line.contains("?"))
scans line to look for a ?, and then the code
line.replace("?", "-");
does the same thing, but this time also replacing any ? with -. You may as well scan line just once:
lns.add(line.replace("?", "-"));
Note that creating an ArrayList just to hold the new lines wastes a fair amount of memory if the file is large. A better pattern would be to write each line, modified if necessary, right after you read in the corresponding line.

Within your while loop you have an if statement checking the line which adds the altered line to the array. You also need to add the unaltered lines to the array.

This should fix your issue:
int lineNum = 0;
while (scanner.hasNextLine()) {
String line = scanner.nextLine();
lineNum++;
if (line.contains("?")) {
line = line.replace("?", "-");
lns.add(line);
// System.out.println("I found it on line " + lineNum);
}
else{
lns.add(line);
}
Previously, you were only adding the line to your ArrayList if it contained a "?" character. You need to add the line to the ArrayList whether or not it contains "?"

I would use a different approach if I'm trying to work on the functionality you want to implement, please check this approach and tell me if this helps you :)
public void saveReplacedFile() {
//1. Given a file in your system
File file = new File("D:\\hl_sv\\L09MF.txt");
try {
//2. I will read it, not necessarily with Scanner, but use a BufferedReader instead
BufferedReader bufferedReader = new BufferedReader(new FileReader(file));
//3. Define a variable that will hold the value of each line
String line = null;
//and also the information of your file
StringBuilder contentHolder = new StringBuilder();
//why not get your line separator based on your O.S?
String lineSeparator = System.getProperty("line.separator");
//4. Check your file line by line
while ((line = bufferedReader.readLine()) != null) {
contentHolder.append(line);
contentHolder.append(lineSeparator);
}
//5. By this point, your contentHolder will contain all the data of your text
//But it is still a StringBuilder type object, why not convert it to a String?
String contentAsString = contentHolder.toString();
//6. Now we can replace your "?" with "-"
String replacedString = contentAsString.replace("?", "-");
//7. Now, let's save it in a new file using BufferedWriter :)
File fileToBeSaved = new File("D:\\hl_sv\\L09MF2.txt");
BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter(fileToBeSaved));
bufferedWriter.write(replacedString);
//Done :)
} catch (FileNotFoundException e) {
// Exception thrown if the file does not exist in your system
e.printStackTrace();
} catch (IOException e) {
// Exception thrown due to an issue with IO
e.printStackTrace();
}
}
Hope this is helpful. Happy coding :)

If you can use Java 8 then your code can be simplified to
try (PrintStream ps = new PrintStream("D:\\hl_sv\\L09MF2.txt");
Stream<String> stream = Files.lines(Paths.get("D:\\hl_sv\\L09MF.txt"))) {
stream.map(line -> line.replace('?', '-')).forEach(ps::println);
} catch (IOException e) {
e.printStackTrace();
}

how to read from a huge file and write to a new file by java

What I am doing is to read one file line by line, format every line, then write to a new file. But the problem is that the file is huge, nearly 178 MB. But always getting error message: IO console updater error, java heap space. Here is my code:
public class fileFormat {
public static void main(String[] args) throws IOException{
String strLine;
FileInputStream fstream = new FileInputStream("train_final.txt");
BufferedReader reader = new BufferedReader(new InputStreamReader(fstream));
BufferedWriter writer = new BufferedWriter(new FileWriter("newOUTPUT.txt"));
while((strLine = reader.readLine()) != null){
List<String> numberBox = new ArrayList<String>();
StringTokenizer st = new StringTokenizer(strLine);
while(st.hasMoreTokens()){
numberBox.add(st.nextToken());
}
for (int i=1; i< numberBox.size(); i++){
String head = numberBox.get(0);
String tail = numberBox.get(i);
String line = head + " "+tail ;
System.out.println(line);
writer.write(line);
writer.newLine();
}
numberBox.clear();
}
reader.close();
writer.close();
}
}
How can I avoid this error message? Moreover, I have set the VM preference: -xms1024m

Remove the line
System.out.println(line);
This is a workaround the fialing console updater, which otherwise runs out of memory.

The program looks okay. I suspect the problem is that you run this inside of Eclipse, and System.out is collected by Eclipse in memory (to be displayed in that Console window).
System.out.println(line);
Try to run it outside of Eclipse, change Eclipse settings to pipe System.out somewhere, or remove the line.

This part of the code:
for (int i=1; i< numberBox.size(); i++){
String head = numberBox.get(0);
String tail = numberBox.get(i);
String line = head + " "+tail ;
System.out.println(line);
writer.write(line);
writer.newLine();
}
Can be translated to:
String head = numberBox.get(0);
for (int i=1; i< numberBox.size(); i++){
String tail = numberBox.get(i);
System.out.print(head);
System.out.print(" ");
System.out.println(tail);
writer.write(head);
writer.write(" ");
writer.write(tail);
writer.newLine();
}
This may add a little code duplication but it avoids creating a lot of objects.
Also there if you merge this for loop with the loop contructing the numberBox, you won't need numberBox structure at all.

If you read whole file the heap memory will occupy so better option in to read the file in chuck. See my below code. It will start reading from the offset given in argument and will return the end offset . You need to pass number of lines to be read.
Please remember: You can use any collection to store these read lines and clear the collection before calling this method to read next chunk.
FileInputStream fis = new FileInputStream(file);
InputStreamReader streamReader = new InputStreamReader(fis, "UTF-8");
LineNumberReader reader = new LineNumberReader(streamReader);
//call this below method recursively until the file does not reaches to the end
public int getParsedLines(LineNumberReader reader, int iLineNumber_Start, int iNumberOfLinesToBeRead) {
int iLineNumber_End = 0;
int iReadUptoLines = iLineNumber_Start + iNumberOfLinesToBeRead;
try {
reader.mark(iLineNumber_Start);
reader.setLineNumber(iLineNumber_Start);
do {
String str = reader.readLine();
if (str == null) {
break;
}
// your code
iLineNumber_End = reader.getLineNumber();
} while (iLineNumber_End != iReadUptoLines);
} catch (Exception ex) {
// exception handling
}
return iLineNumber_End;
}