Develop Reference

Why does this Java code not write to file?

There are so many Input/Output Classes in Java.
It is really a mess. You do not know which to use.
Which functions does operating system offer ? There will be one
to read one byte of a file or many bytes of a file I guess.
So for example if I use this.
String path = "C:\\Users\\myName\\test.txt";
FileOutputStream fos = new FileOutputStream(path);
fos.write(333);
If I open it with a text editor it shows me letter "G" . Already I do not understand this.
And this code does not write anything, the file is empty weirdly.
String path = "C:\\Users\\myName\\test.txt";
FileOutputStream fos = new FileOutputStream(path);
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(fos));
out.write("something");
All these I/O classes just confuse me. What does buffered mean. It reads 1000 Bytes at once. So
there is operating function to straight away read 1000 Bytes of a file I guess.

You need to close the instances of BufferedWriter out and FileOutputStream fos, after invoking the out.write("something"), then only the file gets created successfully with the contents you are trying to write.
public static void main(String[] args) throws IOException {
String path = "C:\\Users\\myName\\test.txt";
FileOutputStream fos = new FileOutputStream(path);
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(fos));
out.write("something");
out.close(); // Closes the stream, flushing it first.
fos.close(); // Closes this file output stream and releases any system resources associated with this stream.
}

Closing the instances of BufferedWriter and FileOutputStream will solve the issue.
fos.write(333) => The number has been written to the file and when you open the file it opens in ASCII format. You can use below code.
public static void main(String[] args) throws IOException {
FileWriter fw=new FileWriter("D:\\test.txt");
fw.write("Hello! This is a sample text");
System.out.println("Writing successful");
fw.close();
/* your code
String path = "D:\\test1.txt";
FileOutputStream fos = new FileOutputStream(path);
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(fos));
out.write("Hello! This is a sample text");
out.close();
fos.close();
*/
}

There are so many Input/Output Classes in Java. It is really a mess. You do not know which to use.
The Files class is by far the easiest to use. For instance,
Files.writeString(Paths.get("test.txt"), "hello world!");
creates a text file named "test.txt" containing the text "hello world!".
The other classes are only needed if you want to do something fancy (for instance, deal with files too big to fit in main memory). For instance, suppose you wanted to read a huge log file (hundreds of gigabytes long) and wanted to write each line containing a particular word to another file. If you were to open the file with
Files.readAllLines(Paths.get("huge.log"));
you'd receive an OutOfMemoryError because the file doesn't fit in main memory. To work around that, we must read the file piece-wise, and that is what all those Reader and Writer classes (or InputStream and OutputStream, if you're dealing with binary files) are good for:
try (
var reader = Files.newBufferedReader(Paths.get("huge.log"));
var writer = Files.newBufferedWriter(Paths.get("interesting.log"));
) {
String line;
while ((line = reader.readLine()) != null) {
if (line.contains(searchWord)) {
writer.write(line);
writer.write('\n');
}
}
}
As you can see, their use is quite a bit more complicated. For one, we must close the Reader and Writer once we are done with them, which is easiest accomplished with the try with resources statement shown above.
Closing is necessary because most operating systems limit the number of files that can be open at once. Closing also gives any Buffered* classes the opportunity to empty their buffers, ensuring that any data still in buffers is passed on to the file system.
If we fail to close, as you did in your example code, the file remains open until our program exits, upon which time any data in the buffers is lost, resulting in the incomplete file you found.

Java FileWriter lost special characters [duplicate]

This question already has answers here:
Java BufferedWriter object with utf-8
(4 answers)
Closed 7 years ago.
I've this code
//write a file in a specific directory
public static void writeFile(String comment, String outputFileName) {
FileWriter fWriter = null;
BufferedWriter writer = null;
try {
fWriter = new FileWriter(outputFileName,true);
writer = new BufferedWriter(fWriter);
writer.write(comment);
writer.newLine();
writer.close();
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
But when I save the file in outputFileName, it lost every special character.
File output format is .txt
Some solution?
Thanks a lot

FileWriter uses the platform-default encoding. It's very rarely a good idea to use that class, IMO.
I would suggest you use an OutputStreamWriter specifying the encoding you want - where UTF-8 is almost always a good choice, when you have a choice.
If you're using Java 7 or higher, Files.newBufferedWriter is your friend - and with Java 8, there's an overload without a Charset parameter, in which case UTF-8 is used automatically.

should I use PrintWriter to wrap BufferedWriter?

in the api,java said that,'Unless prompt output is required, it is advisable to wrap a BufferedWriter around any Writer whose write() operations may be costly, such as FileWriters and OutputStreamWriters'.I tried the comparation many times,it turns out that,it costs less time if I do not use PrintWriter to wrap BufferedWriter.
so does it mean that,I do not have to wrap on the 'console'?
there is my simple code that compare the two.
public static void PrintWrite(String path, String content) throws IOException {
PrintWriter pw = new PrintWriter(new BufferedWriter(new FileWriter(path)));
pw.write(content);
pw.close();
}
public static void BufferedWrite(String path, String content) throws IOException {
BufferedWriter bw = new BufferedWriter(new FileWriter(path));
bw.write(content);
bw.close();
}
private static void main(String[] args)
{
String s = "";
String path = "D:\\out.txt";
for (int i = 0; i < 10000; i++)
{
s += "kjuatlistened";
}
try
{
long start = System.nanoTime();
//PrintWrite(path,s);
BufferedWrite(path,s);
long end = System.nanoTime();
Printer.println("done,elapse " + (end - start) + " s");
}
catch (IOException ex)
{
ex.printStackTrace();
}
}

You're certainly adding a layer of processing when you wrap one object in another.
The main reason for using PrintWriter is the wealth of convenience functions which do intelligent output of various things. If you don't need that capability, you wouldn't use a PrintWriter at all.
The point of a BufferedWriter is to allow output to be buffered and then written in a batch. I/O to file systems, the Internet, etc. is costly, and a large fraction of that cost is in generating the connection in the first place, and then waiting while each I/O operation completes. If you batch I/O to and from external devices, you can really reduce the overall cost because output (or input) can be batched--and you end up waiting for completion just once per batch, rather than once per I/O operation. For an "interesting" test, try writing characters to your FileWriter one at a time, then do the same thing with a BufferedWriter wrapping your FileWriter. I think in the latter case you'll see a large improvement.

I don't fully understand your underlying question, but...
Wrapping a BufferedWriter with a PrintWriter pretty much has to to be slower, since it is added a layer of calls.
System.out and .err are PrintStream objects, not PrintWriter. Is that what you mean by the console? I'm pretty sure you should not wrap either of them.

Java - multiple threads writing to same file [duplicate]

This question already has answers here:
Writing a file using multiple threads
(3 answers)
Closed 7 years ago.
I am trying to write some content to a file, through multiple threads in Java. Each thread reads a different input file, does some computation and writes some (different) content to the common output file. The problem is that in the end, the output file only contains the content written by the last terminating thread and not the content from the other threads. Relevant code for the threads -
public void run()
{
try
{
File file = new File("/home/output.txt");
if (!file.exists())
{
file.createNewFile();
}
FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
BufferedReader br = new BufferedReader(new FileReader(inputfile)); // each thread reads a different input file
String line="";
while((line=br.readLine())!=null)
{
String id = line.trim(); // fetch id
StringBuffer sb = processId(userId); // process id
synchronized(this){
bw.write(sb.toString() + "\n"); // write to file
}
}
bw.close();
}
catch (IOException e)
{
e.printStackTrace();
}
}
How do i make all threads write their content to the common file ?

Use the constructor for FileWriter that uses append mode
FileWriter fw = new FileWriter(file.getAbsoluteFile(), true);

Have a single file writer thread which keeps reading from blocking queue and writing to a file. All other 20 threads will just put the data in blocking queue. This will prevent contention between 20 writer threads.

Few points related to your code :
1. The way you are creating the FileWriter is not correct . If you want to append data to the file, use the constructor that contains an additional boolean argument ( Make it true ):
public FileWriter(File file,boolean append) throws IOException
For example :
FileWriter fw = new FileWriter(file.getAbsoluteFile(),true);
2. You are talking about multiple threads that will share a common file , but I couldn't see any synchronized block in your code . Use synchronization to ensure that only one thread can access the shared resource at a time.

FileWriter fw = new FileWriter(file.getAbsoluteFile(),true);
should be used which indicates append mode.

I'm a newbie. SO I MIGHT BE WRONG! But your code MAY or MAY NOT work. Is that the run of a Runnable or thread? Note this:
MyClass implements Runnable {
public void run() {
....synchronized(this)..
}
}
Thread t1 = new Thread(new MyClass());
Thread t2 = new Thread(new MyClass());
Many of us do it like that(I did it, do it:), and yey! Each thread will obtain a lock on different obj, and you can end up with unexpected results, unless the OS uses some mechanism to sync writes to the same file (I don't know). If it does, then your synchonized is useless(useless anyway in my example), if it doesn't you're pretty.... Also as a note, there may be another process using this output.txt of which we have no idea...complicated stuff, at least for me
My opinion is that synchronized is useful if you have one instance of the class which SHARES, MUTABLE, STATE. (Let's forget about synchronized and static for now)
Does MyClass have state? No. Is it shared? NO, not in my example. Will it work? Not like this. Probably like this?
MyClass mc = new MyClass()
Thread t1 = new Thread(mc);
Thread t2 = new Thread(mc);
PS: I forgot to call start on every thread.

How to append/write huge data file text in Java

I have a database with 150k records. I want to write this to file as fast as possible. I've tried many approaches, but all seem slow. How do I make this faster?
I read these records in blocks of 40k. So first I read 40k then another 40k and so on.
After reading the records, this process returns a StringBuilder which contains 40k lines. Then we write this StringBuilder to a file.
private static void write(StringBuilder sb, Boolean append) throws Exception {
File file = File.createTempFile("foo", ".txt");
FileWriter writer = new FileWriter(file.getAbsoluteFile(), append);
PrintWriter out = new PrintWriter(writer);
try {
out.print(sb);
out.flush();
writer.flush();
} finally {
writer.close();
out.close();
}
}
I read this other example but it is equally slow: Fastest way to write huge data in text file Java
I also tried it with NIO api:
private static void write(StringBuilder sb, Boolean append)) throws Exception {
FileChannel rwChannel = new FileOutputStream("textfile.txt", true).getChannel();
ByteBuffer bb = ByteBuffer.wrap(sb.toString().getBytes("UTF-8"));
rwChannel.write(bb);
rwChannel.close();
}
Which is the best method to write/append huge data into file?

You don’t need a PrintWriter here. If you have whatever kind of Writer (e.g. a FileWriter) you can simply invoke append(sb) on it. And you don’t need to flush, close implies flushing.
private static void write(StringBuilder sb, Boolean append) throws Exception {
File file = File.createTempFile("foo", ".txt");
try(FileWriter writer = new FileWriter(file.getAbsoluteFile(), append)) {
writer.append(sb);
}
}
On my system I encountered a small performance improvement using a Channel rather than an OutputStream:
private static void write0a(StringBuilder sb, Boolean append) throws Exception {
File file = File.createTempFile("foo", ".txt");
try(Writer writer = Channels.newWriter(new FileOutputStream(
file.getAbsoluteFile(), append).getChannel(), "UTF-8")) {
writer.append(sb);
}
}
However these are only slight improvements. I don’t see much possibilities here as all the code ends up calling the same routines. What could really improve your performance is keeping the Writer alive during the invocations and not flushing every record.

If you have a huge amount of data, it's better that you don't store it to StringBuilder and then write it to file at once.
This is the best scenario:
1) Before you start process on the data create FileInputStream
FileOutputStream fos = new FileOutputStream("/path/of/your/file");
2) Create and OutputStreamWriter from this file
OutputStreamWriter w = new OutputStreamWriter(fos, "UTF-8");
3) Create BufferedWriter (Improve file writing performance)
BufferedWriter bw = new BufferedWriter(w);
4) Pass bw to your process function and then flush/close
bw.flush();
bw.close();
The functionality of StringBuilder and BufferedWriter is almost same, So you do not need to change your code so much. The only negative point of this scenario is that, your process will involve all the time that the data are writing to file, but if you don't process the data in different thread, it is not an issue.
In this way, it doesn't matter how large data is it

You are using a FileWriter (or a FileOutputStream in the second example). These are not buffered! So they write single chars resp. bytes to the disk.
That means, you should wrap the FileWriter in a BufferedWriter (or the FileOutputSystem in a BufferedOutputSystem).
private static void write(StringBuilder sb, Boolean append) throws Exception {
File file = File.createTempFile("foo", ".txt");
Writer writer = new BufferedWriter(new FileWriter(file.getAbsoluteFile(), append));
PrintWriter out = new PrintWriter(writer);
try {
out.print(sb);
out.flush();
writer.flush();
} finally {
writer.close();
out.close();
}
}

You are opening the file, writing one line, then closing it. It's the opening and closing that takes the time here. Find a way to keep the output file open.

Did you try Apache IO, is the performance still the same?