I want to get a file contains a letter from other files in random order. I must to do it with Threads. And I don't know why I have in output file content from 1 file, after it content from 2 file and after that content from 3 file. I have Main:
public static void main(final String[] args) throws Exception {
for(int i=1;i<args.length; i++) {
new Thread1( args[i], args[0]).start();
}
}
And class Thread1:
public class Thread1 extends Thread {
String path;
FileWriter fw;
private String desc;
public Thread1( String path, String desc) {
super();
this.desc=desc;
this.path=path;
}
#Override
public void run() {
FileReader f = null;
try {
f = new FileReader(path);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
int c;
try {
fw = new FileWriter(desc, true);
while((c = f.read()) != -1) {
fw.write(c);
}
fw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Please, explain me, why it don't work in way, what I think it should be work.
Please, explain me, why it don't work in way, what I think it should be work.
Your problem is that all of your threads are appending to the same file but using different FileWriter instances. This sounds like it would work but they are all overwriting each other. When you open a file for appending it opens it and positions the write marker at the end of the file. When two threads do this, they both will be at the same marker. If thread #1 writes a character then thread #2 will write a character and overwrite the first.
You could use a single FileWriter and share it with each of your threads. Then you synchronize on it for mutex purposes and do the write.
public Thread1( String path, String desc, FileWriter fw) {
this.fw = fw;
...
}
...
// when you write to it, you will need to synchronize on the writer
sychronized (fw) {
fw.write(c);
}
// don't close it in the threads but close it later after you have joined with
// the threads
Another option is to share a PrintStream which is already synchronized internally.
Related
I have a class that is the processor component of a reader/processor/writer ETL.
It gets work streamed to it, one wrapper at a time, and it is supposed to write to file. However, I think there is a bottleneck writing to the file, as I can see the filesize going up slowly 3kb at a time, whereas I know the process class gets called much more frequently than that. Can I implement some sort of blocking queue here to wait for more wrappers to arrive before writing them to file?
public class RiskToFsProcessor extends Processor<> {
/**
* Default implementation
*/
#Override
protected List<Object> process(Object wrapper) {
FileUtils.writeLinesToFile(wrapper);
return null;
}
writeLinesToFile (appends to file everytime without creating new file):
public static void writeLinesToFile(String fileName, String[] lines) {
try {
// create dir if not exist
File file = new File(fileName);
file.getParentFile().mkdirs();
PrintWriter writer = new PrintWriter(new BufferedWriter(new FileWriter(fileName, true)));
for (String line : lines) {
writer.println(line.trim());
}
writer.close();
LOG.debug("Write to {} finished.", fileName);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
I written a listener which listen to queue continuously, as message arrive it should write that message to file.
#Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body) throws IOException {
String response = new String(body);
String routingKey = envelope.getRoutingKey();
String contentType = properties.getContentType();
String correlationId = properties.getCorrelationId();
System.out.println("response "+ counter+ " :: "+ response);
try {
ResponseWriter.responseWrite(response, correlationId);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Runnable task = new VariableLengthTask(this, envelope.getDeliveryTag(), channel, 1000);
executorService.submit(task);
}
line in above method
ResponseWriter.responseWrite(response, correlationId);
will call below method:
File file =new File(defaultExcelPath);
FileOutputStream fop = null;
PrintWriter writer = null;
synchronized(file) {
try {
List<String> datalist = new ArrayList<String>();
// add details to datalist....
// write to the file ..
.....
.....
.....
fop = new FileOutputStream(defaultExcelPath, true);
writer = new PrintWriter(fop);
for(String data : datalist) {
writer.println(data);
}
writer.close();
} catch (Exception e) {
// TODO Auto-generated catch block
//e.printStackTrace();
throw e;
} finally {
if(fop!= null) fop.close();
if(writer!= null) writer.close();
}
}
but due to handler method listing more than 100 messages in some seconds, file writing method throws below exception
java.io.FileNotFoundException: C:\Files\performanceEvaluator.csv (The process cannot access the file because it is being used by another process)
how I can make the code which write to the file concurrently (or any other way..)
Another way would be to just sync when needed on a well known lock your own class. Alternatively you can declare a static variable of type object and lock on that.
class MyClass{
File file =new File(defaultExcelPath);
FileOutputStream fop = null;
PrintWriter writer = null;
{
try {//better to move this to its own fn. one fn one thing and probably dont need to sync this
List<String> datalist = new ArrayList<String>();
// add details to datalist....
// write to the file ..
.....
.....
.....
}catch ...//compelte earlier try
synchronized(MyClass.class) {
try{
// fop = ;
writer = new PrintWriter(new FileOutputStream(defaultExcelPath, true));
for(String data : datalist) {
writer.println(data);
}
// writer.close();//dont close here
} catch (Exception e) {
//no point throwing error what will the calling code do?
//jjust log it
logger.warn(e,e);//if you dont have logging framework get it or make system.out
} finally {
if(writer!= null) try{
writer.close();
}catch
//complete
// if(fop!= null) writer.close();
//in this case dont need to open two objects, just wrap it in
//but if u do have 2 object add seperate try-catch to close each inside the final
}
}
}
// not tested code please close braces etc
This is where the keyword synchronized comes into play. Declaring a function in this: public synchronized void function() manner does two things:
First, it is not possible for two invocations of synchronized methods on the same object to interleave. When one thread is executing a synchronized method for an object, all other threads that invoke synchronized methods for the same object block (suspend execution) until the first thread is done with the object.
Second, when a synchronized method exits, it automatically establishes a happens-before relationship with any subsequent invocation of a synchronized method for the same object. This guarantees that changes to the state of the object are visible to all threads.
From the Java Tutorials.
I'm new in semaphore and i have any questions. I have a thread which start and read lines from text file A and write them in to other text file B. I wrote this code, but I'm not sure whether thread block critical section and sync properly. Because and others threads can operation with these files.
public static void main(String[] args) {
Thread thread = new Thread(new ThreadManager());
thread.start();
}
Thread class:
public class ThreadManager extends Thread {
private Semaphore semaphore;
public ThreadManager() {
this.semaphore = new Semaphore(1);
}
public void run() {
try {
this.semaphore.acquire();
BufferedReader br = null;
String line;
String fileNme = "threadLog.txt";
ArrayList<String> fileLines = new ArrayList<String>();
int numLine = 0;
File outFile = new File("$$$$$$$$.tmp");
// input
FileInputStream fis = null;
PrintWriter out = null;
try {
fis = new FileInputStream(fileNme);
// output
FileOutputStream fos = new FileOutputStream(outFile);
out = new PrintWriter(fos);
} catch (FileNotFoundException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
}
BufferedReader in = new BufferedReader(new InputStreamReader(fis));
try {
while ((line = in.readLine()) != null) {
fileLines.add(line);
}
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
if (!fileLines.isEmpty()) {
int middleLine = (int) Math.round(fileLines.size() / 2);
fileLines.add(middleLine, Thread.currentThread().getName());
for (int i = 0; i < fileLines.size(); i++) {
out.println(fileLines.get(i));
}
out.flush();
out.close();
try {
in.close();
new File(fileNme).delete();
outFile.renameTo(new File(fileNme));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
this.semaphore.release();
} catch (InterruptedException e3) {
// TODO Auto-generated catch block
e3.printStackTrace();
}
}
You cannot guarantee synchronized access to a file, given only its filename, using semaphores or synchronized blocks (or whatever). Any other thread (or process) can still open, read, or modify that file, e.g. by creating its own FileOutputStream and passing the same file name.
While you could certainly structure your code to encourage synchronized access to a file, at least within your process, you cannot guarantee it. So you'll have to make some assumptions about the possibility of other processes accessing your file, and define (and document) some rules as to how other threads can access the file and adhere to them.
It looks like you're just making a temporary file so you could also consider File.createTempFile() to reduce the possibility of the same file name being used; adding uniqueness to the file name could help.
While I could definitely go into more detail about specific options, though, your precise use isn't clear from your question, and without more information, the best I can tell you is that synchronization primitives can't be used to 100% guarantee that nothing else is accessing that file at the same time - you have to consider your situation and find a balance between providing this protection through code vs. good documentation and sticking to rules that you define.
By the way, a semaphore with 1 permit is the same as a mutex, in which case you might find that synchronized blocks provide a more appealing syntax.
Also don't forget about ReadWriteLock, another useful tool depending on your access patterns and performance requirements.
All, I am trying to read a file which will be written by multiple threads, I am going to use BufferedReader to read that file in a thread.
The code looks like below.
FileReader reader = new FileReader(file);
BufferedReader br = new BufferedReader(reader);
String detail;
while ((detail =br.readLine()) != null)
{
...
}
Currently It seems works fine. But I have some questions about it.
If the question sound silly. please don't laugh at me . thanks.
Is it possible that the loop never been broken ? because the other threads are writing into the file.So maybe the readLine() may never return null?
Updated
Let's say there are 3 threads(T1,T2,T3).
T1 and T2 are writer.
T3 is reader.
The code runs in below sequence.
1.The current file lines number is 100.
2.T1 write a line to file.(file lines increase to 101)
3.T3 reads the last line of file(101). next read will get null.
4.T2 append a line to file.(file lines increase to 102)
5.T3 read again....(Does it return null or not? because T2 just added a new line into file before T3 read again.)
Please help to review it .thanks in advance.
Yes, it is possible that the loop will never end (at least until you run out of memory). Here's some code to prove it:
public class test {
public static void main(String[] args) {
// start thread to write to file
new Thread(new Runnable() {
#Override
public void run() {
FileWriter writer;
try {
int i = 1;
writer = new FileWriter("D:\\text.txt");
writer.append("line"+ i++ + "\n");
writer.flush();
while (true)
{
writer.append("line"+ i++ + "\n");
writer.flush();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
try {
Thread.sleep(500);
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
// start thread to read file
new Thread(new Runnable() {
#Override
public void run() {
try {
FileReader reader = new FileReader("D:\\text.txt");
BufferedReader br = new BufferedReader(reader);
String detail;
while ((detail =br.readLine()) != null)
{
System.out.println(detail);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
}
I did some experiment for it .
One eclipse run a program as writer .
public class Main {
private static Logger log = Logger.getLogger(Main.class);
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
PropertyConfigurator.configure("log4j.properties");
log.warn("Test test test ");
}
}
Another eclipse run the program as reader.
public class Main {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
StringBuffer intiLine = new StringBuffer("");
FileReader reader = new FileReader("D:\\logs\\notify-subscription.log");
BufferedReader br = new BufferedReader(reader);
String detail;
while ((detail =br.readLine()) != null)//debug and set breakpoint here
{
System.out.println(detail);
}
}
}
before I began to test them. The original log file content is empty.
I ran the reader program at first. the result of br.readLine() supposed to be null. But I set break point at the code line while ((detail =br.readLine()) != null) run, before it run , I ran the writer program. So the file contains test test test. and br.readLine() will not be null.
You are absolutely Correct .There will be a chance for Deadlock also if you keep creating threads for writing content to the file.Because if threads are keep on writing to the file there wont be any chance of exiting from loop it goes to infinite state
Consider the following test code.
I am trying to find out if I can use piped streams like "normal" I/O streams, together with the commonly used Reader and Writer implementations (specifically, another part of the code base I am working on demands that I use OutputStreamWriter).
The problem here is that nothing appears to show up on the read end. The program at least appears to correctly write the message to the write-end of the pipe, but when trying to read from the other end I block indefinetly, or if I (as in this case) check for available bytes, the call returns 0.
What am I doing wrong?
public class PipeTest {
private InputStream input;
private OutputStream output;
public PipeTest() throws IOException {
input = new PipedInputStream();
output = new PipedOutputStream((PipedInputStream)input);
}
public void start() {
Stuff1 stuff1 = new Stuff1(input);
Stuff2 stuff2 = new Stuff2(output);
Thread thread = new Thread(stuff1);
thread.start();
Thread thread2 = new Thread(stuff2);
thread2.start();
}
public static void main(String[] args) throws IOException {
new PipeTest().start();
}
private static class Stuff1 implements Runnable {
InputStream inputStream;
public Stuff1(InputStream inputStream) {
this.inputStream = inputStream;
}
#Override
public void run() {
String message;
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
try {
//message = reader.readLine();
System.out.println("Got message!");
System.out.println(inputStream.available());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
private static class Stuff2 implements Runnable {
OutputStream outputStream;
public Stuff2(OutputStream outputStream) {
this.outputStream = outputStream;
}
#Override
public void run() {
String message = "Hej!!\n";
OutputStreamWriter writer = new OutputStreamWriter(outputStream);
try {
writer.write(message);
System.out.println("Wrote message!");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
As you are never reading from the read end, it is impossible to see how you could possibly arrive at that conclusion, and any such conclusion is therefore baseless and invalid.
All you are doing is printing available() at an arbitrary point in time, which isn't sufficient to prove that nothing ever shows up at the read end.