I'm new in semaphore and i have any questions. I have a thread which start and read lines from text file A and write them in to other text file B. I wrote this code, but I'm not sure whether thread block critical section and sync properly. Because and others threads can operation with these files.
public static void main(String[] args) {
Thread thread = new Thread(new ThreadManager());
thread.start();
}
Thread class:
public class ThreadManager extends Thread {
private Semaphore semaphore;
public ThreadManager() {
this.semaphore = new Semaphore(1);
}
public void run() {
try {
this.semaphore.acquire();
BufferedReader br = null;
String line;
String fileNme = "threadLog.txt";
ArrayList<String> fileLines = new ArrayList<String>();
int numLine = 0;
File outFile = new File("$$$$$$$$.tmp");
// input
FileInputStream fis = null;
PrintWriter out = null;
try {
fis = new FileInputStream(fileNme);
// output
FileOutputStream fos = new FileOutputStream(outFile);
out = new PrintWriter(fos);
} catch (FileNotFoundException e2) {
// TODO Auto-generated catch block
e2.printStackTrace();
}
BufferedReader in = new BufferedReader(new InputStreamReader(fis));
try {
while ((line = in.readLine()) != null) {
fileLines.add(line);
}
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
if (!fileLines.isEmpty()) {
int middleLine = (int) Math.round(fileLines.size() / 2);
fileLines.add(middleLine, Thread.currentThread().getName());
for (int i = 0; i < fileLines.size(); i++) {
out.println(fileLines.get(i));
}
out.flush();
out.close();
try {
in.close();
new File(fileNme).delete();
outFile.renameTo(new File(fileNme));
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
this.semaphore.release();
} catch (InterruptedException e3) {
// TODO Auto-generated catch block
e3.printStackTrace();
}
}
You cannot guarantee synchronized access to a file, given only its filename, using semaphores or synchronized blocks (or whatever). Any other thread (or process) can still open, read, or modify that file, e.g. by creating its own FileOutputStream and passing the same file name.
While you could certainly structure your code to encourage synchronized access to a file, at least within your process, you cannot guarantee it. So you'll have to make some assumptions about the possibility of other processes accessing your file, and define (and document) some rules as to how other threads can access the file and adhere to them.
It looks like you're just making a temporary file so you could also consider File.createTempFile() to reduce the possibility of the same file name being used; adding uniqueness to the file name could help.
While I could definitely go into more detail about specific options, though, your precise use isn't clear from your question, and without more information, the best I can tell you is that synchronization primitives can't be used to 100% guarantee that nothing else is accessing that file at the same time - you have to consider your situation and find a balance between providing this protection through code vs. good documentation and sticking to rules that you define.
By the way, a semaphore with 1 permit is the same as a mutex, in which case you might find that synchronized blocks provide a more appealing syntax.
Also don't forget about ReadWriteLock, another useful tool depending on your access patterns and performance requirements.
Related
I am working on a simple project: Server executes a slave (ReaderF Class) inside a thread that reads a file and then extracts its content and saves it inside the Server himself. The server then needs to execute a pool to send the content to a client when he connects. I started by writing the ReaderF to extract the content. Problem: it never edits the String variable in which he is supposed to stock the content. Here is what I did:
public class Serveur {
private ServerSocket serverSocket;
private Socket socket;
public String res=null; //This is what my thread is supposed to be editing
ExecutorService pool = null;
public static final int port = 33333;
Serveur(int port, int size){
try {
pool = Executors.newFixedThreadPool(5);
serverSocket = new ServerSocket(port, size);
} catch (IOException ex) {
Logger.getLogger(Serveur.class.getName()).log(Level.SEVERE, null, ex);
}
}
void manage () throws IOException {
while(true) {
ReaderF S = null;
try {
S = new ReaderF(serverSocket.accept(), this);
} catch (IOException e) {
e.printStackTrace();
}
Thread t=new Thread(S);
t.start();
}
}
And this is the slave that reads and edit the res variable.
public class ReaderF implements Runnable {
private final Socket socket;
private Serveur serverMaitre;
ReaderF(Socket socket, Serveur serverMaitre) {
this.socket = socket;
this.serverMaitre = serverMaitre;
}
public void run() {
String fileName = "/home/abdou/1.txt";
FileReader fileReader = null;
Writer output =null;
try {
fileReader = new FileReader(fileName);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try (BufferedReader bufferedReader = new BufferedReader(fileReader)) {
output = new OutputStreamWriter(socket.getOutputStream(), "8859_1");
String line;
String res="";
while((line = bufferedReader.readLine()) != null) {
res+=line+"\n";
}
serverMaitre.res=res; // This is where i tell him to edit res inside the server
socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
When I execute this main I see nothing on the screen. I know that it is due to the fact that the Server is stuck in the while loop.
public class Main{
public static void main(String[] args) throws IOException {
Serveur serveur = new Serveur(Serveur.port, 1);
serveur.manage();
System.out.println("The res variable content is"+serveur.res);
}}
How would I get out of it please. And Is my thread really changing the res variable ? Thank you
When multiple threads work with the same variables, then they are allowed to work on a local copy of the variable (in their stack) instead of the original variable (in the heap). This means that one thread may not see changes to a variable that another thread did (he is not 'refreshing' its local data), or the other thread is not 'publishing' (technical term: flushing) the updated value. So, when your ReaderF assign a value to 'res', the main thread (which is checking it) might not notice this. Solution: Make the variable volatile:
public volatile String res=null;
This forces the ReaderF to flush the updated value after assigning, and the main thread to refresh it when reading it.
But you might have another problem here, because you have multiple threads that may be all writing the 'res' variable (Depends on how active your socket is). You can get 'lost updates' or other 'race conditions' here. So maybe you even need a synchronized somewhere. But it would be too much to explain Multi-Threading here. Please google about it. Multi-Threading is not trivial unfortunately, you cannot just make new threads as you wish and expect everything to work. And adding a volatile is not the solution most of the time. It might just be enough for what you wanna do here.
I written a listener which listen to queue continuously, as message arrive it should write that message to file.
#Override
public void handleDelivery(String consumerTag,
Envelope envelope,
AMQP.BasicProperties properties,
byte[] body) throws IOException {
String response = new String(body);
String routingKey = envelope.getRoutingKey();
String contentType = properties.getContentType();
String correlationId = properties.getCorrelationId();
System.out.println("response "+ counter+ " :: "+ response);
try {
ResponseWriter.responseWrite(response, correlationId);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Runnable task = new VariableLengthTask(this, envelope.getDeliveryTag(), channel, 1000);
executorService.submit(task);
}
line in above method
ResponseWriter.responseWrite(response, correlationId);
will call below method:
File file =new File(defaultExcelPath);
FileOutputStream fop = null;
PrintWriter writer = null;
synchronized(file) {
try {
List<String> datalist = new ArrayList<String>();
// add details to datalist....
// write to the file ..
.....
.....
.....
fop = new FileOutputStream(defaultExcelPath, true);
writer = new PrintWriter(fop);
for(String data : datalist) {
writer.println(data);
}
writer.close();
} catch (Exception e) {
// TODO Auto-generated catch block
//e.printStackTrace();
throw e;
} finally {
if(fop!= null) fop.close();
if(writer!= null) writer.close();
}
}
but due to handler method listing more than 100 messages in some seconds, file writing method throws below exception
java.io.FileNotFoundException: C:\Files\performanceEvaluator.csv (The process cannot access the file because it is being used by another process)
how I can make the code which write to the file concurrently (or any other way..)
Another way would be to just sync when needed on a well known lock your own class. Alternatively you can declare a static variable of type object and lock on that.
class MyClass{
File file =new File(defaultExcelPath);
FileOutputStream fop = null;
PrintWriter writer = null;
{
try {//better to move this to its own fn. one fn one thing and probably dont need to sync this
List<String> datalist = new ArrayList<String>();
// add details to datalist....
// write to the file ..
.....
.....
.....
}catch ...//compelte earlier try
synchronized(MyClass.class) {
try{
// fop = ;
writer = new PrintWriter(new FileOutputStream(defaultExcelPath, true));
for(String data : datalist) {
writer.println(data);
}
// writer.close();//dont close here
} catch (Exception e) {
//no point throwing error what will the calling code do?
//jjust log it
logger.warn(e,e);//if you dont have logging framework get it or make system.out
} finally {
if(writer!= null) try{
writer.close();
}catch
//complete
// if(fop!= null) writer.close();
//in this case dont need to open two objects, just wrap it in
//but if u do have 2 object add seperate try-catch to close each inside the final
}
}
}
// not tested code please close braces etc
This is where the keyword synchronized comes into play. Declaring a function in this: public synchronized void function() manner does two things:
First, it is not possible for two invocations of synchronized methods on the same object to interleave. When one thread is executing a synchronized method for an object, all other threads that invoke synchronized methods for the same object block (suspend execution) until the first thread is done with the object.
Second, when a synchronized method exits, it automatically establishes a happens-before relationship with any subsequent invocation of a synchronized method for the same object. This guarantees that changes to the state of the object are visible to all threads.
From the Java Tutorials.
All, I am trying to read a file which will be written by multiple threads, I am going to use BufferedReader to read that file in a thread.
The code looks like below.
FileReader reader = new FileReader(file);
BufferedReader br = new BufferedReader(reader);
String detail;
while ((detail =br.readLine()) != null)
{
...
}
Currently It seems works fine. But I have some questions about it.
If the question sound silly. please don't laugh at me . thanks.
Is it possible that the loop never been broken ? because the other threads are writing into the file.So maybe the readLine() may never return null?
Updated
Let's say there are 3 threads(T1,T2,T3).
T1 and T2 are writer.
T3 is reader.
The code runs in below sequence.
1.The current file lines number is 100.
2.T1 write a line to file.(file lines increase to 101)
3.T3 reads the last line of file(101). next read will get null.
4.T2 append a line to file.(file lines increase to 102)
5.T3 read again....(Does it return null or not? because T2 just added a new line into file before T3 read again.)
Please help to review it .thanks in advance.
Yes, it is possible that the loop will never end (at least until you run out of memory). Here's some code to prove it:
public class test {
public static void main(String[] args) {
// start thread to write to file
new Thread(new Runnable() {
#Override
public void run() {
FileWriter writer;
try {
int i = 1;
writer = new FileWriter("D:\\text.txt");
writer.append("line"+ i++ + "\n");
writer.flush();
while (true)
{
writer.append("line"+ i++ + "\n");
writer.flush();
}
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
try {
Thread.sleep(500);
} catch (InterruptedException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
// start thread to read file
new Thread(new Runnable() {
#Override
public void run() {
try {
FileReader reader = new FileReader("D:\\text.txt");
BufferedReader br = new BufferedReader(reader);
String detail;
while ((detail =br.readLine()) != null)
{
System.out.println(detail);
}
br.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
}
I did some experiment for it .
One eclipse run a program as writer .
public class Main {
private static Logger log = Logger.getLogger(Main.class);
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
PropertyConfigurator.configure("log4j.properties");
log.warn("Test test test ");
}
}
Another eclipse run the program as reader.
public class Main {
/**
* #param args
* #throws IOException
*/
public static void main(String[] args) throws IOException {
// TODO Auto-generated method stub
StringBuffer intiLine = new StringBuffer("");
FileReader reader = new FileReader("D:\\logs\\notify-subscription.log");
BufferedReader br = new BufferedReader(reader);
String detail;
while ((detail =br.readLine()) != null)//debug and set breakpoint here
{
System.out.println(detail);
}
}
}
before I began to test them. The original log file content is empty.
I ran the reader program at first. the result of br.readLine() supposed to be null. But I set break point at the code line while ((detail =br.readLine()) != null) run, before it run , I ran the writer program. So the file contains test test test. and br.readLine() will not be null.
You are absolutely Correct .There will be a chance for Deadlock also if you keep creating threads for writing content to the file.Because if threads are keep on writing to the file there wont be any chance of exiting from loop it goes to infinite state
I want to get a file contains a letter from other files in random order. I must to do it with Threads. And I don't know why I have in output file content from 1 file, after it content from 2 file and after that content from 3 file. I have Main:
public static void main(final String[] args) throws Exception {
for(int i=1;i<args.length; i++) {
new Thread1( args[i], args[0]).start();
}
}
And class Thread1:
public class Thread1 extends Thread {
String path;
FileWriter fw;
private String desc;
public Thread1( String path, String desc) {
super();
this.desc=desc;
this.path=path;
}
#Override
public void run() {
FileReader f = null;
try {
f = new FileReader(path);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
int c;
try {
fw = new FileWriter(desc, true);
while((c = f.read()) != -1) {
fw.write(c);
}
fw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Please, explain me, why it don't work in way, what I think it should be work.
Please, explain me, why it don't work in way, what I think it should be work.
Your problem is that all of your threads are appending to the same file but using different FileWriter instances. This sounds like it would work but they are all overwriting each other. When you open a file for appending it opens it and positions the write marker at the end of the file. When two threads do this, they both will be at the same marker. If thread #1 writes a character then thread #2 will write a character and overwrite the first.
You could use a single FileWriter and share it with each of your threads. Then you synchronize on it for mutex purposes and do the write.
public Thread1( String path, String desc, FileWriter fw) {
this.fw = fw;
...
}
...
// when you write to it, you will need to synchronize on the writer
sychronized (fw) {
fw.write(c);
}
// don't close it in the threads but close it later after you have joined with
// the threads
Another option is to share a PrintStream which is already synchronized internally.
Here's a section of my onCreate, which sometimes is causing exception:
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_tilisting);
_context = getApplicationContext();
SDName = Environment.getExternalStorageDirectory();
//listview = (ListView)findViewById(R.id.TIlistview);
String TIdir = new File(SDName, "/TitaniumBackup/").toString();
final ArrayList<String> apps = new ArrayList<String>();
final StringBuffer done = new StringBuffer();
Command command = new Command(0,"ls -a "+TIdir+"/*.properties") {
#Override
public void output(int arg0, String arg1) {
synchronized(apps) {
apps.add(arg1);
if (!done.toString().equals("")) {
done.append("done");//oh no
}
}
}
};
try {
RootTools.getShell(true).add(command).waitForFinish();
String attrLine = "";
int ind;
backups = new ArrayList<TIBackup>();
synchronized(apps) {
for (String app : apps) {
try {
TIBackup bkup = new TIBackup(app);
FileInputStream fstream = new FileInputStream(app);
BufferedReader atts = new BufferedReader(new InputStreamReader(fstream));
while ((attrLine = atts.readLine()) != null) {
ind = attrLine.indexOf('=');
if (ind !=-1 && !attrLine.substring(0,1).equals("#"))
bkup.prop.put(attrLine.substring(0,ind), attrLine.substring(ind+1));
}
backups.add(bkup);
atts.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
done.append("done");
}
setListAdapter( new StableArrayAdapter(this,backups));
} catch (InterruptedException e) {
//TODO:errors
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
}
The for (String app : apps) { is causing the exception, despite the waitforfinish() before it.
This updated code should fix it, adding data from the output, and waiting for any stragglers with the synchronized in the main code, but if you set a breakpoint on the //oh no line above, it is still getting to this point where it tries to add an item after the UI main code ran. So waitforfinish() is not waiting? How do I prevent this race condition?
I also tried the RootTask code below, but it seems to stop at the last readline?
RootTask getProfile = new RootTask() {
#Override
public void onPostExecute(ArrayList<String> result) {
super.onPostExecute(result);
for (String r : result) {
System.out.println(r);
}
}
};
getProfile.execute("ls /data/data/org.mozilla.firefox/files/mozilla/" );
onPostExecute never runs.
This was partially caused by a design flaw in RootTools. I believe the crux of the issue is that the operation that you are performing on the shell is taking longer than the default timeout that is set for shell commands. When the timeout occurs it simply returns the command as completed which is where the design flaw lies.
I have provided a new jar to use as well as some more information on this. I have also deprecated waitForFinish() as I agree that it was, and is, a poor solution.
https://code.google.com/p/roottools/issues/detail?id=35
Please let me know if you have any questions or problems :)
Output() is to be called during waitForFinish() waits. Something is wrong in the code implementing Command execution.
Most likely: the command executor (RootTools ?) runs the command on shell, gets a bunch of output lines, notifies the calling thread from waiting, and then calls output() of command for each line it got as output. I think it should notify the command thread after output() has been called on command object, for all output lines.
Still you can wrap the list modifying code and list iterating code in synchronized(<some common object>){}.
Update:
So waitForFinish() is not waiting? How do I prevent this race condition?
It does wait, but not for your code. Synchronized keyword merely made sure that output() of Command object is not called at the same time when you are iterating the apps collection. It does not schedule the two threads to run in a particular sequence.
IMHO, waitForFinish() is not a good pattern, making calling thread waiting defeats the point of a separate executor. It better be formulated like an AsyncTask or accept an event listener for each Command object.
Just a rough example, this class:
public class RootTask extends AsyncTask<String,Void,List<String>> {
private boolean mSuccess;
public boolean isSuccess() {
return mSuccess;
}
#Override
protected List<String> doInBackground(String... strings) {
List<String> lines = new ArrayList<String>();
try {
Process p = Runtime.getRuntime().exec("su");
InputStream is = p.getInputStream();
OutputStream os = p.getOutputStream();
os.write((strings[0] + "\n").getBytes());
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
String line;
while ((line = rd.readLine()) != null){
lines.add(line);
}
mSuccess = true;
os.write(("exit\n").getBytes());
p.destroy();
} catch (IOException e) {
mSuccess = false;
e.printStackTrace();
}
return lines;
}
}
can be used as:
RootTask listTask = new RootTask{
#Override
public void onPostExecute(List<String> result){
super.onPostExecute();
apps.addAll(result);
//-- or process the results strings--
}
};
listTask.execute("ls -a "+TIdir+"/*.properties");