I was writing a program to implement for running two different tasks with Executor Framework as part to learn multi-threading. Earlier, I was using synchronized method to fullfill this requirement but its giving wrong results. Then, I learned that using Executor Framework is better approach for thread management.
Below Progam using synchronize methods
import java.io.*;
import java.util.Scanner;
import java.nio.*;
class FileWriteThreadExample implements Runnable{
/*This class needs to write some content into text file*/
public synchronized void run() {
StringBuilder thisProgamMessage = new StringBuilder();
try(FileWriter fw = new FileWriter("C:\\TestNotes.txt", true);
BufferedWriter bw = new BufferedWriter(fw);
PrintWriter out = new PrintWriter(bw))
{
for(int i=1; i<=50;i++){
//Thread.sleep(500);
//System.out.println(i);
thisProgamMessage.append(i+":"+Math.random()+"\n");
}
out.println(thisProgamMessage.toString());
} catch (IOException e) {
//exception handling left as an exercise for the reader
}
}
}
class FileWriteThreadExample2 implements Runnable{
/*This class needs to write some content into text file*/
public synchronized void run() {
StringBuilder thisProgamMessage = new StringBuilder();
try(FileWriter fw = new FileWriter("C:\\TestNotes.txt", true);
BufferedWriter bw = new BufferedWriter(fw);
PrintWriter out = new PrintWriter(bw))
{
System.out.println("Starting Second Write Thread Task");
for(int i=50; i>=1;i--){
//Thread.sleep(500);
//System.out.println(i);
thisProgamMessage.append(i+"====>"+Math.random()+"\n");
}
out.println(thisProgamMessage.toString());
System.out.println("Completing Second Write Thread Task");
}
catch (FileNotFoundException fnfe){
fnfe.printStackTrace();
}
catch(IOException ioex) {
ioex.printStackTrace();
}
/*catch(InterruptedException ie){
ie.printStackTrace();
}*/
}
}
class SynchronizeTest {
public static void main (String[] args) {
FileWriteThreadExample t1 = new FileWriteThreadExample();
FileWriteThreadExample2 t2 = new FileWriteThreadExample2();
t1.start();
t2.start();
}
}
Problem here is I don't know to write code for Executor that execute two tasks. I had implemented code with ExecutorService for running single task i.e.
ExecutorService es = Executors.newFixedThreadPool(5);
public void doStuff() {
es.submit(new MyRunnable());
}
Finally, can someone suggest me to implement two different tasks with Executor Framework ?
PS: Let me know for any confusion on understanding problem statement
You're very close:
ExecutorService es = Executors.newFixedThreadPool(5);
public void doStuff() {
es.submit(new FirstTask()); // FirstTask implements Callable
es.submit(new SecondTask()); // SecondTask implements Callable
}
Or alternatively:
ExecutorService es = Executors.newFixedThreadPool(5);
public void doStuff() {
Collection<Callable> tasks = Arrays.asList(new Callable[]
{ new FirstTask(), new SecondTask() });
es.invokeAll(tasks);
}
Each task may synchronize with each other like normal, just as if you were running the tasks in raw threads yourself.
Do note that ExecutorService requires the Callable interface rather than the Runnable interface.
I don't know your intent of exercise. in your synchronize version. you synchronized nothing. the two threads access TestNotes.txt sequentially, because only one file can open a file for write at a moment.is this your intent?
Related
I am learning about the use of semaphores and multi threading in general but am kind of stuck. I have two threads printing G and H respectively and my objective is to alternate the outputs of each thread so that the output string is like this;
G
H
G
H
G
H
Each of the two classes has a layout similar to the one below
public class ClassA extends Thread implements Runnable{
Semaphore semaphore = null;
public ClassA(Semaphore semaphore){
this.semaphore = semaphore;
}
public void run() {
while(true)
{
try{
semaphore.acquire();
for(int i=0; i<1000; i++){
System.out.println("F");
}
Thread.currentThread();
Thread.sleep(100);
}catch(Exception e)
{
System.out.println(e.toString());
}
semaphore.release();
}
}
}
below is my main class
public static void main(String[] args) throws InterruptedException {
Semaphore semaphore = new Semaphore(1);
ClassA clasA = new ClassA(semaphore);
Thread t1 = new Thread(clasA);
ClassB clasB = new ClassB(semaphore);
Thread t2 = new Thread(clasB);
t1.start();
t2.join();
t2.start();
The output I am getting is way too different from my expected result. can anyone help me please? did I misuse the semaphore? any help?
Semaphores can't help you solve such a task.
As far as I know, JVM doesn't promise any order in thread execution. It means that if you run several threads, one thread can execute several times in a row and have more processor time than any other. So, if you want your threads to execute in a particular order you can, for the simplest example, make a static boolean variable which will play a role of a switcher for your threads. Using wait() and notify() methods will be a better way, and Interface Condition will be the best way I suppose.
import java.io.IOException;
public class Solution {
public static boolean order;
public static void main(String[] args) throws IOException, InterruptedException {
Thread t1 = new ThreadPrint("G", true);
Thread t2 = new ThreadPrint("O", false);
t1.start();
t2.start();
t2.join();
System.out.println("Finish");
}
}
class ThreadPrint extends Thread {
private String line;
private boolean order;
public ThreadPrint(String line, boolean order) {
this.line = line;
this.order = order;
}
#Override
public void run() {
int z = 0;
while (true) {
try {
for (int i = 0; i < 10; i++) {
if (order == Solution.order) {
System.out.print(line + " ");
Solution.order = !order;
}
}
sleep(100);
} catch (Exception e) {
System.out.println(e.toString());
}
}
}
}
BTW there can be another problem cause System.out is usually an Operation System buffer and your OS can output your messages in an order on its own.
P.S. You shouldn't inherit Thread and implement Runnable at the same time
public class ClassA extends Thread implements Runnable{
because Thread class already implements Runnable. You can choose only one way which will be better for your purposes.
You should start a thread then join to it not vice versa.
t1.start();
t2.join();
t2.start();
As others have pointed out, locks themselves do not enforce any order and on top of that, you cannot be certain when a thread starts (calling Thread.start() will start the thread at some point in the future, but this might take a while).
You can, however, use locks (like a Semaphore) to enforce an order. In this case, you can use two Semaphores to switch threads on and off (alternate). The two threads (or Runnables) do need to be aware of each other in advance - a more dynamic approach where threads can "join in" on the party would be more complex.
Below a runnable example class with repeatable results (always a good thing to have when testing multi-threading). I will leave it up to you to figure out why and how it works.
import java.util.concurrent.*;
public class AlternateSem implements Runnable {
static final CountDownLatch DONE_LATCH = new CountDownLatch(2);
static final int TIMEOUT_MS = 1000;
static final int MAX_LOOPS = 10;
public static void main(String[] args) {
ExecutorService executor = Executors.newCachedThreadPool();
try {
AlternateSem as1 = new AlternateSem(false);
AlternateSem as2 = new AlternateSem(true);
as1.setAlternate(as2);
as2.setAlternate(as1);
executor.execute(as1);
executor.execute(as2);
if (DONE_LATCH.await(TIMEOUT_MS, TimeUnit.MILLISECONDS)) {
System.out.println();
System.out.println("Done");
} else {
System.out.println("Timeout");
}
} catch (Exception e) {
e.printStackTrace();
} finally {
executor.shutdownNow();
}
}
final Semaphore sem = new Semaphore(0);
final boolean odd;
AlternateSem other;
public AlternateSem(boolean odd) {
this.odd = odd;
}
void setAlternate(AlternateSem other) { this.other = other; }
void release() { sem.release(); }
void acquire() throws Exception { sem.acquire(); }
#Override
public void run() {
if (odd) {
other.release();
}
int i = 0;
try {
while (i < MAX_LOOPS) {
i++;
other.acquire();
System.out.print(odd ? "G " : "H ");
release();
}
} catch (Exception e) {
e.printStackTrace();
}
DONE_LATCH.countDown();
}
}
While this might look like a repeat question, there is not ONE correct working implementation of the Reader Writer's problem in the internet using semaphores in Java. The closest I found was this where an alternative is given as the answer but not using semaphores.
I am new to multi-threading in Java so please bear with me. Here is my code:
import java.util.concurrent.Semaphore;
class ReaderWritersProblem {
static Semaphore readLock = new Semaphore(1);
static Semaphore writeLock = new Semaphore(1);
static int readCount = 0;
static class Read implements Runnable {
#Override
public synchronized void run() {
try {
readLock.acquire();
readCount++;
if (readCount == 1) {
writeLock.acquire();
}
System.out.println("Thread "+Thread.currentThread().getName() + " is READING");
Thread.sleep(1500);
System.out.println("Thread "+Thread.currentThread().getName() + " has FINISHED READING");
readLock.release();
readCount--;
if(readCount == 0) {
writeLock.release();
}
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
static class Write implements Runnable {
#Override
public synchronized void run() {
try {
writeLock.acquire();
System.out.println("Thread "+Thread.currentThread().getName() + " is WRITING");
Thread.sleep(2500);
System.out.println("Thread "+Thread.currentThread().getName() + " has finished WRITING");
writeLock.release();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
public static void main(String[] args) throws Exception {
Read read = new Read();
Write write = new Write();
Thread t1 = new Thread(read);
t1.setName("thread1");
Thread t2 = new Thread(read);
t2.setName("thread2");
Thread t3 = new Thread(write);
t3.setName("thread2");
Thread t4 = new Thread(read);
t4.setName("thread4");
t1.run();
t2.run();
t3.run();
t4.run();
}
}
I am creating 4 threads, 3 for reading and 1 for writing. However, the output is as such:
Thread main is READING Thread main has FINISHED READING Thread
main is READING Thread main has FINISHED READING Thread main
is WRITING Thread main has finished WRITING Thread main is
READING Thread main has FINISHED READING
The thread name being returned is 'main'. Also, all of these are seemingly not being executed concurrently. How do I correct this implementation of Reader Writers problem using Semaphores only? Using ReentrantReadLock etc would simply this but beat the purpose of the question, which clearly says implement the problem using semaphores. Thanks
Edit: Also, HOW do I show multiple threads reading? I am getting the execution results one by one.
EDIT: CORRECTED CODE: I have corrected the algorithm. Would like some review on this.
import java.util.concurrent.Semaphore;
class ReaderWritersProblem {
static Semaphore readLock = new Semaphore(1);
static Semaphore writeLock = new Semaphore(1);
static int readCount = 0;
static class Read implements Runnable {
#Override
public void run() {
try {
//Acquire Section
readLock.acquire();
readCount++;
if (readCount == 1) {
writeLock.acquire();
}
readLock.release();
//Reading section
System.out.println("Thread "+Thread.currentThread().getName() + " is READING");
Thread.sleep(1500);
System.out.println("Thread "+Thread.currentThread().getName() + " has FINISHED READING");
//Releasing section
readLock.acquire();
readCount--;
if(readCount == 0) {
writeLock.release();
}
readLock.release();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
static class Write implements Runnable {
#Override
public void run() {
try {
writeLock.acquire();
System.out.println("Thread "+Thread.currentThread().getName() + " is WRITING");
Thread.sleep(2500);
System.out.println("Thread "+Thread.currentThread().getName() + " has finished WRITING");
writeLock.release();
} catch (InterruptedException e) {
System.out.println(e.getMessage());
}
}
}
public static void main(String[] args) throws Exception {
Read read = new Read();
Write write = new Write();
Thread t1 = new Thread(read);
t1.setName("thread1");
Thread t2 = new Thread(read);
t2.setName("thread2");
Thread t3 = new Thread(write);
t3.setName("thread3");
Thread t4 = new Thread(read);
t4.setName("thread4");
t1.start();
t3.start();
t2.start();
t4.start();
}
}
I assume that your question is related to why this code is not executing concurrently. This is because you have to call the method t1.start() and not t1.run() (and correspondingly so for all your other threads). t1.run() will run the run() method in the context of main thread and not in a new thread.
And I don't see the point of synchronized methods when you are already guarding the code with a binary semaphore. Two threads will not be able to simultaneously acquire a binary semaphore and as long as threads are not releasing the semaphore blindly your code will be safe. This defeats the whole prupose of having multiple reader threads.
Updated for the new question in comment - If you want multiple reader threads to access the same section then you should not use a binary semaphore but a semaphore with a bigger value (preferably equal to the number of reader threads). All of these threads can acquire the semaphore and release them when they are done. Also special care must be taken to to not starve the writer thread when many reader threads are active. This could be done in many ways - Just google for readers - writers problem and you will find out how you can do it.
I'm writing simple html parser with JSoup. I've got about 50 000 links to check, so I thought it's great chance to learn abut threads and concurnecy. I've got 8 tasks registered with ExecutorService: 6 of them parse links to some data stored in ArrayLists and then add it to the BlockingQueues. Two of the tasks are filewriters based on BufferedWriter. The problem is when my 6 tasks finish prase all links, file writers stop write data from BlockingQueue, so I lose part of data. I'm pretty newbie in java, so if you could give me a hand.... The code:
Main file:
public static void main(String[] args) {
BlockingQueue<ArrayList<String>> units = new ArrayBlockingQueue<ArrayList<String>>(50, true);
BlockingQueue<ArrayList<String>> subjects = new ArrayBlockingQueue<ArrayList<String>>(50, true);
File subjectFile = new File("lekarze.csv");
File unitFile = new File("miejsca.csv");
ExecutorService executor = Executors.newFixedThreadPool(9);
executor.submit(new Thread(new FileSaver(subjects, subjectFile)));
executor.submit(new Thread(new FileSaver(units, unitFile)));
for(int i = 29323; i < 29400; i++){
executor.submit(new ParserDocsThread(i, subjects, units, errors));
}
executor.shutdown();
}
FileSaver class:
package parser;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.concurrent.BlockingQueue;
public class FileSaver implements Runnable {
private BlockingQueue<ArrayList<String>> toWrite = null;
private File outputFile = null;
private BufferedWriter writer = null;
public FileSaver(BlockingQueue<ArrayList<String>> queue, File file){
toWrite = queue;
outputFile = file;
}
public void run() {
try {
writer = new BufferedWriter(new FileWriter(outputFile, true));
while(true){
try{
save(toWrite.take());
} catch(InterruptedException e) {
e.printStackTrace();
}
}
} catch (IOException e) {
e.printStackTrace();
}
}
private void save(ArrayList<String> data){
String temp ="";
int size = data.size();
for(int i = 0; i < size; i++){
temp += data.get(i);
if(i != size - 1) temp += '\t';
}
try {
writer.write(temp);
writer.newLine();
} catch (IOException e) {
e.printStackTrace();
}
}
}
In ParserDocsThread I'm only use put() method to add elements to BlockingQueue.
Your consumer threads don't end cleanly because the take() call is waiting for a new array, and are not closing the buffered writer. The ServiceExecutor gives up on waiting for these threads to finish, and kills them. This is causing the last lines in the writer to not be written out to disk.
You should use poll(10, TimeUnit.SECONDS) (but with an appropriate timeout). After that timeout, your consumers will give up on the producers, and you should make sure you close your buffered writer properly so that the last of the buffer is printed out properly.
try (BufferedWriter writer = new BufferedWriter(new FileWriter(outputFile, true)))
{
while(true){
List<String> data = toWrite.poll(10, TimeUnit.SECONDS);
if (data == null) {
break;
}
save(data, writer);
}
} catch (...) {
}
I've put the buffered writer here into a try-with-resources (so the try here will automatically close the writer) and passed it to your save method, but you can do it your way, and manually close the writer in a finally block if you want:
try {
...
} catch(...) {
} finally {
writer.close(); // Closes and flushes out the remaining lines
}
You may also want to put in a call to awaitTermination on the executor servier (like so: How to wait for all threads to finish, using ExecutorService?) with a wait time greater than your poll timeout.
I have a problem with some threads.
My script
1 - loads like over 10 millions lines into an Array from a text file
2 - creates an ExecutorPool of 5 fixed threads
3 - then it is iterating that list and add some threads to the queue
executor.submit(new MyCustomThread(line,threadTimeout,"[THREAD "+Integer.toString(increment)+"]"));
Now the active threads never bypass 5 fixed threads, which is good, but i obseved that my processor goes into 100% load, and i have debuged a little bit and i saw that MyCustomThread constructor is being called, witch means that no matter if i declare 5 fixed threads, the ExecutorService will still try to create 10 milions objects.
The main question is :
How do i prevent this? I just want to have threads being rejected if they don't have room, not to create 10 million object and run them one by one.
Second question :
How do i get the current active threads? I tried threadGroup.activeCount() but it always give me 5 5 5 5 ....
THE CALLER CLASS :
System.out.println("Starting threads ...");
final ThreadGroup threadGroup = new ThreadGroup("workers");
//ExecutorService executor = Executors.newFixedThreadPool(howManyThreads);
ExecutorService executor = Executors.newFixedThreadPool(5,new ThreadFactory() {
public Thread newThread(Runnable r) {
return new Thread(threadGroup, r);
}
});
int increment = 0;
for(String line : arrayOfLines)
{
if(increment > 10000)
{
//System.out.println("TOO MANY!!");
//System.exit(0);
}
System.out.println(line);
System.out.println(threadGroup.activeCount());
if(threadGroup.activeCount() >= 5)
{
for(int i = 0; i < 10; i++)
{
System.out.println(threadGroup.activeCount());
System.out.println(threadGroup.activeGroupCount());
Thread.sleep(1000);
}
}
try
{
executor.submit(new MyCustomThread(line,threadTimeout,"[THREAD "+Integer.toString(increment)+"]"));
}
catch(Exception ex)
{
continue;
//System.exit(0);
}
increment++;
}
executor.awaitTermination(10, TimeUnit.MILLISECONDS);
executor.shutdown();
THREAD CLASS :
public class MyCustomThread extends Thread
{
private String ip;
private String threadName;
private int threadTimeout = 10;
public MyCustomThread(String ip)
{
this.ip = ip;
}
public MyCustomThread(String ip,int threadTimeout,String threadName)
{
this.ip = ip;
this.threadTimeout = threadTimeout;
this.threadName = threadName;
System.out.prinln("MyCustomThread constructor has been called!");
}
#Override
public void run()
{
// do some stuff that takes time ....
}
}
Thank you.
You are doing it a bit wrong. The philosophy with executors is that you implement the work unit as a Runnable or a Callable (instead of a Thread). Each Runnable or Callable should do one atomic piece of work which is mutually exclusive of other Runnables or Callables.
Executor services internally use a pool of threads so your creating a thread group and Thread is not doing any good.
Try this simple piece:
ExecutorService executor = Executors.newFixedThreadPool(5);`
executor.execute(new MyRunnableWorker());
public class MyRunnableWorker implements Runnable{
private String ip;
private String threadName;
private int threadTimeout = 10;
public MyRunnableWorker(String ip){
this.ip = ip;
}
public MyRunnableWorker(String ip,int threadTimeout,String threadName){
this.ip = ip;
this.threadTimeout = threadTimeout;
this.threadName = threadName;
System.out.prinln("MyRunnableWorker constructor has been called!");
}
#Override
public void run(){ {
// do some stuff that takes time ....
}
}
This would give you what you want. Also try to test you thread code execution using visualVM to see how threads are running and what the load distribution.
I think your biggest problem here is that MyCustomThread should implement Runnable, not extend Thread. When you use an ExecutorService you let it handle the Thread management (i.e. you don't need to create them.)
Here's an approximation of what I think you're trying to do. Hope this helps.
public class FileProcessor
{
public static void main(String[] args)
{
List<String> lines = readFile();
System.out.println("Starting threads ...");
ExecutorService executor = Executors.newFixedThreadPool(5);
for(String line : lines)
{
try
{
executor.submit(new MyCustomThread(line));
}
catch(Exception ex)
{
ex.printStackTrace();
}
}
try
{
executor.shutdown();
executor.awaitTermination(10, TimeUnit.SECONDS);
}
catch (InterruptedException e)
{
System.out.println("A processor took longer than the await time to complete.");
}
executor.shutdownNow();
}
protected static List<String> readFile()
{
List<String> lines = new ArrayList<String>();
try
{
String filename = "/temp/data.dat";
FileReader fileReader = new FileReader(filename );
BufferedReader bufferedReader = new BufferedReader(fileReader);
String line = null;
while ((line = bufferedReader.readLine()) != null) {
lines.add(line);
}
bufferedReader.close();
}
catch (Exception e)
{
e.printStackTrace();
}
return lines;
}
}
public class MyCustomThread implements Runnable
{
String line;
MyCustomThread(String line)
{
this.line = line;
}
#Override
public void run()
{
System.out.println(Thread.currentThread().getName() + " processed line:" + line);
}
}
EDIT:
This implementation does NOT block on the ExecutorService submit. What I mean by this is that a new instance of MyCustomThread is created for every line in the file regardless of whether any previously submitted MyCustomThreads have completed. You could add a blocking / limiting worker queue to prevent this.
ExecutorService executor = new ThreadPoolExecutor(5, 5, 0L, TimeUnit.MILLISECONDS, new LimitedQueue<Runnable>(10));
An example of a blocking / limiting queue implementation can be found here:
The code in this question was clearly too long so I've written an example you can just copy and paste that I think encompasses the conceptual problem I'm having.
Thanks to those who read all the way through the previous version!
Two files
One calls the other and passes an ArrayList.
The 2nd file alters the ArrayList and provides a getter for the first file to access the altered file.
How can I make the first file wait for the processing in the second file, before calling the getter. At the moment this code gives you a NullPointerException because the first file isn't waiting.
join() on the thread of execution doesn't seem to work, and if I use wait(), how do I notify() to the first file from the second file?
Here's the code:
public class Launcher implements Runnable {
private ArrayList al = new ArrayList();
private ArrayProcessor ap;
public Launcher(ArrayList al){
this.al = al;
ArrayProcessor ap = new ArrayProcessor(al);
}
public static void main(String[] args){
ArrayList anArray = new ArrayList();
anArray.add(new Integer(1));
anArray.add(new Integer(13));
anArray.add(new Integer(19));
Launcher l = new Launcher(anArray);
l.liftOff();
}
public void liftOff(){
Thread t = new Thread(new Launcher(al));
synchronized(t){
t.start();
try {
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
this.ap = new ArrayProcessor(al);
System.out.println("the array: " + ap.getArray());
}
public void run() {
ap.doProcess(al);
}
}
and the called file:
public class ArrayProcessor extends Thread{
private ArrayList al;
public ArrayProcessor(ArrayList al){
this.al = al;
}
public void doProcess(ArrayList myAL){
this.start();
}
public void run() {
// this should increment the ArrayList elements by one
for (int i=0; i<al.size(); i++){
int num = ((Integer)al.get(i)).intValue();
al.set(i, new Integer(++num));
}
}
public ArrayList getArray(){
return al;
}
}
If you want one thread wait for another to finish you could use a CountDownLatch. Since there is a lot of code in your samples, here is a small POC:
public static void main(String[] args) throws Exception {
final CountDownLatch latch = new CountDownLatch(1);
new Thread(new Runnable() {
#Override
public void run() {
try {
System.out.println("Thread doing some work...");
Thread.sleep(10 * 1000);
System.out.println("Thread done!");
latch.countDown();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
System.out.println("Main Thread waiting...");
latch.await();
System.out.println("Main Thread can continue");
}