What would be a JUnit based code to run this 3 methods each as 10 concurrent threads.
#RunWith(SpringJUnit4ClassRunner.class
#SpringBootTest
public class TestClass {
#Test
public void readFromDBOneRecord(){
try {
dbService.findOneByID("1");
} catch (Exception error) {
Assert.fail("Unexpected error occured .");
}
}
#Test
public void writeToDBOneRecord(){
try {
dbService.save(entity.builder()
.setID("1").setName("John").build())
} catch (Exception error) {
Assert.fail("Unexpected error occured .");
}
}
#Test
public void deleteDbRecord(){
try {
dbService.delete("1");
} catch (Exception error) {
Assert.fail("Unexpected error occured .");
}
}
}
In some cases some of the methods would throw exceptions. Like if the delete being executed before writeToDBOneRecord.
So the sequence would be say for only 3 threads per method e.g.:
OperationNr|| OperationName || [ThreadNr/total threads per method]OperationType
1. write [2/3]w
2. read [1/3]r
3. read [3/3]r
4. delete [2/3]d
5. read [2/3]r
6. delete [3/3]d ->exception no record
7. write [1/3]w
8. write [3/3]w ->exception record already present
9. delete [1/3]d
What would the code for executing this 3 test methods each in 10 concurrent threads (30 in total)?
As you want to do everything in parallel, I would mix everything and rely on CountDownLatch instances to synchronize the threads as next:
#Test
public void testMultiThreading() throws Exception {
// Total of reader threads
int reader = 5;
// Total of writer threads
int writer = 3;
// Total of remover threads
int remover = 1;
// CountDownLatch used to release all the threads at the same time
final CountDownLatch startSignal = new CountDownLatch(1);
// CountDownLatch used to be notified when all threads did their task
final CountDownLatch doneSignal = new CountDownLatch(reader + writer + remover);
// List in which we collect all the errors
final List<Exception> errors = Collections.synchronizedList(new ArrayList<>());
// Create all the reader threads and start them
for (int i = 0; i < reader; i++) {
Thread thread = new Thread() {
public void run() {
try {
startSignal.await();
dbService.findOneByID("1");
} catch (Exception e) {
errors.add(e);
} finally {
doneSignal.countDown();
}
}
};
thread.start();
}
// Create all the writer threads and start them
for (int i = 0; i < writer; i++) {
Thread thread = new Thread() {
public void run() {
try {
startSignal.await();
dbService.save(entity.builder()
.setID("1").setName("John").build());
} catch (Exception e) {
errors.add(e);
} finally {
doneSignal.countDown();
}
}
};
thread.start();
}
// Create all the remover threads and start them
for (int i = 0; i < remover; i++) {
Thread thread = new Thread() {
public void run() {
try {
startSignal.await();
dbService.delete("1");
} catch (Exception e) {
errors.add(e);
} finally {
doneSignal.countDown();
}
}
};
thread.start();
}
// Release the threads
startSignal.countDown();
// Wait until all threads did their task
doneSignal.await();
// If an error has been collected, print the stack trace and throws the
// first error to make the test fail
if (!errors.isEmpty()) {
for (Exception e : errors) {
e.printStackTrace();
}
throw errors.get(0);
}
}
NB: If you want a given unit test to be executed by several concurrent threads, have a look to contiperf but it won't allow you to mix them as you want to achieve
Related
I've been trying to make a simple implementation of Thread-Pool using Active Objects.
Here is my Main:
public static void main(String[] args){
MyThreadPool tp = new MyThreadPool(100,3);
tp.execute(()->{
try { Thread.sleep(5*1000); } catch (InterruptedException e) {}
System.out.println("42");
});
tp.shutDown();
}
The shutDown method is usually called first through the Main and therefore keeps the Active Objects "alive" unwantedly, but sometimes I get the wanted outcome.
Any idea why there is uncertainty about the result?
Below you can see the rest of the classes:
public class MyThreadPool {
ArrayBlockingQueue<Runnable> q;
ArrayBlockingQueue<ActiveObject> activeObjects;
volatile boolean stop;
AtomicInteger count;
Thread t;
Runnable stopTask;
public MyThreadPool(int capacity, int maxThreads) {
activeObjects = new ArrayBlockingQueue<>(maxThreads);
q = new ArrayBlockingQueue<>(capacity);
count = new AtomicInteger(0);
stopTask = ()->stop = true;
t=new Thread(()->{
//System.out.println("Thread-Pool Started");
while(!stop){
//if queue is empty it is gonna be a blocking call
try {
Runnable task = q.take();
if(task==stopTask)
stopTask.run();
else
//size() is atomic integer
if (count.get() < maxThreads) {
ActiveObject a = new ActiveObject(capacity);
activeObjects.put(a);
count.incrementAndGet();
a.execute(task);
}
//we will assign the next task to the least busy ActiveObject
else {
int minSize = Integer.MAX_VALUE;
ActiveObject choice = null;
for (ActiveObject a : activeObjects) {
if (a.size() < minSize) {
minSize = a.size();
choice = a;
}
}
choice.execute(task);
}
} catch (InterruptedException e) { }
}
//System.out.println("Thread-Pool Ended");
});
t.start();
}
//execute returns right away - just puts into the queue
public void execute(Runnable r ){
// if capacity is full it is gonna be a blocking call
if(!stop)
try { q.put(r); } catch (InterruptedException e) { }
}
public void shutDownNow(){
activeObjects.forEach(a->a.shutDownNow());
stop = true;
t.interrupt();
}
public void shutDown(){
activeObjects.forEach(a->a.shutDown());
execute(stopTask);
}
public class ActiveObject {
ArrayBlockingQueue<Runnable> q;
volatile boolean stop;
Thread t;
public ActiveObject(int capacity) {
q = new ArrayBlockingQueue<>(capacity);
t=new Thread(()->{
//System.out.println("Active Object Started");
while(!stop){
//if queue is empty it is gonna be a blocking call
try {
q.take().run();
} catch (InterruptedException e) { }
}
//System.out.println("Active Object Ended");
});
t.start();
}
//execute returns right away - just puts into the queue
public void execute(Runnable r ){
// if capacity is full it is gonna be a blocking call
if(!stop)
try { q.put(r); } catch (InterruptedException e) { }
}
public void shutDownNow(){
stop = true;
t.interrupt();
}
public void shutDown(){
execute(()->stop=true);
}
public int size(){
return q.size();
}
}
In your main method you create a thread pool (which also creates and starts tp.t thread), enqueue a task into tp.q, and then call tp.shutDown():
MyThreadPool tp = new MyThreadPool(100, 3);
tp.execute(() -> {...});
tp.shutDown();
Imagine that tp.shutDown() in the main thread is executed before the MyThreadPool.t thread processes the enqueued task:
activeObjects.forEach(a -> a.shutDown());
execute(stopTask);
here activeObjects is empty, you enqueue stopTask into tp.q, and main thread finishes.
Now we only have MyThreadPool.t thread, let's see what it does:
while (!stop) {
try {
Runnable task = q.take();
if (task == stopTask)
stopTask.run();
else
if (count.get() < maxThreads) {
ActiveObject a = new ActiveObject(capacity);
activeObjects.put(a);
count.incrementAndGet();
a.execute(task);
}
else {
...
}
} catch (InterruptedException e) {
}
}
At this point q contains 2 tasks: a normal task and stopTask.
In the first loop iteration the normal task is taken from q, and is given for processing to a newly created ActiveObject:
ActiveObject a = new ActiveObject(capacity);
activeObjects.put(a);
count.incrementAndGet();
a.execute(task);
new ActiveObject() also creates and starts its own internal ActiveObject.t thread.
The second loop iteration processes stopTask:
if (task == stopTask)
stopTask.run();
which sets stop = true.
As a result, the next check while (!stop) returns false and MyThreadPool.t thread finishes.
Now we only have ActiveObject.t thread, which hasn't been stopped:
while (!stop) {
try {
q.take().run();
} catch (InterruptedException e) {
}
}
here the thread will keep waiting on q.take() forever.
I have the following class, I usually run about 10 threads of it
public class MyClass implements Runnable {
private volatile Device device = null;
public MyClass(Device device) {
this.device = device;
}
#Override
public void run() {
while (true) { // <--- I do know that the "true" has to be changed to a Boolean
try {
Worker worker = new Worker();
worker.work();
System.out.println("Waiting 6 seconds!");
Thread.sleep(6 * 1000);
System.out.println("------------------------------------");
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("Thread in program ended!");
}
}
and in my main I start the threads like this
for (int i = 0; i < 2; i++) {
(new Thread(new MyClass())).start();
}
This is a console based program. What is the most reliable way to end the program? I think the best way would be to change while (true) to while (Boolean) and somehow change that Boolean for all threads, then when the loop ends, the program will end gracefully.
Here i'm ending it by waiting for a user input but you can change it to fire the stop method from anywhere
public static void main(String[] args) {
List<MyClass> myThreads = new ArrayList<>();
for (int i = 0; i < 2; i++) {
MyClass myClass = new MyClass();
Thread t = new Thread(myClass);
t.start();
myThreads.add(myClass);
}
Scanner in = new Scanner(System.in);
in.next();
for(MyClass t : myThreads){
t.stop();
}
}
class MyClass implements Runnable {
private Boolean flag;
public MyClass() {
this.flag = true;
}
#Override
public void run() {
while (flag) { // <--- I do know that the "true" has to be changed to a Boolean
try {
System.out.println("Waiting 6 seconds!");
Thread.sleep(6 * 1000);
System.out.println("------------------------------------");
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("Thread in program ended!");
}
public void stop(){
this.flag = false;
} }
The easy way would be to store all your threads in a set and make loop joining them at the end.
Be aware that this is not the most ortodox neither the most efficient way to do this.
In your main:
HashSet<Thread> threads = new HashSet();
for (int i = 0; i < 2; i++) {
Thread t = new Thread(new MyClass());
threads.add(t);
t.start();
}
for (Thread thread: threads) {
thread.join();
}
some more material
The following code uses an executor service to fix the number of threads that run at any time, it provides a Future object that also tells you when your thread has shutdown gracefully. They share a shutdown object as well. This offers you a bit more flexibility as the executor service can let you decide how many threads run at any one time gracefully.
First lets created a shared shutdown object that will notify all the threads it is time to shut down. There will be one instance of this and each thread will have a copy.
public static class Shutdown {
private boolean running;
public void shutdown() {
this.running = false;
}
public boolean isRunning() {
return running;
}
}
Next let me just create a dummy thread that does nothing more than sleep forever while it is running. Obviously you can simply replace this with your own thread to do something useful.
public static class MyClass implements Runnable {
final Shutdown shutdown;
public MyClass(Shutdown shutdown) {
this.shutdown = shutdown;
}
#Override
public void run() {
while (shutdown.isRunning()) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
System.out.println("Did not gracefully shut down");
}
}
System.out.println("Thread in program ended!");
}
}
}
Now for the main class which will run everything, this is where the magic happens.
public class Main {
public static void main(String[] args) {
//run exactly 10 threads at a time
ExecutorService executorService = Executors.newFixedThreadPool(10);
//this is how we shut it down
Shutdown globalShutdown = new Shutdown();
//start up the 10 threads
List<Future<?>> futures = new ArrayList<>();
for(int i = 0; i< 10; i++)
futures.add(executorService.submit(new MyClass(globalShutdown)));
//gracefully shut them down
globalShutdown.shutdown();
try {
//wait for them all to shutdown
for(Future<?> future : futures)
future.get();
} catch (InterruptedException e) {
throw new IllegalStateException("This should never happen");
} catch (ExecutionException e) {
throw new IllegalStateException("This should never happen");
}
//everything got shutdown!
}
in practice however you probably also want to handle the case where your thread may not end gracefully due to a bug. Rather than stall forever you might want to add a timeout and if that timeout is exceeded then simply forcibly terminate all remaining threads. To do that replace the above try-catch block with this.
try {
//wait for them all to shutdown
boolean timedout = false;
for(Future<?> future : futures) {
if( !timedout ) {
try {
future.get(30, TimeUnit.SECONDS);
} catch (TimeoutException e) {
timedout = true;
}
}
if(timedout) {
future.cancel(true);
}
}
} catch (InterruptedException | ExecutionException e) {
throw new IllegalStateException("This should never happen");
}
package threadShareResource1;
public class NonSynchro1 {
private int sum = 0;
public static void main(String[] args) {
NonSynchro1 n = new NonSynchro1();
n.task();
System.out.println(n.getSum());
}
public synchronized void sumAddOne(){
sum++;
}
public void task(){
for (int i = 0; i < 100; i++) {
new Thread(new Runnable(){
#Override
public void run() {
sumAddOne();
}
}).start();
/* try {
Thread.sleep(10);
} catch (InterruptedException e) {
e.printStackTrace();
} */
}
}
public int getSum() {
return sum;
}
}
Without the commented part of code, the program has data corruption, which is not 100 every time I run it. But I thought the synchronized keyword should acquires a lock on the sumAddOne method, which is the critical region of my program, allowing one thread accessing this method every time.
I've try to use ExecutorService as well, but it doesn't give 100 all the runs.
public void task(){
ExecutorService s = Executors.newCachedThreadPool();
for (int i = 0; i < 100; i++) {
s.execute(new Thread(new Runnable(){
#Override
public void run() {
sumAddOne();
}
}));
}
s.shutdown();
while(!s.isTerminated()){}
}
In Task(), you start 100 threads (which is a lot) and each one is to add 1 to sum.
But when Task is done all you know is that 100 threads are in some process of having started. You don't block before calling println(), so how do you know all the threads have completed?
The sleep probably "prevents the corruption" just because it gives the system time to finish launching all the threads.
Beyond that you are using Synchronized correctly. Any place multiple threads may write to the same variable you need it and, in general (simplifying), you don't need it if you are only reading.
Synchronised keyword is used correctly, the problem is that you are not waiting for the threads to finish. Here is a possible solution:
public class NonSynchro1 {
private static final ExecutorService executorService = Executors.newCachedThreadPool();
private int sum = 0;
public static void main(String[] args) {
NonSynchro1 n = new NonSynchro1();
n.task();
System.out.println(n.getSum());
executorService.shutdown();
}
public synchronized void sumAddOne() {
sum++;
}
public void task() {
List<Callable<Object>> callables = new ArrayList<>();
for (int i = 0; i < 100; i++) {
callables.add(() -> {
sumAddOne();
return null;
});
}
List<Future<Object>> futures;
try {
futures = executorService.invokeAll(callables);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
futures.forEach(future -> {
try {
future.get();
} catch (ExecutionException | InterruptedException e) {
throw new RuntimeException(e);
}
});
}
public int getSum() {
return sum;
}
}
First we create a list of callables - a list of functions that will be executed in parallel.
Then we invoke them on the executor service. newCachedThreadPool I have used here, by default has 0 threads, it will create as many as necessary to execute all passed callables, the threads will be killed after being idle for a minute.
Finally, in the for-each loop we resolve all futures. get() call will block until the function was executed by the executor service. It will also throw exception if it was thrown inside the function (without calling get() you would not see such exception at all).
Also, it is a good idea to shutdown the executor service when you want to terminate the program gracefully. In this case, it is just executorService.shutdown() at the end of main method. If you don't do this, the program will terminate after a minute when idle threads are killed. However, if different executor service, threads might not be killed when idle, in which case the program would never terminate.
Just for completeness sake: Here's a solution showing how the original program can be made to wait for all threads to finish by joining them:
for (Thread t : n.task())
try {
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
which requires task to return the threads it creates. In this case we don't need to complicate things with caching managers or collections: a simple array will do. Here's the complete class:
public class TestSynchro1 {
private int sum = 0;
public synchronized void sumAddOne() {
sum++;
}
public Thread[] task(int n) {
Thread[] threads = new Thread[n];
for (int i = 0; i < n; i++) {
(threads[i] = new Thread(new Runnable() {
#Override
public void run() {
sumAddOne();
}
})).start();
}
return threads;
}
public static void main(String[] args) {
TestSynchro1 n = new TestSynchro1();
for (Thread t : n.task(100))
try {
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println(n.sum);
}
}
I have come up with following thread 'halo' to make it connect to db (redis, in this case) and in the event that server fails, would wait for a second and try again. In my unit test class, method is executing, and not long after new thread starts, server will fail. But then this new thread 'halo' is immediately shut down. What am I doing wrong?
// almost infinitely large number of sets, interrupted by server seg-fault
// you gotta try company methods
Thread halo = new Thread(new Runnable() {
#Override
public void run() {
int count = 0;
while (count < Integer.MAX_VALUE) {
if (JedisPoolFactory.getStatus()) {
try {
for (int i = 0; i < 10000; i++) {
master.set(String.format("key_%d", count), String.format("value_%d", count));
System.out.println(master.get(String.format("key_%d", count)));
count++;
}
} catch (JedisConnectionException igr) {
try {
Thread.sleep(1000);
} catch (InterruptedException ignore) {}
}
} else {
try {
Thread.sleep(1000);
} catch (InterruptedException ignrod) {}
}
}
}
});
halo.start();
try {
master.debug(DebugParams.SEGFAULT());
halo.join();
} catch (JedisConnectionException ignored) {
} catch (InterruptedException igr) {}
thread joining should be done outside of exceptions, when master goes segfault it invokes jedisconnectionexception.
I have a method createThreads which spawns few new threads. Each of the newly created thread does some work. If I invoke the method `createThreads' in junit, how can i ensure that all the newly spawned threads have also completed successfully.
I am currently calling as below
#Test
public void test() {
createThreads(); // Does not wait until the newly created threads also finish.
}
public void createThreads()
{
ExecutorService executorService = Executors
.newFixedThreadPool(numThreads);
for (int i = 0; i < numThreads; i++) {
executorService.execute(new Runnable() {
#Override
public void run() {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("I have completed execution " + Thread.currentThread().getName());
}
});
}
Note that I cannot modify createThreads
a bit odd but..
you can probably get all the runing threads
through Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
then filter it to identify the thread from the executor service.
then do a .join() on each of those threads.
as i said, a bit odd but it should fit your needs ...
try running this, you'll see that they are quite easy to identify :
public static void main(String[] args) {
int nb = 3;
ExecutorService executorService = Executors.newFixedThreadPool(nb);
for (int i = 0; i < nb; i++) {
executorService.execute(new Runnable() {
#Override
public void run() {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("I have completed execution " + Thread.currentThread().getName());
}
});
}
Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
for (Thread t : threadSet) {
System.out.println(t.getName());
}
}
sorry for a 2nd answer not possible to add such a long code in comment