I have encountered a problem when I want to do a time counting.
Basicly the problem is like this: there is a class A, which initiates a private thread in itself, and I have a instant of A in my class B, and in the main method of B I invoked some methods of A and want to test the time to run these methods.
A a = new A();
//start time counter
for (int i = 0; i < 10; i++){ invoke a.method() that takes some time}
//end time counter and prints the time elapsed
but by doing so the method in the for loop will running in a seperate thread in A and the prints method in the last line would probably be executed before the loop ends. So I want to access the thead in a and invokes a join() to wait until all stuff in the for loop get finished. Could you help me figure how to achieve this? Any ideas would be greatly appreciated.
List All Threads and their Groups
public class Main
{
public static void visit(final ThreadGroup group, final int level)
{
final Thread[] threads = new Thread[group.activeCount() * 2];
final int numThreads = group.enumerate(threads, false);
for (int i = 0; i < numThreads; i++)
{
Thread thread = threads[i];
System.out.format("%s:%s\n", group.getName(), thread.getName());
}
final ThreadGroup[] groups = new ThreadGroup[group.activeGroupCount() * 2];
final int numGroups = group.enumerate(groups, false);
for (int i = 0; i < numGroups; i++)
{
visit(groups[i], level + 1);
}
}
public static void main(final String[] args)
{
ThreadGroup root = Thread.currentThread().getThreadGroup().getParent();
while (root.getParent() != null)
{
root = root.getParent();
}
visit(root, 0);
}
}
Based on your edits, you might can find out what group and name the thread is and get a reference to it that way and do what you need to do.
For your own code in the future
You want to look at ExecutorCompletionService and the other thread management facilities in java.util.concurrent. You should not be managing threads manually in Java anymore, pretty much every case you can imagine is handled one or more of the ExecutorService implementations.
Related
I want to create a thread pool code in java where task will wait until the function called in task in completed. I have gone through many examples but cannot achieve my goal so far.
public class ThreadController {
public static void main(String args[]) {
ExecutorService service = Executors.newFixedThreadPool(5);
List<String> list = new ArrayList<String>();
list.add("john");
list.add("reck");
list.add("moni");
list.add("sasha");
list.add("pely");
for (int p = 0; p < 100; p++) {
for (int r = 0; r < 5; r++) {
Task task = new Task(list.get(r));
service.submit(task);
}
}
}
}
final class Task implements Runnable {
private String taskSimNo;
public Task(String no) {
this.taskSimNo = no;
}
public void run() {
Initiate.startingInitiate(this.taskSimNo);
}
}
The complete idea of this function is to call a function processing() which is a method of mainMethod class. So i want to run 10 threads in parallel but, 11th task should only start when any of the 10 tasks gets finish executing so I need to implement wait function to let the task complete. Any suggestions please.
Your synchronized block with task.wait() does nothing but blocking the loop since there is at no point a call to the notify method.
So you first of all need to remove that.
Secondly, your processing method does not benefit from any multi-threading, because it is called within the constructor and object creation is done by the main thread.
Solution is to move your processing method down inside the run method.
You correctly assigned a limit to the thread pool allowing 10 concurrent tasks to run.
Note: Order is not ensured! Task 11 might run before task 8 for example.
This question is NOT about how to use a ThreadLocal. My question is
about the side effect of the ForkJoinPool continuation of ForkJoinTask.compute() which breaks the ThreadLocal contract.
In a ForkJoinTask.compute(), I pull an arbitrary static ThreadLocal.
The value is some arbitrary stateful object but not stateful beyond the end of the compute() call. In other words, I prepare the threadlocal object/state, use it, then dispose.
In principle you would put that state in the ForkJoinTasK, but just assume this thread local value is in a 3rd party lib I cannot change. Hence the static threadlocal, as it is a resource that all tasks instances will share.
I anticipated, tested and proved that simple ThreadLocal gets initialized only once, of course. This means that due to thread continuation beneath the ForkJoinTask.join() call, my compute() method can get called again before it even exited. This exposes the state of the object being used on the previous compute call, many stackframes higher.
How do you solve that undesirable exposure issue?
The only way I currently see is to ensure new threads for every compute() call, but that defeats the F/J pool continuation and could dangerously explode the thread count.
Isn't there something to do in the JRE core to backup TL that changed since the first ForkJoinTask and revert the entire threadlocal map as if every task.compute is the first to run on the thread?
Thanks.
package jdk8tests;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.ForkJoinWorkerThread;
import java.util.concurrent.RecursiveTask;
import java.util.concurrent.atomic.AtomicInteger;
public class TestForkJoin3 {
static AtomicInteger nextId = new AtomicInteger();
static long T0 = System.currentTimeMillis();
static int NTHREADS = 5;
static final ThreadLocal<StringBuilder> myTL = ThreadLocal.withInitial( () -> new StringBuilder());
static void log(Object msg) {
System.out.format("%09.3f %-10s %s%n", new Double(0.001*(System.currentTimeMillis()-T0)), Thread.currentThread().getName(), " : "+msg);
}
public static void main(String[] args) throws Exception {
ForkJoinPool p = new ForkJoinPool(
NTHREADS,
pool -> {
int id = nextId.incrementAndGet(); //count new threads
log("new FJ thread "+ id);
ForkJoinWorkerThread t = new ForkJoinWorkerThread(pool) {/**/};
t.setName("My FJThread "+id);
return t;
},
Thread.getDefaultUncaughtExceptionHandler(),
false
);
LowercasingTask t = new LowercasingTask("ROOT", 3);
p.invoke(t);
int nt = nextId.get();
log("number of threads was "+nt);
if(nt > NTHREADS)
log(">>>>>>> more threads than prescribed <<<<<<<<");
}
//=====================
static class LowercasingTask extends RecursiveTask<String> {
String name;
int level;
public LowercasingTask(String name, int level) {
this.name = name;
this.level = level;
}
#Override
protected String compute() {
StringBuilder sbtl = myTL.get();
String initialValue = sbtl.toString();
if(!initialValue.equals(""))
log("!!!!!! BROKEN ON START!!!!!!! value = "+ initialValue);
sbtl.append(":START");
if(level>0) {
log(name+": compute level "+level);
try {Thread.sleep(10);} catch (InterruptedException e) {e.printStackTrace();}
List<LowercasingTask> tasks = new ArrayList<>();
for(int i=1; i<=9; i++) {
LowercasingTask lt = new LowercasingTask(name+"."+i, level-1);
tasks.add(lt);
lt.fork();
}
for(int i=0; i<tasks.size(); i++) { //this can lead to compensation threads due to l1.join() method running lifo task lN
//for(int i=tasks.size()-1; i>=0; i--) { //this usually has the lN.join() method running task lN, without compensation threads.
tasks.get(i).join();
}
log(name+": returning from joins");
}
sbtl.append(":END");
String val = sbtl.toString();
if(!val.equals(":START:END"))
log("!!!!!! BROKEN AT END !!!!!!! value = "+val);
sbtl.setLength(0);
return "done";
}
}
}
I don't believe so. Not in general and specially not for the ForkJoinTask where tasks are expected to be pure functions on isolated objects.
Sometimes it is possible to change the order of the task to fork and join at the beginning and before the own task's work. That way the subtask will initialize and dispose the thread-local before returning. If that is not possible, maybe you can treat the thread-local as a stack and push, clear, and restore the value around each join.
I have searched the web for a while now trying to resolve this issue, but have had no success.
In my application, I have a large set of messages that I am attempting to encrypt using a basic commutative encryption scheme. Since the sets are large numbers of BigIntegers, I am attempting to multithread the encryptions to increase performance.
Basically, I take the large set of messages and split it up into subsets that are passed to an encryption thread to do a subset of the encryptions. Then I attempt to extract each subset and aggregate them into the original large set after the threads have all done their parts.
When I iterate over the threads and pull out each of their encryptions, the error is occurring when I attempt to actually addAll of the encryptions to the list of all encryptions and the error it throws is the java.util.ConcurrentModificationException error.
I have attempted to use synchronization, but it isn't helping.
Here is the function call:
protected Set<BigInteger> multiEncrypt(BigInteger key, HashSet<BigInteger> messageSet) {
ArrayList<BigInteger> messages = new ArrayList<BigInteger>(messageSet);
Set<BigInteger> encryptions = Collections.synchronizedSet(new HashSet<BigInteger>());
int cores = Runtime.getRuntime().availableProcessors();
int numMessages = messages.size();
int stride = numMessages/cores;
//create all the threads and run them
ArrayList<EncryptThread> threads = new ArrayList<EncryptThread>();
for (int thread = 0; thread < cores; thread++) {
int start = thread*stride;
//don't want to go over the end
int stop = ((thread+1)*stride >= messages.size()) ? messages.size()-1 : (thread+1)*stride;
List<BigInteger> subList = messages.subList(start, stop);
EncryptThread t = new EncryptThread(encryptionScheme.getPrime(), key, subList);
t.start();
threads.add(t);
}
//pull out the encryptions
synchronized(encryptions){
for (int i=0; i < threads.size()-1; i++) {
EncryptThread thread = threads.get(i);
ArrayList<BigInteger> these = thread.getEncryptions();
encryptions.addAll(these); //<-- Erroring Here
thread.finish();
}
}
And here are the relevant parts of the EncryptThread class I wrote to do the encryptions:
/**
* Constructor
*/
public EncryptThread(BigInteger prime, BigInteger key, List<BigInteger> messages) {
//need a new encryption scheme object for each thread
encryptionScheme = new EncryptionScheme(prime);
encryptions = new ArrayList<BigInteger>();
this.key = key;
this.messages = messages;
wait = true;
}
#Override
public void run() {
encryptMessages(key, messages);
while(wait);
}
/**
* Used to encrypt a set of messages
* #param key
* #param messages
* #return
*/
public void encryptMessages(BigInteger key, List<BigInteger> messages) {
System.out.println("Encrypting stuff");
for (BigInteger m : messages) {
BigInteger em = encryptionScheme.encrypt(key, m);
encryptions.add(m);
}
}
public ArrayList<BigInteger> getEncryptions() {
return encryptions;
}
//call this after encryptions have been pulled to let the thread finish
public void finish() {
wait = false;
}
}
I am not new to Java, but I am new to multi threading in java and so I would appreciate any and all advice. Thanks in advance!
EDIT: As per the suggestions, I added a simple locking mechanism to the EncryptThread class, which makes the thread wait to return the encryptions until they are all done and it works now.
public void encryptMessages(BigInteger key, List<BigInteger> messages) {
System.out.println("Encrypting stuff");
this.lock = true;
for (BigInteger m : messages) {
BigInteger em = encryptionScheme.encrypt(key, m);
//deals with when we have to mark chaff at S2
if (shift) {
em.shiftLeft(1);
if(shiftVal != 0) em.add(BigInteger.ONE);
}
encryptions.add(m);
}
this.lock = false;
}
public ArrayList<BigInteger> getEncryptions() {
while(lock);
return encryptions;
}
EDIT #2 So I ended up using a solution which was suggested to me by someone from my lab. I got rid of the lock and wait booleans, and the finish() function in the EncryptThread class, and instead added a simple thread.join() loop between the start and getEncryption loops:
//create all the threads
ArrayList<EncryptThread> threads = new ArrayList<EncryptThread>();
for (int thread = 0; thread < cores; thread++) {
int start = thread*stride;
//don't want to go over the end
int stop = ((thread+1)*stride >= messages.size()) ? messages.size()-1 : (thread+1)*stride;
List<BigInteger> subList = messages.subList(start, stop);
EncryptThread t = new EncryptThread(encryptionScheme.getPrime(), key, subList, shiftVal);
t.start();
threads.add(t);
}
//wait for them to finish
for( EncryptThread thread: threads) {
try {
thread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
//pull out the encryptions
for (int i=0; i < threads.size()-1; i++) {
EncryptThread thread = threads.get(i);
encryptions.addAll(thread.getEncryptions());
}
I think my main confusion was that I thought a thread class couldn't have its methods called on it after it had finished running. But the above works fine.
ConcurrentModificationException happens when you modify a Collection while you're iterating over it. It has very little to do with multi threading, since you can easily create a single threaded example:
ArrayList<String> myStrings = new ArrayList<>();
myStrings.add("foo");
myStrings.add("bar");
for(String s : myStrings) {
myStrings.add("Hello ConcurrentModificationException!");
If you look at the documentation on List's addAll, it says the following:
Appends all of the elements in the specified collection to the end of this list, in the order that they are returned by the specified collection's iterator (optional operation). The behavior of this operation is undefined if the specified collection is modified while the operation is in progress. (Note that this will occur if the specified collection is this list, and it's nonempty.)
You can see your List being modified while addAll is using it's iterator in your encryptMessages method that one of your threads you spawned is currently executing.
for (BigInteger m : messages) {
BigInteger em = encryptionScheme.encrypt(key, m);
encryptions.add(m); // <-- here
}
I didn't look through all of your code fully, but some of the stuff here is not thread safe. You might do well using a CopyOnWriteArrayList instead of a regular ArrayList to avoid the ConcurrentModificationException, that's if, you are okay with not having everything added to the list in the addAll call, if you aren't, you also then need to be waiting for the threads to finish. You probably want to instead just use tasks with an ExecutorService. There's other improvements to make as well probably.
In additional, the goto book everyone mentions to learn how to write thread safe programs in Java is Concurrency in Practice, I'd recommend that if you are new to concurrency in Java.
you are starting your threads here.
for (int thread = 0; thread < cores; thread++) {
int start = thread*stride;
//don't want to go over the end
int stop = ((thread+1)*stride >= messages.size()) ? messages.size()-1 : (thread+1)*stride;
List<BigInteger> subList = messages.subList(start, stop);
EncryptThread t = new EncryptThread(encryptionScheme.getPrime(), key, subList);
t.start();
threads.add(t);
}
Well. Then you have to wait for all threads to get complete , before start aggregate in this block.
//pull out the encryptions
synchronized(encryptions){
for (int i=0; i < threads.size()-1; i++) {
EncryptThread thread = threads.get(i);
ArrayList<BigInteger> these = thread.getEncryptions();
encryptions.addAll(these); //<-- Erroring Here
thread.finish();
}
}
you are blocking threads which are accessing encryptions only. but the thread you have created is not accessing the set . mean time it will keep on add to its own array List these . So when you call encryptions.addAll(these); these is accessed by two threads ( thread owning encryptions and the thread owning these
And the other answers provided detail about why Concurrent exception in addAll.
You have to wait until all the threads get complete thier work.
You can do this using ExecutorService
Change your starting thread as
ExecutorService es = Executors.newFixedThreadPool(cores);
for(int i=0;i<5;i++)
es.execute(new Runnable() { /* your task */ }); //EncryptThread instance
es.shutdown();
boolean finshed = es.awaitTermination(1, TimeUnit.MINUTES);
Then process your adding back process.
ExecutorService es = Executors.newFixedThreadPool(cores);
for (int thread = 0; thread < cores; thread++) {
int start = thread*stride;
//don't want to go over the end
int stop = ((thread+1)*stride >= messages.size()) ? messages.size()-1 : (thread+1)*stride;
List<BigInteger> subList = messages.subList(start, stop);
EncryptThread t = new EncryptThread(encryptionScheme.getPrime(), key, subList);
es.execute(t);
threads.add(t);
}
es.shutdown();
boolean finshed = es.awaitTermination(1, TimeUnit.MINUTES);
//pull out the encryptions
synchronized(encryptions){
for (int i=0; i < threads.size()-1; i++) {
EncryptThread thread = threads.get(i);
ArrayList<BigInteger> these = thread.getEncryptions();
encryptions.addAll(these); //<-- Erroring Here
thread.finish();
}
}
Assumed, your EncryptThread is Thread right now. you might need to change to implements Runnable. and no other change in getEncryptions
in my program I need to have multiple threads use and edit the same variable, but it doesn't seem to be working. Here is an example of what I mean, this would be my main class.
public class MainClass {
public static int number = 0;
public static String num = Integer.toString(number);
public static void main(String[] args) {
Scanner in = new Scanner(System.in);
System.out.println("Enter number of threads.");
int threads = in.nextInt();
for (int n = 1; n <= threads; n++) {
java.lang.Thread t = new Thread();
t.start();
}
}
}
This would be my Thread class:
public class Thread extends java.lang.Thread
{
public void run()
{
MainClass.number++;
System.out.println("Thread started");
System.out.println(MainClass.num);
}
}
I wrote this code on the spot, so there may be some errors, but thats ok. My program basically needs to do something like this, but instead of printing the number plus 1 every time, all the threads simply print the same number, 0, multiple times. Please help me, thanks.
In my program I need to have multiple threads use and edit the same variable, but it doesn't seem to be working...
Anytime multiple threads are updating the same variable you need to worry about memory synchronization. One of the ways that threads get high performance is because each thread utilizes the local CPU memory cache and so may be working with stale copies of variables. You need to use the synchronized or volatile keywords to force the thread's cache to write any updates to central storage or update its cache from central.
Although this takes care of memory synchronization, it doesn't necessarily protect you from race conditions. It is also important to realize that ++ is actually 3 operations: get the current value, increment it, and store it back again. If multiple threads are trying to do this, there are thread race-conditions which can cause the ++ operations to be missed.
In this case, you should use the AtomicInteger class which wraps a volatile int field. It gives you methods like incrementAndGet() which do the job of incrementing that field in a thread-safe manner.
public static AtomicInteger number = new AtomicInteger(0);
...
MainClass.number.incrementAndGet();
Multiple threads can then be incrementing the same variable safely.
Here You go...
import java.util.Scanner;
import java.util.concurrent.atomic.AtomicInteger;
public class UpdateVariables
{
static int num = 0;
public static AtomicInteger atomicInteger = new AtomicInteger(num);
#SuppressWarnings("resource")
public static void main(String args[])
{
Scanner userInput = new Scanner(System.in);
System.out.println("Enter Number of Threads: ");
int getThreadNumber = userInput.nextInt();
for(int i = 0; i < getThreadNumber; i++)
{
PrintThread p = new PrintThread();
p.start();
}
}
}
class PrintThread extends Thread
{
public void run()
{
System.out.println("Thread Started: ");
System.out.println(UpdateVariables.atomicInteger.incrementAndGet());
}
}
Using java, how can I create 50 threads to make a simple http get request to a particular URL?
I want each thread to make maybe 100-1k requests.
Is it possible to guarantee that all these threads connect at the same time?
I basically want something similiar to Apache bench, but written in java so I can learn some java along the way.
So the input would be:
1. # of requests in total
2. # of threads to use
3. url to make a request with
Update
I guess to keep track of request statistics i.e. how long a particular request took on average I would need a global collection that is thread-safe?
Here is some (incomplete) code:
public class Test {
private static int REQUESTS;
private static int NUM_THREADS;
private static String URL;
private static ArrayList<Statistic> result = new ArrayList<Statistic>();
private static class ThreadTask implements Runnable {
private int tid;
public ThreadTask(int tid) {
this.tid = tid;
}
#Override
public void run() {
Statistic stat = new Statistic();
for(int i = 0; i < REQUESTS; i++) {
// make request
// add results to stat
}
result.add(tid, stat); // no need to lock because each
// thread writes to a dedicated index
}
}
public static void main(String[] args) {
// take command line arguments
REQUESTS = Integer.parseInt(args[0]);
NUM_THREADS = Integer.parseInt(args[1]);
URL = args[2];
Thread[] threads = new Thread[NUM_THREADS];
// start threads
for(int i = 0; i < NUM_THREADS; i++) {
threads[i] = new Thread(new ThreadTask(i));
threads[i].start();
}
// wait for threads to finish
for(int i = 0; i < NUM_THREADS; i++) {
try {
threads[i].join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
Class Statistic is something defined by you to collect whatever statistics you want.
Of course, many improvements can be suggested, this is just what I wrote in 5 minutes. :) Hope it helps.
You want to use a combination of a ThreadPoolExecutor for scheduling threads and a CyclicBarrier for activating all threads at the same time. Both classes are in the java.util.concurrency package.
I have used JMeter for this purpose. http://jmeter.apache.org.
There is a little bit of learning curve for the tool.
In Jmeter, threadgroup allows you to use number of threads, ramp-up period and loop count.
You can create an HTTP request and view results using view results in a Tree or Table. Hope this helps.