Develop Reference

Java FixedThreadPool with resources per thread?

This is a pseudocode version of my current working code:
public class DataTransformer {
private final boolean async = true;
private final ExecutorService executorService = Executors.newSingleThreadExecutor();
public void modifyAsync(Data data) {
if (async) {
executorService.submit(new Runnable() {
#Override
public void run() {
modify(data);
}
});
} else {
modify(data);
}
}
// This should actually be a variable inside modify(byte[] data)
// But I reuse it to avoid reallocation
// This is no problem in this case
// Because whether or not async is true, only one thread is used
private final byte[] temp = new byte[1024];
private void modify(Data data) {
// Do work using temp
data.setReady(true); // Sets a volatile flag
}
}
Please read the comments. But now I want to use Executors.newFixedThreadPool(10) instead of Executors.newSingleThreadExecutor(). This is easily possible in my case by moving the field temp inside modify(Data data), such that each execution has it's own temp array. But that's not what I want to do because i want to reuse the array if possible. Instead I want for each of the 10 threads a temp array. What's the best way to achieve this?

As static variable is shared between all Threads, so you could declare as static. But if you want to use different values then either use Threadlocal or use different object.
With ThreadLocal you could do :
ThreadLocal<byte[]> value = ThreadLocal.withInitial(() -> new byte[1024]);
You could also use object like this:
public class Test {
public static void main(String[] args) {
try {
Test test = new Test();
test.test();
} catch (Exception e) {
e.printStackTrace();
}
}
class Control {
public volatile byte[] temp = "Hello World".getBytes();
}
final Control control = new Control();
class T1 implements Runnable {
#Override
public void run() {
String a = Arrays.toString(control.temp);
System.out.println(a);
}
}
class T2 implements Runnable {
#Override
public void run() {
String a = Arrays.toString(control.temp);
System.out.println(a);
}
}
private void test() {
T1 t1 = new T1();
T2 t2 = new T2();
new Thread(t1).start();
new Thread(t2).start();
}
}

How to make sure each thread works on different process object to avoid thread safety issue?

I have two process as shown below. Each of my process has run and shutdown method
Process processA = new ProcessA("processA", getProcessAProperties());
Process processB = new ProcessB("processB", getProcessBProperties());
I want to have different thread pool configuration for each Process so that ProcessA run in its own thread pool and ProcessB run in its own thread pool independent of each other.
And I cannot share Process object between each threads of its own thread pool.
Below is a simple example of how I am running my ProcessA with its own thread pool. There are three threads and each thread gets its own Process object to work on. Now I want to extend this in a more generic way so that it can work for both my processes as shown above.
public static void main(String[] args) {
int numberOfThreads = 3;
ExecutorService executor = Executors.newFixedThreadPool(numberOfThreads);
final List<Process> processes = new ArrayList<>();
for (int i = 0; i < numberOfThreads; i++) {
// each thread works on different Process object
Process processA = new ProcessA("processA", getProcessAProperties());
processes.add(processA);
executor.submit(processA);
}
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
for (Process process : processes) {
process.shutdown();
}
executor.shutdown();
try {
executor.awaitTermination(5000, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace;
}
}
});
}
So I created a Process handler to solve the above problem in a generic way but there is a thread safety issue here:
public final class ProcessHandler {
private final ExecutorService executorServiceProcess;
private final Process process;
private final Thread shutdownHook = new Thread() {
#Override
public void run() {
process.shutdown();
executorServiceProcess.shutdown();
}
};
// in this constructor my code is reusing the same
// process instance for each thread in the pool
// which is a problem for my application, how to fix this?
public ProcessHandler(Process process, int poolSize) {
this.executorServiceProcess = Executors.newFixedThreadPool(poolSize);
this.process = process;
Runtime.getRuntime().addShutdownHook(shutdownHook);
for (int i = 0; i < poolSize; i++) {
executorServiceProcess.submit(process);
}
}
public void shutdown() {
Runtime.getRuntime().removeShutdownHook(shutdownHook);
shutdownHook.start();
try {
shutdownHook.join();
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
}
And this is the way my main method looks now:
public static void main(String[] args) {
Process processA = new ProcessA("processA", getProcessAProperties());
Process processB = new ProcessB("processB", getProcessBProperties());
// processA will run with three threads in its own thread pool
ProcessHandler processHandlerA = new ProcessHandler (processA, 3);
// processB will run with two threads in its own thread pool
ProcessHandler processHandlerB = new ProcessHandler (processB, 2);
// now I can call shutdown on them
processHandlerA.shutdown();
processHandlerB.shutdown();
}
As you can see in my above ProcessHandler constructor, I am reusing the same process instance for each thread in the pool which is not what I want to do. I want each thread to work on a different instance of Process object just like what I have in my first main method for ProcessA, each thread is working on different Process object.
What is the best way to solve this design problem? I am also open in redesigning my ProcessHandler to solve this problem in a right way.

Maybe try something like this :
// Replace Process process by a list of Process
List<Process> processes = new ArrayList<Process>();
private final Thread shutdownHook = new Thread() {
#Override
public void run() {
for (Process process : processes)
process.shutdown();
executorServiceProcess.shutdown();
}
};
public ProcessHandler(Process process, int poolSize) {
this.executorServiceProcess = Executors.newFixedThreadPool(poolSize);
Runtime.getRuntime().addShutdownHook(shutdownHook);
for (int i = 0; i < poolSize; i++) {
// Get a deep copy of the process
Process p = process.clone();
processes.add(p);
executorServiceProcess.submit(p);
}
}
Another way, without cloning method, is to abstract the Process and add two constructor based on these,
Don't forget to adapt your code based on these elements
public abstract class AProcess extends Process {
private String name;
private Properties properties;
public AProcess(String name, Properties properties)
{
this.setName(name);
this.setProperties(properties);
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public Properties getProperties() {
return properties;
}
public void setProperties(Properties properties) {
this.properties = properties;
}
}
You now have to implements your ProcessA and ProcessB :
public class ProcessA extends AProcess {
public ProcessA(String name, Properties properties)
{
super(name, properties);
}
}
Now create your ProcessHandler :
public final class ProcessHandler {
private final ExecutorService executorServiceProcess;
private final List<AProcess> processes = new ArrayList<AProcess>();
private final Thread shutdownHook = new Thread() {
#Override
public void run() {
for (AProcess process : processes)
process.shutdown();
executorServiceProcess.shutdown();
}
};
public ProcessHandler(ProcessA process, int poolSize) {
this.executorServiceProcess = Executors.newFixedThreadPool(poolSize);
Runtime.getRuntime().addShutdownHook(shutdownHook);
for (int i = 0; i < poolSize; i++) {
ProcessA p = new ProcessA(process.getName(), process.getProperties());
processes.add(p);
executorServiceProcess.submit(p);
}
}
public ProcessHandler(ProcessB process, int poolSize) {
this.executorServiceProcess = Executors.newFixedThreadPool(poolSize);
Runtime.getRuntime().addShutdownHook(shutdownHook);
for (int i = 0; i < poolSize; i++) {
ProcessB p = new ProcessB(process.getName(), process.getProperties());
processes.add(p);
executorServiceProcess.submit(p);
}
}
public void shutdown() {
Runtime.getRuntime().removeShutdownHook(shutdownHook);
shutdownHook.start();
try {
shutdownHook.join();
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
}
And use like that ProcessHandler processHandlerA = new ProcessHandler(processA, 3);

How to run two process independently with their own thread pool and other configuration?

I have two processes as shown below in the List.
public static final ImmutableList<String> processes = ImmutableList.of("processA", "processB");
Now for each process, I have a different Properties object and there is no relation and dependency between those two process at all. They are independent of each other.
I wrote a code that works with only one process for now and I need to extend my design efficiently so that it can work for two process. Each process should have it's own Thread Pool configuration. For example, may be I want to run processA with three threads and processB with two threads.
public class ProcessA implements Runnable {
private final Properties props;
private final String processName;
public ProcessA(String processName, Properties props) {
this.processName = processName;
this.props = props;
}
#Override
public void run() {
List<String> clients = getClients(processName);
try {
// .. some code here which does processing
// calling some classes here as well
} catch (Exception ex) {
// log error
} finally {
// close processA here
}
}
public void shutdown() {
// shutdown processA here
}
}
Below is my main class where I execute my processA. In general I will be executing my both the processes from below class only.
#Singleton
#DependencyInjectionInitializer
public class Initializer {
private final ExecutorService executorServiceProcessA = Executors.newFixedThreadPool(3);
private final List<ProcessA> processAList = new ArrayList<>();
public Initializer() {
logger.logInfo("initializing here called.");
TestUtils.getInstance().initializeData();
}
// this is the entrance point for my code
#PostConstruct
public void postInit() {
for (int i = 0; i < 3; i++) {
ProcessA process =
new ProcessA("processA", properties);
processAList.add(process);
executorServiceProcessA.submit(process);
}
}
#PreDestroy
public void shutdown() {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
for (ProcessA process : processAList) {
process.shutdown();
}
executorServiceProcessA.shutdown();
try {
executorServiceProcessA.awaitTermination(1000, TimeUnit.MILLISECONDS);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
});
}
}
I will have only one Initializer class which will execute both my process from its postInit method and then shutdown both the process from its shutdown method.
Problem Statement:-
Now how can I extend my design so that it can work with two processes efficiently? Each process should have its own thread pool, its own properties object and I should be able to execute them from postInit method and then shutdown later from shutdown method.
What is the best and efficient way to do that?

There is a clear violation of DRY (Don't Repeat Yourself) principle in your code.
In other words, there is lots of Boilerplate code in your Process and Main classes which can be eliminated by using the abstract classes (or use interfaces if you use Java8 with default methods).
So I have created two Process and ProcessHandler abstract classes to reuse the code which is common across each process and process handling.
So now, you can define ProcessA, ProcessB classes which extend Process and ProcessHandlerA, ProcesshandlerB which extend ProcessHandler class.
The key point is this solution can be extended to any number of Process **, i.e., this follows **Open/Closed principle of OOP.
You can refer the below code with comments:
Process class (abstract):
public abstract Process implements Runnable {
private final Properties props;
private final String processName;
public Process(String processName, Properties props) {
this.processName = processName;
this.props = props;
}
//this can also be a non abstract (reusable) method
// to eliminate boiler plate code (if any)
public abstract void shutdown();
}
ProcessA class:
public class ProcessA extends Process {
public ProcessA(String processName, Properties props) {
super(processName, props);
}
#Override
public void run() {
//add run code here
}
#Override
public void shutdown() {
//shut down code
}
}
Process B class:
//Similar to ProcessA with specific details of B
ProcessHandler class (abstract):
public abstract class ProcessHandler {
private final ExecutorService executorServiceProcess;
private final List<Process> processList;
private int poolSize;
protected ProcessHandler(int poolSize) {
executorServiceProcess = Executors.newFixedThreadPool(poolSize);
processList = new ArrayList<>();
this.poolSize = poolSize;
}
public void postInit(Process process) {
for (int i = 0; i < poolSize; i++) {
processList.add(process);
executorServiceProcess.submit(process);
}
}
public void shutdown() {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
for (Process process : processList) {
process.shutdown();
}
executorServiceProcess.shutdown();
try {
executorServiceProcess.
awaitTermination(1000, TimeUnit.MILLISECONDS);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
});
}
}
ProcessHandlerA class:
public class ProcessHandlerA extends ProcessHandler {
public ProcessHandlerA() {
super(3);//configure pool size properly w.r.to ProcessA requirements
}
public void postInit() {
ProcessA processA = new ProcessA("processA", properties);
super(processA);
}
public void shutdown() {
super.shutdown();
}
}
ProcessHandlerB class:
//Similar to ProcessHandlerA with specific details for B

How can i know threads jobs are done?

In class B how can i know jobs of threads are finished? In after properties some worker are running. In class B, I need to know if worker are done?
public class A implements InitializingBean{
public void method1(){
...
}
#Override
public void afterPropertiesSet() throws Exception {
System.out.print("test after properties set");
// send threads to executorService
ExecutorService executorService = Executors
.newFixedThreadPool(4);
for (int i = 0; i < 4; i++) {
Worker worker = new Worker();
executorService.submit(worker);
}
}
}
public class Worker implements Callable<Void>{
#Override
public void call(){
...
}
}
public class B{
public void methodB(){
A a = new A();
a.method1();
///Here How can i know the job of the workers are finished?
}
}

Use a listener/callback pattern to have the thread report completion to a listener. This simple example should show the process:
public interface ThreadCompleteListener {
void workComplete();
}
public class NotifyingThread extends Thread {
private Set<ThreadCompleteListener> listeners;
// setter method(s) for adding/removing listeners to go here
#Override
public void run() {
// do stuff
notifyListeners();
}
private void notifyListeners() {
for (ThreadCompleteListener listener : listeners) {
listener.workComplete(); // notify the listening class
}
}
}
in your listening class:
NotifyingThread t = new NotifyingThread();
t.addListener(new ThreadCompleteListener() {
void workComplete() {
// do something
}
});
t.start();

You could use a Future implementation for your thread. It provides a Future#isDone()
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html#isDone()

In general, it is usually more useful to be notified via a callback when jobs complete. However, since others have posted answers which follow that model, I'll instead post a solution that simply allows you to poll and ask whether the jobs are finished, in case this is what fits the needs of your application better.
public static interface InitializingBean{
public void afterPropertiesSet() throws Exception;
}
public static class A implements InitializingBean{
private List<Future<Void>> submittedJobs = Collections.synchronizedList(new ArrayList<Future<Void>>());
public void method1(){
//do stuff
}
#Override
public void afterPropertiesSet() throws Exception {
System.out.print("test after properties set");
// send threads to executorService
ExecutorService executorService = Executors
.newFixedThreadPool(4);
synchronized (submittedJobs) {
for (int i = 0; i < 4; i++) {
Worker worker = new Worker();
submittedJobs.add(executorService.submit(worker));
}
}
}
/**
* Allows you to poll whether all jobs are finished or not.
* #return
*/
public boolean areAllJobsFinished(){
synchronized (submittedJobs) {
for(Future<Void> task : submittedJobs){
if(!task.isDone()){
return false;
}
}
return true;
}
}
}
public static class Worker implements Callable<Void>{
#Override
public Void call(){
//do worker job
return null; //to satisfy compiler that we're returning something.
}
}
public static class B{
public void methodB(){
A a = new A();
a.method1();
if(a.areAllJobsFinished()){
System.out.println("Congrats, everything is done!");
} else {
System.out.println("There's still some work being done :-(");
}
}
}

If you'd like to wait in that thread that starts the ExecutorService, you can actually use the awaitTermination method.
At the end of you afterPropertiesSet method, you should add:
executorService.shutdown();
After this you then add:
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS)
This causes the thread to wait for all the executorService's tasks to be done and then continues. So place any code you want to execute after the call to awaitTermination.

How to get the output stream from a thread

I currently have several runnable classes, each printing a string upon completion using System.out.println().
In the main() I execute them using a ExecutorService ,executor.execute() for each of them.
I am wondering after executing those threads, how to get the output stream from them for future use ?
Pretty much like using .getInputStream for processes but there's no such method in the Thread class. Thanks!
There's a class which implements runnable interface like this:
public class A implements Runnable {
public void run() {
System.out.println(5); //this thread always print out number 5
}
}
and in the main function I need to get the printed number and store it
public static void main(String[] args) {
ExecutorService ThreadPool = Executors.newFixedThreadPool(1);
ThreadPool.execute(new A()); //This statement will cause the thread object A
//to print out number 5 on the screen
ThreadPool.shutdown();
......
}
Now I need to get the printed number 5 and store it into, say an integer variable.

I think below code will satisfy your requirement.
class MyCallable implements Callable<InputStream>
{
#Override
public InputStream call() throws Exception {
//InputStream inputStreamObject = create object for InputStream
return inputStreamObject;
}
}
class Main
{
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(5);
List<Future<InputStream>> list = new ArrayList<Future<InputStream>>();
for (int i = 0; i < 25; i++) {
Callable<InputStream> worker = new MyCallable();
Future<InputStream> submit = executor.submit(worker);
list.add(submit);
}
InputStream inputStreamObject = null;
for (Future<InputStream> future : list) {
try {
inputStreamObject = future.get();
//use inputStreamObject as your needs
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
executor.shutdown();
}
}

Runnable and Callable in thread:
runnable interface has a method public abstract void run(); void - which means after completing run method, it will not return anything. Callable<V> interface has a method V call() throws Exception; which means after completing call method, it will return Object V that is parametrized as
public class Run_Vs_Call {
public static void main(String...args){
CallableTask call = new CallableTask();
RunnableTask run = new RunnableTask();
try{
FutureTask<String> callTask = new FutureTask<String>(call);
Thread runTask = new Thread(run);
callTask.run();
runTask.start();
System.out.println(callTask.get());
}catch(Exception e){
e.printStackTrace();
}
}
public static class CallableTask implements Callable<String>{
public String call( ){
String stringObject = "Inside call method..!! I am returning this string";
System.out.println(stringObject);
return stringObject;
}
}
public static class RunnableTask implements Runnable{
public void run(){
String stringObject = "Inside Run Method, I can not return any thing";
System.out.println(stringObject);
}
}
}

you can use new static class:
public class Global{
//example
public static ..
public static ..
}