Develop Reference

spring boot and Executor Service

I am using spring boot
public interface StringConsume extends Consumer<String> {
default public void strHandel(String str) {
accept(str);
}
}
Impl
#Component("StrImpl")
public class StringConsumeImpl implements StringConsume {
BlockingQueue<String> queue = new ArrayBlockingQueue<>(500);
final ExecutorService exService = Executors.newSingleThreadExecutor();
Future<?> future = CompletableFuture.completedFuture(true);
#Override
public void accept(String t) {
try {
queue.put(t);
} catch (InterruptedException e) {
e.printStackTrace();
}
while (null != queue.peek()) {
if (future.isDone()) {
future = exService.submit(() -> queue.take());
}
}
}
}
Class
#Component
public class Test {
#Resource(name="StrImpl")
private #Autowired StringConsume handler;
public void insertIntoQueue(String str) {
handler.accept(str);
}
}
In StringConsumeImpl , do I need synchronized while loop? and suppose five time StringConsumeImpl class called, then do while loop will create 5 process or only 1 process ? and what is the best replacement of while loop in StringConsumeImpl , if any ?

There are some problems with that code.
First of all, the consumer doesn't really "consume" anything, it just adds the string to the queue then takes it back out. Let's say for the sake of the argument that it also "consumes" it by printing it to console or something.
Secondly, the consumer will only get called once due to the loop unless it is running in a thread of its own. Eg if you do
public static void main(String[]args) {
StringConsume consumer = new StringConsumeImpl();
consumer.accept("hello");
}
The consumer will put "hello" into the queue, take it out immediately and then stay in the loop, waiting for more elements to take out; however, no one is there to actually add any.
The usual concept of doing what it looks like you're trying to do is "producer/consumer". This means that there is a "producer" that puts items into a queue and a "consumer" taking them out and doing stuff with them.
So in your case what your class does is "consume" the string by putting it into the queue, making it a "producer", then "consuming" the string by taking it back out of the queue. Of course, there's also the "actual" producer of the string, ie the class calling this.
So in general you'd do something like this:
/** Produces random Strings */
class RandomStringProducer {
Random random = new Random();
public String produceString() {
return Double.toString(random.nextDouble());
}
}
/** Prints a String */
class PrintConsumer implements StringConsume {
public void accept(String s) { System.out.println(s); }
}
/** Consumes String by putting it into a queue */
class QueueProducer implements StringConsume {
BlockingQueue<String> queue;
public QueueProducer(BlockingQueue<String> q) { queue = q; }
public void accept(String s) {
queue.put(s);
}
}
public static void main(String[] args) {
// the producer
RandomStringProducer producer = new RandomStringProducer();
// the end consumer
StringConsume printConsumer = new PrintConsumer();
// the queue that links producer and consumer
BlockingQueue<String> queue = new ArrayBlockingQueue<>();
// the consumer putting strings into the queue
QueueProducer queuePutter = new QueueProducer(queue);
// now, let's tie them together
// one thread to produce strings and put them into the queue
ScheduledExecutorService producerService = Executors.newScheduledThreadPool(1);
Runnable createStringAndPutIntoQueue = () -> {
String created = producer.createString();
queuePutter.consume(created);
};
// put string into queue every 100ms
producerService.scheduleAtFixedRate(createStringAndPutIntoQueue, 100, TimeUnit.MILLISECONDS);
// one thread to consume strings
Runnable takeStringFromQueueAndPrint = () -> {
while(true) {
String takenFromQueue = queue.take(); // this will block until a string is available
printConsumer.consume(takenFromQueue);
}
};
// let it run in a different thread
ExecutorService consumerService = Executors.newSingleThreadExecutor();
consumerService.submit(takeStringFromQueueAndPrint);
// this will be printed; we are in the main thread and code is still being executed
System.out.println("the produce/consume has started");
}
So when you run this, there will be three threads: the main thread, the producer thread and the consumer thread. The producer and consumer will be doing their thing concurrently, and the main thread will also continue to run (as exemplified by the System.out.println in the last line).

Task execution in Java web application

I'm developing Spring MVC web application. One of it's functionalities is file converting (uploading file -> converting -> storing on server).
Some files could be too big for converting on-the-fly so I decided to put them in shared queue after upload.
Files will be converted with priority based on upload time, i.e. FIFO.
My idea is to add task to queue in controller after upload.
There would also be service executing all tasks in queue, and if empty, then wait until new task is added. I don't need scheduling - tasks should be executing always when queue is not empty.
I've read about ExecutorService but I didn't find any example that fit to my case.
I'd appreciate any suggestions.
EDIT
Thanks for answers, I need to clarify my problem:
Basically, I know how to execute tasks, I need to manage with handling the queue of tasks. User should be able to view the queue and pause, resume or remove task from queue.
My task class:
public class ConvertTask implements Callable<String> {
private Converter converter;
private File source;
private File target;
private State state;
private User user;
public ConvertTask(Converter converter, File source, File target, User user) {
this.converter = converter;
this.source = source;
this.target = target;
this.user = user;
this.state = State.READY;
}
#Override
public String call() throws Exception {
if (this.state == State.READY) {
BaseConverterService converterService = ConverterUtils.getConverterService(this.converter);
converterService.convert(this.source, this.target);
MailSendServiceUtil.send(user.getEmail(), target.getName());
return "success";
}
return "task not ready";
}
}
I also created class responsible for managing queue/tasks followed by your suggestions:
#Component
public class MyExecutorService {
private LinkedBlockingQueue<ConvertTask> converterQueue = new LinkedBlockingQueue<>();
private ExecutorService executorService = Executors.newSingleThreadExecutor();
public void add(ConvertTask task) throws InterruptedException {
converterQueue.put(task);
}
public void execute() throws InterruptedException, ExecutionException {
while (!converterQueue.isEmpty()) {
ConvertTask task = converterQueue.peek();
Future<String> statusFuture = executorService.submit(task);
String status = statusFuture.get();
converterQueue.take();
}
}
}
My point is, how to execute tasks if queue is not empty and resume when new task is added and queue was previously empty. I think of some code that fits in add(ConvertTask task) method.

Edited after question updates
You don't need to create any queue for the tasks since the ThreadPoolExecutor implementation has its own queue. Here's the source code of Oracle's Java 8 implementation of newSingleThreadExecutor() method:
public static ExecutorService newSingleThreadExecutor() {
return new FinalizableDelegatedExecutorService
(new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()));
}
So you just submit a new task directly and it's getting queued by the ThreadPoolExecutor
#Component
public class MyExecutorService {
private ExecutorService executorService = Executors.newSingleThreadExecutor();
public void add(ConvertTask task) throws InterruptedException {
Future<String> statusFuture = executorService.submit(task);
}
}
If you're worried about the bounds of your queue, you can create a queue instance explicitly and supply it to a ThreadPoolExecutor constructor.
private executorService = new ThreadPoolExecutor(1, 1,
0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>(MAX_SIZE));
Please note that I have removed the line
String status = statusFuture.get();
because get() call is blocking. If you have this line in the same thread where you submit, your code is not asynchronous anymore. You should store the Future objects and check them asynchronously in a different thread. Or you can consider using CompletableFuture introduced in Java 8. Check out this post.

After upload you should return response immediately. The client can't wait for resource too long. However you can change it in the client settings. Anyway if you are running a background task you can do it without interacting with the client or notify the client while execution is in progress. This is an example of callable demo used by the executor service
/**
* Created by Roma on 17.02.2015.
*/
class SumTask implements Callable<Integer> {
private int num = 0;
public SumTask(int num){
this.num = num;
}
#Override
public Integer call() throws Exception {
int result = 0;
for(int i=1;i<=num;i++){
result+=i;
}
return result;
}
}
public class CallableDemo {
Integer result;
Integer num;
public Integer getNumValue() {
return 123;
}
public Integer getNum() {
return num;
}
public void setNum(Integer num) {
this.num = num;
}
public Integer getResult() {
return result;
}
public void setResult(Integer result) {
this.result = result;
}
ExecutorService service = Executors.newSingleThreadExecutor();
public String execute() {
try{
Future<Integer> future = service.submit(new SumTask(num));
result = future.get();
//System.out.println(result);
service.shutdown();
}
catch(Exception e)
{
e.printStackTrace();
}
return "showlinks";
}
}

Better way to implement synchronous repeatable task using Spring/Spring-Batch

I have scenario when I need to poll database for specific result. I cant go on within my code until I get the expected result(except the case of passing the timeout interval)
Step A -> Steb B -> Step C
Simple way of doing this(but doesnt feel right for me) was:
numOfRetry=0;
invokeStepA();
while(true)
{
numOfRetry++
boolen result=invokeStepB();
if(result || numOfRetry==3)
{
break;
}
else
{
Thread.sleep(100000)
}
invokeStepC();
Assume the database polling is occurring on Step B.
It doesnt feel right having this while loop on my Spring bean service while calling those jobs.
Maybe I could implement this better?
Thank you.
Farther explanation about my process:
Step A is Invoking external service to do some logic.
Step B need to poll another service which checking if Step A has finished it's work(In case it has finished I can proceed to StepC else I need to try again in X seconds and to check again)
StepC - another logic which must be accomplished only after StepB returned true.
The logic which Step A is doing happens on external service.

In the asynchronous way it happens like
int count = Runtime.getRuntime().availableProcessors();
ExecutorService threadPool = Executors.newFixedThreadPool(count);
invokeStepA();
for (int i = 0; i < RETRY_COUNT; i++) {
Future f = threadPool.submit(new Callable() {
#Override
public Object call() {
return invokeStepB();
}
}
result = (YOUR_DATA_STRUCTURE) f.get();
if (resultIsOK(result)) {
break;
}
}
However, I think since your task is ordered and assuming you cannot go to do something else, using asynchronous isn't really that effective. Please tell me more about your background in case you have special requirements.
EDIT: I think your new requirement looks like you need a proper way to tell if step A is finished fine. So you can use CountDownLatch to check if A has finished properly. I.e.
private final int count = Runtime.getRuntime().availableProcessors();
private final ExecutorService threadPool = Executors.newFixedThreadPool(count);
// invoke step A
invokeStepA();
// submit step B
final CountDownLatch latch = new CountDownLatch(1);
threadPool.submit(new Runnable() {
#Override
public void run() {
invokeStepB();
latch.countDown();
}
});
// wait for step B
boolean result;
try {
result = latch.await(TIME_OUT_IN_MILLISECONDS, TimeUnit.MILLISECOND);
} catch (InterruptedException e) {
}
// Check result
if (result) {
invokeStepC();
} else {
LOG.error("Timeout waiting for step A.");
}
This assumes your invokeStepA() is a blocking method.

Here's another idea by using an event driven approach. This is just out of my mind and not tested ;)
import org.springframework.context.ApplicationEventPublisher;
#Service
public class JobA {
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
#Scheduled(cron = "0 0 * * * ?")
public void doStepA() {
log.debug("some heavy lifting");
Object someData = ....;
applicationEventPublisher.publishEvent(new JobAEvent("Jo, I'm finished", someData));
}
}
#Service
public class JobB implements ApplicationListener<JobAEvent> {
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
#Override
public void onApplicationEvent(final JobAEvent event) {
log.debug("do something more based on the event data");
Object someMoreData = ....;
applicationEventPublisher.publishEvent(new JobBEvent("Dude, me too", event.getSomeData(), someMoreData));
}
}
#Service
public class JobC implements ApplicationListener<JobBEvent> {
#Autowired
private ApplicationEventPublisher applicationEventPublisher;
#Override
public void onApplicationEvent(final JobBEvent event) {
log.debug("do even more work");
}
}
EDIT:
You can also call the method directly but then it runs synchronosly. Another possibilty is using '#Async'

Spurious ConcurrentModificationException

I'm having some issues with a piece of java code which keeps triggering a ConcurrentModificationException. I can't really figure out what is going on, this is a fairly simple static class, not sure why it would be throwing this exception as everything is synchronized. This piece of code has been heavily used for several years, so it's odd that it would start having issues at this point:
java.util.ConcurrentModificationException
at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:953)
at java.util.LinkedList$ListItr.next(LinkedList.java:886)
at DataSubscriptionManager.sendMessages(DataSubscriptionManager.java:18)
private static HashMap<DataClass,LinkedList<DataSubscriber>> subscriberMap = new HashMap();
public static void sendMessages(LinkedList messages, DataClass dataClass) {
synchronized (subscriberMap) {
LinkedList<DataSubscriber> subscribers = subscriberMap.get(dataClass);
if (subscribers != null) {
for (DataSubscriber sub: subscribers) { *** EXCEPTION HAPPENS HERE***
if (sub != null) {
sub.sendMessages(messages);
}
}
}
}
}
public static void addDataSubscriber(DataSubscriber sub, DataClass dataClass) {
synchronized (subscriberMap) {
LinkedList<DataSubscriber> subscribers = subscriberMap.get(dataClass);
if (subscribers == null) {
subscribers = new LinkedList();
subscriberMap.put(dataClass,subscribers);
}
while (subscribers.remove(sub)) {}
subscribers.add(sub);
}
}
public static void removeDataSubscriber(DataSubscriber sub, DataClass dataClass) {
synchronized (subscriberMap) {
LinkedList<DataSubscriber> subscribers = subscriberMap.get(dataClass);
subscribers.remove(sub);
}
}

What's happening is that your collection is being modified while you are iterating over it.
It's could be another thread, or it's possible one of your subscribers is either unsubscribing, or subscribing to a different dataClass in response to the message it receives.

You can try using Collections.synchronizedList(subscribers) which may help avoiding this problem.

Returning a value from Runnable

The run method of Runnable has return type void and cannot return a value. I wonder however if there is any workaround of this.
I have a method like this:
public class Endpoint {
public method() {
Runnable runcls = new RunnableClass();
runcls.run()
}
}
The method run is like this:
public class RunnableClass implements Runnable {
public JaxbResponse response;
public void run() {
int id = inputProxy.input(chain);
response = outputProxy.input();
}
}
I want to have access to response variable in method. Is this possible?

Use Callable<V> instead of using Runnable interface.
Example:
public static void main(String args[]) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(3);
Set<Future<Integer>> set = new HashSet<>();
for (String word : args) {
Callable<Integer> callable = new WordLengthCallable(word);
Future<Integer> future = pool.submit(callable);
set.add(future);
}
int sum = 0;
for (Future<Integer> future : set) {
sum += future.get();
}
System.out.printf("The sum of lengths is %s%n", sum);
System.exit(sum);
}
In this example, you will also need to implement the class WordLengthCallable, which implements the Callable interface.

public void check() {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<Integer> result = executor.submit(new Callable<Integer>() {
public Integer call() throws Exception {
return 10;
}
});
try {
int returnValue = result.get();
} catch (Exception exception) {
//handle exception
}
}

Have a look at the Callable class. This is usually submited via an executor service
It can return a future object which is returned when the thread completes

Yes, there are workaround. Just use queue and put into it value which you want to return. And take this value from another thread.
public class RunnableClass implements Runnable{
private final BlockingQueue<jaxbResponse> queue;
public RunnableClass(BlockingQueue<jaxbResponse> queue) {
this.queue = queue;
}
public void run() {
int id;
id =inputProxy.input(chain);
queue.put(outputProxy.input());
}
}
public class Endpoint{
public method_(){
BlockingQueue<jaxbResponse> queue = new LinkedBlockingQueue<>();
RunnableClass runcls = new RunnableClass(queue);
runcls.run()
jaxbResponse response = queue.take(); // waits until takes value from queue
}
}

If you add a field to RunnableClass you can set it in run and read it in method_. However, Runnable is a poor (the Java keyword) interface as it tells you nothing about the (the concept) interface (only useful line of the API docs: "The general contract of the method run is that it may take any action whatsoever."). Much better to use a more meaningful interface (that may return something).

One way is, we have to use Future - Callable approach.
Another way is, Instead of returning value, you can hold in object
Example:
class MainThread {
public void startMyThread() {
Object requiredObject = new Object(); //Map/List/OwnClass
Thread myThread = new Thread(new RunnableObject(requiredObject)).start();
myThread.join();
System.out.println(requiredObject.getRequiredValue());
}
}
class RunnableObject implements Runnable {
private Object requiredObject;
public RunnableObject(Object requiredObject) {
this.requiredObject = requiredObject;
}
public void run() {
requiredObject.setRequiredValue(xxxxx);
}
}
Because object scope is in the same scope so that you can pass object to thread and can retrieve in the main scope. But, most important thing is, we have to use join() method. Because main scope should be waiting for thread completion of its task.
For multiple thread case, you can use List/Map to hold the values from threads.

Try the following
public abstract class ReturnRunnable<T> implements Runnable {
public abstract T runForResult();
#Override
public void run() {
runForResult();
}
}

Take a look at the callable interface, perhaps this suites your needs. You can also try to get the value of the response field by calling a setter-method inside of your run() method
public void run() {
int id;
id =inputProxy.input(chain);
response = outputProxy.input();
OuterClass.setResponseData(response);
}