Develop Reference

Terminate threads in a thread pool for Multiple Tenant

I'm creating a Threadpool as shown below for a job.
public class MoveToCherwellThreadPool {
public static ThreadPoolExecutor cherwellMoveThreadPoolExecutor = null;
private static EMLogger logger = EMLogger.getLogger();
private static final String CLASSNAME = "MoveToCherwellThreadPool";
public static void initiateCherwellMoveThreadPool() {
BlockingQueue<Runnable> q = new LinkedBlockingQueue<Runnable>(100000);
cherwellMoveThreadPoolExecutor = new ThreadPoolExecutor(10,20, 20, TimeUnit.SECONDS, q);
cherwellMoveThreadPoolExecutor.setRejectedExecutionHandler(new RejectedExecutionHandler() {
#Override
public void rejectedExecution(Runnable r,
ThreadPoolExecutor executor) {
logger.logDebug(CLASSNAME,"Rejected task cherwellMoveThreadPoolExecutor Active tasks : " + cherwellMoveThreadPoolExecutor.getActiveCount() + ", " + "cherwellMoveThreadPoolExecutor Completed tasks : " + cherwellMoveThreadPoolExecutor.getCompletedTaskCount()+" Waiting for a second !! ");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
executor.execute(r);
}
});
}
}
I'm using this during a process running for multiple customers. For each customer new threadpool will be initialized and threads will be running.
Below is the code where I'm using the threadpool.
for (Object[] objects : relationshipList) {
CherwellRelationshipMoveThread relationshipThread = new CherwellRelationshipMoveThread(objects,
this.customerId, sb, credential,mainCIId,moveUniqueId,this.startTime);
CompletableFuture<?> future = CompletableFuture.runAsync(relationshipThread,
MoveToCherwellThreadPool.cherwellMoveThreadPoolExecutor);
crelationshipList.add(future);
}
crelationshipList.forEach(CompletableFuture::join);
This thread will be created for multiple customers. I'm giving an option to terminate this job in UI. On click of stop process I need to stop/kill only the threads running for that particular customer and other customer's thread shouldn't be harmed and should be keep running.
On click of stop process from UI I'm calling a service where inside the service my code will be
MoveToCherwellThreadPool.cherwellMoveThreadPoolExecutor.shutdownNow();
I'm calling shutdownNow() on the ThreadPoolExecutor.
This is killing all the threads of all the customers. I don't want to kill all the customers process, but only for the customer where I'll click on stop process.

This code doesn't maintain any mapping from a tenant to a thread pool, there's only one static reference to a ThreadPoolExecutor. Each time initiateCherwellMoveThreadPool is called, any existing executor will be replaced with a new one, and the existing one isn't shut down, so it will leak resources. As a result, this will execute tasks from multiple tenants in the same thread pool.
This code is also not thread safe. It's possible (if unlikely) that a thread could schedule a task on a newly-created executor, or even shut it down, before setRejectedExecutionHandler is called.
If you need a separate executor per tenant, this will need to be implemented. A good option might be to use a ConcurrentHashMap with customerId keys and ThreadPoolExecutor values, for example (logging omitted for brevity):
public class MoveToCherwellThreadPool {
public static ConcurrentMap<String, ThreadPoolExecutor> cherwellMoveThreadPoolExecutors = new ConcurrentHashMap<>();
public static ThreadPoolExecutor getCherwellMoveThreadPool(String customerId) {
return cherwellMoveThreadPoolExecutors.computeIfAbsent(customerId, id -> {
BlockingQueue<Runnable> q = new LinkedBlockingQueue<Runnable>(100000);
ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 20, 20, TimeUnit.SECONDS, q);
executor.setRejectedExecutionHandler(new RejectedExecutionHandler() { /*...*/ });
return executor;
});
}
public static List<Runnable> stopCherwellMoveTheadPool(String customerId) {
if (cherwellMoveThreadPoolExecutors.containsKey(customerId)) {
return cherwellMoveThreadPoolExecutors.get(customerId).shutdownNow();
}
return Collections.emptyList();
}
}
This can be used like this:
CompletableFuture<?> future = CompletableFuture.runAsync(relationshipThread,
MoveToCherwellThreadPool.getCherwellMoveThreadPool(customerId));
It's also important to realise that calling shutdownNow can only attempt to cancel currently executing tasks, and "does not wait for actively executing tasks to terminate":
This implementation cancels tasks via Thread.interrupt(), so any task that fails to respond to interrupts may never terminate.
The code implementing CherwellRelationshipMoveThread isn't shown, so this may or may not be the case.

Java - Concurrent processing with a common executor service instance

I have n number of worker threads that retrieve records from a kinesis stream (this is not important for this problem), which are then pushed on to an executor service where the records are processed and persisted to a backend database. This same executor service instance is used for all worker threads.
Now there is a scenario where any given worker loop stops processing records and blocks until all records that were submitted by it are processed completely. This essentially means that there should be no pending/running threads in the executor service for a record from that particular worker thread.
A very trivial example of the implementation is like this:
Worker class
public class Worker {
Worker(Listener listener){
this.listener = listener;
}
//called periodically to fetch records from a kinesis stream
public void processRecords(Record records) {
for (Record record : records) {
listener.handleRecord(record);
}
//if 15 minutes has elapsed, run below code. This is blocking.
listener.blockTillAllRecordsAreProcessed()
}
}
Listener class
public class Listener {
ExecutorService es;
// same executor service is shared across all listeners.
Listener(ExecutorService es){
this.es = es;
}
public void handleRecord(Record record) {
//submit record to es and return
// non blocking
}
public boolean blockTillAllRecordsAreProcessed(){
// this should block until all records are processed
// no clue how to implement this with a common es
}
}
The only approach I could think of is to have a local executor service for each worker and do something like invokeAll for each batch, which would be change the implementation slightly but get the job done. but I feel like there should be a better approach to tackle this problem.

You could use the CountdownLatch class to block as follows:
public void processRecords(List<Record> records) {
CountDownLatch latch = new CountDownLatch(records.size());
for (Record record : records) {
listener.handleRecord(record, latch);
}
//if 15 minutes has elapsed, run below code. This is blocking.
listener.blockTillAllRecordsAreProcessed(latch)
}
public class Listener {
ExecutorService es;
...
public void handleRecord(Record record, CountDownLatch latch) {
//submit record to es and return
// non blocking
es.submit(()->{
someSyncTask(record);
latch.countDown();
})
}
public boolean blockTillAllRecordsAreProcessed(CountDownLatch latch){
System.out.println("waiting for processes to complete....");
try {
//current thread will get notified if all chidren's are done
// and thread will resume from wait() mode.
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Read more here: https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/CountDownLatch.html

`ThreadPoolTaskExecutor` Threads are not killed after execution in Spring

I am trying to change Quartz Sequential execution to Parallel Execution.
It is working fine, Performance wise, it is seems good but Spawned (created) threads are not destroyed.
It is Still in Runnable State; why and How can I fix that?
Please Guide me.
Code is here :
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
logger.error("Result Processing executed");
List<Object[]> lstOfExams = examService.getExamEntriesForProcessingResults();
String timeZone = messageService.getMessage("org.default_timezone", null, Locale.getDefault());
if(lstOfExams!=null&&!lstOfExams.isEmpty()){
ThreadPoolTaskExecutor threadPoolExecuter = new ThreadPoolTaskExecutor();
threadPoolExecuter.setCorePoolSize(lstOfExams.size());
threadPoolExecuter.setMaxPoolSize(lstOfExams.size()+1);
threadPoolExecuter.setBeanName("ThreadPoolTaskExecutor");
threadPoolExecuter.setQueueCapacity(100);
threadPoolExecuter.setThreadNamePrefix("ThreadForUpdateExamResult");
threadPoolExecuter.initialize();
for(Object[] obj : lstOfExams){
if(StringUtils.isNotBlank((String)obj[2]) ){
timeZone = obj[2].toString();
}
try {
Userexams userexams=examService.findUserExamById(Long.valueOf(obj[0].toString()));
if(userexams.getExamresult()==null){
UpdateUserExamDataThread task=new UpdateUserExamDataThread(obj,timeZone);
threadPoolExecuter.submit(task);
}
// testEvaluator.generateTestResultAsPerEvaluator(Long.valueOf(obj[0].toString()), obj[4].toString(), obj[3]==null?null:obj[3].toString(),timeZone ,obj[5].toString() ,obj[1].toString());
// logger.error("Percentage Marks:::::"+result.getPercentageCatScore());
} catch (Exception e) {
Log.error("Exception at ResultProcessingJob extends QuartzJobBean executeInternal(JobExecutionContext context) throws JobExecutionException",e);
continue;
}
}
threadPoolExecuter.shutdown();
}
}
UpdateUserExamDataThread .class
#Component
//#Scope(value="prototype", proxyMode=ScopedProxyMode.TARGET_CLASS)
//public class UpdateUserExamDataThread extends ThreadLocal<String> //implements Runnable {
public class UpdateUserExamDataThread implements Runnable {
private Logger log = Logger.getLogger(UpdateUserExamDataThread.class);
#Autowired
ExamService examService;
#Autowired
TestEvaluator testEvaluator;
private Object[] obj;
private String timeZone;
public UpdateUserExamDataThread(Object[] obj,String timeZone) {
super();
this.obj = obj;
this.timeZone = timeZone;
}
#Override
public void run() {
String threadName=String.valueOf(obj[0]);
log.info("UpdateUserExamDataThread Start For:::::"+threadName);
testEvaluator.generateTestResultAsPerEvaluator(Long.valueOf(obj[0].toString()), obj[4].toString(), obj[3]==null?null:obj[3].toString(),timeZone ,obj[5].toString() ,obj[1].toString());
//update examResult
log.info("UpdateUserExamDataThread End For:::::"+threadName);
}
}
TestEvaluatorImpl.java
#Override
#Transactional
public Examresult generateTestResultAsPerEvaluator(Long userExamId, String evaluatorType, String codingLanguage,String timeZoneFollowed ,String inctenceId ,String userId) {
dbSchema = messageService.getMessage("database.default_schema", null, Locale.getDefault());
try {
//Some Methods
return examResult;
}catch(Exception e){
log.erorr(e);
}
}
I can provide Thread Dump file if needed.

it seems you create a thread pool in the same size of exams which is not quite optimal.
// Core pool size is = number of exams
threadPoolExecuter.setCorePoolSize(lstOfExams.size());
// Max pool size is just 1 + exam size.
threadPoolExecuter.setMaxPoolSize(lstOfExams.size()+1);
You have to consider that:
- If you create a thread pool and started it as many threads as defined in core size started immediately.
The max pool size is only than effective when you submit more than core pool threads can process right now AND when the queue size is full (in this case 100). So that means a new thread will be only then created when the number of submitted tasks exceeded 100+exam size.
In your case I would set the core pool size 5 or 10 (it actually depends on the how many core your target CPU have and/or how IO bound the submitted tasks are).
The max pool size can be double of that but it doesn't effective until the queue is full.
To let the size of live threads decrease after the submitted work done you have to set 2 parameters.
setKeepAliveSeconds(int keepAliveSeconds) : Which let the threads shut down automatically if they are not used along the defined seconds (by default 60 seconds, which is optimal) BUT this is normally only used to shut down threads of non-core pool threads.
To shut down threads of core part after keepAliveSeconds you have to set setAllowCoreThreadTimeOut(boolean allowCoreThreadTimeOut) as true. Which is normally false to keep core pool alive as long as the application is running.
I hope it helps.

I suspect that one of your threads waits indefinitely for an IO request answer. For example, you try to connect to a remote host where you did not set connection timeout and the host does not answer. In this case, you can shutdown all executing tasks forcefully by running shutdownNow method of the underlying ExecutorService then you can analyze InterruptedIOException thrown by the offending threads.
Replace
threadPoolExecuter.shutdown();
with below so you can examine errors.
ExecutorService executorService = threadPoolExecuter.getThreadPoolExecutor();
executorService.shutdownNow();
This will send interrupt signal to all running threads.

The threads do not wait on IO from some remote server, because the executed method on the threads would be in some jdbc driver classes, but they are currently all in UpdateUserExamDataThread.run(), line 37.
Now the question is: what is the code at UpdateUserExamDataThread.java line 37 ?
Unfortunately, the UpdateUserExamDataThread.java given at the moment is incomplete and/or not the version really executed: the package declaration is missing and it ends at line 29.

I suspect the issue is simply that you are calling run() instead of execute() when spawning the task thread using submit(). There is probably some expectation when using submit that threads kill themselves when the task is finished rather than terminating at the end of the run method.

Just Needed to increase the priority of threads and create number of threads as per number of cores in processor.
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
logger.error("Result Processing executed");
List<Object[]> lstOfExams = examService.getExamEntriesForProcessingResults();
String timeZone = messageService.getMessage("org.default_timezone", null, Locale.getDefault());
int cores = Runtime.getRuntime().availableProcessors();
if(lstOfExams!=null&&!lstOfExams.isEmpty()){
ThreadPoolTaskExecutor threadPoolExecuter = new ThreadPoolTaskExecutor();
threadPoolExecuter.setCorePoolSize(cores);
// threadPoolExecuter.setMaxPoolSize(Integer.MAX_VALUE);
threadPoolExecuter.setBeanName("ThreadPoolTaskExecutor");
// threadPoolExecuter.setQueueCapacity(Integer.MAX_VALUE);
threadPoolExecuter.setQueueCapacity(lstOfExams.size()+10);
threadPoolExecuter.setThreadNamePrefix("ThreadForUpdateExamResult");
threadPoolExecuter.setWaitForTasksToCompleteOnShutdown(true);
threadPoolExecuter.setThreadPriority(10);
threadPoolExecuter.initialize();
for(Object[] obj : lstOfExams){
if(StringUtils.isNotBlank((String)obj[2]) ){
timeZone = obj[2].toString();
}
try {
Userexams userexam=examService.findUserExamById(Long.valueOf(obj[0].toString()));
if(userexam.getExamresult()==null){
UpdateUserExamDataThread task=new UpdateUserExamDataThread(obj,timeZone,testEvaluator);
// threadPoolExecuter.submit(task);
threadPoolExecuter.execute(task);
}
// testEvaluator.generateTestResultAsPerEvaluator(Long.valueOf(obj[0].toString()), obj[4].toString(), obj[3]==null?null:obj[3].toString(),timeZone ,obj[5].toString() ,obj[1].toString());
// logger.error("Percentage Marks:::::"+result.getPercentageCatScore());
} catch (Exception e) {
logger.error("Exception at ResultProcessingJob extends QuartzJobBean executeInternal(JobExecutionContext context) throws JobExecutionException",e);
continue;
}
}
threadPoolExecuter.shutdown();
}
}

Is there an out-of-the-box thread pool with multiple queues (that ensure serial processing of each queue)?

Among all my tasks, I have some that must be processed serially (they can never run concurrently and they must be processed in order).
I achieved that creating a separated thread pool with a single thread for each group of tasks that must be executed serially. It works but I don't have the resources for that. I don't control the number of groups, so I might end up with a ridiculous number of threads running simultaneously.
Is there any way I can accomplish that with a single thread pool? Is there a thread pool with multiple blocking queues where I could ensure serial execution for each queue?
EDIT:
Just emphasizing what I've said in my second paragraph: I've solved this with a single threaded thread pool for each group of tasks that must be executed serially. I can't go on with this solution, though. There are way too many groups and I can't have all these threads.
I've found this related question, but since it is not very recent, I still created mine. All I'm doing is trying to avoid reinventing the wheel, but it seems I don't have a choice.
Does Java have an indexable multi-queue thread pool?

If you maintain a queue for each group, you can pull items off each queue and feed them into a thread pool. The code below won't prioritize any one group, it just pulls them in a round-robin fashion. If you need to add prioritization you should easily be able to. The following code will round-robin 4 groups using two threads (plus the thread managing the queue). You can use another queue mechanism. I typically use LinkedBlockingQueue for situations where I want to wait for items to be placed on the queue by another thread, which probably is not what you want - so I'm polling instead of calling take(). Take is the call that waits.
private Future group1Future = null;
private Future group2Future = null;
private Future group3Future = null;
private Future group4Future = null;
private LinkedBlockingQueue<Callable> group1Queue
= new LinkedBlockingQueue<>();
private LinkedBlockingQueue<Callable> group2Queue
= new LinkedBlockingQueue<>();
private LinkedBlockingQueue<Callable> group3Queue
= new LinkedBlockingQueue<>();
private LinkedBlockingQueue<Callable> group4Queue
= new LinkedBlockingQueue<>();
private ExecutorService executor = Executors.newFixedThreadPool(2);
public void startProcessing() {
while (true) {
if (group1Future != null && group1Future.isDone()) {
if (group1Queue.peek() != null) {
group1Future = executor.submit(group1Queue.poll());
}
}
if (group2Future != null && group1Future.isDone()) {
if (group2Queue.peek() != null) {
group2Future = executor.submit(group2Queue.poll());
}
}
if (group3Future != null && group3Future.isDone()) {
if (group3Queue.peek() != null) {
group3Future = executor.submit(group3Queue.poll());
}
}
if (group4Future != null && group4Future.isDone()) {
if (group4Queue.peek() != null) {
group4Future = executor.submit(group4Queue.poll());
}
}
}
}
If a task for that group is not complete, it will skip to the next group. No more than two groups will be processed at a time and no single group will ever run more than one task. The queues will enforce ordered execution.

Akka, as suggested by #SotiriosDelimanolis and #AlexeiKaigorodov seems promising, as well as #Dodd10x second answer, which certainly solves the problem. The only downside is that I'd have to code my own polling strategy to make sure my tasks are eventually added to the executor (like the infinite loop in his example).
On the other hand, the Striped Executor Service suggested by #OldCurmudgeon exactly matches my problem and works out of the box simply as a custom ExecutorService.
This magical thread pool would ensure that all Runnables with the same stripeClass would be executed in the order they were submitted, but StripedRunners with different stripedClasses could still execute independently. He wanted to use a relatively small thread pool to service a large number of Java NIO clients, but in such a way that the runnables would still be executed in-order.
There is even a comment about using a single threaded thread pool for each group (stripe), as it was suggested here:
Several suggestions were made, such as having a SingleThreadExecutor for each stripeClass. However, that would not satisfy the requirement that we could share the threads between connections.
I see this as the best solution for its simplicity and ease of use.

I recently answered a question about a "serial task queue" with a basic implementation as demonstration here. I imagine you have been using a similar solution. It is relatively easy to adapt the implementation to use a map of task lists and still share one (fixed size) executor.
The Striped Executor Service you mention is the better solution, but I show the adapted implementation here to demonstrate decoupling the task queue(s) from the executor. The implementation uses a callback and therefor has no need to do polling or signalling. Since a "critical (stop the world) section" is used, the map with task queues can clean itself: no tasks queued means empty map. Downside of the "critical section" is that throughput is limited: only so many tasks can be added and removed per second.
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.locks.ReentrantLock;
// Copied and updated from https://stackoverflow.com/a/32916943/3080094
public class SerialTaskQueues {
public static void main(String[] args) {
// test the serial task execution using different groups
ExecutorService executor = Executors.newFixedThreadPool(2);
SerialTaskQueues tq = new SerialTaskQueues(executor);
try {
// test running the tasks one by one
tq.add(new SleepSome("1", 30L));
Thread.sleep(5L);
tq.add(new SleepSome("2", 20L));
tq.add(new SleepSome("1", 10L));
Thread.sleep(100L);
// all queues should be empty
System.out.println("Queue size 1: " + tq.size("1")); // should be empty
System.out.println("Queue size 2: " + tq.size("2")); // should be empty
tq.add(new SleepSome("1", 10L));
tq.add(new SleepSome("2", 20L));
// with executor pool size set to 2, task 3 will have to wait for task 1 to complete
tq.add(new SleepSome("3", 30L));
tq.add(new SleepSome("1", 20L));
tq.add(new SleepSome("2", 10L));
Thread.sleep(100L);
} catch (Exception e) {
e.printStackTrace();
} finally {
executor.shutdownNow();
}
}
// all lookups and modifications to the list must be synchronized on the list.
private final Map<String, GroupTasks> taskGroups = new HashMap<>();
// make lock fair so that adding and removing tasks is balanced.
private final ReentrantLock lock = new ReentrantLock(true);
private final ExecutorService executor;
public SerialTaskQueues(ExecutorService executor) {
this.executor = executor;
}
public boolean add(String groupId, Runnable task) {
lock.lock();
try {
GroupTasks gt = taskGroups.get(groupId);
if (gt == null) {
gt = new GroupTasks(groupId);
taskGroups.put(groupId, gt);
}
gt.tasks.add(task);
} finally {
lock.unlock();
}
runNextTask(groupId);
return true;
}
/* Utility method for testing. */
public void add(SleepSome sleepTask) {
add(sleepTask.groupId, sleepTask);
}
private void runNextTask(String groupId) {
// critical section that ensures one task is executed.
lock.lock();
try {
GroupTasks gt = taskGroups.get(groupId);
if (gt.tasks.isEmpty()) {
// only cleanup when last task has executed, prevent memory leak
if (!gt.taskRunning.get()) {
taskGroups.remove(groupId);
}
} else if (!executor.isShutdown() && gt.taskRunning.compareAndSet(false, true)) {
executor.execute(wrapTask(groupId, gt.taskRunning, gt.tasks.remove(0)));
}
} finally {
lock.unlock();
}
}
private CallbackTask wrapTask(final String groupId, final AtomicBoolean taskRunning, Runnable task) {
return new CallbackTask(task, new Runnable() {
#Override
public void run() {
if (!taskRunning.compareAndSet(true, false)) {
System.out.println("ERROR: programming error, the callback should always run in execute state.");
}
runNextTask(groupId);
}
});
}
/** Amount of (active) task groups. */
public int size() {
int size = 0;
lock.lock();
try {
size = taskGroups.size();
} finally {
lock.unlock();
}
return size;
}
public int size(String groupId) {
int size = 0;
lock.lock();
try {
GroupTasks gt = taskGroups.get(groupId);
size = (gt == null ? 0 : gt.tasks.size());
} finally {
lock.unlock();
}
return size;
}
public Runnable get(String groupId, int index) {
Runnable r = null;
lock.lock();
try {
GroupTasks gt = taskGroups.get(groupId);
r = (gt == null ? null : gt.tasks.get(index));
} finally {
lock.unlock();
}
return r;
}
public Runnable remove(String groupId, int index) {
Runnable r = null;
lock.lock();
try {
GroupTasks gt = taskGroups.get(groupId);
r = gt.tasks.remove(index);
// similar to runNextTask - cleanup if there are no tasks (running) for the group
if (gt.tasks.isEmpty() && !gt.taskRunning.get()) {
taskGroups.remove(groupId);
}
} finally {
lock.unlock();
}
return r;
}
/* Helper class for the task-group map. */
class GroupTasks {
final List<Runnable> tasks = new LinkedList<Runnable>();
// atomic boolean used to ensure only 1 task is executed at any given time
final AtomicBoolean taskRunning = new AtomicBoolean(false);
final String groupId;
GroupTasks(String groupId) {
this.groupId = groupId;
}
}
// general callback-task, see https://stackoverflow.com/a/826283/3080094
static class CallbackTask implements Runnable {
private final Runnable task, callback;
public CallbackTask(Runnable task, Runnable callback) {
this.task = task;
this.callback = callback;
}
#Override
public void run() {
try {
task.run();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
callback.run();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
// task that just sleeps for a while
static class SleepSome implements Runnable {
static long startTime = System.currentTimeMillis();
private final String groupId;
private final long sleepTimeMs;
public SleepSome(String groupId, long sleepTimeMs) {
this.groupId = groupId;
this.sleepTimeMs = sleepTimeMs;
}
#Override public void run() {
try {
System.out.println(tdelta(groupId) + "Sleeping for " + sleepTimeMs + " ms.");
Thread.sleep(sleepTimeMs);
System.out.println(tdelta(groupId) + "Slept for " + sleepTimeMs + " ms.");
} catch (Exception e) {
e.printStackTrace();
}
}
private String tdelta(String groupId) { return String.format("% 4d [%s] ", (System.currentTimeMillis() - startTime), groupId); }
}
}

A single thread executor will do
ExecutorService executorService = Executors.newSingleThreadExecutor();
Which internally uses a ThreadPoolExecutor with a LinkedBlockingQueue
new ThreadPoolExecutor(1, 1,0L, TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>()))
So you can use this for your sequential stuff and probably use a multi-threaded executor service for concurrent tasks

Look into Java's built-in thread executor service.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html
There is a single thread executor that will process each task synchronously.
In response to the comments section:
Please read the API before you say this won't work.
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html#newSingleThreadExecutor()
public static ExecutorService newSingleThreadExecutor()
Creates an Executor that uses a single worker thread operating off an unbounded queue. (Note however that if this single thread terminates due to a failure during execution prior to shutdown, a new one will take its place if needed to execute subsequent tasks.) Tasks are guaranteed to execute sequentially, and no more than one task will be active at any given time. Unlike the otherwise equivalent newFixedThreadPool(1) the returned executor is guaranteed not to be reconfigurable to use additional threads.
Note: is states they are guaranteed to execute sequentially.
EDIT:
Now that I understand your question better, I have an idea you could try. If you maintain a queue for each group, you can pull items off each queue and feed them into a thread pool. The code below won't prioritize any one group, it just pulls them in a round robbing fashion. If you need to add prioritization you should easily be able to. The following code will round robbing 4 groups using two threads (plus the thread managing the queue). You can use another queue mechanism. I typically use LinkedBlockingQueue for situations where I want to wait for items to be placed on the queue by another thread, which probably is not what you want - which is why I'm polling instead of calling take(). Take is the call that waits.
private Future group1Future = null;
private Future group2Future = null;
private Future group3Future = null;
private Future group4Future = null;
private LinkedBlockingQueue<Callable> group1Queue
= new LinkedBlockingQueue<>();
private LinkedBlockingQueue<Callable> group2Queue
= new LinkedBlockingQueue<>();
private LinkedBlockingQueue<Callable> group3Queue
= new LinkedBlockingQueue<>();
private LinkedBlockingQueue<Callable> group4Queue
= new LinkedBlockingQueue<>();
private ExecutorService executor = Executors.newFixedThreadPool(2);
public void startProcessing() {
while (true) {
if (group1Future != null && group1Future.isDone()) {
if (group1Queue.peek() != null) {
group1Future = executor.submit(group1Queue.poll());
}
}
if (group2Future != null && group1Future.isDone()) {
if (group2Queue.peek() != null) {
group2Future = executor.submit(group2Queue.poll());
}
}
if (group3Future != null && group3Future.isDone()) {
if (group3Queue.peek() != null) {
group3Future = executor.submit(group3Queue.poll());
}
}
if (group4Future != null && group4Future.isDone()) {
if (group4Queue.peek() != null) {
group4Future = executor.submit(group4Queue.poll());
}
}
}
}
If a task for that group is not complete, it will skip to the next group. No more than two groups will be processed at a time and no single group will ever run more than one task. The queues will enforce ordered execution.

What you need is not a special executor, but means to express dependencies between tasks. Instead of a group of tasks which must be executed serially, think of a task which, at the end of execution, sends a signal to the next task, thus starting its execution. So your task can be coded as an actor which waits for allowing signal to start execution. Consider Akka or any other actor library (e.g. mine df4j).

There is no standard implementation of thread pool with these requirements.
Striped Executor Service mentioned in the accepted answer is a good substitute.
The disadvantages I see are: multiple queues (no way to limit queue capacity, or maintain a submission order), thread per stripe (if you have a lot of stripes, your thread pool will grow).
I decided to create similar implementation with single queue:
GitHub - TaggedThreadPoolExecutor.java
It implements standard ExecutorService interface, maintain single queue, takes a maximum number of threads as a parameter, support different rejection policies (similar to standard ThreadPoolExecutor), unlike ThreadPoolExecutor it starts new thread not when queue is full, but when new task is submitted.

You could maintain a bunch of queues (List or a Map of queues). Each queue hold a task for that specific class, and have a background running thread which will dequeue tasks from each queue serially and submit them on a separate threadpool executor which could be a bigger in size in terms of number of threads!

Java Sockets Concurrency Threads Being Too Slow

I am developing an application in Enterprise JavaBeans 3.1 and I receive data from a Socket. This application acts as a Listener and once data is received it is processed. This application was single threaded and due to it processing it slowly, the application is implemented using Threads which is now a multi threaded application. By doing this, the application now runs much faster.
However, there are two threads and both threads access the database to insert and update the database. I face the problem of concurrency where one thread inserts and the other updates causing problems. To deal with concurrency, I added a synchronized block to lock an object making sure the full block is executed. By doing this application is now very slow as it was with a single threaded application. The insert and update is done through JDBC.
Is there anything else that can be done so it is processed and processed very quickly without slowing down the application. The below is sample code:
#Startup
#Singleton
public class Listener {
private ServerSocket serverSocket;
private Socket socket;
private Object object;
private InetAddress server;
#Resource
private ScheduledExecutorService executor;
#PostConstruct
public void init() {
object = new Object();
serverSocket = new ServerSocket("somePortNumber");
Runnable runnable = new Runnable() {
public void run() {
checkDatabase();
if(!isServerActive()) {
// send e-mail
listen();
}
else {
listen();
}
}
};
executor.scheduleAtFixedRate(runnable, 0, 0, TimeUnit.SECONDS);
}
public void listen() {
if(socket == null) {
socket = serverSocket.accept();
}
else if(socket.isClosed()) {
socket = serverSocket.accept();
}
startThread(socket);
}
public void startThread(Socket socket) {
Runnable runnable = new Runnable() {
public void run() {
processMessage(socket);
}
};
new Thread(runnable).start();
}
public void processMessage(Socket socket) {
synchronized(object) {
// build data from Socket
// insert into database message, sentDate
// do other things
// update processDate
}
}
public void checkDatabase() {
synchronized(object) {
// get data and further update
}
}
public boolean isServerActive() {
boolean isActive = true;
if(server == null) {
sever = InetAddress.getByName("serverName");
}
if(!server.isNotReachable(5000)) {
isActive = false;
if(socket != null) {
socket.close();
}
}
return isActive;
}
}
EDIT:
Table name: Audit
Message: VARCHAR NOT NULL
SentDate: DATE NOT NULL
ProcessedDate: DATE
AnotherDate: DATE
Query: INSERT INTO AUDIT (message, sentDate, processedDate, receivedDate) VALUES (?, java.sql.Timestamp, null, null)
Assuming a record is inserted without the synchronized block inserting the message and sentDate. The other thread will execute causing this record to be found and further update. The problem is that after the initial insert and processedDate should be updated and then the other thread should be executed.
The processMessage() sends the data over HTTPS asynchronously.
One of the reasons to use Threads was because only one piece of data came to Java. So by introducing threads the full set of data comes to Java.

Even with single thread you can get much better speed by using JDBC batching and running any transactions around the batch instead of committing every individual insert/update statement.
In a multi threaded environment you can avoid concurrency problems if you ensure no two threads act on the same database row at the same time. You can use row level locks to avoid multiple threads updating the same row.
It is not possible to give you any more inputs with the information you have given. You may get more ideas if you provide information about the data you are processing.

The application behaved as single threaded because the processMessage & checkDatabase methods have synchronised block on the same class object , the threads that are listening currently will hold the lock and other threads will have to wait until the message is processed,which will cause the application to slow down. instead of putting synchronised in two separate blocks create separate threads outside of the class that checks this condition and try to invoke then separately based on a condition or you could try with wait() and notifyAll in your synchronized blocks also.