Develop Reference

Why does the slow task block the other smaller tasks in this program that uses parallel streams?

Following a question form a colleague about parallel streams I wrote the following code to test something out.
public class Test {
public static void main(String args[]) {
List<Runnable> list = new LinkedList<>();
list.add(() -> {
try {
Thread.sleep(10000);
System.out.println("Time : " + System.nanoTime() + " " + "Slow task");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
for (int i = 0; i < 1000; i++) {
int j = i;
list.add(() -> System.out.println("Time : " + System.nanoTime() + " " + j));
}
list.parallelStream().forEach(r -> r.run());
}
}
Strangely the output is always something like the following.
Time : 4096118049370412 61
Time : 4096118049567530 311
Time : 4096118049480238 217
Time : 4096118049652415 405
Time : 4096118049370678 436
Time : 4096118049370575 155
Time : 4096118049720639 437
Time : 4096118049719368 280
Time : 4096118049804630 281
Time : 4096118049684148 406
Time : 4096118049660398 218
TRUNCATED
Time : 4096118070511768 669
Time : 4096118070675678 670
Time : 4096118070584951 426
Time : 4096118070704143 427
Time : 4096118070714441 428
Time : 4096118070722080 429
Time : 4096118070729569 430
Time : 4096118070736782 431
Time : 4096118070744069 432
Time : 4096118070751286 433
Time : 4096118070758554 434
Time : 4096118070765913 435
Time : 4096118070550370 930
Time : 4096118070800538 931
Time : 4096118070687425 671
Time : 4096118070813669 932
Time : 4096118070827794 672
Time : 4096118070866089 933
Time : 4096118070881358 673
Time : 4096118070895344 934
Time : 4096118070907608 674
Time : 4096118070920712 935
Time : 4096118070932934 675
Time : 4096118070945131 936
Time : 4096118070957850 676
Time : 4096118070982326 677
Time : 4096118070991158 678
Time : 4096118070999002 679
Time : 4096118071006501 680
Time : 4096118071017766 681
Time : 4096118071025766 682
Time : 4096118071033318 683
Time : 4096118071070603 684
Time : 4096118071080240 685
Time : 4096128063025914 Slow task
Time : 4096128063123940 0
Time : 4096128063148135 1
Time : 4096128063173285 2
Time : 4096128063176723 3
Time : 4096128063179939 4
Time : 4096128063183077 5
Time : 4096128063191001 6
Time : 4096128063194156 7
Time : 4096128063197273 8
Time : 4096128063200395 9
Time : 4096128063203581 10
Time : 4096128063206988 11
Time : 4096128063210155 12
Time : 4096128063213285 13
Time : 4096128063216411 14
Time : 4096128063219542 15
Time : 4096128063222733 16
Time : 4096128063232190 17
Time : 4096128063235653 18
Time : 4096128063238827 19
Time : 4096128063241962 20
Time : 4096128063245176 21
Time : 4096128063248296 22
Time : 4096128063251444 23
Time : 4096128063254557 24
Time : 4096128063257705 25
Time : 4096128063261566 26
Time : 4096128063264733 27
Time : 4096128063268115 28
Time : 4096128063272851 29
Process finished with exit code 0
That is, there is always some tasks waiting for the slow task to finish processing, even though all the other tasks have finished. I would assume that the slow task should take only one thread and all the other tasks should finish without any problem and only the slow task should take the full 10 seconds. I have 8 CPUs so the parallelism level is 7.
What could the reason be for this?
To add more information, the code is only for understanding purposes. I am not putting it anywhere in production.

There are some limited capabilities when it comes to work-stealing with streams, so if a single thread has pegged itself for some work in other runners, that work will be blocked until it's finished processing other tasks.
You can visualize this by adding a few more debugging notes around your code...
class Test {
public static void main(String[] args) {
List<Runnable> list = new LinkedList<>();
list.add(() -> {
try {
System.out.println("Long sleep - " + Thread.currentThread().getName());
Thread.sleep(10000);
System.out.println("Time : " + System.nanoTime() + " " + "Slow task");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
for (int i = 0; i < 1000; i++) {
int j = i;
list.add(() -> System.out.println("Time : " + System.nanoTime() + " " + j));
}
list.parallelStream().forEach(r -> {
System.out.println(Thread.currentThread().getName());
r.run();
System.out.println();
});
}
}
Upon running this, I observe the following message come up:
Long sleep - ForkJoinPool.commonPool-worker-4
...and about ten seconds later...
Time : 11525122027429 Slow task
ForkJoinPool.commonPool-worker-4
Time : 11525122204035 0
ForkJoinPool.commonPool-worker-4
Time : 11525122245739 1
ForkJoinPool.commonPool-worker-4
Time : 11525122267015 2
ForkJoinPool.commonPool-worker-4
Time : 11525122286921 3
ForkJoinPool.commonPool-worker-4
Time : 11525122306266 4
ForkJoinPool.commonPool-worker-4
Time : 11525122338787 5
ForkJoinPool.commonPool-worker-4
Time : 11525122357288 6
ForkJoinPool.commonPool-worker-4
Time : 11525122376716 7
ForkJoinPool.commonPool-worker-4
Time : 11525122395218 8
ForkJoinPool.commonPool-worker-4
Time : 11525122414165 9
ForkJoinPool.commonPool-worker-4
Time : 11525122432755 10
ForkJoinPool.commonPool-worker-4
Time : 11525122452805 11
ForkJoinPool.commonPool-worker-4
Time : 11525122472624 12
ForkJoinPool.commonPool-worker-4
Time : 11525122491380 13
ForkJoinPool.commonPool-worker-4
Time : 11525122514417 14
ForkJoinPool.commonPool-worker-4
Time : 11525122534550 15
ForkJoinPool.commonPool-worker-4
Time : 11525122553751 16
So this implies that on my box, worker-4 had some work slated for it which couldn't be stolen based on the fact that there were some uneven chunks. Note: if a thread is processing a task in a chunk, that work isn't going to be broken up any further.
[31, 31, 31, 32, 31, 31, 31, 32, 31, 31, 31, 32, 31, 31, 31, 32, 31, 31, 31, 32, 31, 31, 31, 32, 31, 31, 31, 32, 31, 32, 31, 32, 0]
If you were looking for a threading implementation which could steal work from threads which ran longer, it'd be best to use the work-stealing pool directly.
class Test {
public static void main(String[] args) throws InterruptedException {
List<Runnable> list = new LinkedList<>();
list.add(() -> {
try {
System.out.println("Long sleep - " + Thread.currentThread().getName());
Thread.sleep(10000);
System.out.println("Time : " + System.nanoTime() + " " + "Slow task");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
for (int i = 0; i < 1000; i++) {
int j = i;
list.add(() -> {
System.out.println(Thread.currentThread().getName());
System.out.println("Time : " + System.nanoTime() + " " + j);
System.out.println();
});
}
final ExecutorService stealingPool = Executors.newWorkStealingPool();
list.forEach(stealingPool::execute);
stealingPool.shutdown();
stealingPool.awaitTermination(15, TimeUnit.SECONDS);
}
}
The above prints out a more tenable and more reasonable result at the end of the list:
Time : 12210445469314 Slow task
...which implies that all available work has been processed in the time allotted (15 seconds).

Java Spring - manage threads for sync requests

I have Java web application with REST calls using SPRING.
I want to control the number of threads the application is opening for the requests.
So I added Thread config:
package myPackage;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
#Configuration
public class ThreadConfig {
#Bean
public TaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(1);
executor.setMaxPoolSize(1);
executor.initialize();
return executor;
}
}
I'm using Sync service not Async, I tested it and it doesn't limit the threads handling the requests, it handles them all at the same time.
What I was expecting is when I send 2 requests at a time - either the 2nd request will be thrown or it will wait until the 1st request will finish.
I'm not implementing Thread in my application at all.
This is the relevant code from my controller:
#RestController
public class Module1Controller {
#RequestMapping(method = RequestMethod.GET, path = "/module1")
InterruptedException {
public Module1 Module1() throws InterruptedException {
Date startDate = new Date();
System.out.println("Thread #: " + Thread.currentThread().getId() + " Request received at: " + startDate);
Thread.sleep(10000);
Date endDate = new Date();
long diff = endDate.getTime() - startDate.getTime();
long seconds = TimeUnit.MILLISECONDS.toSeconds(diff);
System.out.println("Thread #: " + Thread.currentThread().getId() + " thread released at: " + endDate + ", total seconds: " + seconds);
return new Module1(new Clock());
}
This is the console result:
Thread #: 34 Request received at: Sun Dec 17 10:16:20 IST 2017
Thread #: 35 Request received at: Sun Dec 17 10:16:21 IST 2017
Thread #: 34 thread released at: Sun Dec 17 10:16:30 IST 2017, total seconds: 10
Thread #: 35 thread released at: Sun Dec 17 10:16:31 IST 2017, total seconds: 10
What am I missing here?

The problem is that the creation of a TaskExecutor in a configuration bean has no effect on your RestController.
The easiest way to make your RestController process only 1 request at a time is to make the handling method synchronized, e.g. like this:
#RequestMapping(method = RequestMethod.GET, path = "/module1")
public synchronized Module1 getModule1() throws InterruptedException {
If you want a certain maximum number of requests to be processed simultaneously you can use a FixedThreadPool, e.g. like this:
// allow only 2 requests at a time, more requests are automatically placed in a queue
private final ExecutorService es = Executors.newFixedThreadPool(2);
#RequestMapping(method = RequestMethod.GET, path = "/module1")
public Module1 getModule1() throws ExecutionException, InterruptedException {
Future<Module1> result = es.submit(new Callable<Module1>() {
#Override
public String call() throws Exception {
try {
//.... do your work here....
return Module1()
} catch (InterruptedException e) {
return null;
}
}
});
return result.get();
}
I'm not sure why you would want to do this. Limiting the number of requests will result in bad performance and users are not going to like this.

You can not control the threads of request in application instead in container. Maybe you want to run some tasks in limited threads in application. You can do like this:
#RestController
public class ThreadController {
#Autowired
private TaskExecutor taskExecutor;
#RequestMapping(method = RequestMethod.GET, path = "/thread")
public void Module1() {
taskExecutor.execute(new Runnable() {
#Override
public void run() {
Date startDate = new Date();
System.out.println("Thread #: " + Thread.currentThread().getId() +
" Request received at: " + startDate);
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
Date endDate = new Date();
long diff = endDate.getTime() - startDate.getTime();
long seconds = TimeUnit.MILLISECONDS.toSeconds(diff);
System.out.println("Thread #: " + Thread.currentThread().getId() +
" thread released at: " + endDate + ", total seconds: " + seconds);
}
});
}
}
The result:
Thread #: 55 Request received at: Sun Dec 17 22:40:57 CST 2017
Thread #: 55 thread released at: Sun Dec 17 22:41:07 CST 2017, total seconds: 10
Thread #: 55 Request received at: Sun Dec 17 22:41:16 CST 2017
Thread #: 55 thread released at: Sun Dec 17 22:41:26 CST 2017, total seconds: 10
Thread #: 55 Request received at: Sun Dec 17 22:41:32 CST 2017
Thread #: 55 thread released at: Sun Dec 17 22:41:42 CST 2017, total seconds: 10

schedule a task after a delay in executor service

I want a task to be executed after a delay of 3 seconds and my one task takes 2 seconds to finish.
The output i am getting is showing interval of 5 seconds
Note:The Student class implements Callable interface
I have the following queries
Why there is delay of 5 seconds coming.How can make a delay of 3
seconds Why are thread 1 is showed in second execution ,it should be
thread two
The output i am getting is
The time is : Sat Nov 26 15:08:02 IST 2016
Doing a task during : prerna - Time - Sat Nov 26 15:08:06 IST 2016
pool-1-thread-1 Helloprerna
Doing a task during : abc - Time - Sat Nov 26 15:08:11 IST 2016
pool-1-thread-1 Helloabc
Doing a task during : def - Time - Sat Nov 26 15:08:16 IST 2016
pool-1-thread-2 Hellodef
Doing a task during : xyz - Time - Sat Nov 26 15:08:21 IST 2016
pool-1-thread-1 Helloxyz
Doing a task during : ritu - Time - Sat Nov 26 15:08:26 IST 2016
pool-1-thread-3 Helloritu
Doing a task during : babita - Time - Sat Nov 26 15:08:31 IST 2016
pool-1-thread-2 Hellobabita
The code:
private String display(String name2) {
try {
// System.out.println(Thread.currentThread().getName());
name2=Thread.currentThread().getName()+" Hello"+ name;
System.out.println("Doing a task during : " + name + " - Time - " + new Date());
Thread.sleep(000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return name2;
}
#Override
public String call() throws Exception {
// TODO Auto-generated method stub
if (name == "archana") {
throw new Exception();
}
/*} catch (Exception e) {
// TODO Auto-generated catch block
// e.printStackTrace();
}finally{
return "error";
}*/
return display(name);
}
public class ExecutorScheduleDemo {
public static void main(String args[]) throws InterruptedException{
ScheduledExecutorService executor= Executors.newScheduledThreadPool(5);
ArrayList<Student> list = new ArrayList<Student>();
list.add(new Student("prerna"));
list.add(new Student("abc"));
//list.add(new Student("archana"));
list.add(new Student("def"));
list.add(new Student("xyz"));
list.add(new Student("ritu"));
list.add(new Student("babita"));
System.out.println("The time is : " + new Date());
List<Future<String>> resultList= new ArrayList<Future<String>>();
for(Student s:list){
Future<String> f=executor.schedule(s, 3, TimeUnit.SECONDS);
try {
System.out.println(f.get());
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}

Use scheduleAtFixedRate(Runnable, long initialDelay, long period, TimeUnit timeunit) instead of schedule(Runnable task, long delay, TimeUnit timeunit).
scheduleAtFixedRate (Runnable, long initialDelay, long period,
TimeUnit timeunit)
Creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given period; that is executions will commence after initialDelay then initialDelay+period, then initialDelay + 2 * period, and so on. If any execution of the task encounters an exception, subsequent executions are suppressed. Otherwise, the task will only terminate via cancellation or termination of the executor. If any execution of this task takes longer than its period, then subsequent executions may start late, but will not concurrently execute.
next execution.

To complete eeedev's answer, since your object seems to be a Callable:
You can simply create a new FutureTask by passing your Callable to the constructor, as described in the oracle docs
Note that the FutureTask's type parameter must be the same as the Callable's.
Example:
class Main {
public static void main(String[] args) {
Foo foo = new Foo();
FutureTask<String> fooFutureTask = new FutureTask<>(foo);
}
}
class Foo implements Callable<String> {
#Override
public String call() throws Exception {
return "Calling";
}
}
You can then schedule your newly created FutureTask for execution as described by eeedev.

Quartz Scheduler: How to Group Jobs together?

I wanted to ask if anybody had the same problem with the Quartz scheduler. I created Jobs with Trigger and JobKeys where i set the Groupnames. But when i print out the group that has been set it is always DEFAULT.
How can i Set this groupname to finally be able to group Jobs together and most importantly cancel only specified groups? With a Code similar like this:
public void unscheduleByGroupname(String groupName) throws SchedulerException {
for (JobKey jobKey : scheduler.getJobKeys(GroupMatcher.jobGroupEquals(groupName))) {
scheduler.unscheduleJob(new TriggerKey(jobKey.getName(), jobKey.getGroup()));
}
}
Input:
TriggerKey tKey = new TriggerKey("Trigger:" + jobName + "-Somename:" + object.toString(),
"Group:" + jobName + "-Somename:" + object.toString());
JobKey jKey = new JobKey("Job:" + jobName + "-Somename:" + object.toString(),
"Group:" + jobName + "-Somename:" + object.toString());
JobDetail job = JobBuilder.newJob(Somename.class).withDescription("Somename")
.withIdentity(jKey).build();
Trigger trigger = TriggerBuilder.newTrigger().forJob(jKey).startAt(new Date()).withIdentity(tKey).build();
Output Function:
for (String groupName : scheduler.getJobGroupNames()) {
for (JobKey jobKey : scheduler.getJobKeys(GroupMatcher.jobGroupEquals(groupName))) {
String jobName = jobKey.getName();
String jobGroup = jobKey.getGroup();
// get job's trigger
List<Trigger> triggers = (List<Trigger>) scheduler.getTriggersOfJob(jobKey);
Date nextFireTime = triggers.get(0).getNextFireTime();
System.out.println("[jobName] : " + jobName + " [groupName] : " + jobGroup + " - " + nextFireTime);}
Output:
[jobName] : Job:-Somename:13 [groupName] : DEFAULT - Tue Jul 19 13:48:40 CEST 2016
[jobName] : Job:-Somename:14 [groupName] : DEFAULT - Tue Jul 19 13:49:11 CEST 2016
[jobName] : Job:-Somename:15 [groupName] : DEFAULT - Tue Jul 19 13:49:41 CEST 2016
[jobName] : Job:-Somename:16 [groupName] : DEFAULT - Tue Jul 19 13:50:11 CEST 2016

When you are seeing up the job identity, you can add the group info. I chain the methods like below, and it works for me (I can see that the group is the desired name that I set):
JobDetail job = JobBuilder.newJob(ScheduledJob.class)
.withIdentity("JOB KEY", "GROUP NAME")
.withDescription("Job description")
.usingJobData(dataMap)
.build();

Java 8 forEach Stream() vs old forEach Loop

I was trying a example code with spring. And a part of code is like below;
private List<Point> points;
long timeTakeninMilis = System.currentTimeMillis();
public List<Point> getPoints() {
return points;
}
public void setPoints(List<Point> points) {
this.points = points;
}
public void drawJava8() {
points.stream().forEachOrdered(
point -> System.out.println("Point : (" + point.getX() + ", "
+ point.getY() + ")"));
System.out.println("Total Time Taken drawJava8(): "
+ (System.currentTimeMillis() - timeTakeninMilis)
+ " miliseconds");
}
public void draw() {
for (Point point : points) {
System.out.println("Point = (" + point.getX() + ", " + point.getY()
+ " )");
}
System.out.println("Total Time Taken draw(): "
+ (System.currentTimeMillis() - timeTakeninMilis)
+ " miliseconds");
}
The OUTPUT,
Jun 30, 2015 11:30:53 AM org.springframework.context.support.ClassPathXmlApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext#7daf6ecc: startup date [Tue Jun 30 11:30:53 IST 2015]; root of context hierarchy
Jun 30, 2015 11:30:53 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring.xml]
Point = (0, 0 )
Point = (-50, 0 )
Point = (0, 50 )
Total Time Taken draw(): 70 miliseconds
Point : (0, 0)
Point : (-50, 0)
Point : (0, 50)
Total Time Taken drawJava8(): 124 miliseconds
Jun 30, 2015 11:30:54 AM org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext#7daf6ecc: startup date [Tue Jun 30 11:30:53 IST 2015]; root of context hierarchy
Why it is taking more time?
Or i am doing something wrong?
I was expecting it to be faster or of similar speed...
Please help me understand what is the benefit of the Lambda Expressions?
INFO: I did it in two different programs. Times are taken from those. I merged them here to make it short.

Adding this as an analysis per original poster's request.
We can not really predict the sophisticated analysis and transformation that the modern JIT compiler performs on running code. Hence while benchmarking items such as these, you should not conclude it just by running two method calls.
Instead, create various sample input sets (boundary cases) and check the perofmance by repeatedly calling your test cases without shutting down JVM. In this case for example :
for (int i=0;i<100;i++){draw(); drawJava8();}
Once you have the results, find out average execution and you can safely ignore first execution result as it might not have had optimizations.
So the conclusion you have drawn from your tests is not completely correct.