Develop Reference

How do I complete numerous jobs with a few threads giving each the same timeout to complete?

An idea I am trying to implement is the following.
I have 1000 urls to download data from to use it for post processing (say, calculating some statistics).
I don't really need all of the downloads to finish successfully, but as many as possible.
I assume that some of the locations might be unavailable, either responding nothing valuable (e.g., HTTP 503) or taking more that TO=10 seconds of time to process a request.
I have T=5 threads to process the urls in parallel, giving the equal timeout TO to each.
As soon as one completes (what I expect to happen far earlier that TO exceeds) I aggregate some statistics (what is a very fast operation) and start the next download (if any).
The solution I have come up so far with is
ExecutorService executorService = Executors.newFixedThreadPool(T);
ExecutorCompletionService<MyResult> completionService = new ExecutorCompletionService<>(executorService);
urls.forEach(url -> {
Callable<MyResult> callable = () -> new MyResult(url);
completionService.submit(callable);
});
for (int i = 0; i < urls.size(); i++) {
Future<MyResult> resultFuture = completionService.poll(TO, TimeUnit.SECONDS);
if (resultFuture == null)
continue;
MyResult myResult = resultFuture.get();
myAggregate(myResult.getRate());
}
It looks like somewhat I am trying to achieve. But it for instance neither gives every download the same timeout nor cancels the Futures properly. So, what is the correct solution?

Try using the invokeAll-Method, you simply put your Callables in a List and then call invokeAll() on your ExecutorService giving it a timeout as second and third argument.
executorService.invokeAll(callableList, 20, TimeUnit.SECONDS);

concurrent http request to independent web services

i'm trying to find a simple way to send http request concurrently to diferent web services. each request is completely independent of each other.
currently, my implementation look like this ( just a simplification, don't pay attention to design )
let's say a i have a List queries;
public class Service {
private List<HttpClient> httpClients; // one for each web service
public List<QueryResult> doQueries(List<Query> queries) {
ExecutorService service = Executors.... ;
List<Callable<QueryResult>> .... ;
for ( Query q : queries ) {
Future<> .....
}
service.invokeAll(...) ;
***// what should i do from here ?
// how should i wait all those tasks to finish ?***
}
}
my question is specifically that.
how do i wait ?

You seem to create a list of Callable and each callable will return result of type QueryResult as clear from List<Callable<QueryResult>>. You will get Future after submitting them to ExecutorService. So use code in this way:
List<Future<QueryResult >> futures = executorService.invokeAll(callables);
for(Future<QueryResult> future : futures){
System.out.println("future.get = " + future.get());
}
executorService.shutdown();
If you want to set some maximum time to wait for result you can use awaitTermination method as well. IMO ExecutorCompletionService is more suited for your requirements and you can read about it in my article at dzone.

You have 3 choices:
execute each request on a separate thread. Since each thread consumes a lot of memory, you can get OutOfMemoreError if >100 requests run in parallel.
limit the number of threads as akhil_mittal suggested. The number of concurrent requests will be also limited.
Use an async io library, e.g. nio2. They allow thousands of simultaneous requests with moderate memory consumption.

How to improve the performance while using ExecutorService with thread timeout capabilities?

I am not a Multithreading Expert but I am seeing some performance issues with my current code which is using ExecutorService.
I am working on a project in which I need to make a HTTP URL call to my server and if it is taking too long time to respond then timeout the call. Currently it is returning simple JSON String back..
Current requirement I have is for 10 ms. Within 10 ms it should be able to get the data back from the server. I guess its possible since it is just an HTTP call to server within the same datacenter.
My client program and actual servers are within same datacenter and ping time latency is 0.5 ms between them so it should be doable for sure..
I am using RestTemplate for this to make the URL call.
Below is my code which I have wrote for me which uses ExecutorService and Callables -
public class URLTest {
private ExecutorService executor = Executors.newFixedThreadPool(10);
public String getData() {
Future<String> future = executor.submit(new Task());
String response = null;
try {
System.out.println("Started..");
response = future.get(100, TimeUnit.MILLISECONDS);
System.out.println("Finished!");
} catch (TimeoutException e) {
System.out.println("Terminated!");
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
return response;
}
}
Below is my Task class which implements Callable interface -
class Task implements Callable<String> {
private RestTemplate restTemplate = new RestTemplate();
public String call() throws Exception {
// TimerTest timer = TimerTest.getInstance(); // line 3
String response = restTemplate.getForObject(url, String.class);
// timer.getDuration(); // line 4
return response;
}
}
And below is my code in another class DemoTest which calls the getData method in URLTest class 500 times and measure the 95th percentile of it end to end -
public class DemoTest {
public static void main(String[] args) {
URLTest bc = new URLTest();
// little bit warmup
for (int i = 0; i <= 500; i++) {
bc.getData();
}
for (int i = 0; i <= 500; i++) {
TimerTest timer = TimerTest.getInstance(); // line 1
bc.getData();
timer.getDuration(); // line 2
}
// this method prints out the 95th percentile
logPercentileInfo();
}
}
With the above code as it is, I am always seeing 95th percentile as 14-15 ms (which is bad for my use case as it is end to end flow and that's what I need to measure).
I am surprised why? Is ExectuorFramework adding all the latency here?. May be Each task is submitted, and the submitting thread is waiting (via future.get) until the task is finished..
My main goal is to reduce the latency here as much as possible.. My use case is simple, Make a URL call to one of my server with a TIMEOUT feature enabled, meaning if the server is taking lot of time to response, then Timeout the whole call. Customer will call our code from there application which can be multithreaded as well..
Is there anything I am missing or some other flavors of ExecutorService I need to use? How can I improve my performance here? Any suggestions will be of great help..
Any example will be greatly appreciated.. I was reading about ExecutorCompletionService not sure whether I should use this or something else..

As for your observation that you are measuring 15 ms on the outside, but only 3 ms on the inside, my bet is that the construction of the RestTemplate takes the difference. This could be fixed by refactoring.
Note that RestTemplate is a heavyweight, thread-safe object, and is designed to be deployed as an application-wide singleton. Your current code is in critical violation of this intent.
If you need asynchronous HTTP requests, you should really use an asynchronous HTTP library such an AsyncHttpClient, based on Netty underneath, which is again based on Java NIO. That means that you don't need to occupy a thread per an outstanding HTTP request. AsyncHttpClient also works with Futures so you'll have an API you are used to. It can also work with callbacks, which is preferred for the asynchronous approach.
However, even if you keep your current synchronous library, you should at the very least configure a timeout on the REST client instead of letting it run its course.

then run the program again it will start giving me 95th percentile as 3 ms. So not sure why end to end flow gives me 95th percentile as 14-15 ms
You are generating the tasks faster than you can process them. This means the longer you run the tests, the further behind it gets as it is queuing them up. I would expect if you made this 2000 requests you would see latencies up to 4x what you do now. The bottlneck could be on the client side (in which case more threads would help) but quite likely the bottleneck is on the server side in which case more threads could make it worse.
The default behaviour for HTTP is to establish a new TCP connection for each request. The connection time for a new TCP connection can be up to 20 ms easily even if you have two machines side by side. I suggest looking at using HTTp/1.1 and maintain a persistent connection.
BTW You can ping from one side of London to the other in 0.5 ms. However, getting below 1 ms with HTTP reliably is tricky as the protocol is not designed for low latency. It is designed for use on high latency networks.
Note: you cannot see latencies below 25 ms and 100 ms is plenty fast enough for most web requests. It is with these sort of assumption that HTTP was designed.

Scheduled Task using ExecutorService

I would like to use ExecutorService in JAVA to schedule the insertion into database every 5 min.
this the task that i want to execute it:
MyClass{
a counter that count a handled data received from a Thread
I receive usually 300 data line/second
Apply a treatment on these data and keep the results in the memory
every five minutes {
update the database with the counters saved in memory*
}
}
Basically it's calling the task each time he get a data from thread running in the background.
and as i have more then 300 data/sec it's impossible to use it in this way.
So what i am trying to do id to handle the received tasks and keep a counter in the memory and update the database only each 5 min.
My question, is it possible to use these java function ScheduledExecutorService to do this, and how can we do it ( i don't want to block my JAVA application on the task for 5min, i want that that application run normally but without executing the task every time, i appreciate if you can show me an example of usage for these function ?

ScheduledExecutorService service = Executors.newScheduledThreadPool(1);
service.scheduleAtFixedRate(command, 5, 5, TimeUnit.MINUTES);
Where command is:
Runnable command = new Runnable() {
public void run() {
// update database
}
}

java : execute a method over a maximum period of time

I am using the JavaMail API , and there is a method in the Folder class called "search" that sometimes take too long to execute. What i want is to execute this method over a maximum period of time( say for example 15 seconds in maximum) , that way i am sure that this method will not run up more than 15 seconds.
Pseudo Code
messages = maximumMethod(Folder.search(),15);
Do I have to create a thread just to execute this method and in the main thread use the wait method ?

The best way to do this is create a single threaded executor which you can submit callables with. The return value is a Future<?> which you can get the results from. You can also say wait this long to get the results. Here is sample code:
ExecutorService service = Executors.newSingleThreadExecutor();
Future<Message[]> future = service.submit(new Callable<Message[]>() {
#Override
public Message[] call() throws Exception {
return Folder.search(/*...*/);
}
});
try {
Message[] messages = future.get(15, TimeUnit.SECONDS);
}
catch(TimeoutException e) {
// timeout
}

You could
mark current time
launch a thread that will search in the folder
while you get the result (still in thread) don't do anything if current time exceeds time obtained in 1 plus 15 seconds. You won't be able to stop the connection if it is pending but you could just disgard a late result.
Also, if you have access to the socket used to search the folder, you could set its timeout but I fear it's gonna be fully encapsulated by javamail.
Regards,
Stéphane

This SO question shows how to send a timeout exception to the client code: How do I call some blocking method with a timeout in Java?
You might be able to interrupt the actual search using Thread.interrupt(), but that depends on the method's implementation. You may end up completing the action only to discard the results.