Develop Reference

How to get the execution results of ExecutorService without blocking the current code path?

I have a service which adds a bunch of requests to Callables and then prints the results of the executions. Currently the service request is blocked until I print all the Future results from the execution. However I want to return 200 to the requestor and run these requests in parallel without blocking the request. How can I achieve this? Below is my code.
Below is my code to run parallel code.
public void runParallelFunctions(Callable<Map<String, String>> invokerTask) {
List<Callable<Map<String, String>>> myTasks = new ArrayList<>();
for (int i = 0; i < invocationCount; i++) {
myTasks.add(invokerTask);
}
List<Future<Map<String, String>>> results = null;
try {
results = executorService.invokeAll(myTasks);
} catch (InterruptedException e) {
}
this.printResultsFromParallelInvocations(results);
}
Below is how I print the results from the Futures.
private void printResultsFromParallelInvocations(List<Future<Map<String, String>>> results) {
results.forEach(executionResults -> {
try {
executionResults.get().entrySet().forEach(entry -> {
LOGGER.info(entry.getKey() + ": " + entry.getValue());
});
} catch (InterruptedException e) {
} catch (ExecutionException e) {
}
});
}
Below is how I'm invoking the above methods when someone places a request to the service.
String documentToBeIndexed = GSON.toJson(indexDocument);
int documentId = indexMyDocument(documentToBeIndexed);
createAdditionalCandidatesForFuture(someInput);
return true;
In the above code, I call the createAdditionalCandidatesForFuture and then return true. But the code still waits for the printResultsFromParallelInvocations method to complete. How can I make the code return after invoking createAdditionalCandidatesForFuture without waiting for the results to print? Do I have to print the results using another executor thread or is there another way? Any help would be much appreciated

The answer is CompletableFuture.
Updated runParallelFunctions:
public void runParallelFunctions(Callable<Map<String, String>> invokerTask) {
// write a wrapper to handle exception outside CompletableFuture
Supplier<Map<String, String>> taskSupplier = () -> {
try {
// some task that takes a long time
Thread.sleep(4000);
return invokerTask.call();
} catch (Exception e) {
System.out.println(e);
}
// return default value on error
return new HashMap<>();
};
for (int i = 0; i < 5; i++) {
CompletableFuture.supplyAsync(taskSupplier, executorService)
.thenAccept(this::printResultsFromParallelInvocations);
}
// main thread immediately comes here after running through the loop
System.out.println("Doing other work....");
}
And, printResultsFromParallelInvocations may look like:
private void printResultsFromParallelInvocations(Map<String, String> result) {
result.forEach((key, value) -> System.out.println(key + ": " + value));
}
Output:
Doing other work....
// 4 secs wait
key:value

Calling get on a Future will block the thread until the task is completed, so yes, you will have to move the printing of the results to another thread/Executor service.
Another option is that each task prints its results upon completion, provided they are supplied with the necessary tools to do so (Access to the logger, etc). Or putting it in another way, each task is divided into two consecutive steps: execution and printing.

I have tried to optimize (memory) my program, but GC is still making it lag

I have written a piece of software in Java that checks if proxies are working by sending a HTTP request using the proxy.
It takes around 30,000 proxies from a database, then attempts to check if they are operational. The proxies received from the database used to be returned as an ArrayList<String>, but have been changed to Deque<String> for reasons stated below.
The way the program works is there is a ProxyRequest object that stores the IP & Port as a String and int respectively. The ProxyRequest object has a method isWorkingProxy() which attempts to send a request using a proxy and returns a boolean on whether it was successful.
This ProxyRequest object is wrapped around by a RunnableProxyRequest object that calls super.isWorkingProxy() in the overrided run() method. Based on the response from super.isWorkingProxy(), the RunnableProxyRequest object updates a MySQL database.
Do note that the updating of the MySQL database is synchronized().
It runs on 750 threads using a FixedThreadPool (on a VPS), but towards
the end, it becomes very slow (stuck on ~50 threads), which obviously
implies the garbage collector is working. This is the problem.
I have attempted the following to improve the lag, it does not seem to work:
1) Using a Deque<String> proxies and using Deque.pop() to obtain the String in which the proxy is. This (I believe), continuously makes the Deque<String> smaller, which should improve lag caused by the GC.
2) Set the con.setConnectTimeout(this.timeout);, where this.timeout = 5000; This way, the connection should return a result in 5 seconds. If not, the thread is completed and should no longer be active in the threadpool.
Besides this, I don't know any other way I can improve performance.
Can anyone recommend a way for me to improve performance to avoid / stop lagging towards the end of the threads by the GC? I know there is a Stackoverflow question about this (Java threads slow down towards the end of processing), but I have tried everything in the answer and it has not worked for me.
Thank you for your time.
Code snippets:
Loop adding threads to the FixedThreadPool:
//This code is executed recursively (at the end, main(args) is called again)
//Create the threadpool for requests
//Threads is an argument that is set to 750.
ThreadPoolExecutor executor = (ThreadPoolExecutor)Executors.newFixedThreadPool(threads);
Deque<String> proxies = DB.getProxiesToCheck();
while(proxies.isEmpty() == false) {
try {
String[] split = proxies.pop().split(":");
Runnable[] checks = new Runnable[] {
//HTTP check
new RunnableProxyRequest(split[0], split[1], Proxy.Type.HTTP, false),
//SSL check
new RunnableProxyRequest(split[0], split[1], Proxy.Type.HTTP, true),
//SOCKS check
new RunnableProxyRequest(split[0], split[1], Proxy.Type.SOCKS, false)
//Add more checks to this list as time goes...
};
for(Runnable check : checks) {
executor.submit(check);
}
} catch(IndexOutOfBoundsException e) {
continue;
}
}
ProxyRequest class:
//Proxy details
private String proxyIp;
private int proxyPort;
private Proxy.Type testingType;
//Request details
private boolean useSsl;
public ProxyRequest(String proxyIp, String proxyPort, Proxy.Type testingType, boolean useSsl) {
this.proxyIp = proxyIp;
try {
this.proxyPort = Integer.parseInt(proxyPort);
} catch(NumberFormatException e) {
this.proxyPort = -1;
}
this.testingType = testingType;
this.useSsl = useSsl;
}
public boolean isWorkingProxy() {
//Case of an invalid proxy
if(proxyPort == -1) {
return false;
}
HttpURLConnection con = null;
//Perform checks on URL
//IF any exception occurs here, the proxy is obviously bad.
try {
URL url = new URL(this.getTestingUrl());
//Create proxy
Proxy p = new Proxy(this.testingType, new InetSocketAddress(this.proxyIp, this.proxyPort));
//No redirect
HttpURLConnection.setFollowRedirects(false);
//Open connection with proxy
con = (HttpURLConnection)url.openConnection(p);
//Set the request method
con.setRequestMethod("GET");
//Set max timeout for a request.
con.setConnectTimeout(this.timeout);
} catch(MalformedURLException e) {
System.out.println("The testing URL is bad. Please fix this.");
return false;
} catch(Exception e) {
return false;
}
try(
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
) {
String inputLine = null; StringBuilder response = new StringBuilder();
while((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
//A valid proxy!
return con.getResponseCode() > 0;
} catch(Exception e) {
return false;
}
}
RunnableProxyRequest class:
public class RunnableProxyRequest extends ProxyRequest implements Runnable {
public RunnableProxyRequest(String proxyIp, String proxyPort, Proxy.Type testingType, boolean useSsl) {
super(proxyIp, proxyPort, testingType, useSsl);
}
#Override
public void run() {
String test = super.getTest();
if(super.isWorkingProxy()) {
System.out.println("-- Working proxy: " + super.getProxy() + " | Test: " + test);
this.updateDB(true, test);
} else {
System.out.println("-- Not working: " + super.getProxy() + " | Test: " + test);
this.updateDB(false, test);
}
}
private void updateDB(boolean success, String testingType) {
switch(testingType) {
case "SSL":
DB.updateSsl(super.getProxyIp(), super.getProxyPort(), success);
break;
case "HTTP":
DB.updateHttp(super.getProxyIp(), super.getProxyPort(), success);
break;
case "SOCKS":
DB.updateSocks(super.getProxyIp(), super.getProxyPort(), success);
break;
default:
break;
}
}
}
DB class:
//Locker for async
private static Object locker = new Object();
private static void executeUpdateQuery(String query, String proxy, int port, boolean toSet) {
synchronized(locker) {
//Some prepared statements here.
}
}

Thanks to Peter Lawrey for guiding me to the solution! :)
His comment:
#ILoveKali I have found network libraries are not aggressive enough in
shutting down a connection when things go really wrong. Timeouts tend
to work best when the connection is fine. YMMV
So I did some research, and found that I had to also use the method setReadTimeout(this.timeout);. Previously, I was only using setConnectTimeout(this.timeout);!
Thanks to this post (HttpURLConnection timeout defaults) that explained the following:
Unfortunately, in my experience, it appears using these defaults can
lead to an unstable state, depending on what happens with your
connection to the server. If you use an HttpURLConnection and don't
explicitly set (at least read) timeouts, your connection can get into
a permanent stale state. By default. So always set setReadTimeout to
"something" or you might orphan connections (and possibly threads
depending on how your app runs).
So the final answer is: The GC was doing just fine, it was not responsible for the lag. The threads were simply stuck FOREVER at a single number because I did not set the read timeout, and so the isWorkingProxy() method never got a result and kept reading.

An Efficient concurrent data structure to wait for a computed value (or timeout)

I'm hoping some concurrency experts can advise as I'm not looking to rewrite something that likely exists.
Picture the problem; I have a web connection that comes calling looking for their unique computed result (with a key that they provide in order to retrieve their result) - however the result may not have been computed YET so I would like for the connection to wait (block) for UP TO n seconds before giving up and telling them I don't (yet) have their result (computation time to calculate value is non deterministic). something like;
String getValue (String key)
{
String value = [MISSING_PIECE_OF_PUZZLE].getValueOrTimeout(key, 10, TimeUnit.SECONDS)
if (value == null)
return "Not computed within 10 Seconds";
else
return "Value was computed and was " + value;
}
and then have another thread (the computation threads)that is doing the calculations - something like ;
public void writeValues()
{
....
[MISSING_PIECE_OF_PUZZLE].put(key, computedValue)
}
In this scenario, there are a number of threads working in the background to compute the values that will ultimately be picked up by a web connections. The web connections have NO control or authority over what is computed and when the computations execute - as I've said - this is being done in a pool in the background but these thread can publish when the computation has completed (how they do is the gist of this question). The publish message maybe consumed or not - depending if any subscribers are interested in this computed value.
As these are web connections that will be blocking - i could potentially have 1000s of concurrent connections waiting (subscribing) for their specific computed value so such a solution needs to be very light on blocking resources. The closest i've came to is this SO question which I will explore further but wanted to check i'm not missing something blindly obvious before writing this myself?

I think you should use a Future it gives an ability to compute data in a separate thread and block for the requested time period while waiting for an answer. Notice how it throws an exception if more then 3 seconds passed
public class MyClass {
// Simulates havy work that takes 10 seconds
private static int getValueOrTimeout() throws InterruptedException {
TimeUnit.SECONDS.sleep(10);
return 123;
}
public static void main(String... args) throws InterruptedException, ExecutionException {
Callable<Integer> task = () -> {
Integer val = null;
try {
val = getValueOrTimeout();
} catch (InterruptedException e) {
throw new IllegalStateException("task interrupted", e);
}
return val;
};
ExecutorService executor = Executors.newFixedThreadPool(1);
Future<Integer> future = executor.submit(task);
System.out.println("future done? " + future.isDone());
try {
Integer result = future.get(3, TimeUnit.SECONDS);
System.out.print("Value was computed and was : " + result);
} catch (TimeoutException ex) {
System.out.println("Not computed within 10 Seconds");
}
}
}

After looking in changes in your question I wanted to suggest a different approach using BlockingQueue in such case the producer logic completely separated from the consumer so you could do something like this
public class MyClass {
private static BlockingQueue<String> queue = new ArrayBlockingQueue<>(10);
private static Map<String, String> dataComputed = new ConcurrentHashMap<>();
public static void writeValues(String key) {
Random r = new Random();
try {
// Simulate working for long time
TimeUnit.SECONDS.sleep(r.nextInt(11));
String value = "Hello there fdfsd" + Math.random();
queue.offer(value);
dataComputed.putIfAbsent(key, value);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private static String getValueOrTimeout(String key) throws InterruptedException {
String result = dataComputed.get(key);
if (result == null) {
result = queue.poll(10, TimeUnit.SECONDS);
}
return result;
}
public static void main(String... args) throws InterruptedException, ExecutionException {
String key = "TheKey";
Thread producer = new Thread(() -> {
writeValues(key);
});
Thread consumer = new Thread(() -> {
try {
String message = getValueOrTimeout(key);
if (message == null) {
System.out.println("No message in 10 seconds");
} else {
System.out.println("The message:" + message);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
});
consumer.start();
producer.start();
}
}
With that said I have to agree with #earned that making the client thread to wait is not a good approach instead I would suggest using a WebSocket which gives you an ability to push data to the client when it is ready you can find lots of tutorials on WebSocket here is one for example ws tutorial

Retrieving data from multiple tasks from different threads

Good evening,
I have a List of different URLs (about 500) which content I get from this method
public static String getWebContent(URL url){
// create URL, build HTTPConnection, getContent of page
}
after this I have another method where the content is fetched for values etc.
At this time I do it like this:
List<URL> urls = new ArrayList<>();
List<String> webcontents = new ArrayList<>();
for(URL url : urls){
webcontents.add(getWebContent(url));
}
// Futher methods to extract values from the webcontents
But it actually takes a lot of time, because there is only one Thread doing it. I wanted to make it multithreaded, but I am not sure what's the best way how to do it.
First, I need the return value of every Thread, should I implement Callable instead of Runnable for it?
And how do I run the method with different Threads, should there be one starting with index 0, one with index 50, etc.? And when they are done with one URL they set a flag to true? That would be my way, but it is not very effective I think. If the first website has a lot of content, the first Thread might take much longer then the others.
And when every Thread is done, how I can my data back to one list? Like this?
List<String> webcontent = new ArrayList<>();
if(!t1.isAlive() && !t2.isAlive()){
webcontent.add(t1.getData());
webcontent.add(t2.getData());
}
I hope you can understand my problem and can give me a tip :) Many thanks

You could use an ExecutorCompletionService to retrieve your tasks as they complete.
List<URL> urls = ...; // Create this list somehow
ExecutorCompletionService<String> service =
new ExecutorCompletionService<String>(Executors.newFixedThreadPool(10));
for (URL url: urls) {
service.submit(new GetWebContentCallable(url)); // you need to define the GetWebContentCallable
}
int remainingTasks = urls.size();
while (remainingTasks > 0) {
String nextResult = service.take();
processResult(nextResult); // you define processResult
remainingTasks -= 1;
}

Perhaps you could try something like:
public static List<String> getWebContents(final int threads, final URL... urls){
final List<Future<String>> futures = new LinkedList<>();
final ExecutorService service = Executors.newFixedThreadPool(threads);
Arrays.asList(urls).forEach(
url -> {
final Callable<String> callable = () -> {
try{
return getWebContent(url);
}catch(IOException ex){
ex.printStackTrace();
return null;
}
};
futures.add(service.submit(callable));
}
);
final List<String> contents = new LinkedList<>();
futures.forEach(
future -> {
try{
contents.add(future.get());
}catch(Exception ex){
ex.printStackTrace();
}
}
);
service.shutdown();
return contents;
}
Of if you're not using Java 8:
public static List<String> getWebContents(final int threads, final URL... urls){
final List<Future<String>> futures = new LinkedList<Future<String>>();
final ExecutorService service = Executors.newFixedThreadPool(threads);
for(final URL url : urls){
final Callable<String> callable = new Callable<String>(){
public String call(){
try{
return getWebContent(url);
}catch(IOException ex){
ex.printStackTrace();
return null;
}
}
};
futures.add(service.submit(callable));
}
final List<String> contents = new LinkedList<String>();
for(final Future<String> future : futures){
try{
contents.add(future.get());
}catch(Exception ex){
ex.printStackTrace();
}
}
service.shutdown();
return contents;
}

Instead of retrieving values from working threads, let working threads put results in a resulting collection (be it List<String> webcontent or anything else). Note this may require synchronization.

Concurrent requests via Apache Http Client (and java.util.concurrent) seem to finish correctly but crash my OS

i am currently learning to use the the concurrent features of Java provided by the package java.util.concurrent. As an exercise i tried to write a little program that could be used to performance test a HTTP API. But somehow my program is not terminating correctly very often. It even crashes my OS.
Following is the pseudo code of my program:
Instantiate Request Objects, that query an HTTP API (In the example i just query one random site).
Instantiate multiple Callables, where each one represents a represents an Http Call.
Iterate over the Callables and schedule them via a ScheduledExecutorService (how many requests should be performed per second can be configured at the begin of the code).
After scheduling all Callables, i am beginning to iterate over the Futures. If a futures is done, retrieve the response. Do this every second. If no new Future was finished, quit the loop.
What problems am i experiencing in detail?
Lots of times, the program is not finishing correctly. I see all log prints in the console, as if the program is finishing correctly. But actually i am seeing that stop button in eclipse still remains active . If i click it, it says that the program could not be terminated correctly. It does not finish no matter how i long i wait (NOTE: I am starting the program inside eclipse).
I can provoke the error easily if i am increasing the number of Requests. If am turning up to 2000, this will happen for sure. If it happens my OS even crashes, i can still use eclipse, but other apps do not work anymore.
My Environment is Eclipse 3.7 on Mac OS X 10.7 with Java 1.6 and Apache httpclient 4.2.2
Do you spot any major erros in my code? Before i have never had such issues in a java program with crashing my OS and seeing no exceptions at all.
The code:
public class ConcurrentHttpRequestsTest {
/**
* #param args
*/
public static void main(String[] args) {
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(25);
Integer standardTimeout = 5000;
Float numberOfRequestsPerSecond = 50.0f;
Integer numberOfRequests = 500;
Integer durationBetweenRequests = Math.round(1000 / numberOfRequestsPerSecond);
// build Http Request
HttpGet request = null;
request = new HttpGet("http://www.spiegel.de");
// request.addHeader("Accept", "application/json");
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, standardTimeout);
HttpConnectionParams.setSoTimeout(params, standardTimeout);
request.setParams(params);
// setup concurrency logic
Collection<Callable<Long>> callables = new LinkedList<Callable<Long>>();
for (int i = 1; i <= numberOfRequests; i++) {
HttpClient client = new DefaultHttpClient();
callables.add(new UriCallable(request, client));
}
// start performing requests
int i = 1;
Collection<Future<Long>> futures = new LinkedList<Future<Long>>();
for (Callable<Long> callable : callables) {
ScheduledFuture<Long> future = scheduledExecutorService.schedule(callable, i * durationBetweenRequests, TimeUnit.MILLISECONDS);
futures.add(future);
i++;
}
// process futures (check wether they are ready yet)
Integer maximumNoChangeCount = 5;
boolean futuresAreReady = false;
int noChangeCount = 0;
int errorCount = 0;
List<Long> responses = new LinkedList<Long>();
while (!futuresAreReady) {
boolean allFuturesAreDone = true;
boolean atLeast1FutureIsDone = false;
Iterator<Future<Long>> iterator = futures.iterator();
while (iterator.hasNext()) {
Future<Long> future = iterator.next();
allFuturesAreDone = allFuturesAreDone && (future.isDone());
if (future.isDone()) {
try {
atLeast1FutureIsDone = true;
responses.add(future.get());
iterator.remove();
} catch (Exception e) {
// remove failed futures (e.g. timeout)
// System.out.println("Reached catch of future.get()" +
// e.getClass() + " " + e.getCause().getClass() + " " +
// e.getMessage());
iterator.remove();
errorCount++;
}
}
if (future.isCancelled()) {
// this code is never reached. Just here to make sure that
// this is not the cause of problems.
System.out.println("Found a cancelled future. Will remove it.");
iterator.remove();
}
}
if (!atLeast1FutureIsDone) {
System.out.println("At least 1 future was not done. Current noChangeCount:" + noChangeCount);
noChangeCount++;
} else {
// reset noChangeCount
noChangeCount = 0;
}
futuresAreReady = allFuturesAreDone;
// log the current state of responses, errors and remaining futures
System.out.println("Size of responses :" + responses.size() + "; Size of futures:" + futures.size() + " Errors:" + errorCount);
if (noChangeCount >= maximumNoChangeCount) {
System.out.println("Breaking while loop becauce no new future finished in the last " + maximumNoChangeCount + " iterations");
break;
}
// check every second
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
for (Long responsetime : responses) {
// analyze responsetimes or whatever
}
// clean up
// .shutdown() made even more problems than shutdownNow()
scheduledExecutorService.shutdownNow();
System.out.println("Executors have been shutdown - Main Method finished. Will exit System.");
System.out.flush();
System.exit(0);
}
private static class UriCallable implements Callable<Long> {
private HttpUriRequest request;
private HttpClient client;
public UriCallable(HttpUriRequest request, HttpClient client) {
super();
this.request = request;
this.client = client;
}
public Long call() throws Exception {
Long start = System.currentTimeMillis();
HttpResponse httpResponse = client.execute(request);
Long end = System.currentTimeMillis();
return end - start;
}
}
}

Never do this in a loop:
} catch (InterruptedException e) {
e.printStackTrace();
}
It might cause problems on shutdown.
Also, most of your code could be replaced by a single call to ExecutorService.invokeAll(), so try that and see if you have more luck.
Lastly, when you don't know what your Java application is doing, run jconsole, attach to the application, and look at the thread stacks to see what code is currently in progress.