Develop Reference

How to effectively terminate or cancle the timed out thread in java?

I am looking for best practices to properly close all the child processes running in a thread.
I have an abstract class
public class AbstractMonitoredTask {
protected abstract ReturnStatus doTask(final RequestObject req);
public ReturnStatus run(final RequestObject req) {
String failureReason = "";
Integer timeout = 0;
Future<String> future = null;
ExecutorService executor = null;
try {
executor = Executors.newSingleThreadExecutor();
timeout = req.getTimeout();
future = executor.submit(() -> doTask(req));
if (nonNull(timeout) && timeout > 0) {
failureReason = future.get(timeout, TimeUnit.MINUTES);
} else {
failureReason = future.get();
}
} catch (TimeoutException ex) {
failureReason = "Task timed out.";
} catch (Exception ex) {
failureReason = "Task failed due to: " + ex.getMessage();
} finally {
if (isNotEmpty(failureReason)) {
if (nonNull(future)) {
future.cancel(true);
}
if (nonNull(executor)) {
List<Runnable> runnableList = executor.shutdownNow();
runnableList.forEach(runnable -> {
System.out.println("Task running was " + runnable);
});
}
}
}
return ReturnStatus.of(failureReason);
}
}
And its implementation looks like
public class MonitoredTask extends AbstractMonitoredTask {
#Override
protected ReturnStatus doTask(final RequestObject req) {
String failureReason = "";
PipedOutputStream outputStream = new PipedOutputStream();
PipedInputStream inputStream = new PipedInputStream();
FileDownloader downloader = FileDownloader.getDownloader(req);
FileUploader uploader = FileUploader.getUploader(req);
try(outputStream) {
outputStream.connect(inputStream);
new Thread(() -> downloader.downloadFile(req, outputStream)).start();
failureReason = uploader.uploadFile(req, inputStream);
} catch (Exception ex) {
failureReason = e.getMessage();
}
return ReturnStatus.of(failureReason);
}
}
So far, the future.get(timeout, TimeUnit.MINUTES); thread is throwing an error which is handled but, the upload and download process threads are still running but they are failing with the Pipe closed error after a long time.
Please suggest how to implement the abstract class and how I can trigger the interrupt/signal to close the streams. As the outputStream and inputStream are closed by the uploader and downloader.

Right way to run tasks concurrently

I am currently facing the following issue with Java concurrency. I want to parse a friend list on a website, then search for the friends of the friends etc. recursively.
Here it is represented visually:
0
/|\
0 0 0
/|\
0 0 0
I came up with a solution but it does not perform as well as I expected, I assume my logic might be somewhat faulty.
private ArrayList<String> getUserFriendsProfileURLs(final String uri, final int indexOfDeep, int foldenesLevel)
throws IOException {
var usersURIs = getUsers(uri); //long network call
ArrayList<String> uris = new ArrayList<>();
uris.addAll(usersURIs);
if (indexOfDeep != foldenesLevel) {
List<CompletableFuture<ArrayList<String>>> futures = new ArrayList<>();
usersURIs.forEach(useruri -> {
CompletableFuture<ArrayList<String>> future = CompletableFuture.supplyAsync(new Supplier<>() {
public ArrayList<String> get() {
var friendsOfUser = new ArrayList<String>();
try {
friendsOfUser = getUserFriendsProfileURLs(useruri, indexOfDeep, 1 + foldenesLevel);
} catch (IOException e) {
throw new IllegalStateException(e);
}
return friendsOfUser;
}
});
futures.add(future);
});
CompletableFuture<Void> allFuturesResult = CompletableFuture
.allOf(futures.toArray(new CompletableFuture[futures.size()]));
var res = allFuturesResult.thenApply(v -> futures.stream().map(CompletableFuture::join).toList());
try {
uris = (ArrayList<String>) res.get().stream().flatMap(Collection::stream).collect(Collectors.toList());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return uris;
}
I wrote 2nd version of function:
/**
* getUserFriendsProfileURLs returns list of friends of given steamid DFS
* algorithm is used
*
*/
private ArrayList<String> getUserFriendsProfileURLs(final int indexOfDeep, int foldenesLevel,
ArrayList<String> usersURIs) throws IOException {
ArrayList<String> uris = new ArrayList<>();
uris.addAll(usersURIs);
if (indexOfDeep != foldenesLevel) {
List<CompletableFuture<ArrayList<String>>> futures = new ArrayList<>();
usersURIs.forEach(useruri -> {
CompletableFuture<ArrayList<String>> future = CompletableFuture.supplyAsync(new Supplier<>() {
public ArrayList<String> get() {
ArrayList<String> ur = null;
try {
ur = getUsers(useruri); // long network call
} catch (IOException e1) {
e1.printStackTrace();
}
return ur;
}
});
futures.add(future);
});
CompletableFuture<Void> allFuturesResult = CompletableFuture
.allOf(futures.toArray(new CompletableFuture[futures.size()]));
var res = allFuturesResult.thenApply(v -> futures.stream().map(CompletableFuture::join).toList());
try {
uris = (ArrayList<String>) res.get().stream().flatMap(Collection::stream).collect(Collectors.toList());
uris = getUserFriendsProfileURLs(indexOfDeep, 1 + foldenesLevel, uris);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return uris;
}

Spring Boot application blocking StreamingResponseBody

I have a Spring Boot application where I created a POST method that sends data in a streaming fashion to the caller. Code below:
#RequestMapping(value = "/mapmatchstreaming", method = RequestMethod.POST)
public ResponseEntity<StreamingResponseBody> handleRequest(#RequestParam(value = "data", required = true) String data, #RequestParam(value = "mnr", required = true) Boolean mnr) {
logger.info("/mapmatchstreaming endpoint");
try {
Semaphore semaphore = new Semaphore(1);
ObjectMapper mapper = new ObjectMapper();
StreamingResponseBody responseBody = new StreamingResponseBody() {
#Override
public void writeTo (OutputStream outputStream) throws IOException {
// For each map
DataReader dataReader = new DataReader(data, "2020.06.011");
for(String mapRoot: dataReader.getMapsFolders()) {
dataReader = new DataReader(data, "2020.06.011");
DistributedMapMatcherStreaming distributedMapMatcher = new DistributedMapMatcherStreaming(dataReader.getTraces(), mapRoot, dataReader.getBoundingBox());
distributedMapMatcher.mapMatchBatch(new DistributedMapMatcherResult() {
#Override
public void onCorrectlyMapMatched(MapMatchedTrajectory mapMatchedTrajectory) {
try {
semaphore.acquire();
outputStream.write(mapper.writeValueAsString(mapMatchedTrajectory).getBytes());
outputStream.flush();
}
catch (Exception e) {
e.printStackTrace();
logger.error(String.format("Writing to output stream error: %s", e.getMessage()));
} finally{
semaphore.release();
}
}
});
}
}
};
return new ResponseEntity<StreamingResponseBody>(responseBody, HttpStatus.OK);
}
catch (Exception e) {
logger.error(String.format("Map-matching result ERROR: %s", ExceptionUtils.getStackTrace(e)));
return new ResponseEntity<StreamingResponseBody>(HttpStatus.BAD_REQUEST);
}
}
It works nicely, but the problem is that if multiple calls arrive to this method, all of them are run in parallel even if I have set server.tomcat.threads.max=1. In the non-streaming version, every next call waits for the current one to complete.
Is it possible to have blocking streaming calls in Spring? Thanks.
EDIT: I temporarily solved by using a global semaphore with only 1 permit, but I don't think this is the ideal solution.

Async HTTP post retry and exhaust after max retries

How can I exhaust retries after a number of failed async request calls?
I am using AsyncHttpClient to send requests to our server. In case of request timeouts, connection exceptions, etc., I would like the client to retry N times and throw a custom exception. The calling method should receive this exception or it can be left unhandled.
// calls post
public void call(String data) throws CustomException {
asyncHttpClient.post(data, 10);
}
// posts data to http endpoint
public void post(String data, int retries) throw CustomException {
// if retries are exhausted, throw CustomException to call()
if (retry <= 0) {
throw new CustomException("exc");
}
BoundRequest request = httpClient.preparePost("http_endpoint");
ListenableFuture<Response> responseFuture = httpClient.post(request);
responseFuture.addListener(() -> {
Response response = null;
int status = 0;
try {
response = responseFuture.get();
status = response.getStatusCode();
// HTTP_ACCEPTED = {200, 201, 202}
if (ArrayUtils.contains(HTTP_ACCEPTED, status)) {
// ok
} else {
sleep(10);
post(data, retry - 1);
}
} catch (InterruptedException e) {
sleep(10);
post(data, retry - 1);
} catch (ExecutionException e) {
// ConnectionException
// RequestTimeoutException
sleep(10); // 10 seconds
post(data, retry - 1);
} catch (Exception e) {
sleep(10); // 10 seconds
post(data, retry - 1 );
} finally {
responseFuture.done();
}
}, Runnable::run);
}
This approach has a few problems:
uses recursive calls to retry.
the CustomException seems like it is never thrown and after retries == 0, the control goes back to the finally block.
...
} catch (ExecutionException e) {
// ConnectionException
// RequestTimeoutException
sleep(10); // 10 seconds
try {
post(data, retry - 1);
} catch (CustomException e) {
}
}
...

There is a predefined function in the AsyncHttpClient to handle MaxRetries,
The code below shows a simple implementation
AsyncHttpClientConfig cf = new DefaultAsyncHttpClientConfig.Builder().setMaxRequestRetry(5).setKeepAlive(true).build()
final AsyncHttpClient asyncHttpClient = new DefaultAsyncHttpClient(cf);
You can remove the retry logic of yours and let AsyncHttpClient to handle the same.

Alright, so tried to reproduce what you were trying to achieve with your code, but immediately realized that your CustomException only works if it is of type RuntimeException. The reason why is that you want to throw the exception during runtime and in another thread.
The code below shows a simple implementation of the Exception. Keep in mind that not all RuntimeExceptions stop the program. This is explained in this thread. So if you want to terminate the program, you have to manually stop it.
public class CustomException extends RuntimeException {
public CustomException(String msg) {
super(msg);
// print your exception to the console
// optional: exit the program
System.exit(0);
}
}
I changed the remaining of your implementation, so that you don't have to make recursive calls anymore. I removed the callback method and instead call the get() method, which waits for the request to finish. But since I am executing all of this in a separate thread it should be running in the background and not main thread.
public class Main {
private final AsyncHttpClient httpClient;
private final int[] HTTP_ACCEPTED = new int[]{200, 201, 202};
private final static String ENDPOINT = "https://postman-echo.com/post";
public static void main(String[] args) {
String data = "{message: 'Hello World'}";
Main m = new Main();
m.post(data, 10);
}
public Main() {
httpClient = asyncHttpClient();
}
public void post(final String data, final int retry) {
Runnable runnable = () -> {
int retries = retry;
for (int i = 0; i < retry; i++) {
Request request = httpClient.preparePost(ENDPOINT)
.addHeader("Content-Type", "application/json")
.setBody(data)
.build();
ListenableFuture<Response> responseFuture = httpClient.executeRequest(request);
try {
Response response = responseFuture.get();
int status = response.getStatusCode();
if (ArrayUtils.contains(HTTP_ACCEPTED, status)) {
System.out.println("Successful! Breaking Loop");
break;
} else {
Thread.sleep(10);
}
} catch (InterruptedException | ExecutionException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
retries--;
}
System.out.println("Remaining retries: " + retries);
if (retries <= 0) {
throw new CustomException("exc");
}
};
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(runnable);
}
}
Alternative
You could use the same Runnable to make asynchronous calls, without having to wait for future.get(). Each listener will conveniently be called in the same thread, which makes it more efficient for your use case.
public void post2(final String data, final int retry) {
Request request = httpClient.preparePost(ENDPOINT)
.addHeader("Content-Type", "application/json")
.setBody(data)
.build();
ListenableFuture<Response> future = httpClient.executeRequest(request);
MyRunnable runnable = new MyRunnable(retry, future, request);
future.addListener(runnable, null);
}
public class MyRunnable implements Runnable {
private int retries;
private ListenableFuture<Response> responseFuture;
private final Request request;
public MyRunnable(int retries, ListenableFuture<Response> future, Request request) {
this.retries = retries;
this.responseFuture = future;
this.request = request;
}
#Override
public void run() {
System.out.println("Remaining retries: " + this.retries);
System.out.println("Thread ID: " + Thread.currentThread().getId());
try {
Response response = responseFuture.get();
int status = response.getStatusCode();
if (ArrayUtils.contains(HTTP_ACCEPTED, status)) {
System.out.println("Success!");
//do something here
} else if (this.retries > 0) {
Thread.sleep(10);
this.execute();
} else {
throw new CustomException("Exception!");
}
} catch (InterruptedException | ExecutionException e) {
this.execute();
}
}
private void execute() {
this.retries -= 1;
this.responseFuture = httpClient.executeRequest(this.request);
this.responseFuture.addListener(this, null);
}
}

How to add batching implicit for client?

Lets consider following code:
Client code:
public class MyClient {
private final MyClientSideService myClientSideService;
public MyClient(MyClientSideService myClientSideService) {
this.myClientSideService = myClientSideService;
}
public String requestRow(Integer req) {
return myClientSideService.requestSingleRow(req);
}
}
Client side service:
public class MyClientSideService {
private final MyServerSideService myServerSideService;
public MyClientSideService(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
}
public String requestSingleRow(int req) {
return myServerSideService.requestRowBatch(Arrays.asList(req)).get(0);
}
}
Server side service:
#Slf4j
public class MyServerSideService {
//single threaded bottleneck service
public synchronized List<String> requestRowBatch(List<Integer> batchReq) {
log.info("Req for {} started");
try {
Thread.sleep(100);
return batchReq.stream().map(String::valueOf).collect(Collectors.toList());
} catch (InterruptedException e) {
return null;
} finally {
log.info("Req for {} finished");
}
}
}
And main:
#Slf4j
public class MainClass {
public static void main(String[] args) {
MyClient myClient = new MyClient(new MyClientSideService(new MyServerSideService()));
for (int i = 0; i < 20; i++) {
new Thread(() -> {
for (int m = 0; m < 100; m++) {
int k = m;
log.info("Response is {}", myClient.requestRow(k));
}
}).start();
}
}
}
According the logs it takes approximately 4 min 22 sec but it too much. Ithink it might be improved dramatically. I would like to implement implicit batching. So MyClientSideService should collect requests and when it becomes 50(it is preconfigured batch size) or some preconfigured timeout expired then to request MyServerSideService and back route result to the clients. Protocol should be synchronous so clients must be blocked until result getting.
I tried to write code using CountDownLatches and CyclicBarriers but my attempts were far from success.
How can I achieve my goal?
P.S.
If to replace requestRowBatch return type List<String> from to Map<Integer, String> to delegate request and response mapping to server following works with limititations. It works only if I send <=25 requests
#Slf4j
public class MyClientSideService {
private final Integer batchSize = 25;
private final Integer maxTimeoutMillis = 5000;
private final MyServerSideService myServerSideService;
private final Queue<Integer> queue = new ArrayBlockingQueue(batchSize);
private final Map<Integer, String> responseMap = new ConcurrentHashMap();
private final AtomicBoolean started = new AtomicBoolean();
private CountDownLatch startBatchRequestLatch = new CountDownLatch(batchSize);
private CountDownLatch awaitBatchResponseLatch = new CountDownLatch(1);
public MyClientSideService(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
}
public String requestSingleRow(int req) {
queue.offer(req);
if (!started.compareAndExchange(false, true)) {
log.info("Start batch collecting");
startBatchCollecting();
}
startBatchRequestLatch.countDown();
try {
log.info("Awaiting batch response latch for {}...", req);
awaitBatchResponseLatch.await();
log.info("Finished awaiting batch response latch for {}...", req);
return responseMap.get(req);
} catch (InterruptedException e) {
e.printStackTrace();
return "EXCEPTION";
}
}
private void startBatchCollecting() {
new Thread(() -> {
try {
log.info("Await startBatchRequestLatch");
startBatchRequestLatch.await(maxTimeoutMillis, TimeUnit.MILLISECONDS);
log.info("await of startBatchRequestLatch finished");
} catch (InterruptedException e) {
e.printStackTrace();
}
responseMap.putAll(requestBatch(queue));
log.info("Releasing batch response latch");
awaitBatchResponseLatch.countDown();
}).start();
}
public Map<Integer, String> requestBatch(Collection<Integer> requestList) {
return myServerSideService.requestRowBatch(requestList);
}
}
Update
According Malt answer I was able to develop following:
#Slf4j
public class MyClientSideServiceCompletableFuture {
private final Integer batchSize = 25;
private final Integer maxTimeoutMillis = 5000;
private final MyServerSideService myServerSideService;
private final Queue<Pair<Integer, CompletableFuture>> queue = new ArrayBlockingQueue(batchSize);
private final AtomicInteger counter = new AtomicInteger(0);
private final Lock lock = new ReentrantLock();
public MyClientSideServiceCompletableFuture(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
}
public String requestSingleRow(int req) {
CompletableFuture<String> future = new CompletableFuture<>();
lock.lock();
try {
queue.offer(Pair.of(req, future));
int counter = this.counter.incrementAndGet();
if (counter != 0 && counter % batchSize == 0) {
log.info("request");
List<Integer> requests = queue.stream().map(p -> p.getKey()).collect(Collectors.toList());
Map<Integer, String> serverResponseMap = requestBatch(requests);
queue.forEach(pair -> {
String response = serverResponseMap.get(pair.getKey());
CompletableFuture<String> value = pair.getValue();
value.complete(response);
});
queue.clear();
}
} finally {
lock.unlock();
}
try {
return future.get();
} catch (Exception e) {
return "Exception";
}
}
public Map<Integer, String> requestBatch(Collection<Integer> requestList) {
return myServerSideService.requestRowBatch(requestList);
}
}
But it doesn't work if size is not multiple of batch size

If to replace requestRowBatch return type from List<String> with Map<Integer, String> to delegate request and response mapping to server I was able to crete following solution:
#Slf4j
public class MyClientSideServiceCompletableFuture {
private final Integer batchSize = 25;
private final Integer timeoutMillis = 5000;
private final MyServerSideService myServerSideService;
private final BlockingQueue<Pair<Integer, CompletableFuture>> queue = new LinkedBlockingQueue<>();
private final Lock lock = new ReentrantLock();
private final Condition requestAddedCondition = lock.newCondition();
public MyClientSideServiceCompletableFuture(MyServerSideService myServerSideService) {
this.myServerSideService = myServerSideService;
startQueueDrainer();
}
public String requestSingleRow(int req) {
CompletableFuture<String> future = new CompletableFuture<>();
while (!queue.offer(Pair.of(req, future))) {
log.error("Can't add {} to the queue. Retrying...", req);
}
lock.lock();
try {
requestAddedCondition.signal();
} finally {
lock.unlock();
}
try {
return future.get();
} catch (Exception e) {
return "Exception";
}
}
private void startQueueDrainer() {
new Thread(() -> {
log.info("request");
while (true) {
ArrayList<Pair<Integer, CompletableFuture>> requests = new ArrayList<>();
if (queue.drainTo(requests, batchSize) > 0) {
log.info("drained {} items", requests.size());
Map<Integer, String> serverResponseMap = requestBatch(requests.stream().map(Pair::getKey).collect(Collectors.toList()));
requests.forEach(pair -> {
String response = serverResponseMap.get(pair.getKey());
CompletableFuture<String> value = pair.getValue();
value.complete(response);
});
} else {
lock.lock();
try {
while (queue.size() == 0) {
try {
log.info("Waiting on condition");
requestAddedCondition.await(timeoutMillis, TimeUnit.MILLISECONDS);
log.info("Waking up on condition");
} catch (InterruptedException e) {
e.printStackTrace();
}
}
} finally {
lock.unlock();
}
}
}
}).start();
}
public Map<Integer, String> requestBatch(Collection<Integer> requestList) {
return myServerSideService.requestRowBatch(requestList);
}
}
It looks like a working solution. But I am not sure if it is optimal.

Your MyClientSideServiceCompletableFuture solution, will send the requests to the server every time you add something to the queue and doesnt wait for requests to be batch sized. You are using BlockingQueue and adding the uneccessary blocking condition and locks. BlockingQueue has blocking-timeout capabilites so no addition condition is neccessary.
You can simplify your solution like this:
It sends requests to server only when the batch is full or the timeout passed and batch is not empty.
private void startQueueDrainer() {
new Thread(() -> {
log.info("request");
ArrayList<Pair<Integer, CompletableFuture>> batch = new ArrayList<>(batchSize);
while (true) {
try {
batch.clear(); //clear batch
long timeTowWait = timeoutMillis;
long startTime = System.currentTimeMillis();
while (timeTowWait > 0 && batch.size() < batchSize) {
Pair<Integer, CompletableFuture> request = queue.poll(timeTowWait , TimeUnit.MILLISECONDS);
if(request != null){
batch.add(request);
}
long timeSpent = (System.currentTimeMillis() - startTime);
timeTowWait = timeTowWait - timeSpent;
}
if (!batch.isEmpty()) {
// we wait at least timeoutMillis or batch is full
log.info("send {} requests to server", batch.size());
Map<Integer, String> serverResponseMap = requestBatch(batch.stream().map(Pair::getKey).collect(Collectors.toList()));
batch.forEach(pair -> {
String response = serverResponseMap.get(pair.getKey());
CompletableFuture<String> value = pair.getValue();
value.complete(response);
});
} else {
log.info("We wait {} but the batch is still empty", System.currentTimeMillis() - startTime);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
}
Change the method requestSingleRow to not use lock
public String requestSingleRow(int req) {
CompletableFuture<String> future = new CompletableFuture<>();
while (!queue.offer(Pair.of(req, future))) {
log.error("Can't add {} to the queue. Retrying...", req);
}
try {
return future.get();
} catch (Exception e) {
return "Exception";
}
}

You could use CompletableFuture.
Have threads calling MyClientSideService put their request in a Queue (possibly BlockingQueue, and get a new CompletableFuture in return. The calling thread can call CompletableFuture.get() to block until a result is ready, or go on doing other things.
That CompletableFuture will be stored together with the request in MyClientSideService. When you reach 50 requests (and therefore 50 CompletableFuture instances), have the client service send the batch request.
When the request is complete, use the CompletableFuture.complete(value) method of each ComplatableFuture instance in the queue to notify the client thread that the response is ready. This will unblock the client if it has called blocking method like CompletableFuture.get(), or make it return instantly with value if called later.