Say I have two endpoints in my API (in Spring as previously):
#RequestMapping("/async")
public CompletableFuture<String> g(){
CompletableFuture<String> f = new CompletableFuture<>();
f.runAsync(() -> {
try {
Thread.sleep(5000);
f.complete("Finished");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
Thread.sleep(1000);
return f;
}
#RequestMapping("/sync")
public String h() throws InterruptedException {
Thread.sleep(5000);
Thread.sleep(1000);
return "Finished";
}
When I send 2 get requests (just single get requests) to:
localhost:8080/async --> response in 5024ms
localhost:8080/sync --> response in '6055ms`
This makes sense because we are sending just a single request. Now things get interesting when I do a load test with Siege involving 255 concurrent users.
In this case, my async API endpoint isn't able to handle many connections.
So async is not as scalable.
Does this depend on my hardware? Say I had hardware able to handle more thread-handlers, then with heavy hardware, would the async one be able to handle more transactions since there are more threads?
You're still using the ForkJoinPool.commonPool() for your async invocations. I told you it's small and it will get filled up. Try this (I fixed your CompletableFuture code too, as it's totally wrong, it just doesn't show in your example).
CompletableFuture<Void> f = CompletableFuture.runAsync(() -> {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}, Executors.newSingleThreadExecutor());
return f;
Now each async call gets its own executor, so it doesn't choke on the common pool. Of course since all async calls get their own executor, this is a bad example. You'd want to use a shared pool, but something larger than the common pool.
It has nothing (well, very little) to do with your hardware. It has everything to do with long running operations intermingled with short running operations.
Related
I want to make web calls to 2 different services simultaneously. At the end, I zip the 2 Response objects into one stream. I'm using a Callable, but I'm not sure I'm going about this in the correct way. It seems as though I'm still going to be blocked by the first get() call to the Future, right? Can someone tell me if I'm on the right track? This is what I have so far:
// submit the 2 calls to the thread pool
ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
Future<Mono<Response<ProcessInstance>>> processFuture =
executorService.submit(() -> getProcessInstances(processDefinitionKey, encryptedIacToken));
Future<Mono<Response<Task>>> taskFuture =
executorService.submit(() -> getTaskResponses(processDefinitionKey, encryptedIacToken, 100, 0));
// get the result of the 2 calls
Optional<Tuple2<Response<ProcessInstance>, Response<Task>>> tuple;
try {
Mono<Response<ProcessInstance>> processInstances = processFuture.get();
Mono<Response<Task>> userTasks = taskFuture.get();
tuple = processInstances.zipWith(userTasks).blockOptional();
} catch (InterruptedException e) {
log.error("Exception while processing response", e);
// Restore interrupted state...
Thread.currentThread().interrupt();
return emptyProcessResponseList;
} catch (ExecutionException e) {
log.error("Exception while processing response", e);
return emptyProcessResponseList;
}
Given: You need to wait until both tasks are complete.
If processFuture ends first, you'll immediately fall through and wait until taskFuture ends. If taskFuture ends first you'll block until processFuture ends, but the taskFuture.get() call will return instantly since that task is done. In either case the result is the same.
You could use CompletableFutures instead and then CompletableFuture.allOf() but for something this simple what you have works fine. See also Waiting on a list of Future
Your code will block until the processFuture is finished, then it will block until the taskFuture is finished.
The callables will be processed in parallel, so here you are saving time (assuming thread pool size >= 2).
I think mistakenly guys compared take() vs poll(), but I found that it is reasonable to compare take() vs poll(time, unit) as both provided by BlockingQueue and both are blocking tell queue not Empty "and in case or poll or time-out", OK lets start comparing, usually I'm using take() for BlockingQueue but I was facing issues about:
handling interrupt inside loop.
waiting till be interrupted from outside.
how to stop looping on queue "using Kill-Bill or interrupt thread"
specially when work with Java 8 streams, then I got idea about I need to stop retrieving data from queue and close it in better way, so I thought to make waiting for sometime after that I can stop retrieve data then I found poll(time, unit) and it will fit for this idea check code below:
public static void main(String[] args) throws InterruptedException {
BlockingQueue<Integer> q = new LinkedBlockingQueue<Integer>();
ExecutorService executor = Executors.newCachedThreadPool();
executor.submit(() -> {
IntStream.range(0, 1000).boxed().forEach(i -> {
try {
q.put(i);
} catch (InterruptedException e) {
currentThread().interrupt();
throw new RuntimeException(e);
}
});
});
....
// Take
Future fTake = executor.submit(() -> {
try {
while (!Thread.currentThread().isInterrupted()) {
System.out.println(q.take());
}
} catch (InterruptedException e) {
currentThread().interrupt();
throw new RuntimeException(e);
}
});
//to stop it I have to do below code "Expecting that execution will take 1 sec"
executor.shutdown();
sleep(1000);
fTake.cancel(true);
....
// poll there is no need to expect time till processing will be done
Future fPoll = executor.submit(() -> {
try {
Integer i;
while ((i = q.poll(100, TimeUnit.MILLISECONDS)) != null)
System.out.println(i);
} catch (InterruptedException e) {
currentThread().interrupt();
throw new RuntimeException(e);
}
});
executor.shutdown();
}
I think the poll code is more clean and there is no need to depend on interrupt and also no need to estimate execution time or make code to determined when to interrupt thread, what do you think?
Note 1: I'm sure that 2nd solution also have drawbacks like not getting data till time-out but I think you are going to know what is suitable time-out for your case.
Note 2: if use case requires waiting for ever and producer is low frequency provide data, I think take solution is better.
I wanted to prototype an example where I call a ServiceC using a value returned by ServiceA using Spring Reactor Stream API. So I wrote code like this
final ExecutorService executor = new ThreadPoolExecutor(4, 4, 10, TimeUnit.MINUTES, new LinkedBlockingQueue<Runnable>());
Streams.defer(executor.submit(new CallToRemoteServiceA()))
.flatMap(s -> Streams.defer(executor.submit(new CallToRemoteServiceC(s))))
.consume(s -> System.out.println("End Result : " + s));
To simulate the latency involved in ServiceA and ServiceC the call() methods of CallToRemoteServiceA and CallToRemoteServiceC has Thread.sleep() methods. The problem is that when I comment out the Thread.sleep() method i.e. the service method calls have no latency which is not true in the real world the consume method gets called. If the Thread.sleep() methods are kept in place then the consume method doesn't get called. I understand that the Streams.defer() returns a cold stream and hence it probably only executes the consume method for items accepted after it's registration but then I was wondering how I could create a HotStream from a Future returned by the ExecutorService?
I believe this is because of a bug in the reactor.rx.stream.FutureStream.subscribe() method. In this line:
try {
// Bug in the line below since unit is never null
T result = unit == null ? future.get() : future.get(time, unit);
buffer.complete();
onNext(result);
onComplete();
} catch (Throwable e) {
onError(e); <-- With default constructor this gets called if time == 0 and
future has as yet not returned
}
In this case when the default FutureStream(Future) constructor is called the unit is never null and hence the above code always calls future.get(0, TimeUnit.SECONDS) leading to an immediate timeout exception in the catch(Throwable) block. If you guys agree that this is a bug I can make a pull request with a fix for this issue??
I think what you want is to use Streams.just. You can optionally .dispatchOn(Dispatcher) if you want, but since you're already in the thread of the thread pool, you'll probably want to use the sync Dispatcher. Here's a quick test to illustrate:
#Test
public void streamsDotJust() throws InterruptedException {
ExecutorService executor = Executors.newSingleThreadExecutor();
Streams
.just(executor.submit(() -> "Hello World!"))
.map(f -> {
try {
return f.get();
} catch (Exception e) {
throw new IllegalStateException(e);
}
})
.consume(System.out::println);
Thread.sleep(100);
}
I want to implement something like this.
1.A background process which will be running forever
2.The background process will check the database for any requests in pending state. If any found,will assign a separate thread to process the request.So one thread per request.Max threads at any point of time should be 10. Once the thread has finished execution,the status of the request will be updated to something,say "completed".
My code outline looks something like this.
public class SimpleDaemon {
private static final int MAXTHREADS = 10;
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(MAXTHREADS);
RequestService requestService = null; //init code omitted
while(true){
List<Request> pending = requestService.findPendingRequests();
List<Future<MyAppResponse>> completed = new ArrayList<Future<MyAppResponse>>(pending.size());
for (Request req:pending) {
Callable<MyAppResponse> worker = new MyCallable(req);
Future<MyAppResponse> submit = executor.submit(worker);
completed.add(submit);
}
// Now retrieve the result
for (Future<MyAppResponse> future : completed) {
try {
requestService.updateStatus(future.getRequestId());
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
try {
Thread.sleep(10000); // Sleep sometime
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Can anyone spend sometime to review this and comment any suggestion/optimization (from multi threading perspective) ? Thanks.
Using a max threads of ten seems somewhat arbitrary. Is this the maximum available connections to your database?
I'm a little confused as to why you are purposefully introducing latency into your applications. Why aren't pending requests submitted to the Executor immediately?
The task submitted to the Executor could then update the RequestService, or you could have a separate worker Thread belonging to the RequestService which calls poll on a BlockingQueue of Future<MyAppResponse>.
You have no shutdown/termination strategy. Nothing indicates that main is run on a Thread that is set to Daemon. If it is, I think the ExecutorService's worker threads will inherit the daemon status, but then your application could shutdown with live connection to the database, no? Isn't that bad?
If the thread isn't really a Daemon, then you need to handle that InterruptedException and treat it as an indication that you are being asked to exit the application.
Your calls to requestService appear to be single threaded resulted in any long running queries preventing completed queries from being completed.
Unless the updateStatus has to be called in a specific order, I suggest you call this as part of your query in MyCallable. This could simplify your code and allow results to be processed as they become available.
You need to handle the potential throwing of a RejectedExecutionException by executor.submit() because the thread-pool has a finite number of threads.
You'd probably be better off using an ExecutorCompletionService rather than an ExecutorService because the former can tell you when a task completes.
I strongly recommend reading Brian Goetz's book "Java Concurrency in Practice".
I'm trying to understand how to ensure that a specific action completes in a certain amount of time. Seems like a simple job for java's new util.concurrent library. However, this task claims a connection to the database and I want to be sure that it properly releases that connection upon timeout.
so to call the service:
int resultCount = -1;
ExecutorService executor = null;
try {
executor = Executors.newSingleThreadExecutor();
FutureTask<Integer> task = new CopyTask<Integer>();
executor.execute(task);
try {
resultCount = task.get(2, TimeUnit.MINUTES);
} catch (Exception e) {
LOGGER.fatal("Migrate Events job crashed.", e);
task.cancel(true);
return;
}
} finally {
if (executor != null) {
executor.shutdown();
}
The task itself simply wrapps a callable, here is the call method:
#Override
public Integer call() throws Exception {
Session session = null;
try {
session = getSession();
... execute sql against sesssion ...
}
} finally {
if (session != null) {
session.release();
}
}
}
So, my question for those who've made it this far, is: Is session.release() garaunteed to be called in the case that the task fails due to a TimeoutException? I postulate that it is no, but I would love to be proven wrong.
Thanks
edit: The problem I'm having is that occasionally the sql in question is not finishing due to wierd db problems. So, what I want to do is simply close the connection, let the db rollback the transaction, get some rest and reattempt this at a later time. So I'm treating the get(...) as if it were like killing the thead. Is that wrong?
When you call task.get() with a timeout, that timeout only applies to the attempt to obtain the results (in your current thread), not the calculation itself (in the worker thread). Hence your problem here; if a worker thread gets into some state from which it will never return, then the timeout simply ensures that your polling code will keep running but will do nothing to affect the worker.
Your call to task.cancel(true) in the catch block is what I was initially going to suggest, and this is good coding practice. Unfortunately this only sets a flag on the thread that may/should be checked by well-behaved long-running, cancellable tasks, but it doesn't take any direct action on the other thread. If the SQL executing methods don't declare that they throw InterruptedException, then they aren't going to check this flag and aren't going to be interruptable via the typical Java mechanism.
Really all of this comes down to the fact that the code in the worker thread must support some mechanism of stopping itself if it's run for too long. Supporting the standard interrupt mechanism is one way of doing this; checking some boolean flag intermittently, or other bespoke alternatives, would work too. However there is no guaranteed way to cause another thread to return (short of Thread.stop, which is deprecated for good reason). You need to coordinate with the running code to signal it to stop in a way that it will notice.
In this particular case, I expect there are probably some parameters you could set on the DB connection so that the SQL calls will time out after a given period, meaning that control returns to your Java code (probably with some exception) and so the finally block gets called. If not, i.e. there's no way to make the database call (such as PreparedStatement.execute()) return control after some predetermined time, then you'll need to spawn an extra thread within your Callable that can monitor a timeout and forcibly close the connection/session if it expires. This isn't very nice though and your code will be a lot cleaner if you can get the SQL calls to cooperate.
(So ironically despite you supplying a good amount of code to support this question, the really important part is the bit you redacted: "... execute sql against sesssion ..." :-))
You cannot interrupt a thread from the outside, so the timeout will have no effect on the code down in the JDBC layer (perhaps even over in JNI-land somewhere.) Presumably eventually the SQL work will end and the session.release() will happen, but that may be long after the end of your timeout.
The finally block will eventually execute.
When your Task takes longer then 2 minutes, a TimeoutException is thrown but the actual thread continues to perform it's work and eventually it will call the finally block. Even if you cancel the task and force an interrupt, the finally block will be called.
Here's a small example based in your code. You can test these situations:
public static void main(String[] args) {
int resultCount = -1;
ExecutorService executor = null;
try {
executor = Executors.newSingleThreadExecutor();
FutureTask<Integer> task = new FutureTask<Integer>(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
try {
Thread.sleep(10000);
return 1;
} finally {
System.out.println("FINALLY CALLED!!!");
}
}
});
executor.execute(task);
try {
resultCount = task.get(1000, TimeUnit.MILLISECONDS);
} catch (Exception e) {
System.out.println("Migrate Events job crashed: " + e.getMessage());
task.cancel(true);
return;
}
} finally {
if (executor != null) {
executor.shutdown();
}
}
}
Your example says:
copyRecords.cancel(true);
I assume this was meant to say:
task.cancel(true);
Your finally block will be called assuming that the contents of the try block are interruptible. Some operations are (like wait()), some operations are not (like InputStream#read()). It all depends on the operation that that the code is blocking on when the task is interrupted.