I am trying to decide whether convertSendAndReceive is going to work for the following use case:
I have a RESTful web service that needs to make RPC calls and get a response back in order to service the request. I've never used the reply-to functionality in spring-amqp or RabbitMQ for that matter.
Will this work or am I headed down a path fraught with peril?
EDIT: My concern is whether the thread producing the message will get the correct corresponding response back and not get another thread's response.
I have added to a test that's listed in the spring-amqp documentation called JavaConfigFixedReplyQueueTests. I added the following test case:
(My connectionFactory bean is different but that's just to specify our SSL configuration for my company's rabbitmq instance so I'm not listing that here. All the existing tests passed with that change.)
#Test
public void testReplyContainer_multiple_threads() throws Exception
{
fixedReplyQRabbitTemplate.setReplyTimeout(-1);
// limit the number of actual threads
int poolSize = 100;
ExecutorService service = Executors.newFixedThreadPool(poolSize);
List<Future<?>> futures = new ArrayList<>();
for(int n = 0; n < 1000; n++)
{
Future<?> f = service.submit(makeNumberedRunnable(n));
futures.add(f);
}
// wait for all tasks to complete before continuing
for(Future<?> f : futures)
{
f.get();
}
// shut down the executor service so that this thread can exit
service.shutdownNow();
}
private Runnable makeNumberedRunnable(int n)
{
return new Runnable()
{
#Override
public void run()
{
assertEquals("FOO" + n, fixedReplyQRabbitTemplate.convertSendAndReceive("foo" + n));
}
};
}
Basically my test is creating many threads and making sure that the correct thread specific response is returned. The test passes just fine which gives me a little confidence in my approach.
I'm hoping Gary Russell can chime in or maybe Artem Bilan to give me their expert opinion. However, I welcome anyone knowledgeable in this area to give me their advice.
Thanks for your time.
All of the ...sendAndReceive(...) methods will properly correlate the request/reply so that the proper reply is returned on the requesting thread.
Related
After spending the day of learning about the java Concurrency API, I still dont quite get how could I create the following functionality with the help of CompletableFuture and ExecutorService classes:
When I get a request on my REST endpoint I need to:
Start an asynchronous task (includes DB query, filtering, etc.), which will give me a list of String URLs at the end
In the meanwhile, responde back to the REST caller with HTTP OK, that the request was received, I'm working on it
When the asynchronous task is finished, I need to send HTTP requests (with the payload, the REST caller gave me) to the URLs I got from the job. At most the number of URLs would be around a 100, so I need these to happen in parallel.
Ideally I have some syncronized counter which counts how many of the http requests were a success/fail, and I can send this information back to the REST caller (the URL I need to send it back to is provided inside the request payload).
I have the building blocks (methods like: getMatchingObjectsFromDB(callerPayload), getURLs(resultOfgetMachingObjects), sendHttpRequest(Url, methodType), etc...) written for these already, I just cant quite figure out how to tie step 1 and step 3 together. I would use CompletableFuture.supplyAsync() for step 1, then I would need the CompletableFuture.thenComponse method to start step 3, but it's not clear to me how parallelism can be done with this API. It is rather intuitive with ExecutorService executor = Executors.newWorkStealingPool(); though, which creates a thread pool based on how much processing power is available and the tasks can be submitted via the invokeAll() method.
How can I use CompletableFutureand ExecutorService together? Or how can I guarantee parallel execution of a list of tasks with CompletableFuture? Demonstrating code snippet would be much appreciated. Thanks.
You should use join() to wait for all thread finish.
Create Map<String, Boolean> result to store your request result.
In your controller:
public void yourControllerMethod() {
CompletableFuture.runAsync(() -> yourServiceMethod());
}
In your service:
// Execute your logic to get List<String> urls
List<CompletableFuture> futures = urls.stream().map(v ->
CompletableFuture.supplyAsync(url -> requestUrl(url))
.thenAcceptAsync(requestResult -> result.put(url, true or false))
).collect(toList()); // You have list of completeable future here
Then use .join() to wait for all thread (Remember that your service are executed in its own thread already)
CompletableFuture.allOf(futures).join();
Then you can determine which one success/fail by accessing result map
Edit
Please post your proceduce code so that other may understand you also.
I've read your code and here are the needed modification:
When this for loop was not commented out, the receiver webserver got
the same request twice,
I dont understand the purpose of this for loop.
Sorry in my previous answer, I did not clean it up. That's just a temporary idea on my head that I forgot to remove at the end :D
Just remove it from your code
// allOf() only accepts arrays, so the List needed to be converted
/* The code never gets over this part (I know allOf() is a blocking call), even long after when the receiver got the HTTP request
with the correct payload. I'm not sure yet where exactly the code gets stuck */
Your map should be a ConcurrentHashMap because you're modifying it concurrently later.
Map<String, Boolean> result = new ConcurrentHashMap<>();
If your code still does not work as expected, I suggest to remove the parallelStream() part.
CompletableFuture and parallelStream use common forkjoin pool. I think the pool is exhausted.
And you should create your own pool for your CompletableFuture:
Executor pool = Executors.newFixedThreadPool(10);
And execute your request using that pool:
CompletableFuture.supplyAsync(YOURTASK, pool).thenAcceptAsync(Yourtask, pool)
For the sake of completion here is the relevant parts of the code, after clean-up and testing (thanks to Mạnh Quyết Nguyễn):
Rest controller class:
#POST
#Path("publish")
public Response publishEvent(PublishEvent eventPublished) {
/*
Payload verification, etc.
*/
//First send the event to the right subscribers, then send the resulting hashmap<String url, Boolean subscriberGotTheRequest> back to the publisher
CompletableFuture.supplyAsync(() -> EventHandlerService.propagateEvent(eventPublished)).thenAccept(map -> {
if (eventPublished.getDeliveryCompleteUri() != null) {
String callbackUrl = Utility
.getUri(eventPublished.getSource().getAddress(), eventPublished.getSource().getPort(), eventPublished.getDeliveryCompleteUri(), isSecure,
false);
try {
Utility.sendRequest(callbackUrl, "POST", map);
} catch (RuntimeException e) {
log.error("Callback after event publishing failed at: " + callbackUrl);
e.printStackTrace();
}
}
});
//return OK while the event publishing happens in async
return Response.status(Status.OK).build();
}
Service class:
private static List<EventFilter> getMatchingEventFilters(PublishEvent pe) {
//query the database, filter the results based on the method argument
}
private static boolean sendRequest(String url, Event event) {
//send the HTTP request to the given URL, with the given Event payload, return true if the response is positive (status code starts with 2), false otherwise
}
static Map<String, Boolean> propagateEvent(PublishEvent eventPublished) {
// Get the event relevant filters from the DB
List<EventFilter> filters = getMatchingEventFilters(eventPublished);
// Create the URLs from the filters
List<String> urls = new ArrayList<>();
for (EventFilter filter : filters) {
String url;
try {
boolean isSecure = filter.getConsumer().getAuthenticationInfo() != null;
url = Utility.getUri(filter.getConsumer().getAddress(), filter.getPort(), filter.getNotifyUri(), isSecure, false);
} catch (ArrowheadException | NullPointerException e) {
e.printStackTrace();
continue;
}
urls.add(url);
}
Map<String, Boolean> result = new ConcurrentHashMap<>();
Stream<CompletableFuture> stream = urls.stream().map(url -> CompletableFuture.supplyAsync(() -> sendRequest(url, eventPublished.getEvent()))
.thenAcceptAsync(published -> result.put(url, published)));
CompletableFuture.allOf(stream.toArray(CompletableFuture[]::new)).join();
log.info("Event published to " + urls.size() + " subscribers.");
return result;
}
Debugging this was a bit harder than usual, sometimes the code just magically stopped. To fix this, I only put code parts into the async task which was absolutely necessary, and I made sure the code in the task was using thread-safe stuff. Also I was a dumb-dumb at first, and my methods inside the EventHandlerService.class used the synchronized keyword, which resulted in the CompletableFuture inside the Service class method not executing, since it uses a thread pool by default.
A piece of logic marked with synchronized becomes a synchronized block, allowing only one thread to execute at any given time.
I am not a Multithreading Expert but I am seeing some performance issues with my current code which is using ExecutorService.
I am working on a project in which I need to make a HTTP URL call to my server and if it is taking too long time to respond then timeout the call. Currently it is returning simple JSON String back..
Current requirement I have is for 10 ms. Within 10 ms it should be able to get the data back from the server. I guess its possible since it is just an HTTP call to server within the same datacenter.
My client program and actual servers are within same datacenter and ping time latency is 0.5 ms between them so it should be doable for sure..
I am using RestTemplate for this to make the URL call.
Below is my code which I have wrote for me which uses ExecutorService and Callables -
public class URLTest {
private ExecutorService executor = Executors.newFixedThreadPool(10);
public String getData() {
Future<String> future = executor.submit(new Task());
String response = null;
try {
System.out.println("Started..");
response = future.get(100, TimeUnit.MILLISECONDS);
System.out.println("Finished!");
} catch (TimeoutException e) {
System.out.println("Terminated!");
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
return response;
}
}
Below is my Task class which implements Callable interface -
class Task implements Callable<String> {
private RestTemplate restTemplate = new RestTemplate();
public String call() throws Exception {
// TimerTest timer = TimerTest.getInstance(); // line 3
String response = restTemplate.getForObject(url, String.class);
// timer.getDuration(); // line 4
return response;
}
}
And below is my code in another class DemoTest which calls the getData method in URLTest class 500 times and measure the 95th percentile of it end to end -
public class DemoTest {
public static void main(String[] args) {
URLTest bc = new URLTest();
// little bit warmup
for (int i = 0; i <= 500; i++) {
bc.getData();
}
for (int i = 0; i <= 500; i++) {
TimerTest timer = TimerTest.getInstance(); // line 1
bc.getData();
timer.getDuration(); // line 2
}
// this method prints out the 95th percentile
logPercentileInfo();
}
}
With the above code as it is, I am always seeing 95th percentile as 14-15 ms (which is bad for my use case as it is end to end flow and that's what I need to measure).
I am surprised why? Is ExectuorFramework adding all the latency here?. May be Each task is submitted, and the submitting thread is waiting (via future.get) until the task is finished..
My main goal is to reduce the latency here as much as possible.. My use case is simple, Make a URL call to one of my server with a TIMEOUT feature enabled, meaning if the server is taking lot of time to response, then Timeout the whole call. Customer will call our code from there application which can be multithreaded as well..
Is there anything I am missing or some other flavors of ExecutorService I need to use? How can I improve my performance here? Any suggestions will be of great help..
Any example will be greatly appreciated.. I was reading about ExecutorCompletionService not sure whether I should use this or something else..
As for your observation that you are measuring 15 ms on the outside, but only 3 ms on the inside, my bet is that the construction of the RestTemplate takes the difference. This could be fixed by refactoring.
Note that RestTemplate is a heavyweight, thread-safe object, and is designed to be deployed as an application-wide singleton. Your current code is in critical violation of this intent.
If you need asynchronous HTTP requests, you should really use an asynchronous HTTP library such an AsyncHttpClient, based on Netty underneath, which is again based on Java NIO. That means that you don't need to occupy a thread per an outstanding HTTP request. AsyncHttpClient also works with Futures so you'll have an API you are used to. It can also work with callbacks, which is preferred for the asynchronous approach.
However, even if you keep your current synchronous library, you should at the very least configure a timeout on the REST client instead of letting it run its course.
then run the program again it will start giving me 95th percentile as 3 ms. So not sure why end to end flow gives me 95th percentile as 14-15 ms
You are generating the tasks faster than you can process them. This means the longer you run the tests, the further behind it gets as it is queuing them up. I would expect if you made this 2000 requests you would see latencies up to 4x what you do now. The bottlneck could be on the client side (in which case more threads would help) but quite likely the bottleneck is on the server side in which case more threads could make it worse.
The default behaviour for HTTP is to establish a new TCP connection for each request. The connection time for a new TCP connection can be up to 20 ms easily even if you have two machines side by side. I suggest looking at using HTTp/1.1 and maintain a persistent connection.
BTW You can ping from one side of London to the other in 0.5 ms. However, getting below 1 ms with HTTP reliably is tricky as the protocol is not designed for low latency. It is designed for use on high latency networks.
Note: you cannot see latencies below 25 ms and 100 ms is plenty fast enough for most web requests. It is with these sort of assumption that HTTP was designed.
I am building a set of web services with the intent to aggregate similar data sets across multiple backends (through db calls and service calls). Some of the queries could take more than a couple of seconds to run, and if I stack these requests sequentially, there is a chance the total run time would be outside of the desired response time.
I am hoping to make the calls in parallel, collect all results and then aggregate. What is the best approach to tackling this?
The services will be deployed to Websphere 6.1 (so java 5, j2ee 1.4).
Any information would be greatly appreciated.
Look into the java.util.concurrent API. In particular, you can create a thread Executor, pass in Callables, and get back Futures, that will be run asynchronously.
Your code will look something like this:
ExecutorService exec = Executors.newCachedThreadPool();
Future<ReplyA> raFuture = exec.submit(new Callable<ReplyA>() {
public ReplyA call() {
// call remote service here.
return new ReplyA(...);
}});
Future<ReplyB> rbFuture = exec.submit(new Callable<ReplyB>() {
public ReplyB call() {
// call remote service here.
return new ReplyB(...);
}});
ReplyA replyA = raFuture.get();
ReplyB replyB = rbFuture.get();
exec.shutdown();
You can also use the timeout versions of get() so you can do something reasonable if the responses are taking too long. If you decide to take this path, you would probably be better served with the ExecutorService's invokeAll method, so the timeout will apply to all of the Callables as a group:
Callable<Reply> taskA = new Callable<ReplyA>() { ... };
Callable<Reply> taskB = new Callable<ReplyB>() { ... };
List<Callable<Reply>> tasks = Arrays.asList(taskA, taskB);
List<Future<Reply>> futures = exec.invokeAll(tasks, 20, TimeUnit.SECONDS);
for(Future<Reply> future: futures) {
if(replyFuture.isCancelled()) {
// deal with it
} else {
Reply reply = future.get();
// do something with the reply.
}
}
Start daemon threads, let them do the job, join() them all, aggregate.
Another possible way is to place the queries into a Queue, and let MDBs handle one request at a time. Aggregation is a bit more confusing though -- have to get results from another queue, and accumulate them between replies, and handle the errors... uh, common, just do the threads! :)
RequestAThread rath = new RequestAThread(dataForRequestA);
RequestBThread rbth = new RequestBThread(dataForRequestB);
...
rath.start();
rbth.start();
...
rath.join();
ReplyA ra = rath.getReply();
rbth.join();
ReplyB rb = rbth.getReply();
Result r = aggregate(ra,rb);
Error handling by taste.
Hi I have a webapp - and in one method I need to encrypt part of data from request and store them on disk and return response.
Response is in no way related to encryption.
The encryption is quite time demanding however. How to make threads or so properly in this problem?
I tried something like
Thread thread ...
thread.start();
or
JobDetail job = encryptionScheduler.getJobDetail(jobDetail.getName(), jobDetail.getGroup());
encryptionScheduler.scheduleJob(jobDetail,TriggerUtils.makeImmediateTrigger("encryptionTrigger",1,1)
I tried servlet where before encryption I close the outpuStream.
or: Executors.newFixedThreadPool(1);
But whatever I tried a client has to wait longer.
btw: why is that so? Can it be faster?
I haven't tried to start thread after context initalization and wait somehow for method needing encryption.
how to speed up this?
thank you
--------------EDIT:
//I use axis 1.4, where I have Handler, which in invoke method encrypt a value:
try {
LogFile logFile = new LogFile(strategy,nodeValue,path, new Date());
LogQueue.queue.add(logFile);
}
catch (Exception e) {
log.error(e.getMessage(),e);
}
EExecutor.executorService.execute(new Runnable() {
public void run() {
try {
LogFile poll = LogQueue.queue.poll();
String strategy = poll.getStrategy();
String value = poll.getNodeValue();
value = encrypt(strategy,value);
PrintWriter writer = new PrintWriter(new OutputStreamWriter(new BufferedOutputStream(new FileOutputStream(poll.getPath(), true )),"UTF-8"));
writer.print(value);
writer.close();
}catch (IOException e ) {
log.error(e.getMessage(),e);
}
}
});
} catch (Throwable e ) {
log.error(e.getMessage(),e);
}
//besides I have executor service
public class EExecutor { public static ExecutorService executorService = Executors.newCachedThreadPool();}
//and what's really interesting.. when I move encryption from this handler away into another handler which is called
last when I send response! It's faster. But when I leave it in one of the first handlers when I receive request. It's even slower without using threads/servlet etc.
Threads only help you if parts of your task can be done in parallel. It sounds like you're waiting for the encryption to finish before returning the result. If it's necessary for you to do that (e.g., because the encrypted data is the result) then doing the encryption on a separate thread won't help you here---all it will do is introduce the overhead of creating and switching to a different thread.
Edit: If you're starting a new thread for each encryption you do, then that might be part of your problem. Creating new threads is relatively expensive. A better way is to use an ExecutorService with an unbounded queue. If you don't care about the order in which the encryption step happens (i.e., if it's ok that the encryption which started due to a request at time t finishes later than one which started at time t', and t < t'), then you can let the ExecutorService have more than a single thread. That will give you both greater concurrency and save you the overhead of recreating threads all the time, since an ExecutorService pools and reuses threads.
The proper way to do something like this is to have a message queue, such as the standard J2EE JMS.
In a message queue, you have one software component whose job it is to receive messages (such as requests to encrypt some resource, as in your case), and make the request "durable" in a transactional way. Then some independent process polls the message queue for new messages, takes action on them, and transactionally marks the messages as received.
Edit
This question has gone through a few iterations by now, so feel free to look through the revisions to see some background information on the history and things tried.
I'm using a CompletionService together with an ExecutorService and a Callable, to concurrently call the a number of functions on a few different webservices through CXF generated code.. These services all contribute different information towards a single set of information I'm using for my project. The services however can fail to respond for a prolonged period of time without throwing an exception, prolonging the wait for the combined set of information.
To counter this I'm running all the service calls concurrently, and after a few minutes would like to terminate any of the calls that have not yet finished, and preferably log which ones weren't done yet either from within the callable or by throwing an detailed Exception.
Here's some highly simplified code to illustrate what I'm doing already:
private Callable<List<Feature>> getXXXFeatures(final WiwsPortType port,
final String accessionCode) {
return new Callable<List<Feature>>() {
#Override
public List<Feature> call() throws Exception {
List<Feature> features = new ArrayList<Feature>();
//getXXXFeatures are methods of the WS Proxy
//that can take anywhere from second to never to return
for (RawFeature raw : port.getXXXFeatures(accessionCode)) {
Feature ft = convertFeature(raw);
features.add(ft);
}
if (Thread.currentThread().isInterrupted())
log.error("XXX was interrupted");
return features;
}
};
}
And the code that concurrently starts the WS calls:
WiwsPortType port = new Wiws().getWiws();
List<Future<List<Feature>>> ftList = new ArrayList<Future<List<Feature>>>();
//Counting wrapper around CompletionService,
//so I could implement ccs.hasRemaining()
CountingCompletionService<List<Feature>> ccs =
new CountingCompletionService<List<Feature>>(threadpool);
ftList.add(ccs.submit(getXXXFeatures(port, accessionCode)));
ftList.add(ccs.submit(getYYYFeatures(port accessionCode)));
ftList.add(ccs.submit(getZZZFeatures(port, accessionCode)));
List<Feature> allFeatures = new ArrayList<Feature>();
while (ccs.hasRemaining()) {
//Low for testing, eventually a little more lenient
Future<List<Feature>> polled = ccs.poll(5, TimeUnit.SECONDS);
if (polled != null)
allFeatures.addAll(polled.get());
else {
//Still jobs remaining, but unresponsive: Cancel them all
int jobsCanceled = 0;
for (Future<List<Feature>> job : ftList)
if (job.cancel(true))
jobsCanceled++;
log.error("Canceled {} feature jobs because they took too long",
jobsCanceled);
break;
}
}
The problem I'm having with this code is that the Callables aren't actually canceled when waiting for port.getXXXFeatures(...) to return, but somehow keep running. As you can see from the if (Thread.currentThread().isInterrupted()) log.error("XXX was interrupted"); statements the interrupted flag is set after port.getFeatures returns, this is only available after the Webservice call completes normally, instead of it having been interrupted when I called Cancel.
Can anyone tell me what I am doing wrong and how I can stop the running CXF Webservice call after a given time period, and register this information in my application?
Best regards, Tim
Edit 3 New answer.
I see these options:
Post your problem on the Apache CXF as feature request
Fix ACXF yourself and expose some features.
Look for options for asynchronous WS call support within the Apache CXF
Consider switching to a different WS provider (JAX-WS?)
Do your WS call yourself using RESTful API if the service supports it (e.g. plain HTTP request with parameters)
For über experts only: use true threads/thread group and kill the threads with unorthodox methods.
The CXF docs have some instructions for setting the read timeout on the HTTPURLConnection:
http://cwiki.apache.org/CXF20DOC/client-http-transport-including-ssl-support.html
That would probably meet your needs. If the server doesn't respond in time, an exception is raised and the callable would get the exception. (except there is a bug where is MAY hang instead. I cannot remember if that was fixed for 2.2.2 or if it's just in the SNAPSHOTS right now.)