I have a service which process a request from a user.
And this service call another external back-end system(web services). but I need to execute those back-end web services in parallel. How would you do that? What is the best approach?
thanks in advance
-----edit
Back-end system can run requests in parallel, we use containers like (tomcat for development) and websphere finally for production.
So I'm already in one thread(servlet) and need to spawn two tasks and possibly run them in parallel as close together as possible.
I can imagine using either quartz or thread with executors or let it be on Servlet engine. What is proper path to take in such a scenario?
You can use Threads to run the requests in parallel.
Depending on what you want to do, it may make sense to build on some existing technology like Servlets, that do the threading for you
The answer is to run the tasks in separate threads.
For something like this, I think you should be using a ThreadPoolExecutor with a bounded pool size rather than creating threads yourself.
The code would look something like this. (Please note that this is only a sketch. Check the javadocs for details, info on what the numbers mean, etc.)
// Create the executor ... this needs to be shared by the servlet threads.
Executor exec = new ThreadPoolExecutor(1, 10, 120, TimeUnit.SECONDS,
new ArrayBlockingQueue(100), ThreadPoolExecutor.CallerRunsPolicy);
// Prepare first task
final ArgType someArg = ...
FutureTask<ResultType> task = new FutureTask<ResultType>(
new Callable<ResultType>() {
public ResultType call() {
// Call remote service using information in 'someArg'
return someResult;
}
});
exec.execute(task);
// Repeat above for second task
...
exec.execute(task2);
// Wait for results
ResultType res = task.get(30, TimeUnit.SECONDS);
ResultType res2 = task2.get(30, TimeUnit.SECONDS);
The above does not attempt to handle exceptions, and you need to do something more sophisticated with the timeouts; e.g. keeping track of the overall request time and cancelling tasks if we run over time.
This is not a problem that Quartz is designed to solve. Quartz is a job scheduling system. You just have some tasks that you need to be executed ASAP ... possibility with the facility to cancel them.
Heiko is right that you can use Threads. Threads are complex beasts, and need to be treated with care. The best solution is to use a standard library, such as java.util.concurrent. This will be a more robust way of managing parallel operations. There are performance benefits which coming with this approach, such as thread pooling. If you can use such a solution, this would be the recommended way.
If you want to do it yourself, here is a very simple way of executing a number of threads in parallel, but probably not very robust. You'll need to cope better with timeouts and destruction of threads, etc.
public class Threads {
public class Task implements Runnable {
private Object result;
private String id;
public Task(String id) {
this.id = id;
}
public Object getResult() {
return result;
}
public void run() {
System.out.println("run id=" + id);
try {
// call web service
Thread.sleep(10000);
result = id + " more";
} catch (InterruptedException e) {
// TODO do something with the error
throw new RuntimeException("caught InterruptedException", e);
}
}
}
public void runInParallel(Runnable runnable1, Runnable runnable2) {
try {
Thread t1 = new Thread(runnable1);
Thread t2 = new Thread(runnable2);
t1.start();
t2.start();
t1.join(30000);
t2.join(30000);
} catch (InterruptedException e) {
// TODO do something nice with exception
throw new RuntimeException("caught InterruptedException", e);
}
}
public void foo() {
Task task1 = new Task("1");
Task task2 = new Task("2");
runInParallel(task1, task2);
System.out.println("task1 = " + task1.getResult());
System.out.println("task2 = " + task2.getResult());
}
}
Related
Let's assume that I have a grpc-java server with code as something like this:
#Override
public void getData(RequestValue requestValue, StreamObserver<ResponseValue>responseObserver) {
ResponseValue rv = ... // blocking code here
responseObserver.onNext(rv);
responseObserver.onCompleted();
}
So I have a responseValue as a result of blocking code (data from database or other service).
I want to avoid blocking my current thread using another thread-pool for my blocking tasks. For example, in Netty I can use specific EventExecutorGroup for such tasks.
How can I manage it properly with grpc-java service?
The easiest way is to do this is pass the responseObserver to the long running task:
#Override
public void getData(RequestValue requestValue, StreamObserver<ResponseValue> responseObserver) {
Runnable r = () -> {
try {
ResponseValue rv = ... // blocking code here
responseObserver.onNext(rv);
responseObserver.onCompleted();
} catch (Exception e) {
responseObserver.onError(e);
}
executor.schedule(r);
}
It is important that you complete the call at some time, even if an unexpected error occurs. Otherwise you will leak calls (that remain open until the timeout occurs, if ever).
I've been developing Rest API's in Java. I want to convert them it into Async. The Two options I see is DeferredResult and CompletableFeature.
I don't seem to be find the difference between these two, and when to chose over another.
Any real time examples would be appreciated.
DeferredResult is spring class and it is just a container of the result (as its name implies) so we need to explicitly use some kind of thread pool (ForkJoinPool for example) to run our processing asynchronously. CompletableFuture is part of java.util.concurrent and allow to run the processing asynchronously. It implements Future and basically have the ability to compose, combine and execute asynchronous computation steps.
Simple example of both options:
#GetMapping(value = "/deferredResult")
public DeferredResult<Boolean> useDeferredResult() {
DeferredResult<Boolean> deferredResult = new DeferredResult<>();
deferredResult.onCompletion(() -> logResult((Boolean)deferredResult.getResult()));
ForkJoinPool.commonPool().submit(() -> {
deferredResult.setResult(processRequest());
});
return deferredResult;
}
#GetMapping(value = "/completableFuture")
public CompletableFuture<Boolean> useCompletableFuture() {
return CompletableFuture.supplyAsync(this::processRequest)
.thenApplyAsync(this::logResult);
}
private boolean logResult(Boolean result) {
System.out.println("Result: " + result);
return true;
}
private boolean processRequest() {
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
return true;
}
Notes:
By default, Spring will execute the CompletableFuture actions by
ForkJoinPool (can be configured).
In the case of DeferredResult, the logResult will be executed by the servlet container (for example Tomcat) worker thread - not nessecrally the one got the request at the beginning.
You can (while I don't see any
reason to) run processing asynchronously with CompletableFuture and
return DeferredResult.
With DeferredResult you can register more callbacks, like onCompleted - For example onError, etc. See here.
CompletableFuture has a lot options to compose actions. See here.
IMHO, CompletableFuture is more elegant and has more capabilities.
Also, here you have a working example project.
I have a dagger-singleton-wrapper handling my basic Realm requests. One of which looks like this:
public void insertOrUpdateAsync(final List<RealmMessage> messages, #Nullable final OnInsertListener listener) {
Realm instance = getRealmInstance();
instance.executeTransactionAsync(realm -> {
List<RealmMessage> newMessages = insertOrUpdateMessages(realm, messages);
},
() -> success(listener, instance),
error -> error(listener, error, instance));
}
private List<RealmMessage> insertOrUpdateMessages(#NonNull Realm realm, #NonNull final List<RealmMessage> messages) {
...
return realm.copyToRealmOrUpdate(unattendedMessages);
}
Which works great.
However there is a corner case where - long story short - I launch insertOrUpdateAsynch() many, many times. And after some requests I get this:
Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask#b7b848 rejected from io.realm.internal.async.RealmThreadPoolExecutor#80f96e1[Running, pool size = 17, active threads = 17, queued tasks = 100, completed tasks = 81]
My question is: how should I handle this without rebuilding whole application flow.
My idea was to queue incoming requests via RxJava. Am I right? Which operators should I consider and educate myself?
Or am I approaching this in a completely wrong way?
From most of my googling I've noticed that mostly the problem is in launching method like mine in a loop. I'm not using any. In my case problem is that this method is launched by multiple responses, and changing that is kind of impossible because of the current backend implementation.
If you do not want to redesign your application you may use a counting semaphore. You will see that two Threads will instantly acquire the the lock. The other thread will block until some call will release one lock. It is not recommanded to use acquire() without an Timeout.
In order to use RxJava you would have to change the design of your application and rate-limiting in RxJava is not that easy, because it is all about throughoutput.
private final Semaphore semaphore = new Semaphore(2);
#Test
public void name() throws Exception {
Thread t1 = new Thread(() -> {
doNetworkStuff();
});
Thread t2 = new Thread(() -> {
doNetworkStuff();
});
Thread t3 = new Thread(() -> {
doNetworkStuff();
});
t1.start();
t2.start();
t3.start();
Thread.sleep(1500);
}
private void doNetworkStuff() {
try {
System.out.println("enter doNetworkStuff");
semaphore.acquire();
System.out.println("acquired");
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace(); // Don't do this!!
} finally {
semaphore.release();
}
}
I need to send a email during registration process , so for this reason i am using Java Mail API , this is working fine , but observed that
the email process is taking nearly 6 seconds (which is too long ) so Ajax call making the user wait too long for response
so for this reason i have decided to use background thread for sending email so the user need not wait for the Ajax call response (Jersey REST Web Service call)
My question is it a good practice to creating threads in a webapplication for every request ??
#Path("/insertOrUpdateUser")
public class InsertOrUpdateUser {
final static Logger logger = Logger.getLogger(InsertOrUpdateUser.class);
#GET
#Consumes("application/text")
#Produces("application/json")
public String getSalesUserData(#QueryParam(value = "empId") String empId
)
throws JSONException, SQLException {
JSONObject final_jsonobject = new JSONObject();
ExecutorService executorService = Executors.newFixedThreadPool(10);
executorService.execute(new Runnable() {
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed",e);
}
}
});
}
} catch (SQLException e) {
} catch (Exception e) {
}
finally {
}
return response;
}
}
And this is my Utility class for sending email
public class SendEmailUtility
{
public static String sendmail(String sendto)
throws IOException
{
String result = "fail";
Properties props_load = getProperties();
final String username = props_load.getProperty("username");
final String password = props_load.getProperty("password");
Properties props_send = new Properties();
props_send.put("mail.smtp.auth", "true");
props_send.put("mail.smtp.starttls.enable", "true");
props_send.put("mail.smtp.host", props_load.getProperty("mail.smtp.host"));
props_send.put("mail.smtp.port", props_load.getProperty("mail.smtp.port"));
Session session = Session.getInstance(props_send,
new javax.mail.Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication(username, password);
}
});
try {
Message message = new MimeMessage(session);
message.setFrom(new InternetAddress(props_load.getProperty("setFrom")));
message.setRecipients(Message.RecipientType.TO, InternetAddress.parse(sendto));
message.setText("Some Text to be send in mail");
Transport.send(message);
result = "success";
} catch (MessagingException e) {
result = "fail";
logger.error("Exception Occured - sendto: " + sendto, e);
}
return result;
}
}
Could you please let me know if this is best practice to do in a web application ??
There are host of ways you can handle it, so it all depends on whether your application server has that much resources (memory, threads etc.) to handle your implementation, so it makes you best person to decide on which approach to go.
As such it is not bad practice to spawn parallel threads for doing something if it is justified by design, but typically you should go with controlled threads.
Please note that whether you use newSingleThreadExecutor() or newFixedThreadPool(nThreads), under-the-hoods there will always be a ThreadPoolExecutor object created.
My recommendation will be to use seconds option in below list i.e. "Controlled number of threads", and in that specify max thread count as you see fir.
One thread for each request
In this approach one thread will be created for each incoming request from GUI, so if you are getting 10 requests for inserting/updating user then 10 threads will be spawned which will send emails.
Downside of this approach is that there is no control on number of threads so you can end with StackOverflowException or may be memory issue.
Please make sure to shutdown your executor service else you will end up wasting JVM resources.
// inside your getSalesUserData() method
ExecutorService emailExecutor = Executors.newSingleThreadExecutor();
emailExecutor.execute(new Runnable() {
#Override
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed", e);
}
}
});
emailExecutor.shutdown(); // it is very important to shutdown your non-singleton ExecutorService.
Controlled number of threads
In this approach, some pre-defined number of threads will be present and those will process your email sending requirement. In below example I am starting a thread pool with max of 10 threads, then I am using a LinkedBlockingQueue implementation so this will ensure that if there are more than 10 requests and currently all my 10 threads are busy then excess of requests will be queued and not lost, this is the advantage you get with LinkedBlockingQueue implementation of Queue.
You can initialize you singleton ThreadPoolExecutor upon application server start, if there are no requests then no threads will be present so it is safe to do so. In fact I use similar configuration for my prod application.
I am using time to live seconds as 1 seconds so if a thread is ideal in JVM for more than 1 seconds then it will die.
Please note that since same thread pool is used for processing all you requests, so it should be singleton and do not shutdown this thread pool else your tasks will never be executed.
// creating a thread pool with 10 threads, max alive time is 1 seconds, and linked blocking queue for unlimited queuing of requests.
// if you want to process with 100 threads then replace both instances of 10 with 100, rest can remain same...
// this should be a singleton
ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 10, 1, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>());
// inside your getSalesUserData() method
executor.execute(new Runnable() {
#Override
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed", e);
}
}
});
Java's default cached thread pool
This approach is much like above, only that Java will initialize the ThreadPoolExecutor for you as ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
Here max number of threads will be Integer.MAX_VALUE, so threads will be created as needed and time to live will be 60 seconds.
If you want to use this way then below is the way.
// this should be a singleton
ExecutorService emailExecutor = Executors.newCachedThreadPool();
// from you getSalesUserData() method
emailExecutor.execute(new Runnable() {
#Override
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed", e);
}
}
});
Manually creating of ExecutorService on java web serer is bad idea. In your implementation for each request you create 10 threads.
Better solution is to use ManagedExecutorService (example) if you work with JEE7 or ThreadPoolTaskExecutor if you work with Spring(docs).
If you work with Tomcat you should read this thread.
The best practice is to use a single ExecutorService to provide a thread pool for all requests. You probably want to configure the ExecutorService with a non-zero, but limited, number of threads.
The idea here is that you will have some threads that are reused throughout the lifetime of the application. You get the added benefit that if there is a temporary slowdown (or halt) in sending emails, you don't end up with a growing number of threads Instead, you end up with a growing number of pieces of work (emails to send) to be executed, which is much less resource intensive than extra threads.
I am using Java EmailSender class.
I simply started a new thread to send mail because it was blocking the main thread and I was getting Time out an exception.
String link = "http://localhost:PORT/api/v1/registration/confirm?token=" +token;
//Sending mail in thread beacause it block main thread
new Thread(
() -> emailSender.sendMail(request.getEmail(),buildEmail(request.getFirstName(),
link))).start();
I need to develop a web service operation with CXF 3 hosted by Tomcat 7. Our model layer is Spring 3.
This operation calls 16 other web services hosted by distant servers. We need to wait all responses in order to construct the response of our own operation.
We currently call each distant operations sequentially. Of course, we have response time issue. I think we should parallelize our operation inner calls and synchronize the different responses.
What kind of multithreading implementation can be safe? What can we do to make it better?
I'd use Java's generic Futures and a Spring's #Async methods in a #Service.
In short, you call the services sequentially and get all results as Futures, and then you simply check whether all the futures have finished proccessing. You can also do some work with partial data if there is such possibility.
Here's a simple example on how to do it. A sample service from the link:
#Service
public class GitHubLookupService {
RestTemplate restTemplate = new RestTemplate();
#Async
public Future<User> findUser(String user) throws InterruptedException {
System.out.println("Looking up " + user);
User results = restTemplate.getForObject("https://api.github.com/users/" + user, User.class);
// Artificial delay of 1s for demonstration purposes
Thread.sleep(1000L);
return new AsyncResult<User>(results);
}
}
And a method using it:
#Override
public void run(String... args) throws Exception {
// Start the clock
long start = System.currentTimeMillis();
// Kick of multiple, asynchronous lookups
Future<User> page1 = gitHubLookupService.findUser("PivotalSoftware");
Future<User> page2 = gitHubLookupService.findUser("CloudFoundry");
Future<User> page3 = gitHubLookupService.findUser("Spring-Projects");
// Wait until they are all done
while (!(page1.isDone() && page2.isDone() && page3.isDone())) {
Thread.sleep(10); //10-millisecond pause between each check
}
// Print results, including elapsed time
System.out.println("Elapsed time: " + (System.currentTimeMillis() - start));
System.out.println(page1.get());
System.out.println(page2.get());
System.out.println(page3.get());
}
I would use a traditional approach using join() to wait for the threads to finish instead of polling (I don't like polling pattern too much).
Kind of this for a generic thread to replicate:
public class ThreadedWebServiceRetrieval extends Thread {
private List<ResultObject> resultList;
private GenericWebServiceStub stub;
public ThreadedWebServiceRetrieval (List<ResultObject> resultList, GenericWebServiceStub stub) {
this.resultList = resultList;
this.stub = stub;
}
public void run() {
resultList.add(stub.retrieveData());
}
}
And this for the parallel retrieval code:
// ... Controller/Service stuff
List<ResultObject> resultList = new LinkedList<>();//Diamond operator
List<Thread> webServiceList = new LinkedList<>();
webServiceList.add(new ThreadedWebServiceRetrieval(resultList, stub1));
//...
webServiceList.add(new ThreadedWebServiceRetrieval(resultList, stubN));
for (Thread thread : webServiceList) {
thread.start();
}
for (Thread thread : webServiceList) {
thread.join();
}
// resultList is fulfilled
Time of this approach should be +/- longest retrieval.
I made the code VERY generic (overall in the Thread implementation) but it's intentional to fit most cases.
Enjoy!