Develop Reference

Async API giving worse performance

Interesting, I would think have 255 concurrent users, an async API would have better performance. Here are 2 of my endpoints in my Spring server:
#RequestMapping("/async")
public CompletableFuture<String> g(){
CompletableFuture<String> f = new CompletableFuture<>();
f.runAsync(() -> {
try {
Thread.sleep(500);
f.complete("Finished");
} catch (InterruptedException e) {
e.printStackTrace();
}
});
return f;
}
#RequestMapping("/sync")
public String h() throws InterruptedException {
Thread.sleep(500);
return "Finished";
}
In the /async it runs it on a different thread. I am using Siege for load testing as follows:
siege http://localhost:8080/sync --concurrent=255 --time=10S > /dev/null
For the async endpoint, I got a transaction number of 27 hits
For the sync endpoint, I got a transaction number of 1531 hits
So why is this? Why isnt the async endpoint able to handle more transactions?

Because the async endpoint is using a shared (the small ForkJoinPool.commonPool()) threadpool to execute the sleeps, whereas the sync endpoint uses the larger threadpool of the application server. Since the common pool is so small, you're running maybe 4-8 operations (well, if you call sleeping an operation) at a time, while others are waiting for their turn to even get in the pool. You can use a bigger pool with CompletableFuture.runAsync(Runnable, Executor) (you're also calling the method wrong, it's a static method that returns a CompletableFuture).
Async isn't a magical "make things faster" technique. Your example is flawed as all the requests take 500ms and you're only adding overhead in the async one.

Interrupt a long running Jersey Client operation

I am using the Oracle Jersey Client, and am trying to cancel a long running get or put operation.
The Client is constructed as:
JacksonJsonProvider provider = new JacksonJsonProvider(new ObjectMapper());
ClientConfig clientConfig = new DefaultClientConfig();
clientConfig.getSingletons().add(provider);
Client client = Client.create(clientConfig);
The following code is executed on a worker thread:
File bigZipFile = new File("/home/me/everything.zip");
WebResource resource = client.resource("https://putfileshere.com");
Builder builder = resource.getRequestBuilder();
builder.type("application/zip").put(bigZipFile); //This will take a while!
I want to cancel this long-running put. When I try to interrupt the worker thread, the put operation continues to run. From what I can see, the Jersey Client makes no attempt to check for Thread.interrupted().
I see the same behavior when using an AsyncWebResource instead of WebResource and using Future.cancel(true) on the Builder.put(..) call.
So far, the only solution I have come up with to interrupt this is throwing a RuntimeException in a ContainerListener:
client.addFilter(new ConnectionListenerFilter(
new OnStartConnectionListener(){
public ContainerListener onStart(ClientRequest cr) {
return new ContainerListener(){
public void onSent(long delta, long bytes) {
//If the thread has been interrupted, stop the operation
if (Thread.interrupted()) {
throw new RuntimeException("Upload or Download canceled");
}
//Report progress otherwise
}
}...
I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.

I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.
Yeah, interrupting the thread will only work if the code is watching for the interrupts or calling other methods (such as Thread.sleep(...)) that watch for it.
Throwing an exception out of listener doesn't sound like a bad idea. I would certainly create your own RuntimeException class such as TimeoutRuntimeException or something so you can specifically catch and handle it.
Another thing to do would be to close the underlying IO stream that is being written to which would cause an IOException but I'm not familiar with Jersey so I'm not sure if you can get access to the connection.
Ah, here's an idea. Instead of putting the File, how about putting some sort of extension on a BufferedInputStream that is reading from the File but also has a timeout. So Jersey would be reading from the buffer and at some point it would throw an IOException if the timeout expires.

As of Jersey 2.35, the above API has changed. A timeout has been introduces in the client builder which can set read timeout. If the server takes too long to respond, the underlying socket will timeout. However, if the server starts sending the response, it shall not timeout. This can be utilized, if the server does not start sending partial response, which depends on the server implementation.
client=(JerseyClient)JerseyClientBuilder
.newBuilder()
.connectTimeout(1*1000, TimeUnit.MILLISECONDS)
.readTimeout(5*1000, TimeUnit.MILLISECONDS).build()
The current filters and interceptors are for data only and the solution posted in the original question will not work with filters and interceptors (though I admit I may have missed something there).
Another way is to get hold of the underlying HttpUrlConnection (for standard Jersey client configuration) and it seems to be possible with org.glassfish.jersey.client.HttpUrlConnectorProvider
HttpUrlConnectorProvider httpConProvider=new HttpUrlConnectorProvider();
httpConProvider.connectionFactory(new CustomHttpUrlConnectionfactory());
public static class CustomHttpUrlConnectionfactory implements
HttpUrlConnectorProvider.ConnectionFactory{
#Override
public HttpURLConnection getConnection(URL url) throws IOException {
System.out.println("CustomHttpUrlConnectionfactory ..... called");
return (HttpURLConnection)url.openConnection();
}//getConnection closing
}//inner-class closing
I did try the connection provider approach, however, I could not get that working. The idea would be to keep reference to the connection by some means (thread id etc.) and close it if the communication is taking too long. The primary problem was I could not find a way to register the provider with the client. The standard
.register(httpConProvider)
mechanism does not seem to work (or perhaps it is not supposed to work like that) and the documentation is a bit sketchy in that direction.

How to test my httpClient can close idled connections?

I'm using http-client 4.3.x in my project. In order to close "idled" connections, I started a thread to monitor the connection pool and try to close connections which have been idled for 30s every one second.
The connectionManager:
PoolingHttpClientConnectionManager connectionManager =
new PoolingHttpClientConnectionManager(registry);
The thread to monitor it and close idled connections:
class ClosingIdleConnectionThread extends Thread {
private final PoolingHttpClientConnectionManager connectionManager;
private final int timeout;
public ClosingIdleConnectionThread(PoolingHttpClientConnectionManager connectionManager, int timeout) {
this.connectionManager = connectionManager;
this.timeout = timeout;
}
#Override
public void run() {
while (!Thread.currentThread().isInterrupted()) {
connectionManager.closeIdleConnections(timeout, TimeUnit.MILLISECONDS);
connectionManager.closeExpiredConnections();
try {
TimeUnit.SECONDS.sleep(1);
} catch (InterruptedException e) {
return;
}
}
}
}
But now I have two problems:
Actually, I'm not sure what does "idled" connection means
I have to write some tests to make sure my code is working, but I don't know how to test it
Thanks for your help ~

(1) A persistent connection is considered idle if it has not been leased from the pool. Idle for 5 seconds = has not been leased for 5 seconds.
(2) This is a tough one. I see two options here
Use PowerMock, instrument SocketImpl abstract class and 'spy' on its #close method. Disclaimer: I have not tried it. It's just an idea and it may not work.
Use Mockito or similar toolkit and spy on PlainConnectionSocketFactory instance instead. Register Mockito enhanced implementation with the connection pool manager. Count how many times #connectSocket method has been called. If the method has been invoked upon new connection lease after a long period of inactivity, that basically means one of two things: the pool was either empty or idle connections were not in a re-usable state. It is not a perfect test, but it could be easier to implement given it does not require classloader magic of PowerMock.

Java Multithreaded - Better way to cancel Future task with database and http connections?

I am having difficulty trying to correctly program my application in the way I want it to behave.
Currently, my application (as a Java Servlet) will query the database for a list of items to process. For every item in the list, it will submit an HTTP Post request. I am trying to create a way where I can stop this processing (and even terminate the HTTP Post request in progress) if the user requests. There can be simultaneous threads that are separately processing different queries. Right now, I will stop processing in all threads.
My current attempt involves implementing the database query and HTTP Post in a Callable class. Then I submit the Callable class via the Executor Service to get a Future object.
However, in order properly to stop the processing, I need to abort the HTTP Post and close the database's Connection, Statement and ResultSet - because the Future.cancel() will not do this for me. How can I do this when I call cancel() on the Future object? Do I have to store a List of Arrays that contains the Future object, HttpPost, Connection, Statement, and ResultSet? This seems overkill - surely there must be a better way?
Here is some code I have right now that only aborts the HttpPost (and not any database objects).
private static final ExecutorService pool = Executors.newFixedThreadPool(10);
public static Future<HttpClient> upload(final String url) {
CallableTask ctask = new CallableTask();
ctask.setFile(largeFile);
ctask.setUrl(url);
Future<HttpClient> f = pool.submit(ctask); //This will create an HttpPost that posts 'largefile' to the 'url'
linklist.add(new tuple<Future<HttpClient>, HttpPost>(f, ctask.getPost())); //storing the objects for when I cancel later
return f;
}
//This method cancels all running Future tasks and aborts any POSTs in progress
public static void cancelAll() {
System.out.println("Checking status...");
for (tuple<Future<HttpClient>, HttpPost> t : linklist) {
Future<HttpClient> f = t.getFuture();
HttpPost post = t.getPost();
if (f.isDone()) {
System.out.println("Task is done!");
} else {
if (f.isCancelled()) {
System.out.println("Task was cancelled!");
} else {
while (!f.isDone()) {
f.cancel(true);
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("!Aborting Post!");
try {
post.abort();
} catch (Exception ex) {
System.out.println("Aborted Post, swallowing exception: ");
ex.printStackTrace();
}
}
}
}
}
}
Is there an easier way or a better design? Right now I terminate all processing threads - in the future, I would like to terminate individual threads.

I think keeping a list of all the resources to be closed is not the best approach. In your current code, it seems that the HTTP request is initiated by the CallableTask but the closing is done by somebody else. Closing resources is the responsibility of the one who opened it, in my opinion.
I would let CallableTask to initiate the HTTP request, connect to database and do it's stuff and, when it is finished or aborted, it should close everything it opened. This way you have to keep track only the Future instances representing your currently running tasks.

I think your approach is correct. You would need to handle the rollback yourself when you are canceling the thread
cancel() just calls interrupt() for already executing thread. Have a look here
http://docs.oracle.com/javase/tutorial/essential/concurrency/interrupt.html:
As it says
An interrupt is an indication to a thread that it should stop what it
is doing and do something else. It's up to the programmer to decide
exactly how a thread responds to an interrupt, but it is very common
for the thread to terminate.
Interrupted thread would throw InterruptedException
when a thread is waiting, sleeping, or otherwise paused for a long
time and another thread interrupts it using the interrupt() method in
class Thread.
So you need to explicitly code for scenarios such as you mentioned in executing thread where there is a possible interruption.

connection pooling timeout problems in multithreading environment

My team has to make some changes and renew an old web application. This application has one main thread and 5 to 15 daemon threads used as workers to retrieve and insert data in a DB.
All those threads have this design (here simplified for convenience):
public MyDaemon implements Runnable {
// initialization and some other stuffs
public void run() {
...
while(isEnabled) {
Engine.doTask1();
Engine.doTask2();
...
Thread.sleep(someTime);
}
}
}
The Engine class provides a series of static methods used to maipulate other methods of DataAccessor classes, some of those methods been static:
public Engine {
public static doTask1() {
ThisDataAccessor.retrieve(DataType data);
// some complicated operations
ThisDataAccessor.insertOrUpdate(DataType data);
}
public static doTask2() {
ThatDataAccessor da = new ThatDataAccessor();
da.retrieve(DataType data);
// etc.
}
...
}
DataAccessor classes usually interact with DB using simple JDBC statements enclosed in synchronized methods (static for some classes). DataSource is configured in the server.
public ThatDataAccessor {
public synchronized void retrieve(DataType data) {
Connection conn = DataSource.getConnection();
// JDBC stuff
conn.close();
}
...
}
The problem is that the main thread needs to connect to DB and when these daemon threads are working we run easily out of available connections from the pool, getting "waiting for connection timeout" exceptions. In addition, sometimes even those daemon threads get the same exception.
We have to get rid of this problem.
We have a connection pool configured with 20 connections, and no more can be added since that "20" is our production environment standard. Some blocks of code need to be synchronized, even if we plan to move the "synchronized" keyword only where really needed. But I don't think that it would make really the difference.
We are not experienced in multithreading programming and we've never faced this connection pooling problem before, that's why I'm asking: is the problem due to the design of those threads? Is there any flaw we haven't noticed?
I have profiled thread classes one by one and as long as they are not running in parallel it seems that there's no bottleneck to justify those "waiting for connection timeout".
The app is running on WebSphere 7, using Oracle 11g.

You are likely missing a finally block somewhere to return the connections back to the pool. With hibernate, I think this is probably done when you call close() or possibly for transactions, when you call rollback(). But I would call close anyway.
For example, I wrote a quick and dirty pool myself to extend an old app to make it multithreaded, and here is some of the handling code (which should be meaningless to you except the finnally block):
try {
connection = pool.getInstance();
connection.beginTransaction();
processFile(connection, ...);
connection.endTransaction();
logger_multiThreaded.info("Done processing file: " + ... );
} catch (IOException e) {
logger_multiThreaded.severe("Failed to process file: " + ... );
e.printStackTrace();
} finally {
if (connection != null) {
pool.releaseInstance(connection);
}
}
It is fairly common for people to fail to use finally blocks properly... For example, look at this hibernate tutorial, and skip to the very bottom example. You will see that in the try{} he uses tx.commit() and in the catch{} he uses tx.rollback(), but he has no session.close(), and no finally. So even if he added a "session.close()" in try and in catch, if his try block threw something other than a RuntimeException, or his catch caused an additional Exception before the try or a non-HibernateException before the rollback(), his connection would not be closed. And without session.close(), I don't think that is actually very good code. But even if the code is seemingly working, a finally gives you assurance that you are protected from this type of problem.
So I would rewrite his methods that use Session to match the idiom shown on this hibernate documentation page. (and also I don't recommend his throwing a RuntimeException, but that is a different topic).
So if you are using Hibernate, I think the above is good enough. But otherwise, you'll need to be more specific if you want specific code help, but otherwise the simple idea that you should use a finally to ensure the connection is closed is enough.