I'm trying to write a non-blocking proxy with netty 4.1. I have a "FrontHandler" which handles incoming connections, and then a "BackHandler" which handles outgoing ones. I'm following the HexDumpProxyHandler (https://github.com/netty/netty/blob/ed4a89082bb29b9e7d869c5d25d6b9ea8fc9d25b/example/src/main/java/io/netty/example/proxy/HexDumpProxyFrontendHandler.java#L67)
In this code I have found:
#Override
public void channelRead(final ChannelHandlerContext ctx, Object msg) {
if (outboundChannel.isActive()) {
outboundChannel.writeAndFlush(msg).addListener(new ChannelFutureListener() {, I've seen:
Meaning that the incoming message is only written if the outbound client connection is already ready. This is obviously not ideal in a HTTP proxy case, so I am thinking what would be the best way to handle it.
I am wondering if disabling auto-read on the front-end connection (and only trigger reads manually once the outgoing client connection is ready) is a good option. I could then enable autoRead over the child socket again, in the "channelActive" event of the backend handler. However, I am not sure about how many messages would I get in the handler for each "read()" invocation (using HttpDecoder, I assume I would get the initial HttpRequest, but I'd really like to avoid getting the subsequent HttpContent / LastHttpContent messages until I manually trigger the read() again and enable autoRead over the channel).
Another option would be to use a Promise to get the Channel from the client ChannelPool:
private void setCurrentBackend(HttpRequest request) {
pool.acquire(request, backendPromise);
backendPromise.addListener((FutureListener<Channel>) future -> {
Channel c = future.get();
if (!currentBackend.compareAndSet(null, c)) {
pool.release(c);
throw new IllegalStateException();
}
});
}
and then do the copying from input to output thru that promise. Eg:
private void handleLastContent(ChannelHandlerContext frontCtx, LastHttpContent lastContent) {
doInBackend(c -> {
c.writeAndFlush(lastContent).addListener((ChannelFutureListener) future -> {
if (future.isSuccess()) {
future.channel().read();
} else {
pool.release(c);
frontCtx.close();
}
});
});
}
private void doInBackend(Consumer<Channel> action) {
Channel c = currentBackend.get();
if (c == null) {
backendPromise.addListener((FutureListener<Channel>) future -> action.accept(future.get()));
} else {
action.accept(c);
}
}
but I'm not sure about how good it is to keep the promise there forever and do all the writes from "front" to "back" by adding listeners to it. I'm also not sure about how to instance the promise so that the operations are performed in the right thread... right now I'm using:
backendPromise = group.next().<Channel> newPromise(); // bad
// or
backendPromise = frontCtx.channel().eventLoop().newPromise(); // OK?
(where group is the same eventLoopGroup as used in the ServerBootstrap of the frontend).
If they're not handled thru the right thread, I assume it could be problematic to have the "else { }" optimization in the "doInBackend" method to avoid using the Promise and write to the channel directly.
The no-autoread approach doesn't work by itself, because the HttpRequestDecoder creates several messages even if only one read() was performed.
I have solved it by using chained CompletableFutures.
I have worked on a similar proxy application based on the MQTT protocol. So it was basically used to create a real-time chat application. The application that I had to design however was asynchronous in nature so I naturally did not face any such problem. Because in case the
outboundChannel.isActive() == false
then I can simply keep the messages in a queue or a persistent DB and then process them once the outboundChannel is up. However, since you are talking about an HTTP application, so this means that the application is synchronous in nature meaning that the client cannot keep on sending packets until the outboundChannel is up and running. So the option you suggest is that the packet will only be read once the channel is active and you can manually handle the message reads by disabling the auto read in ChannelConfig.
However, what I would like to suggest is that you should check if the outboundChannel is active or not. In case the channel is active, send he packet forward for processing. In case the channel is not active, you should reject the packet by sending back a response similar to Error404
Along with this you should configure your client to keep on retrying sending the packets after certain intervals and accordingly handle what needs to be done in case the channel takes too long a time to become active and become readable. Manually handling channelRead is generally not preferred and is an anti pattern. You should let Netty handle that for you in the most efficient way.
Related
I am trying to use connection pooling in netty, but I am having issues writing some of the error handling. My original code had something like the following:
ChannelFuture connectFuture = bootstrap.connect(...);
connectFuture.addListener(ChannelFutureListener.CLOSE_ON_FAILURE);
However, ChannelPool.acquire returns a Future<Channel>. This means on operation failure, there's no access to the channel, so I'm not aware of a way to close it. Is it important to close the failed channel? I assume it may still hold onto some system resources, even if it wasn't able to connect.
I think the relevant code is in the netty class SimpleChannelPool in notifyConnect:
private void notifyConnect(ChannelFuture future, Promise<Channel> promise) throws Exception {
if (future.isSuccess()) {
Channel channel = future.channel();
handler.channelAcquired(channel);
if (!promise.trySuccess(channel)) {
// Promise was completed in the meantime (like cancelled), just release the channel again
release(channel);
}
} else {
promise.tryFailure(future.cause());
}
}
Here we can see the promise returned to the caller is failed, but the channel is not propagated.
This ended up being very simple, you just need to override SimpleChannelPool.connectChannel()` and add the listener there.
For the last week I read documentation about vertx. What i don't get it's how vertx handlers are work? For example
public class Client extends AbstractVerticle{
#Override
public void start() throws Exception {
final HttpClient httpClient = this.vertx.createHttpClient();
this.vertx.setPeriodic(1000, handler->{
httpClient.getNow(8080, "localhost", "/", responseHandler -> {
System.out.println("response");
});
});
}
}
And server is:
public class JdbcVertx extends AbstractVerticle{
#Override
public void start() throws Exception {
JDBCClient client = JDBCClient.createNonShared(this.vertx, new JsonObject()
.put("url", "jdbc:postgresql://localhost:5432/test")
.put("user", "user")
.put("password", "password")
.put("driver_class", "org.postgresql.Driver")
.put("max_pool_size", 30));
this.vertx.createHttpServer()
.requestHandler(r -> {
client.getConnection(handler -> {
final SQLConnection connection = handler.result();
connection.execute(execute(), hndlr -> {
connection.close(closehndlr -> {
r.response().putHeader("content-type", "text/html").end("Response");
});
});
});
}).listen(8080);
}
private String execute(){
return "insert into rubish (name) values ('test')";
}
}
(P.S i know that i firstly should to check if handler is succeded and then make some action but i remove this checking to simplify code and also from official docs if there is no any response during 30 sec there will be an exception in handler)
From the code above, client send request each second and doesn't wait for response , but it has a handler that will be executed when response was comming.
'JdbcVertx' listen on port 8080 , get request, make insertion to db with sleep for example 3 s (i put 1_000_000 rows to db and create index to slow down insertion time) and then send response, therefore each request is non blocking.
As i know , vertx has only one thread named EventLoop event loop from jdbcVertx get reqests but doesn't return response immediately , instead it put a handler that will be executed when db insertion was succeed. How event loop know when IO action is done. I think that it use somthing like this
if(Thread.currentThread().getState != 'blocked' && sec != 30){
this.object.getHandler().execute();
} else if(sec == 30){
Thread.currentThread.inerrupt();
} else{
sec++;
}
But we have only one thread, and when we have blocking call it doesn't has a thread, only handler.
The problem is , how event loop know when blocking operation is ended and it's time to execute handler
But we have only one thread, and when we have blocking call it doesn't has a thread, only handler. How it work , and why do we need to use Worker Verticle if we can use handlers instead?
The handlers are just actions triggered upon receipt of an eventbus message or an http call. They are not designed to handle scalability for you. If you only use handlers and if your actions starts to take a long time or if you have any increase in the number of requests, you will block the eventloop of your verticle and will have a lot of Thread xxxx has been blocked warns.
To answer on how handler works and why the event loop doesn't wait the end of a handler to start another, according to this : https://vertx.io/docs/vertx-core/java/#_reactor_and_multi_reactor
Instead of a single event loop, each Vertx instance maintains several event loops. By default we choose the number based on the number of available cores on the machine, but this can be overridden.
This means a single Vertx process can scale across your server, unlike Node.js.
We call this pattern the Multi-Reactor Pattern to distinguish it from the single threaded reactor pattern.
But that's not enough to handle all scalability and thread blocking problematics for you in my opinion, you schould read this too : https://vertx.io/docs/vertx-core/java/#golden_rule
There are many ways to design verticles but you have to stay as non-blocking as possible. In my opinion, using vert.x with a traditional blocking approach (like blocking restfull endpoints for example) is not relevant.
Personally I'd design my verticles as follows :
verticle A : which expose a restfull endpoint and take a callback url (whatever the action GET/POST/PUT/PATCH/DELETE). The verticle always respond a 202 Accepted immediatly without result and send a message in the eventbus to a verticle B.
verticle B : get the message, do the action (eventually invoke other verticles asynchronously with the eventbus and waiting the replies) and reply invoking the callback url.
I'd avoid to use worker verticle or the executeBlocking method or even creating a pool of thread. I'd privilege multiplying the instances of my verticles B (in seperate pids) that listen to the same eventbus cluster (and eventually verticle A with a http reverse proxy). We can even imagine having a variable number of verticle B instances (in seperate pids) depending on the number of requests in real time.
P.S : sometimes I use a more powerfull message broker tool like Apache Kafka instead of the native eventbus (when I need to respect a sort of message, or when I need to replay some messages).
Answering the question:
how event loop know when blocking operation is ended and it's time to execute handler?
According to the non-blocking model, the event-loop upon the call
connection.execute( execute(), hndlr )
spawns a new thread, executes your blocking piece of code and upon it's completion (something like Thread.join()) invokes the hndlr callback in the event-loop thread. Thus the main loop doesn't get blocked although the blocking code can be executed.
I am using the Oracle Jersey Client, and am trying to cancel a long running get or put operation.
The Client is constructed as:
JacksonJsonProvider provider = new JacksonJsonProvider(new ObjectMapper());
ClientConfig clientConfig = new DefaultClientConfig();
clientConfig.getSingletons().add(provider);
Client client = Client.create(clientConfig);
The following code is executed on a worker thread:
File bigZipFile = new File("/home/me/everything.zip");
WebResource resource = client.resource("https://putfileshere.com");
Builder builder = resource.getRequestBuilder();
builder.type("application/zip").put(bigZipFile); //This will take a while!
I want to cancel this long-running put. When I try to interrupt the worker thread, the put operation continues to run. From what I can see, the Jersey Client makes no attempt to check for Thread.interrupted().
I see the same behavior when using an AsyncWebResource instead of WebResource and using Future.cancel(true) on the Builder.put(..) call.
So far, the only solution I have come up with to interrupt this is throwing a RuntimeException in a ContainerListener:
client.addFilter(new ConnectionListenerFilter(
new OnStartConnectionListener(){
public ContainerListener onStart(ClientRequest cr) {
return new ContainerListener(){
public void onSent(long delta, long bytes) {
//If the thread has been interrupted, stop the operation
if (Thread.interrupted()) {
throw new RuntimeException("Upload or Download canceled");
}
//Report progress otherwise
}
}...
I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.
I am wondering if there is a better solution (perhaps when creating the Client) that correctly handles interruptible I/O without using a RuntimeException.
Yeah, interrupting the thread will only work if the code is watching for the interrupts or calling other methods (such as Thread.sleep(...)) that watch for it.
Throwing an exception out of listener doesn't sound like a bad idea. I would certainly create your own RuntimeException class such as TimeoutRuntimeException or something so you can specifically catch and handle it.
Another thing to do would be to close the underlying IO stream that is being written to which would cause an IOException but I'm not familiar with Jersey so I'm not sure if you can get access to the connection.
Ah, here's an idea. Instead of putting the File, how about putting some sort of extension on a BufferedInputStream that is reading from the File but also has a timeout. So Jersey would be reading from the buffer and at some point it would throw an IOException if the timeout expires.
As of Jersey 2.35, the above API has changed. A timeout has been introduces in the client builder which can set read timeout. If the server takes too long to respond, the underlying socket will timeout. However, if the server starts sending the response, it shall not timeout. This can be utilized, if the server does not start sending partial response, which depends on the server implementation.
client=(JerseyClient)JerseyClientBuilder
.newBuilder()
.connectTimeout(1*1000, TimeUnit.MILLISECONDS)
.readTimeout(5*1000, TimeUnit.MILLISECONDS).build()
The current filters and interceptors are for data only and the solution posted in the original question will not work with filters and interceptors (though I admit I may have missed something there).
Another way is to get hold of the underlying HttpUrlConnection (for standard Jersey client configuration) and it seems to be possible with org.glassfish.jersey.client.HttpUrlConnectorProvider
HttpUrlConnectorProvider httpConProvider=new HttpUrlConnectorProvider();
httpConProvider.connectionFactory(new CustomHttpUrlConnectionfactory());
public static class CustomHttpUrlConnectionfactory implements
HttpUrlConnectorProvider.ConnectionFactory{
#Override
public HttpURLConnection getConnection(URL url) throws IOException {
System.out.println("CustomHttpUrlConnectionfactory ..... called");
return (HttpURLConnection)url.openConnection();
}//getConnection closing
}//inner-class closing
I did try the connection provider approach, however, I could not get that working. The idea would be to keep reference to the connection by some means (thread id etc.) and close it if the communication is taking too long. The primary problem was I could not find a way to register the provider with the client. The standard
.register(httpConProvider)
mechanism does not seem to work (or perhaps it is not supposed to work like that) and the documentation is a bit sketchy in that direction.
I have a ServerBootstrap configured with a fairly standard Http-Codec ChannelInitializer.
On shutdown my server waits for a grace period where it can still handle incoming requests. My server supports keep-alive, but on shutdown I want to make sure every HttpResponse sent closes the connection with HTTP header "Connection: close" and that the channel is closed after the write. This is only necessary on server shutdown.
I have a ChannelHandler to support that:
#ChannelHandler.Sharable
public class CloseConnectionHandler extends ChannelOutboundHandlerAdapter {
#Override
public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
HttpResponse response = (HttpResponse) msg;
if (isKeepAlive(response)) {
setKeepAlive(response, false);
promise.addListener(ChannelFutureListener.CLOSE);
}
ctx.write(msg, promise);
}
I keep a track of all connected clients using a ChannelGroup, so I can dynamically modify the pipeline of each client at the point of shutdown to include my CloseConnectionHandler, this works no problem.
However, new connections in the grace period have their pipeline configuration provided by the original ServerBootstrap ChannelInitializer, and I can't see a way of dynamically re-configuring that?
As a work-around I can have the CloseConnectionHandler configured in the standard pipeline and turned off with a boolean, only activating it on shutdown. But I'd rather avoid that if possible, seems a bit unnecessary.
there is currently no way to "replace" the initializer at run-time. So using a flag etc would be the best bet.
I have an Akka actor that owns an AsyncHttpClient. This actor must handles a lot of asynchronous requests. Because my system cannot handle thousands of requests simultaneously, I need to limit the number of concurrent requests.
Right now, I'm doing this :
AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder().setAllowPoolingConnection(true)
.addRequestFilter(new ThrottleRequestFilter(32))
.setMaximumConnectionsPerHost(16)
.setMaxRequestRetry(5)
.build();
final AsyncHttpClient httpClient = new AsyncHttpClient(new NettyAsyncHttpProvider(config));
When my actor receives a message, I use the client like this :
Future<Integer> f = httpClient.prepareGet(url).execute(
new AsyncCompletionHandler<Integer>() {
#Override
public Integer onCompleted(Response response) throws Exception {
// handle successful request
}
#Override
public void onThrowable(Throwable t){
// handle failed request
}
}
);
The problem is that requests are never put in the client queue and are all processed like the configuration doesn't matter. Why doesn't this work as it should?
From the maintainer:
setMaxConnectionsPerHost only caps the number of connections that can be open to a given host. There's no built-in queuing mechanism for requests that might need a connection while there's none available.
So basically, it's a hard limit. Also, in versions of the library prior to, I believe, 1.9.10, the maximumConnectionsPerHost field was not being properly utilized by the code to limit the number of concurrent connections per host. Instead, there was a bug where the client only looked at the maximumConnectionsTotal field.
Link to issue referenced on GitHub