Develop Reference

RxNetty reuse the connection

I want to use Netflix-Ribbon as TCP client load balancer without Spring Cloud,and i write test code.
public class App implements Runnable
{
public static String msg = "hello world";
public BaseLoadBalancer lb;
public RxClient<ByteBuf, ByteBuf > client;
public Server echo;
App(){
lb = new BaseLoadBalancer();
echo = new Server("localhost", 8000);
lb.setServersList(Lists.newArrayList(echo));
DefaultClientConfigImpl impl = DefaultClientConfigImpl.getClientConfigWithDefaultValues();
client = RibbonTransport.newTcpClient(lb, impl);
}
public static void main( String[] args ) throws Exception
{
for( int i = 40; i > 0; i--)
{
Thread t = new Thread(new App());
t.start();
t.join();
}
System.out.println("Main thread is finished");
}
public String sendAndRecvByRibbon(final String data)
{
String response = "";
try {
response = client.connect().flatMap(new Func1<ObservableConnection<ByteBuf, ByteBuf>,
Observable<ByteBuf>>() {
public Observable<ByteBuf> call(ObservableConnection<ByteBuf, ByteBuf> connection) {
connection.writeStringAndFlush(data);
return connection.getInput();
}
}).timeout(1, TimeUnit.SECONDS).retry(1).take(1)
.map(new Func1<ByteBuf, String>() {
public String call(ByteBuf ByteBuf) {
return ByteBuf.toString(Charset.defaultCharset());
}
})
.toBlocking()
.first();
}
catch (Exception e) {
System.out.println(((LoadBalancingRxClientWithPoolOptions) client).getMaxConcurrentRequests());
System.out.println(lb.getLoadBalancerStats());
}
return response;
}
public void run() {
for (int i = 0; i < 200; i++) {
sendAndRecvByRibbon(msg);
}
}
}
i find it will create a new socket everytime i callsendAndRecvByRibbon even though the poolEnabled is setting to true. So,it confuse me,i miss something?
and there are no option to configure the size of the pool,but hava a PoolMaxThreads and MaxConnectionsPerHost.
My question is how to use a connection pool in my simple code, and what's wrong with my sendAndRecvByRibbon,it open a socket then use it only once,how can i reuse the connection?thanks for your time.
the server is just a simple echo server writing in pyhton3,i comment outconn.close() because i want to use long connection.
import socket
import threading
import time
import socketserver
class ThreadedTCPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
conn = self.request
while True:
client_data = conn.recv(1024)
if not client_data:
time.sleep(5)
conn.sendall(client_data)
# conn.close()
class ThreadedTCPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
pass
if __name__ == "__main__":
HOST, PORT = "localhost", 8000
server = ThreadedTCPServer((HOST, PORT), ThreadedTCPRequestHandler)
ip, port = server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = True
server_thread.start()
server.serve_forever()
and the pom of mevan,i just add two dependency in IED's auto generated POM.
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>com.netflix.ribbon</groupId>
<artifactId>ribbon</artifactId>
<version>2.2.2</version>
</dependency>
the code for printing src_port
#Sharable
public class InHandle extends ChannelInboundHandlerAdapter {
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
System.out.println(ctx.channel().localAddress());
super.channelRead(ctx, msg);
}
}
public class Pipeline implements PipelineConfigurator<ByteBuf, ByteBuf> {
public InHandle handler;
Pipeline() {
handler = new InHandle();
}
public void configureNewPipeline(ChannelPipeline pipeline) {
pipeline.addFirst(handler);
}
}
and change the client = RibbonTransport.newTcpClient(lb, impl);to Pipeline pipe = new Pipeline();client = RibbonTransport.newTcpClient(lb, pipe, impl, new DefaultLoadBalancerRetryHandler(impl));

So, your App() constructor does the initialization of lb/client/etc.
Then you're starting 40 different threads with 40 different RxClient instances (each instance has own pool by default) by calling new App() in the first for loop. To make things clear - the way you spawn multiple RxClient instances here does not allow them to share any common pool. Try to use one RxClient instance instead.
What if you change your main method like below, does it stop creating extra sockets?
public static void main( String[] args ) throws Exception
{
App app = new App() // Create things just once
for( int i = 40; i > 0; i--)
{
Thread t = new Thread(()->app.run()); // pass the run()
t.start();
t.join();
}
System.out.println("Main thread is finished");
}
If above does not help fully (at least it will reduce created sockets count in 40 times) - can you please clarify how exactly do you determine that:
i find it will create a new socket everytime i call sendAndRecvByRibbon
and what are your measurements after you update constructor with this line:
DefaultClientConfigImpl impl = DefaultClientConfigImpl.getClientConfigWithDefaultValues();
impl.set(CommonClientConfigKey.PoolMaxThreads,1); //Add this one and test
Update
Yes, looking at the sendAndRecvByRibbon it seems that it lacks marking the PooledConnection as no longer acquired by calling close once you don't expect any further reads from it.
As long as you expect the only single read event, just change this line
connection.getInput()
to the
return connection.getInput().zipWith(Observable.just(connection), new Func2<ByteBuf, ObservableConnection<ByteBuf, ByteBuf>, ByteBuf>() {
#Override
public ByteBuf call(ByteBuf byteBuf, ObservableConnection<ByteBuf, ByteBuf> conn) {
conn.close();
return byteBuf;
}
});
Note, that if you'd design more complex protocol over TCP, then input bytebuf can be analyzed for your specific 'end of communication' sign which indicates the connection can be returned to the pool.

Netty channelAcquired is not getting called

I'm using netty channel pool for a http client and in the ChannelPoolHandler implementation channelAcquired is not getting called when the channelPool.acquire() invoked. I'm using netty 4.0.32.Final. Here's how I created the chanelpool. I just followed the simple example listed at netty.io. If someone can just explain what I've done wrong or if there is a bug that'll be very helpful. Thanks.
EventLoopGroup group = new NioEventLoopGroup();
final Bootstrap b = new Bootstrap();
b.group(group).channel(NioSocketChannel.class);
AbstractChannelPoolMap<InetSocketAddress, SimpleChannelPool> poolMap = new AbstractChannelPoolMap<InetSocketAddress, SimpleChannelPool>() {
#Override
protected SimpleChannelPool newPool(InetSocketAddress key) {
return new SimpleChannelPool(b.remoteAddress(key), new HttpClientPoolHandler());
}
};
final SimpleChannelPool simpleChannelPool = poolMap.get(new InetSocketAddress(uri.getHost(), uri.getPort()));
final Future<Channel> acquire = simpleChannelPool.acquire();
acquire.addListener(new FutureListener<Channel>() {
public void operationComplete(Future<Channel> f) throws Exception {
if (f.isSuccess()) {
final Channel ch = f.getNow();
// Send the HTTP request.
ChannelFuture channelFuture = ch.writeAndFlush(request);
channelFuture.addListener(new ChannelFutureListener() {
public void operationComplete(ChannelFuture channelFuture) throws Exception {
if (channelFuture.isSuccess()) {
simpleChannelPool.release(ch);
} else {
}
}
});
} else {
System.out.println("ERROR : " + f.cause());
}
}
});

The channelAcquiredmethod will only be called if you "acquire" a previous created channel. In your case there is not channel yet in the pool so it will call channelCreated.

ChannelHandlerContext.attr is not accessible from inside userEventTriggered

I am using netty for developing my server.
I am also implementing the Idle state handling in netty.
I got it working but an issue I recently found out.
I can't access the channel context attributes inside the userEventTriggered method.
here is my code and can anybody tell me why it is not possible.
I am setting it like
public static final AttributeKey<Agent> CLIENT_MAPPING = AttributeKey.valueOf("clientMapping");
...
ctx.attr(CLIENT_MAPPING).set(agent);
and inside handler, I am getting the value like (this is working perfectly)
Agent agent = ctx.attr(CLIENT_MAPPING).get();
But inside userEventTriggered it is returning null. (I am sure that it is set before this function is being called.)
public class Server
{
...
public void run() throws Exception
{
...
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup).
channel(NioServerSocketChannel.class).
childHandler(new SslServerInitializer());
...
}
}
class SslServerInitializer extends ChannelInitializer<SocketChannel>
{
#Override
public void initChannel(SocketChannel ch) throws Exception
{
ChannelPipeline pipeline = ch.pipeline();
....
pipeline.addLast("idleStateHandler", new IdleStateHandler(0, 0, Integer.parseInt(Main.configurations.get("netty.idleTimeKeepAlive.ms"))));
pipeline.addLast("idleTimeHandler", new ShelloidIdleTimeHandler());
}
}
class ShelloidIdleTimeHandler extends ChannelDuplexHandler
{
#Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception
{
if (evt instanceof IdleStateEvent)
{
try
{
// This I am getting null, but I confirmed that I set the attribute from my handler and is accessible inside handler.
Agent agt = ctx.attr(WebSocketSslServerHandler.CLIENT_MAPPING).get();
ctx.channel().writeAndFlush(new TextWebSocketFrame("{\"type\":\"PING\", \"userId\": \"" + agt.getUserId() + "\"}"));
}
catch (Exception ex)
{
ctx.disconnect();
ex.printStackTrace();
}
}
}
}

Are you sure you set and get it in the same ChannelHandler? If you want to set and get it in different ChannelHandler you need to use Channel.attr(...)

Netty 5 HexDumpProxy issue

I am facing some problem in HexDumpProxy usage. I am using netty lib netty-all-5.0.0.Alpha1.jar.
In HexDumpProxyInitializer class's initChannel method, I am having
#Override
public void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline cp = ch.pipeline();
cp.addLast(new LoggingHandler(LogLevel.INFO));
cp.addLast("decoder", new StringDecoder());
cp.addLast("encoder", new StringEncoder());
cp.addLast(new HexDumpProxyFrontendHandler(remoteHost, remotePort));
}
and in HexDumpProxyFrontendHandler class, I want to process the incoming message as follows,
here I am converting Object msg to String and want to change the value, and facing problem in sending modified string.
#Override
public void channelRead(final ChannelHandlerContext ctx, Object msg) throws Exception {
String s1 = ((ByteBuf) msg).toString(Charset.defaultCharset());
if (outboundChannel.isActive()) {
outboundChannel.writeAndFlush(s1).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (future.isSuccess()) {
// was able to flush out data, start to read the next
// chunk
ctx.channel().read();
} else {
future.channel().close();
}
}
});
}
}
If I am sending object itself without any modification, its working fine. But if I want to send String, its not throwing any exception, also not working.
After that I have enabled String encoder and decoder in initChannel method, then I am getting the following error,
Proxying *:9999 to 192.168.1.27:8554 ...
java.lang.ClassCastException: java.lang.String cannot be cast to io.netty.buffer.ByteBuf
at ivz.proxy.HexDumpProxyFrontendHandler.channelRead(HexDumpProxyFrontendHandler.java:68)
at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:74)
at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:138)
at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.ChannelHandlerInvokerUtil.invokeChannelReadNow(ChannelHandlerInvokerUtil.java:74)
at io.netty.channel.DefaultChannelHandlerInvoker.invokeChannelRead(DefaultChannelHandlerInvoker.java:138)
at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:320)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:127)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:485)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:452)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:346)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:794)
at java.lang.Thread.run(Unknown Source)
Except the above mentioned methods, I have not changed anything in the code. So, my problem is, I want to change some values in Object msg before sending to server, How can I achive this? Or in other words, is it possible to send String in writeAndFlush method?

I didn't get the logic behind outboundChannel part, but I think there are couple of issues with your code
cp.addLast(new LoggingHandler(LogLevel.INFO));
cp.addLast("decoder", new StringDecoder());
cp.addLast("encoder", new StringEncoder());
cp.addLast(new HexDumpProxyFrontendHandler(remoteHost, remotePort));
Since you have StringDecoder before HexDumpProxyFrontendHandler, so object received in the channelRead will be always String,
When you write the String object back, it will be encoded to ByteBuff by StringEncoder.
In ChannelFutureListener implementation, no need to call ctx.channel().read()
Netty will automatically call your handler's channelRead() method
Update:
channel().read() is required if you want to throttle/control the reads, to do that you have set auto read false in channel options, by default auto read is true.

It's quite late, but I also struggled with this.
So here is how I solved it for my case.
In the HexDumpProxyFrontendHandler class I made the following changes to channelActive():
#Override
public void channelActive(ChannelHandlerContext ctx) {
final Channel inboundChannel = ctx.channel();
// Start the connection attempt.
Bootstrap b = new Bootstrap();
b.group(inboundChannel.eventLoop()).channel(ctx.channel().getClass())
.handler(new HexDumpProxyBackendHandler(inboundChannel))
.option(ChannelOption.AUTO_READ, false);
ChannelFuture f = b.connect(remoteHost, remotePort);
outboundChannel = f.channel();
outboundChannel.pipeline().addFirst(new StringDecoder()); // this 2 lines
outboundChannel.pipeline().addFirst(new StringEncoder());
f.addListener(new ChannelFutureListener() {
public void operationComplete(ChannelFuture future) {
if (future.isSuccess()) {
// connection complete start to read first data
inboundChannel.read();
} else {
// Close the connection if the connection attempt has
// failed.
inboundChannel.close();
}
}
});
}
So after that, in the channelRead callbacks (for HexDumpProxyFrontendHandler and HexDumpProxyBackendHandler), the messages will be String

Netty 4 read/write in handler multiple times

I'm new in Netty, and I decided to start with 4.0.0, because I thought it should be better, because it's newer. My server application should receive data from gps devices, and the process is like this - at first I'm receiving 2 bytes, which are length of device imei, and then I'm receiving imei with that length, then I should send 0x01 to device if I want to accept data from it. After my answer device sends me gps data with AVL protocol. Now my server is working without Netty, and I want to change it to work with netty.
This is what I have done:
I have created server class like this
public class BusDataReceiverServer {
private final int port;
private final Logger LOG = LoggerFactory.getLogger(BusDataReceiverServer.class);
public BusDataReceiverServer(int port) {
this.port = port;
}
public void run() throws Exception {
LOG.info("running thread");
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try{
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new BusDataReceiverInitializer());
b.bind(port).sync().channel().closeFuture().sync();
}catch (Exception ex){
LOG.info(ex.getMessage());
}
finally {
LOG.info("thread closed");
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
new BusDataReceiverServer(3129).run();
}
}
and created initializer class
public class BusDataReceiverInitializer extends ChannelInitializer<SocketChannel> {
#Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast("imeiDecoder", new ImeiDecoder());
pipeline.addLast("busDataDecoder", new BusDataDecoder());
pipeline.addLast("encoder", new ResponceEncoder());
pipeline.addLast("imeiHandler", new ImeiReceiverServerHandler());
pipeline.addLast("busDataHandler", new BusDataReceiverServerHandler());
}
}
then I have created decoders and encoder and 2 handlers. My imeiDecoder and encoder, and ImeiReceiverServerHandler are working. This is my ImeiReceiverServerHandler
public class ImeiReceiverServerHandler extends ChannelInboundHandlerAdapter {
private final Logger LOG = LoggerFactory.getLogger(ImeiReceiverServerHandler.class);
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageList<Object> msgs) throws Exception {
MessageList<String> imeis = msgs.cast();
String imei = imeis.get(0);
ctx.write(Constants.BUS_DATA_ACCEPT);
ctx.fireMessageReceived(msgs);
}
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
super.channelInactive(ctx); //To change body of overridden methods use File | Settings | File Templates.
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
super.exceptionCaught(ctx, cause); //To change body of overridden methods use File | Settings | File Templates.
}
}
Now, after accepting I don't understand how to continue receive gps data and forward it to handler BusDataReceiverServerHandler.
If anyone could help me with this or could offer me useful documentation, I will be very grateful. Or if it is possible to do this with Netty 3, for this I will also be thankful.

I have not used Netty 4, so I am not sure if my answer will be 100% accurate or the best way to do things in Netty 4, but what you need to do is track the state of your connection / client session in order to know when to forward messages to your second handler.
E.g.
private enum HandlerState { INITIAL, IMEI_RECEIVED; }
private HandlerState state = HandlerState.INITIAL;
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageList<Object> msgs) throws Exception
{
if (state == HandlerState.INITIAL)
{
MessageList<String> imeis = msgs.cast();
String imei = imeis.get(0);
ctx.write(Constants.BUS_DATA_ACCEPT);
state = HandlerState.IMEI_RECEIVED;
} else
{
// Forward message to next handler...
// Not sure exactly how this is done in Netty 4
// Maybe: ctx.fireMessageReceived(msgs);
// Or maybe it is:
// ctx.nextInboundMessageBuffer().add(msg);
// ctx.fireInboundBufferUpdated();
// I believe you could also remove the IMEI handler from the
// pipeline instead of having it keep state, if it is not going to do anything
// further.
}
}
So either track state in the handler, or remove the handler from the pipeline once it has finished if it will not be used further. When tracking state, you can either keep the state in the handler itself (as shown above), or keep the state variables in the context / attribute map (however that is done in netty 4).
The reason to not keep the state in the handler itself would be if you were going to make the handler shareable (one instance used across multiple channels). It is not necessary to do this, but there could be some resource savings if you have a large number of concurrent channels.