Develop Reference

Memory leak due to ByteBuffers in Netty

I created a small Netty server to calculate the factorial of a BigInteger and send the results. The code is as follows.
Factorial.java
public class Factorial {
private int port;
public Factorial(int port) {
this.port = port;
}
public void run(int threadcount) throws Exception {
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup(threadcount);
try {
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast(new FactorialHandler());
}
})
.option(ChannelOption.SO_BACKLOG, 128)
.childOption(ChannelOption.SO_KEEPALIVE, true);
ChannelFuture f = b.bind(port).sync();
f.channel().closeFuture().sync();
} finally {
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
int port = 15000;
new Factorial(port).run(Integer.parseInt(args[0]));
}
}
FactorialHandler.java
public class FactorialHandler extends ChannelInboundHandlerAdapter {
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
BigInteger result = BigInteger.ONE;
String resultString;
for (int i=2000; i>0; i--)
result = result.multiply(BigInteger.valueOf(i));
resultString = result.toString().substring(0, 3)+"\n";
ByteBuf buf = Unpooled.copiedBuffer(resultString.getBytes());
ctx.write(buf);
ctx.flush();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
ctx.close();
}
}
When I ran this I got the following error
Jun 08, 2018 5:28:09 PM io.netty.util.ResourceLeakDetector reportTracedLeak
SEVERE: LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records:
As explained in the given link, I released the ByteBuffer by calling buf.release() in the channelRead method after ctx.flush().
But when I do that, the server starts throwing the following exception
io.netty.util.IllegalReferenceCountException: refCnt: 0, increment: 1
Can someone please tell me how to fix this issue?

The problem is not the outbound ByteBuf. Outbound ByteBufs are always taken care of for you (See OutboundMessages). The problem is the inbound ByteBuf. I'm looking at you, FactorialHandler. It extends ChannelInboundHandlerAdapter. Note this from the JavaDoc:
Be aware that messages are not released after the
channelRead(ChannelHandlerContext, Object) method returns
automatically. If you are looking for a ChannelInboundHandler
implementation that releases the received messages automatically,
please see SimpleChannelInboundHandler.
Your handler has a signature like this:
public void channelRead(ChannelHandlerContext ctx, Object msg)
That msg, (which you don't use, by the way) is actually a ByteBuf, which is exactly what the JavaDoc note above is warning you about. (In the absence of any other ChannelHandlers, messages will always be instances of ByteBuf.)
So your options are:
Use a SimpleChannelInboundHandler which will clean up that reference for you.
At the end of your handler, release the inbound ByteBuf using ReferenceCountUtil.release(java.lang.Object msg).

Its because you dont call msg.release() (msg is an instance of ByteBuf).

Netty 4 Calls being blocked , Simple Telnet Server

I am trying to play around with netty api using Netty Telnet server to check if the true asynchronous behaviour could be observed or not.
Below are the three classes being used
TelnetServer.java
public class TelnetServer {
public static void main(String[] args) throws InterruptedException {
// TODO Auto-generated method stub
EventLoopGroup bossGroup = new NioEventLoopGroup(1);
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.handler(new LoggingHandler(LogLevel.INFO))
.childHandler(new TelnetServerInitializer());
b.bind(8989).sync().channel().closeFuture().sync();
} finally {
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
}
TelnetServerInitializer.java
public class TelnetServerInitializer extends ChannelInitializer<SocketChannel> {
private static final StringDecoder DECODER = new StringDecoder();
private static final StringEncoder ENCODER = new StringEncoder();
private static final TelnetServerHandler SERVER_HANDLER = new TelnetServerHandler();
final EventExecutorGroup executorGroup = new DefaultEventExecutorGroup(2);
public TelnetServerInitializer() {
}
#Override
public void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
// Add the text line codec combination first,
pipeline.addLast(new DelimiterBasedFrameDecoder(8192, Delimiters.lineDelimiter()));
// the encoder and decoder are static as these are sharable
pipeline.addLast(DECODER);
pipeline.addLast(ENCODER);
// and then business logic.
pipeline.addLast(executorGroup,"handler",SERVER_HANDLER);
}
}
TelnetServerHandler.java
/**
* Handles a server-side channel.
*/
#Sharable
public class TelnetServerHandler extends SimpleChannelInboundHandler<String> {
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
// Send greeting for a new connection.
ctx.write("Welcome to " + InetAddress.getLocalHost().getHostName() + "!\r\n");
ctx.write("It is " + new Date() + " now.\r\n");
ctx.flush();
ctx.channel().config().setAutoRead(true);
}
#Override
public void channelRead0(ChannelHandlerContext ctx, String request) throws Exception {
// Generate and write a response.
System.out.println("request = "+ request);
String response;
boolean close = false;
if (request.isEmpty()) {
response = "Please type something.\r\n";
} else if ("bye".equals(request.toLowerCase())) {
response = "Have a good day!\r\n";
close = true;
} else {
response = "Did you say '" + request + "'?\r\n";
}
// We do not need to write a ChannelBuffer here.
// We know the encoder inserted at TelnetPipelineFactory will do the conversion.
ChannelFuture future = ctx.write(response);
Thread.sleep(10000);
// Close the connection after sending 'Have a good day!'
// if the client has sent 'bye'.
if (close) {
future.addListener(ChannelFutureListener.CLOSE);
}
}
#Override
public void channelReadComplete(ChannelHandlerContext ctx) {
ctx.flush();
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
cause.printStackTrace();
ctx.close();
}
}
Now when i connect through telnet client and send commands hello hello hello three times
the request doesn't reach channelRead until first response to channelRead is being done is there any way i can make it asynchronous completely as to receive three hello as soon as they are available on socket.

Netty uses 1 thread max for every incoming read per handler, meaning that the next call to channelRead will only be dispatched after the previous call has been completed. This is required to for correct working of most handlers, including the sending back of messages in the proper order. If the amount of computation is really complex, another solution is using a custom thread pool for the messages.
If the other operation is instead a other kind of connection, you should use that as a async connection too. You can only get asynchronous if every part does this correctly.

Netty encoder not executed

I made a server with Netty but I'm having a problem. The encoder that i created is not being executed.
My pipeline on the server:
bootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast("encoder", new PacketEncoder());
ch.pipeline().addLast("decoder", new PacketDecoder());
ch.pipeline().addLast(new LoginServerChannel());
}
});
My encoder:
public class PacketEncoder extends MessageToByteEncoder<Packet> {
#Override
protected void encode(ChannelHandlerContext channelHandlerContext, Packet packet, ByteBuf byteBuf) throws Exception {
System.out.println("Encoding");
}
My decoder:
public class PacketDecoder extends ReplayingDecoder<DecoderState> {
#Override
protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
...
}
My channel handler:
public class LoginServerChannel extends SimpleChannelInboundHandler<Packet> {
#Override
protected void channelRead0(ChannelHandlerContext channelHandlerContext, Packet packet) throws Exception {
channelHandlerContext.writeAndFlush(packet.encode());
System.out.println("Sending");
}
The decoder is called and then the channel handler but not the encoder. I have tried to change the order in the pipeline but same problem also try to used fireChannelRead(...) and same problem.
Thanks

Your encoder extends MessageToByteEncoder<Packet> so it is called when a Packet is received in input to encode it.
In your logic handler, you do
channelHandlerContext.writeAndFlush( packet.encode() );
I suppose that encode() returns a byte array, so your encoder ignore it and do nothing.
You probably should do something like that:
channelHandlerContext.writeAndFlush( packet );

Netty Encoder Not Being Called

Using Netty 4.0.27 & Java 1.8.0_20
So I am attempting to learn how Netty works by building a simple chat server (the typical networking tutorial program, I guess?). Designing my own simple protocol, called ARC (Andrew's Relay Chat)... so that's why you see ARC in the code a lot. K, so here's the issue.
So here I start the server and register the various handlers...
public void start()
{
System.out.println("Registering handlers...");
ArcServerInboundHandler inboundHandler = new ArcServerInboundHandler(this);
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try
{
ServerBootstrap bootstrap = new ServerBootstrap();
bootstrap.group(bossGroup, workerGroup).channel(NioServerSocketChannel.class).childHandler(new ChannelInitializer<SocketChannel>()
{
#Override
public void initChannel(SocketChannel ch) throws Exception
{
ch.pipeline().addLast(new ArcDecoder(), inboundHandler);
ch.pipeline().addLast(new ArcEncoder());
}
}).option(ChannelOption.SO_BACKLOG, 128).childOption(ChannelOption.SO_KEEPALIVE, true);
try
{
System.out.println("Starting Arc Server on port " + port);
ChannelFuture f = bootstrap.bind(port).sync();
f.channel().closeFuture().sync();
}
catch(InterruptedException e)
{
e.printStackTrace();
}
}
finally
{
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
}
}
My "inboundHandler" does get called when the user connects.
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception
{
System.out.println("CLIENT CONNECTED"); // THIS PRINTS, REACHES THIS POINT
ArcPacket packet = new ArcPacket();
packet.setArc("PUBLIC_KEY");
packet.setField("KEY", Crypto.bytesToHex(server.getRsaKeys().getPublic().getEncoded()));
ctx.writeAndFlush(packet);
}
This is my encoder, which does not seem to get called at all...
public class ArcEncoder extends MessageToByteEncoder<ArcPacket>
{
#Override
protected void encode(ChannelHandlerContext ctx, ArcPacket msg, ByteBuf out) throws Exception
{
System.out.println("ENCODE"); // NEVER GETS HERE
String message = ArcPacketFactory.encode(msg);
byte[] data = message.getBytes("UTF-8");
out.writeBytes(data);
System.out.println("WROTE");
}
#Override
public boolean acceptOutboundMessage(Object msg) throws Exception
{
System.out.println("ACCEPT OUTBOUND MESSAGE"); // NEVER GETS HERE
return msg instanceof ArcPacket;
}
}
So,
The code that calls ctx.writeAndFlush(packet); is run, but it doesn't seem to invoke the encoder at any point. Am I missing something obvious? Perhaps I'm adding the encoder incorrectly? Though it looks right when I compare it to other examples I've seen.
Thanks for any help.

Your encoder (ArcEncoder) is placed after your inbound handler. It means, the ctx.*() method calls will never be evaluated by the encoder. To fix your problem, you have to move the ArcEncoder before the inbound handler:
ch.pipeline().addLast(new ArcDecoder(), new ArcEncoder(), inboundHandler);
For more information about the event evaluation order, please read the API documentation of ChannelPipeline.

I think the problem is that you're using the ChannelHandlerContext to write to the Channel. What this does is to insert the message in the pipeline at the point of your handler, going outbound. But since your decoder is added before your encoder in the pipeline this means that anything you write using the decoder context will be inserted after the encoder in the pipeline.
The correct way to do it to ensure that the encoder is called is to call:
ctx.channel.writeAndFlush()

How could a Netty server send data to different clients?

I wanna make something like a simple chat.
Server have to add new clients in a list of clients, and when one client sends message to the server, server have to resend this message to the other clients.
I know, how to read message from client, but I don't know, how to send message from server to the client. And i don't sure where should be list of clients, but guess that in handler class.
Here is my main class which initializes the server class
package firstPackage;
public class main {
public static void main(String[] args) throws Exception
{
Server server = new Server(9050);
server.run();
}
}
Here is the Server class
package firstPackage;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.channel.socket.SocketChannel;
public class Server {
private int port;
public Server(int port)
{
this.port=port;
}
public void run() throws Exception
{
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try{
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup,workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception{
ch.pipeline().addLast(new DiscardServerHandler());
}
})
.option(ChannelOption.SO_BACKLOG,128)
.childOption(ChannelOption.SO_KEEPALIVE, true);
ChannelFuture f = b.bind(port).sync();
f.channel().closeFuture().sync();
}
finally {
workerGroup.shutdownGracefully();
bossGroup.shutdownGracefully();
}
}
}
And here the Handler class
package firstPackage;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.util.ReferenceCountUtil;
public class DiscardServerHandler extends ChannelInboundHandlerAdapter {
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
ByteBuf in = (ByteBuf) msg;
try {
while (in.isReadable()) {
System.out.print((char) in.readByte());
System.out.flush();
}
System.out.println();
ctx.writeAndFlush("hey"); // вот здесь я думал, что сообщение будет отправлятся клиенту, от которого я получил сообщение, но не отправляется
} finally {
ReferenceCountUtil.release(msg);
}
}
#Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
System.out.println("channel is active");
}
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
System.out.println("channel is invactive");
}
#Override
public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
System.out.println("handler added");
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
cause.printStackTrace();
ctx.close();
}
}
Actually now I have not list of clients, because i even don't know what type of object this list must contain, in C# it's was Socket object, so what in Netty?

I'm betting this example should help you out (it's a YouTube video, so spare the comments about broken links please). In particular, it uses a ChannelGroup to solve the problem of which you ask. And, yes, it goes in the handler on the server side.
Edit:
Notice too that in the example the ChannelGroup is static. I might argue it would be better to not use a static member and inject a ChannelGroup into the handler from the server class, but the simplicity of the static member may be preferable if you just want to get something working quickly.