I have created a fairly straight forward server using Netty 4. I have been able to scale it up to handle several thousand connections and it never climbs above ~40 threads.
In order to test it out, I have also created a test client that creates thousands of connections. Unfortunately this creates as many threads as it makes connections. I was hoping to minimize threads for the clients. I have looked at many posts for this. Many examples show single connection setup. This and this say to share NioEventLoopGroup across clients, which I do. I'm getting a limited number of nioEventLoopGroup, but getting a thread per connection elsewhere. I am not purposely creating threads in the pipeline and don't see what could be.
Here is a snippet from the setup of my client code. It seems that it should maintain a fixed thread count based on what I've researched so far. Is there something I'm missing that I should be doing to prevent a thread per client connection?
Main
final EventLoopGroup group = new NioEventLoopGroup();
for (int i=0; i<100; i++)){
MockClient client = new MockClient(i, group);
client.connect();
}
MockClient
public class MockClient implements Runnable {
private final EventLoopGroup group;
private int identity;
public MockClient(int identity, final EventLoopGroup group) {
this.identity = identity;
this.group = group;
}
#Override
public void run() {
try {
connect();
} catch (Exception e) {}
}
public void connect() throws Exception{
Bootstrap b = new Bootstrap();
b.group(group)
.channel(NioSocketChannel.class)
.handler(new MockClientInitializer(identity, this));
final Runnable that = this;
// Start the connection attempt
b.connect(config.getHost(), config.getPort()).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (future.isSuccess()) {
Channel ch = future.sync().channel();
} else {
//if the server is down, try again in a few seconds
future.channel().eventLoop().schedule(that, 15, TimeUnit.SECONDS);
}
}
});
}
}
As has happened to me many times before, explaining the problem in detail made me think about it more and I came across the issue. I wanted to provide it here should anyone else come across the same issue with creating thousands of Netty clients.
I have one path in my pipeline that will create a timeout task to simulate a client connection rebooting. It turns out it was this timer task that was creating the extra threads per connection whenever it received a 'reboot' signal from the server (which happens every so often) up until there was a thread per connection.
Handler
private final HashedWheelTimer timer;
#Override
protected void channelRead0(ChannelHandlerContext ctx, Packet msg) throws Exception {
Packet packet = reboot();
ChannelFutureListener closeHandler = new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
RebootTimeoutTask timeoutTask = new RebootTimeoutTask(identity, client);
timer.newTimeout(timeoutTask, SECONDS_FOR_REBOOT, TimeUnit.SECONDS);
}
};
ctx.writeAndFlush(packet).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (future.isSuccess()) {
future.channel().close().addListener(closeHandler);
} else {
future.channel().close();
}
}
});
}
Timeout Task
public class RebootTimeoutTask implements TimerTask {
public RebootTimeoutTask(...) {...}
#Override
public void run(Timeout timeout) throws Exception {
client.connect(identity);
}
}
Related
I need to create a RMI service which can notify events to clients.
Each client register itself on the server, the client can emit an event and the server will broadcast it to all other clients.
The program works, but, the client reference on the server is never garbage collected, an the thread which the server uses to check if the client reference will never terminate.
So each time a client connects to the server, a new thread is created and never terminated.
The Notifier class can register and unregister a listener.
The broadcast method call each registered listener and send the message back.
public class Notifier extends UnicastRemoteObject implements INotifier{
private List<IListener> listeners = Collections.synchronizedList(new ArrayList());
public Notifier() throws RemoteException {
super();
}
#Override
public void register(IListener listener) throws RemoteException{
listeners.add(listener);
}
#Override
public void unregister(IListener listener) throws RemoteException{
boolean remove = listeners.remove(listener);
if(remove) {
System.out.println(listener+" removed");
} else {
System.out.println(listener+" NOT removed");
}
}
#Override
public void broadcast(String msg) throws RemoteException {
for (IListener listener : listeners) {
try {
listener.onMessage(msg);
} catch (RemoteException e) {
e.printStackTrace();
}
}
}
}
The listener is just printing each received message.
public class ListenerImpl extends UnicastRemoteObject implements IListener {
public ListenerImpl() throws RemoteException {
super();
}
#Override
public void onMessage(String msg) throws RemoteException{
System.out.println("Received: "+msg);
}
}
The RunListener client subscribes a listener wait few seconds to receive a message and then terminates.
public class RunListener {
public static void main(String[] args) throws Exception {
Registry registry = LocateRegistry.getRegistry();
INotifier notifier = (INotifier) registry.lookup("Notifier");
ListenerImpl listener = new ListenerImpl();
notifier.register(listener);
Thread.sleep(6000);
notifier.unregister(listener);
UnicastRemoteObject.unexportObject(listener, true);
}
}
The RunNotifier just publish the service and periodically sends a message.
public class RunNotifier {
static AtomicInteger counter = new AtomicInteger();
public static void main(String[] args) throws RemoteException, AlreadyBoundException, NotBoundException {
Registry registry = LocateRegistry.createRegistry(1099);
INotifier notifier = new Notifier();
registry.bind("Notifier", notifier);
ScheduledExecutorService executor = Executors.newScheduledThreadPool(1);
executor.scheduleAtFixedRate(new Runnable() {
#Override
public void run() {
try {
int n = counter.incrementAndGet();
System.out.println("Broadcasting "+n);
notifier.broadcast("Hello ("+n+ ")");
} catch (RemoteException e) {
e.printStackTrace();
}
}
},5 , 5, TimeUnit.SECONDS);
try {
System.in.read();
} catch (IOException e) {
}
executor.shutdown();
registry.unbind("Notifier");
UnicastRemoteObject.unexportObject(notifier, true);
}
}
I've seen many Q&A on stack overflow about RMI, but none addresses this kind of problem.
I guess I'm doing some very big mistake, but I can't spot it.
As you can see in the picture, a new RMI RenewClean thread is created for each incoming connection, and this thread will never terminate.
Once the client disconnects, and terminates, the RenewClean thread will silently swallow all ConnectionException thrown and will keep polling a client which will never reply.
As a side note, I even tried to keep just weak reference of the IListener in the Notifier class, and still the results are the same.
This may not be very helpful if you are stuck on JDK1.8, but when I test on JDK17 the multiple rmi server threads created for each incoming client RMI RenewClean-[IPADDRESS:PORT] are cleaned up on the server, and not showing "will never terminate" behaviour you may have observed on JDK1.8. It may be a JDK1.8 issue, or simply that you are not waiting long enough for the threads to end.
For quicker cleanup, try adjusting the system property for client thread garbage collection setting from the default (3600000 = 1 hour):
java -Dsun.rmi.dgc.client.gcInterval=3600000 ...
On my server I added this in one of the API callbacks:
Function<Thread,String> toString = t -> t.getName()+(t.isDaemon() ? " DAEMON" :"");
Set<Thread> threads = Thread.getAllStackTraces().keySet();
System.out.println("-".repeat(40)+" Threads x "+threads.size());
threads.stream().map(toString).forEach(System.out::println);
After RMI server startup it printed names of threads and no instances of "RMI RenewClean":
---------------------------------------- Threads x 12
After connecting many times from a client, the server reported corresponding instances of "RMI RenewClean":
---------------------------------------- Threads x 81
Leaving the RMI server for a while, these gradually shrank back - not to 12 threads -, but low enough to suggest that RMI thread handling is not filling up with many unnecessary daemon threads:
---------------------------------------- Threads x 20
After about an hour all the remaining "RMI RenewClean" were removed - probably due to housekeeping performed at the interval defined by the VM setting sun.rmi.dgc.client.gcInterval=3600000:
---------------------------------------- Threads x 13
Note also that RMI server shutdown is instant at any point - the "RMI RenewClean" daemon threads do not hold up rmi server shutdown.
I have a situation like: My Netty Server will be getting data from a Client at a blazing speed. I think the client is using somewhat PUSH mechanism for that speed. I don't know what exactly PUSH - POP mechanism is, but I do feel that the Client is using some mechanism for sending data at a very high speed.Now my requirement is, I wrote a simple TCP Netty server that receives data from the client and just adds to the BlockingQueue implemented using ArrayBlockingQueue. Now , as Netty is event based, the time taken to accept the data and store it in a queue is some what more , this is raising an exception at the client side that the Netty server is not running.But my server is running perfectly, but the time to accept single data and store it in the queue is more. How can I prevent this? Is there any fastest queue for this situation? I nam using BlockingQueue as another thread will take data from the queue and process it. So I need a synchronized queue. How can I improve the performance of the Server or is there any way to insert data at a very high speed? All I care about is latency. The latency needs to be as low as possible.
My Server code:
public class Server implements Runnable {
private final int port;
static String message;
Channel channel;
ChannelFuture channelFuture;
int rcvBuf, sndBuf, lowWaterMark, highWaterMark;
public Server(int port) {
this.port = port;
rcvBuf = 2048;
sndBuf = 2048;
lowWaterMark = 1024;
highWaterMark = 2048;
}
#Override
public void run() {
try {
startServer();
} catch (Exception ex) {
System.err.println("Error in Server : "+ex);
Logger.error(ex.getMessage());
}
}
public void startServer() {
// System.out.println("8888 Server started");
EventLoopGroup group = new NioEventLoopGroup();
try {
ServerBootstrap b = new ServerBootstrap();
b.group(group)
.channel(NioServerSocketChannel.class)
.localAddress(new InetSocketAddress(port))
.childOption(ChannelOption.SO_RCVBUF, rcvBuf * 2048)
.childOption(ChannelOption.SO_SNDBUF, sndBuf * 2048)
.childOption(ChannelOption.WRITE_BUFFER_WATER_MARK, new WriteBufferWaterMark(lowWaterMark * 2048, highWaterMark * 2048))
.childOption(ChannelOption.TCP_NODELAY, true)
.childHandler(new ChannelInitializer<SocketChannel>() {
#Override
public void initChannel(SocketChannel ch) throws Exception {
channel = ch;
System.err.println("OMS connected : " + ch.localAddress());
ch.pipeline().addLast(new ReceiveFromOMSDecoder());
}
});
channelFuture = b.bind(port).sync();
this.channel = channelFuture.channel();
channelFuture.channel().closeFuture().sync();
} catch (InterruptedException ex) {
System.err.println("Exception raised in SendToOMS class"+ex);
} finally {
group.shutdownGracefully();
}
}
}
My ServerHandler code:
#Sharable
public class ReceiveFromOMSDecoder extends MessageToMessageDecoder<ByteBuf> {
private Charset charset;
public ReceiveFromOMSDecoder() {
this(Charset.defaultCharset());
}
/**
* Creates a new instance with the specified character set.
*/
public ReceiveFromOMSDecoder(Charset charset) {
if (charset == null) {
throw new NullPointerException("charset");
}
this.charset = charset;
}
#Override
protected void decode(ChannelHandlerContext ctx, ByteBuf msg, List<Object> out) throws Exception {
String buffer = msg.toString(charset);
if(buffer!=null){
Server.sq.insertStringIntoSendingQueue(buffer); //inserting into queue
}
else{
Logger.error("Null string received"+buffer);
}
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
// Logger.error(cause.getMessage());
System.err.println(cause);
}
}
Three quickies:
Doesn't look like you're sending a response. You probably should.
Don't block the IO thread. Use an EventExecutorGroup to dispatch the handling of the incoming payload. i.e. something like ChannelPipeline.addLast(EventExecutorGroup group, String name, ChannelHandler handler).
Just don't block in general. Ditch your ArrayBlockingQueue and take a look at JCTools or some other implementation to find a non-blocking analog.
I have a scenario where I am establishing TCP connection using netty NIO, suppose server went down than how can I automatically connect to server when it comes up again ?
Or Is there any way to attach availability listener on server ?
You can have a DisconnectionHandler, as the first thing on your client pipeline, that reacts on channelInactive by immediately trying to reconnect or scheduling a reconnection task.
For example,
public class DisconnectionHandler extends ChannelInboundHandlerAdapter {
#Override
public void channelInactive(final ChannelHandlerContext ctx) throws Exception {
Channel channel = ctx.channel();
/* If shutdown is on going, ignore */
if (channel.eventLoop().isShuttingDown()) return;
ReconnectionTask reconnect = new ReconnectionTask(channel);
reconnect.run();
}
}
The ReconnectionTask would be something like this:
public class ReconnectionTask implements Runnable, ChannelFutureListener {
Channel previous;
public ReconnectionTask(Channel c) {
this.previous = c;
}
#Override
public void run() {
Bootstrap b = createBootstrap();
b.remoteAddress(previous.remoteAddress())
.connect()
.addListener(this);
}
#Override
public void operationComplete(ChannelFuture future) throws Exception {
if (!future.isSuccess()) {
// Will try to connect again in 100 ms.
// Here you should probably use exponential backoff or some sort of randomization to define the retry period.
previous.eventLoop()
.schedule(this, 100, MILLISECONDS);
return;
}
// Do something else when success if needed.
}
}
Check here for an example of Exponential Backoff library.
I'm trying to implement a UDP server with Netty. The idea is to bind only once (therefore creating only one Channel). This Channel is initialized with only one handler that dispatches processing of incoming datagrams among multiple threads via an ExecutorService.
#Configuration
public class SpringConfig {
#Autowired
private Dispatcher dispatcher;
private String host;
private int port;
#Bean
public Bootstrap bootstrap() throws Exception {
Bootstrap bootstrap = new Bootstrap()
.group(new NioEventLoopGroup(1))
.channel(NioDatagramChannel.class)
.option(ChannelOption.ALLOCATOR, PooledByteBufAllocator.DEFAULT)
.handler(dispatcher);
ChannelFuture future = bootstrap.bind(host, port).await();
if(!future.isSuccess())
throw new Exception(String.format("Fail to bind on [host = %s , port = %d].", host, port), future.cause());
return bootstrap;
}
}
#Component
#Sharable
public class Dispatcher extends ChannelInboundHandlerAdapter implements InitializingBean {
private int workerThreads;
private ExecutorService executorService;
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
DatagramPacket packet = (DatagramPacket) msg;
final Channel channel = ctx.channel();
executorService.execute(new Runnable() {
#Override
public void run() {
//Process the packet and produce a response packet (below)
DatagramPacket responsePacket = ...;
ChannelFuture future;
try {
future = channel.writeAndFlush(responsePacket).await();
} catch (InterruptedException e) {
return;
}
if(!future.isSuccess())
log.warn("Failed to write response packet.");
}
});
}
#Override
public void afterPropertiesSet() throws Exception {
executorService = Executors.newFixedThreadPool(workerThreads);
}
}
I have the following questions:
Should the DatagramPacket received by the channelRead method of the Dispatcher class be duplicated before being used by the worker thread? I wonder if this packet is destroyed after the channelRead method returns, even if a reference is kept by the worker thread.
Is it safe to share the Channel among all the worker threads and let them call writeAndFlush concurrently?
Thanks!
Nope. If you need the object to live longer you either turn it into something else or use ReferenceCountUtil.retain(datagram) and then ReferenceCountUtil.release(datagram) once you're done with it. You also shouldn't be doing await() at the executor service as well, you should register a handler for whatever happens.
Yes, channel objects are thread safe and they can be called from many different threads.
I am using Netty 3.2.7. I am trying to write functionality in my client such that if no messages are written after a certain amount of time (say, 30 seconds), a "keep-alive" message is sent to the server.
After some digging, I found that WriteTimeoutHandler should enable me to do this. I found this explanation here: https://issues.jboss.org/browse/NETTY-79.
The example given in the Netty documentation is:
public ChannelPipeline getPipeline() {
// An example configuration that implements 30-second write timeout:
return Channels.pipeline(
new WriteTimeoutHandler(timer, 30), // timer must be shared.
new MyHandler());
}
In my test client, I have done just this. In MyHandler, I also overrided the exceptionCaught() method:
public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) {
if (e.getCause() instanceof WriteTimeoutException) {
log.info("Client sending keep alive!");
ChannelBuffer keepAlive = ChannelBuffers.buffer(KEEP_ALIVE_MSG_STR.length());
keepAlive.writeBytes(KEEP_ALIVE_MSG_STR.getBytes());
Channels.write(ctx, Channels.future(e.getChannel()), keepAlive);
}
}
No matter what duration the client does not write anything to the channel, the exceptionCaught() method I have overridden is never called.
Looking at the source of WriteTimeoutHandler, its writeRequested() implementation is:
public void writeRequested(ChannelHandlerContext ctx, MessageEvent e)
throws Exception {
long timeoutMillis = getTimeoutMillis(e);
if (timeoutMillis > 0) {
// Set timeout only when getTimeoutMillis() returns a positive value.
ChannelFuture future = e.getFuture();
final Timeout timeout = timer.newTimeout(
new WriteTimeoutTask(ctx, future),
timeoutMillis, TimeUnit.MILLISECONDS);
future.addListener(new TimeoutCanceller(timeout));
}
super.writeRequested(ctx, e);
}
Here, it seems that this implementation says, "When a write is requested, make a new timeout. When the write succeeds, cancel the timeout."
Using a debugger, it does seem that this is what is happening. As soon as the write completes, the timeout is cancelled. This is not the behavior I want. The behavior I want is: "If the client has not written any information to the channel for 30 seconds, throw a WriteTimeoutException."
So, is this not what WriteTimeoutHandler is for? This is how I interpreted it from what I've read online, but the implementation does not seem to work this way. Am I using it wrong? Should I use something else? In our Mina version of the same client I am trying to rewrite, I see that the sessionIdle() method is overridden to achieve the behavior I want, but this method is not available in Netty.
For Netty 4.0 and newer, you should extend ChannelDuplexHandler like in example from IdleStateHandler documentation :
// An example that sends a ping message when there is no outbound traffic
// for 30 seconds. The connection is closed when there is no inbound traffic
// for 60 seconds.
public class MyChannelInitializer extends ChannelInitializer<Channel> {
#Override
public void initChannel(Channel channel) {
channel.pipeline().addLast("idleStateHandler", new IdleStateHandler(60, 30, 0));
channel.pipeline().addLast("myHandler", new MyHandler());
}
}
// Handler should handle the IdleStateEvent triggered by IdleStateHandler.
public class MyHandler extends ChannelDuplexHandler {
#Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
if (evt instanceof IdleStateEvent) {
IdleStateEvent e = (IdleStateEvent) evt;
if (e.state() == IdleState.READER_IDLE) {
ctx.close();
} else if (e.state() == IdleState.WRITER_IDLE) {
ctx.writeAndFlush(new PingMessage());
}
}
}
}
I would suggest to add the IdleStateHandler and then add your custom implementation of IdleStateAwareUpstreamHandler which can react on the idle state. This works out very well for me on many different projects.
The javadocs list the following example, that you could use as the base of your implementation:
public class MyPipelineFactory implements ChannelPipelineFactory {
private final Timer timer;
private final ChannelHandler idleStateHandler;
public MyPipelineFactory(Timer timer) {
this.timer = timer;
this.idleStateHandler = new IdleStateHandler(timer, 60, 30, 0);
// timer must be shared.
}
public ChannelPipeline getPipeline() {
return Channels.pipeline(
idleStateHandler,
new MyHandler());
}
}
// Handler should handle the IdleStateEvent triggered by IdleStateHandler.
public class MyHandler extends IdleStateAwareChannelHandler {
#Override
public void channelIdle(ChannelHandlerContext ctx, IdleStateEvent e) {
if (e.getState() == IdleState.READER_IDLE) {
e.getChannel().close();
} else if (e.getState() == IdleState.WRITER_IDLE) {
e.getChannel().write(new PingMessage());
}
}
}
ServerBootstrap bootstrap = ...;
Timer timer = new HashedWheelTimer();
...
bootstrap.setPipelineFactory(new MyPipelineFactory(timer));
...