Develop Reference

Jersey SSE - eventOutput.write throws nullpointer after first message is sent

I have implemented a Restful web interface using Jersey for sending messages received from an internal JMS publisher to external clients via HTTP. I have managed to get a test message out to a Java client, but the Thread throws a null pointer exception before completing the write() execution, closing the connection and preventing further communication.
Here is my resource class:
#GET
#Path("/stream_data")
#Produces(SseFeature.SERVER_SENT_EVENTS)
public EventOutput getServerSentEvents(#Context ServletContext context){
final EventOutput eventOutput = new EventOutput();
new Thread( new ObserverThread(eventOutput, (MService) context.getAttribute("instance")) ).start();
return eventOutput;
}
And here is my thread's run method:
public class ObserverThread implements Observer, Runnable {
//constructor sets eventOutput & mService objects
//mService notifyObservers() called when JMS message received
//text added to Thread's message queue to await sending to client
public void run() {
try {
String message = "{'symbol':'test','entryType'='0','price'='test'}";
Thread.sleep(1000);
OutboundEvent.Builder builder = new OutboundEvent.Builder();
builder.mediaType(MediaType.APPLICATION_JSON_TYPE);
builder.data(String.class, message);
OutboundEvent event = builder.build();
eventOutput.write(event);
System.out.println(">>>>>>SSE CLIENT HAS BEEN REGISTERED!");
mService.addObserver(this);
while(!eventOutput.isClosed()){
if(!updatesQ.isEmpty()){
pushUpdate(updatesQ.dequeue());
}
}
System.out.println("<<<<<<<SSE CLIENT HAS BEEN DEREGISTERED!");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Here is my client code:
Client client = ClientBuilder.newBuilder().register(SseFeature.class).build();
WebTarget target = client.target(url);
EventInput eventInput = target.request().get(EventInput.class);
try {
while (!eventInput.isClosed()) {
eventInput.setChunkType(MediaType.WILDCARD_TYPE);
final InboundEvent inboundEvent = eventInput.read();
if (inboundEvent != null) {
String theString = inboundEvent.readData();
System.out.println(theString + "\n");
}
}
} catch (Exception e) {
e.printStackTrace();
}
I am getting the "{'symbol':'test','entryType'='0','price'='test'}" test message printed to the client console, but the server then prints a NullPointerException before it can print the ">>>>SSE Client registered" message. This closes the connection so the client exits the while loop and stops listening for updates.
I converted the project to a webapp 3.0 version facet in order to add an async-supported tag to the web.xml but i am receiving the same null pointer error. I am inclined to think that it is caused by the servlet ending the Request/Response objects once the first message is returned, evidence is shown in the stack trace:
Exception in thread "Thread-20" java.lang.NullPointerException
at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:741)
at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:434)
at org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:299)
at org.apache.coyote.http11.Http11Processor.action(Http11Processor.java:981)
at org.apache.coyote.Response.action(Response.java:183)
at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:314)
at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:288)
at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.java:98)
at org.glassfish.jersey.message.internal.CommittingOutputStream.flush(CommittingOutputStream.java:292)
at org.glassfish.jersey.server.ChunkedOutput$1.call(ChunkedOutput.java:241)
at org.glassfish.jersey.server.ChunkedOutput$1.call(ChunkedOutput.java:192)
at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
at org.glassfish.jersey.internal.Errors.process(Errors.java:242)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:345)
at org.glassfish.jersey.server.ChunkedOutput.flushQueue(ChunkedOutput.java:192)
at org.glassfish.jersey.server.ChunkedOutput.write(ChunkedOutput.java:182)
at com.bpc.services.service.ObserverThread.run(MarketObserverThread.java:32)
at java.lang.Thread.run(Thread.java:745)
<<<<<<<SSE CLIENT HAS BEEN DEREGISTERED!
I have attempted to test an sse broadcaster as well. In this case I am not seeing any exceptions thrown, but the connection is closed once the first message has been received, leading me to believe it is something in the servlet forcing the connection to close. Can anyone advise me on how to debug this on the server-side?

I had a similar issue from what seems to be a long standing bug in Jersey's #Context injection for ExecutorService instances. In their current implementation of Sse (version 2.27),
class JerseySse implements Sse {
#Context
private ExecutorService executorService;
#Override
public OutboundSseEvent.Builder newEventBuilder() {
return new OutboundEvent.Builder();
}
#Override
public SseBroadcaster newBroadcaster() {
return new JerseySseBroadcaster(executorService);
}
}
the executorService field is never initialized, so the JerseySseBroadcaster raises a NullPointerException in my case. I worked around the bug by explicitly triggering the injection.
If you're using HK2 for CDI (Jersey's default), a rough sketch of a solution to the question above could look similar to the following:
#Singleton
#Path("...")
public class JmsPublisher {
private Sse sse;
private SseBroadcaster broadcaster;
private final ExecutorService executor;
private final BlockingQueue<String> jmsMessageQueue;
...
#Context
public void setSse(Sse sse, ServiceLocator locator) {
locator.inject(sse); // Inject sse.executorService
this.sse = sse;
this.broadcaster = sse.newBroadcaster();
}
...
#GET
#Path("/stream_data")
#Produces(MediaType.SERVER_SENT_EVENTS)
public void register(SseEventSink eventSink) {
broadcaster.register(eventSink);
}
...
#PostConstruct
private void postConstruct() {
executor.submit(() -> {
try {
while(true) {
String message = jmsMessageQueue.take();
broadcaster.broadcast(sse.newEventBuilder()
.mediaType(MediaType.APPLICATION_JSON_TYPE)
.data(String.class, message)
.build());
}
} catch(InterruptedException e) {
Thread.currentThread().interrupt();
}
});
}
#PreDestroy
private void preDestroy() {
executor.shutdownNow();
}
}

Netty 4 read/write in handler multiple times

I'm new in Netty, and I decided to start with 4.0.0, because I thought it should be better, because it's newer. My server application should receive data from gps devices, and the process is like this - at first I'm receiving 2 bytes, which are length of device imei, and then I'm receiving imei with that length, then I should send 0x01 to device if I want to accept data from it. After my answer device sends me gps data with AVL protocol. Now my server is working without Netty, and I want to change it to work with netty.
This is what I have done:
I have created server class like this
public class BusDataReceiverServer {
private final int port;
private final Logger LOG = LoggerFactory.getLogger(BusDataReceiverServer.class);
public BusDataReceiverServer(int port) {
this.port = port;
}
public void run() throws Exception {
LOG.info("running thread");
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try{
ServerBootstrap b = new ServerBootstrap();
b.group(bossGroup, workerGroup)
.channel(NioServerSocketChannel.class)
.childHandler(new BusDataReceiverInitializer());
b.bind(port).sync().channel().closeFuture().sync();
}catch (Exception ex){
LOG.info(ex.getMessage());
}
finally {
LOG.info("thread closed");
bossGroup.shutdownGracefully();
workerGroup.shutdownGracefully();
}
}
public static void main(String[] args) throws Exception {
new BusDataReceiverServer(3129).run();
}
}
and created initializer class
public class BusDataReceiverInitializer extends ChannelInitializer<SocketChannel> {
#Override
protected void initChannel(SocketChannel socketChannel) throws Exception {
ChannelPipeline pipeline = socketChannel.pipeline();
pipeline.addLast("imeiDecoder", new ImeiDecoder());
pipeline.addLast("busDataDecoder", new BusDataDecoder());
pipeline.addLast("encoder", new ResponceEncoder());
pipeline.addLast("imeiHandler", new ImeiReceiverServerHandler());
pipeline.addLast("busDataHandler", new BusDataReceiverServerHandler());
}
}
then I have created decoders and encoder and 2 handlers. My imeiDecoder and encoder, and ImeiReceiverServerHandler are working. This is my ImeiReceiverServerHandler
public class ImeiReceiverServerHandler extends ChannelInboundHandlerAdapter {
private final Logger LOG = LoggerFactory.getLogger(ImeiReceiverServerHandler.class);
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageList<Object> msgs) throws Exception {
MessageList<String> imeis = msgs.cast();
String imei = imeis.get(0);
ctx.write(Constants.BUS_DATA_ACCEPT);
ctx.fireMessageReceived(msgs);
}
#Override
public void channelInactive(ChannelHandlerContext ctx) throws Exception {
super.channelInactive(ctx); //To change body of overridden methods use File | Settings | File Templates.
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
super.exceptionCaught(ctx, cause); //To change body of overridden methods use File | Settings | File Templates.
}
}
Now, after accepting I don't understand how to continue receive gps data and forward it to handler BusDataReceiverServerHandler.
If anyone could help me with this or could offer me useful documentation, I will be very grateful. Or if it is possible to do this with Netty 3, for this I will also be thankful.

I have not used Netty 4, so I am not sure if my answer will be 100% accurate or the best way to do things in Netty 4, but what you need to do is track the state of your connection / client session in order to know when to forward messages to your second handler.
E.g.
private enum HandlerState { INITIAL, IMEI_RECEIVED; }
private HandlerState state = HandlerState.INITIAL;
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageList<Object> msgs) throws Exception
{
if (state == HandlerState.INITIAL)
{
MessageList<String> imeis = msgs.cast();
String imei = imeis.get(0);
ctx.write(Constants.BUS_DATA_ACCEPT);
state = HandlerState.IMEI_RECEIVED;
} else
{
// Forward message to next handler...
// Not sure exactly how this is done in Netty 4
// Maybe: ctx.fireMessageReceived(msgs);
// Or maybe it is:
// ctx.nextInboundMessageBuffer().add(msg);
// ctx.fireInboundBufferUpdated();
// I believe you could also remove the IMEI handler from the
// pipeline instead of having it keep state, if it is not going to do anything
// further.
}
}
So either track state in the handler, or remove the handler from the pipeline once it has finished if it will not be used further. When tracking state, you can either keep the state in the handler itself (as shown above), or keep the state variables in the context / attribute map (however that is done in netty 4).
The reason to not keep the state in the handler itself would be if you were going to make the handler shareable (one instance used across multiple channels). It is not necessary to do this, but there could be some resource savings if you have a large number of concurrent channels.

Netty 4.0 - StringDecoder and ChannelInboundMessageHandlerAdapter<String> not working

I'm using netty 4.0.0-CR3, following the example on server-side:
https://github.com/netty/netty/blob/master/example/src/main/java/io/netty/example/telnet/TelnetServerPipelineFactory.java
I've constructed my pipeline as follows:
private static final StringDecoder DECODER = new StringDecoder(CharsetUtil.UTF_8);
#Override
public void initChannel(SocketChannel ch) throws Exception {
ChannelPipeline pipeline = ch.pipeline();
pipeline.addLast("decoder", DECODER);
// and then business logic
pipeline.addLast("serverHandler", new ServerHandler());
}
And handler:
public class ServerHandler extends ChannelInboundMessageHandlerAdapter<String> {
private static final Logger LOGGER = LoggerFactory.getLogger(ServerHandler.class);
public void messageReceived(ChannelHandlerContext ctx, String request)
throws Exception {
// Displays the message
LOGGER.info("Received: " + request);
}
#Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause)
throws Exception {
LOGGER.error("Unexpected exception from downstream.", cause);
ctx.close();
}
}
I created a simple C# client that encodes String into bytes, and send to the server. However, I don't see EITHER StringDecoder's decode() OR handler's messageReceived() called.
I then removed StringDecoder() in pipeline, and changed the handler to be:
public class Handler extends ChannelInboundByteHandlerAdapter {
#Override
protected void inboundBufferUpdated(ChannelHandlerContext ctx, ByteBuf in)
throws Exception {
System.out.println("called " + in.toString(CharsetUtil.UTF_8));
}
}
Now it works properly. Functionally both pipelines should work right? Why is the first setup not working? The client code is the same.
Thanks a lot!

The documentation for StringDecoder clearly states that it must be used in conjunction with a ByteToMessageDecoder if used over a stream connection (such as TCP). The example you refer to has such a handler in front of the StringDecoder.

Thanks guys! so I added the following:
pipeline.addLast("framer", new DelimiterBasedFrameDecoder(8192, Delimiters.nulDelimiter()));
And this still didn't work until I explicitly add '\0' to the end to String in C# :
ASCIIEncoding encoder = new ASCIIEncoding();
int index = random.Next(0, 2);
byte[] buffer = encoder.GetBytes(list[index] +　"\0");
The weird thing is that, I was using Netty 3.6 previously, and everything worked fine even without a FrameDecoder (only StringDecoder was there / client code was same) but now I have to do the steps above to make it to work..........?

Netty 4 - Outbound message at head of pipeline discarded

I am using Netty 4 RC1. I initialize my pipeline at the client side:
public class NodeClientInitializer extends ChannelInitializer<SocketChannel> {
#Override
protected void initChannel(SocketChannel sc) throws Exception {
// Frame encoding and decoding
sc.pipeline()
.addLast("logger", new LoggingHandler(LogLevel.DEBUG))
// Business logic
.addLast("handler", new NodeClientHandler());
}
}
NodeClientHandler has the following relevant code:
public class NodeClientHandler extends ChannelInboundByteHandlerAdapter {
private void sendInitialInformation(ChannelHandlerContext c) {
c.write(0x05);
}
#Override
public void channelActive(ChannelHandlerContext c) throws Exception {
sendInitialInformation(c);
}
}
I connect to the server using:
public void connect(final InetSocketAddress addr) {
Bootstrap bootstrap = new Bootstrap();
ChannelFuture cf = null;
try {
// set up the pipeline
bootstrap.group(new NioEventLoopGroup())
.channel(NioSocketChannel.class)
.handler(new NodeClientInitializer());
// connect
bootstrap.remoteAddress(addr);
cf = bootstrap.connect();
cf.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture op) throws Exception {
logger.info("Connect to {}", addr.toString());
}
});
cf.channel().closeFuture().syncUninterruptibly();
} finally {
bootstrap.shutdown();
}
}
So, what I basically want to do is to send some initial information from the client to the server, after the channel is active (i.e. the connect was successful). However, when doing the c.write() I get the following warning and no package is send:
WARNING: Discarded 1 outbound message(s) that reached at the head of the pipeline. Please check your pipeline configuration.
I know there is no outbound handler in my pipeline, but I didn't think I need one (at this point) and I thought Netty would take care to transport the ByteBuffer over to the server. What am I doing wrong here in the pipeline configuration?

Netty only handle messages of type ByteBuf by default if you write to the Channel. So you need to wrap it in a ByteBuf. See also the Unpooled class with its static helpers to create ByteBuf instances.

How to pass application/user data to the ChannelHandler Netty

When you design an a client that is going to connect to a lot of servers, like a crawler.
You will code something like that :
// the pipeline
public class CrawlerPipelineFactory implements ChannelPipelineFactory {
public ChannelPipeline getPipeline() throws Exception {
return Channels.pipeline(new CrawlerHandler());
}
}
// the channel handler
public class CrawlerHandler extends SimpleChannelHandler {
#Override
public void channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) throws Exception {
// ...
}
}
// the main :
public static void main(){
ChannelFactory factory = new NioClientSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool());
ClientBootstrap scannerBootstrap = new ClientBootstrap(factory);
scannerBootstrap.setPipelineFactory(new CrawlerPipelineFactory());
while(true){
MyURL url = stack.pop();
ChannelFuture connect = scannerBootstrap.connect(url.getSocketAddress());
}
}
Now when you are in your ApplicationHandler, the stuff that implements your SimpleChannelHandler or WhatEverStreamHandler, (CrawlerHander in the example) the only piece of information you get is the socketAdress you are connecting to that you can recover in "public void channelConnected()" function.
Ok but what if I want to recover some user data, like the MyURL object you see in my code example ?
I use a dirty hack, I use a Map<"ip:port",MyURL> so I can retrieve the associated data in channelConnected because I know ip:port i'm connected on.
This hack is really dirty, it won't work if you are connecting simultaneously to the same server (or you'll have to bind to a local port and use a key like "localport:ip:remoteport" but it's so dirty).
So I'm seeking what is the good way to pass data the the CrawlerHander ?
It would be cool if we could pass this data via the connect() method of the bootstrap. I know I can pass argument in my ChannelPipelineFactory.getPipeline() because it's invoked via connect(). But now we can't, so here is another dirty hack I use :
EDIT:
// the main
while(!targets.isEmpty()){
client.connect("localhost",111); // we will never connect to localhost, it's a hack
}
// the pipleline
public ChannelPipeline getPipeline() throws Exception {
return Channels.pipeline(
new CrawlerHandler(targets.pop()) // I specify each new host to connect here
);
}
// in my channel handler
// Now I have the data I want in the constructor, so I m sure I get them before everything is called
public class CrawlerHandler extends SimpleChannelHandler {
ExtraParameter target;
public CrawlerHandler(ExtraParameter target) {
this.target = target;
// but, and it's the most dirty part, I have to abort the connection to localhost, and reinit a new connection to the real target
boolean bFirstConnect=true;
#Override
public void connectRequested(ChannelHandlerContext ctx, ChannelStateEvent e) throws Exception {
if(bFirstConnect){
bFirstConnect = false;
ctx.getChannel().connect(target.getSocketAddr());
}

You can pass variables to Channel via Bootstrap.
Netty.io 4.1 & SO - Adding an attribute to a Channel before creation
Update to this answer while very late.

You can pass the data to the newly connected channel/channel handler using ChannelLocal or in ChannelHandlerContext (or in the Channel it self in latest Netty 3.x) using a connect future listener. In below example, ChannelLocal is used.
public class ChannelDataHolder {
public final static ChannelLocal<String> CHANNEL_URL = new ChannelLocal<String>(true);
}
// for each url in bootstrap
MyURL url = ....;
ChannelFuture cf = scannerBootstrap.connect(url.getSocketAddress());
final String urlString = url.getUrl();
cf.addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) throws Exception {
ChannelDataHolder.CHANNEL_URL.set(future.getChannel(), urlString);
}
});
//In the handler
public class CrawlerHandler extends SimpleChannelHandler {
#Override
public void channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) throws Exception {
String urlString = ChannelDataHolder.CHANNEL_URL.get(ctx.getChannel());
// ...use the data here
}
}
Note: instead of ChannelLocal, you can set and get the data using
ChannelHandlerContext.setAttachment()/getAttachment()
Channel.setAttachment()/getAttachment() in latest 3.x version of Netty
but both approaches does not support type safety.