Develop Reference

Reactor / WebFlux implement a reactive http news ticker

I have a request that is rather simple to formulate, but I cannot pull it of without leaking resources.
I want to return a response of type application/stream+json, featuring news events someone posted. I do not want to use Websockets, not because I don't like them, I just want to know how to do it with a stream.
For this I need to return a Flux<News> from my restcontroller, that is continuously fed with news, once someone posts any.
My attempt for this was creating a Publisher:
public class UpdatePublisher<T> implements Publisher<T> {
private List<Subscriber<? super T>> subscribers = new ArrayList<>();
#Override
public void subscribe(Subscriber<? super T> s) {
subscribers.add(s);
}
public void pushUpdate(T message) {
subscribers.forEach(s -> s.onNext(message));
}
}
And a simple News Object:
public class News {
String message;
// Constructor, getters, some properties omitted for readability...
}
And endpoints to publish news respectively get the stream of news
// ...
private UpdatePublisher<String> updatePublisher = new UpdatePublisher<>();
#GetMapping(value = "/news/ticker", produces = "application/stream+json")
public Flux<News> getUpdateStream() {
return Flux.from(updatePublisher).map(News::new);
}
#PutMapping("/news")
public void putNews(#RequestBody News news) {
updatePublisher.pushUpdate(news.getMessage());
}
This WORKS, but I cannot unsubscribe, or access any given subscription again - so once a client disconnects, the updatePublisher will just continue to push onto a growing number of dead channels - as I have no way to call the onCompleted() handler on the subscriptions.
TL;DL:
Can one push messages onto a possible endless Flux from a different thread and still terminate the Flux on demand without relying on a reset by peer exception or something along those lines?

You should never try to implement yourself the Publisher interface, as it boils down to getting the reactive streams implementation right. This is exactly the issue you're facing here.
Instead you should use one of the generator operators provided by Reactor itself (this is actually a Reactor question, nothing specific to Spring WebFlux).
In this case, Flux.create or Flux.push are probably the best candidates, given your code uses some type of event listener to push events down the stream. See the reactor project reference documentation on that.
Without more details, it's hard to give you a concrete code sample that solves your problem. Here are a few pointers though:
you might want to .share() the stream of events for all subscribers if you'd like some multicast-like communication pattern
pay attention to the push/pull/push+pull model that you'd like to have here; how is the backpressure supposed to work here? What if we produce more events that the subscribers can handle?
this model would only work on a single application instance. If you'd like this to work on multiple application instances, you might want to look into messaging patterns using a broker

transfer data dynamically with netty

I've been learning Netty for a while, and from the Netty's tutorials(the MEAP book), almostly the examples are based on a fixed framework, like the EventLoop, Bootstrap, it seems that only the implementations of the handlers in the channelPipeline are the things we really should be concerned about.
Here I wanna design a simple chess game, based on a Server/Client mode, where two players are on different computers. And the background data I want to use Netty to transmit.(I just wanna practice using netty)
And in such a game, the front GUI detect the player put a chessman and then make some change to the data. Then, I need to deliver this data to the other player. And here comes the question.
I don't know how to implement a ChannelHandler in this situation, because in most examples, it seems that the data are not added dynamically by the handler. For instance, the data was created when channel was active by the method channelActive() or something else. All these methods was auto-invoked by Netty itself.
The only method I think is the write(). However, it seems that I have to call this method by myself if I implements this method, I don't know where I can get the parameter ChannelHandlerContext.
So, how to solve problems like this?
p.s.
I'm not so familiar with java network programming, nor the Netty. All the things I learn is based on the book, which I haven't finished reading yet. :)

Channel hander of Netty looks
package netty_sample;
import org.jboss.netty.channel.ChannelHandlerContext;
import org.jboss.netty.channel.MessageEvent;
import org.jboss.netty.channel.SimpleChannelHandler;
/**
* Server side action
*/
public class EchoServerHandler extends SimpleChannelHandler {
/**
* This method will be invoked when server recieved a message
*/
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent event) {
String msg = (String) event.getMessage(); // extract a message received
// You can write any code which handles the message, changes data, and create message for client, etc.
ctx.getChannel().write(someMessageToClient); // send back to client
}
}
As I understand, handler routine is invoked dynamically (in event-driven) when server received a message.
So code in the handler works dynamically, and you can write anything in the code.

How to execute asynchronous database insert in Actor's onReceive method?

I have a Play framework 2 application that also uses Akka. I have an Actor that receives messages from a remote system, the amount of such messages can be very huge. After a message is received, i log it into the database (using the built-in Ebean ORM) and then continue to process it. I don't care, how fast this database logging works, but it definitely should not block the further processing. Here is a simplified code sample:
public class MessageReceiver extends UntypedActor {
#Override
public void onReceive(Object message) throws Exception {
if (message instanceof ServerMessage) {
ServerMessage serverMessage = (ServerMessage) message;
ServerMessageModel serverMessageModel = new ServerMessageModel(serverMessage);
serverMessageModel.save();
//now send the message to another actor for further processing
} else {
unhandled(message);
}
}
}
As i understand, database inserting is blocking in this realization, so it does not meet my needs. But i can't figure out how to make it unblocking. I've read about the Future class, but i can't get it to work, since it should return some value, and serverMessageModel.save(); returns void.I understand that writing a lot of messages one-by-one into the database is unefficient, but that is not the issue at the moment.
Am i right that this implementation is blocking? If it is, how can i make it run asynchronously?

Future solution seems good to me. I haven't used Futures from Java, but you can just return arbitrary Integer or String if you definitely need some return value.
Other option is to send that message to some other actor which would do the saving to the DB. Then you should make sure that the mailbox of that actor would not overfill.
Have you considered akka-persistence for this? Maybe that would suit your use-case.

If you wish to use Future - construct an Akka Future with a Callable (anonymous class), whose apply() will actually implement the db save code. You can actually put all of this (future creation and apply()) in your ServerMessageModel class -- maybe call it asynchSave(). Your Future maybe Future where status is the result of asynchSave...
public Future<Status> asyncSave(...) { /* should the params be ServerMessageModel? */
return future(new Callable<Status>() {
public Status call() {
/* do db work here */
}
}
In your onReceive you can go ahead with tell to the other actor. NOTE: if you want to make sure that you are firing the tell to the other actor after this future returns, then you could use Future's onSuccess.
Future<Status> f = serverMessageModel.asyncSave();
f.onSuccess(otherActor.tell(serverMessage, self());
You can also do failure handling... see http://doc.akka.io/docs/akka/2.3.4/java/futures.html for further details.
Hope that helps.

Persist actor state with Martin Krassers akka-persistence extension and my jdbc persistence provider akka persistence jdbc https://github.com/dnvriend/akka-persistence-jdbc

Desktop program communicating with server

I am in the process of moving the business logic of my Swing program onto the server.
What would be the most efficient way to communicate client-server and server-client?
The server will be responsible for authentication, fetching and storing data, so the program will have to communication frequently.

it depends on a lot of things. if you want a real answer, you should clarify exactly what your program will be doing and exactly what falls under your definition of "efficient"
if rapid productivity falls under your definition of efficient, a method that I have used in the past involves serialization to send plain old java objects down a socket. recently I have found that, in combination with the netty api, i am able to rapidly prototype fairly robust client/server communication.
the guts are fairly simple; the client and server both run Netty with an ObjectDecoder and ObjectEncoder in the pipeline. A class is made for each object designed to handle data. for example, a HandshakeRequest class and HandshakeResponse class.
a handshake request could look like:
public class HandshakeRequest extends Message {
private static final long serialVersionUID = 1L;
}
and a handshake response may look like:
public class HandshakeResponse extends Message {
private static final long serialVersionUID = 1L;
private final HandshakeResult handshakeResult;
public HandshakeResponse(HandshakeResult handshakeResult) {
this.handshakeResult = handshakeResult;
}
public HandshakeResult getHandshakeResult() {
return handshakeResult;
}
}
in netty, the server would send a handshake request when a client connects as such:
#Override
public void channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) {
Channel ch = e.getChannel();
ch.write(new HandshakeRequest();
}
the client receives the HandshakeRequest Object, but it needs a way to tell what kind of message the server just sent. for this, a Map<Class<?>, Method> can be used. when your program is run, it should iterate through the Methods of a class with reflection and place them in the map. here is an example:
public HashMap<Class<?>, Method> populateMessageHandler() {
HashMap<Class<?>, Method> temp = new HashMap<Class<?>, Method>();
for (Method method : getClass().getMethods()) {
if (method.getAnnotation(MessageHandler.class) != null) {
Class<?>[] methodParameters = method.getParameterTypes();
temp.put(methodParameters[1], method);
}
}
return temp;
}
this code would iterate through the current class and look for methods marked with an #MessageHandler annotation, then look at the first parameter of the method (the parameter being an object such as public void handleHandshakeRequest(HandshakeRequest request)) and place the class into the map as a key with the actual method as it's value.
with this map in place, it is very easy to receive a message and send the message directly to the method that should handle the message:
#Override
public void messageReceived(ChannelHandlerContext ctx, MessageEvent e) {
try {
Message message = (Message) e.getMessage();
Method method = messageHandlers.get(message.getClass());
if (method == null) {
System.out.println("No handler for message!");
} else {
method.invoke(this, ctx, message);
}
} catch(Exception exception) {
exception.printStackTrace();
}
}
there's not really anything left to it. netty handles all of the messy stuff allowing us to send serialized objects back and forth with ease. if you decide that you do not want to use netty, you can wrap your own protocol around java's Object Output Stream. you will have to do a little bit more work overall, but the simplicity of communication remains intact.

It's a bit hard to say which method is "most efficient" in terms of what, and I don't know your use cases, but here's a couple of options:
The most basic way is to simply use "raw" TCP-sockets. The upside is that there's nothing extra moving across the network and you create your protocol yourself, the latter being also a downside; you have to design and implement your own protocol for the communication, plus the basic framework for handling multiple connections in the server end (if there is a need for such).
Using UDP-sockets, you'll probably save a little latency and bandwidth (not much, unless you're using something like mobile data, you probably won't notice any difference with TCP in terms of latency), but the networking code is a bit harder task; UDP-sockets are "connectionless", meaning all the clients messages will end up in the same handler and must be distinguished from one another. If the server needs to keep up with client state, this can be somewhat troublesome to implement right.
MadProgrammer brought up RMI (remote method invocation), I've personally never used it, and it seems a bit cumbersome to set up, but might be pretty good in the long run in terms of implementation.
Probably one of the most common ways is to use http for the communication, for example via REST-interface for Web services. There are multiple frameworks (I personally prefer Spring MVC) to help with the implementation, but learning a new framework might be out of your scope for now. Also, complex http-queries or long urls could eat your bandwidth a bit more, but unless we're talking about very large amounts of simultaneous clients, this usually isn't a problem (assuming you run your server(s) in a datacenter with something like 100/100MBit connections). This is probably the easiest solution to scale, if it ever comes to that, as there're lots of load-balancing solutions available for web servers.

Preventing race conditions for message processing

I have a J2EE application that receives messages (events) via a web service. The messages are of varying types (requiring different processing depending on type) and sent in a specific sequence. It have identified a problem where some message types take longer to process than others. The result is that a message received second in a sequence may be processed before the first in the sequence. I have tried to address this problem by placing a synchronized block around the method that processes the messages. This seems to work, but I am not confident that this is the "correct" approach? Is there perhaps an alternative that may be more appropriate or is this "acceptable"? I have included a small snippit of code to try to explain more clearly. .... Any advice / guidance appreciated.
public class EventServiceImpl implements EventService {
public String submit (String msg) {
if (msg == null)
return ("NAK");
EventQueue.getInstance().submit(msg);
return "ACK";
}
}
public class EventQueue {
private static EventQueue instance = null;
private static int QUEUE_LENGTH = 10000;
protected boolean done = false;
BlockingQueue<String> myQueue = new LinkedBlockingQueue<String>(QUEUE_LENGTH);
protected EventQueue() {
new Thread(new Consumer(myQueue)).start();
}
public static EventQueue getInstance() {
if(instance == null) {
instance = new EventQueue();
}
return instance;
}
public void submit(String event) {
try {
myQueue.put(event);
} catch (InterruptedException ex) {
}
}
class Consumer implements Runnable {
protected BlockingQueue<String> queue;
Consumer(BlockingQueue<String> theQueue) { this.queue = theQueue; }
public void run() {
try {
while (true) {
Object obj = queue.take();
process(obj);
if (done) {
return;
}
}
} catch (InterruptedException ex) {
}
}
void process(Object obj) {
Event event = new Event( (String) obj);
EventHandler handler = EventHandlerFactory.getInstance(event);
handler.execute();
}
}
// Close queue gracefully
public void close() {
this.done = true;
}

I am not sure what is the framework (EJB(MDB)/JMS) you are working with. Generally using synchronization inside a Managed Environment like that of EJB/JMS should be avoided(its not a good practice). One way to get around is
the client should wait for the acknowledgement from the server before it sends the next message.
this way you client itself will control the sequence of events.
Please note this won't work if there are multiple client submitting the messages.
EDIT:
You have a situation wherein the client of the web service sends message in sequence without taking into account the message processing time. It simply dumps the message one after another. This is a good case for Queue ( First In First Out ) based solution. I suggest following two ways to accomplish this
Use JMS . This will have an additional overhead of adding a JMS providers and writing some plumbing code.
Use some multitheading pattern like Producer-Consumer wherein your web service handler will be dumping the incoming message in a Queue and a single threaded consumer will consume one message at a time. See this example using java.util.concurrent package.
Use database. Dump the incoming messages into a database. Use a different scheduler based program to scan the datbase (based on sequence number) and process the messages accordingly.
First and third solution is very standard for these type of problems. The second approach would be quick and won't need any additional libraries in your code.

If the events are to be processed in a specific sequence, then why not try adding "eventID" and 'orderID' fields to the messages? This way your EventServiceImpl class can sort, order and then execute in the proper order (regardless of the order they are created and/or delivered to the handler).
Synchronizing the handler.execute() block will not get the desired results, I expect. All the synchronized keyword does is prevent multiple threads from executing that block at the same time. It does nothing in the realm of properly ordering which thread goes next.
If the synchronized block does seem to make things work, then I assert you are getting very lucky in that the messages are being created, delivered and then acted upon in the proper order. In a multithread environment, this is not assured! I'd take steps to assure you are controlling this, rather than relying on good fortune.
Example:
Messages are created in the order 'client01-A', 'client01-C',
'client01-B', 'client01-D'
Messages arrive at the handler in the order 'client01-D',
'client01-B', 'client01-A', 'client01-C'
EventHandler can distinquish messages from one client to another and starts to cache 'client01' 's messages.
EventHandler recv's 'client01-A' message and knows it can process this and does so.
EventHandler looks in cache for message 'client01-B', finds it and processes it.
EventHandler cannot find 'client01-C' because it hasn't arrived yet.
EventHandler recv's 'client01-C' and processes it.
EventHandler looks in cache for 'client01-D' finds it, processes it, and considers the 'client01' interaction complete.
Something along these lines would assure proper processing and would promote good use of multiple threads.