Develop Reference

How to send elements to a Source.actorRef or a Source.queue in Java

I'm currently working with Akka Streams (in Java) for a personal project and I'm having a hard time understanding how to send element to a Source.
The idea is to use a WebSocket to push content into the user's web browser. I've managed to use Akka Streams to create a request-response system, following the Akka HTTP documentation, but this is not what I want to do.
Looking into the Akka Streams documentation, I saw that there is Source.queue and Source.actorRef. But I don't understand how to put an element into the Source. Source.queue and Source.actorRef return a Source, which doesn't have the method offer (for Source.queue) or tell (for Source.actorRef).
My question is: how do I get the ActorRef for the Source created by Source.actorRef or the SourceQueueWithComplete for a Source created with Source.queue, to be able to send elements to my Source?
I searched the various Akka documentation but found no method to do that. And the majority of the code I found on the Internet is written in Scala, which doesn't seem to have the same problem.

The actor and queue from Source.actorRef and Source.queue, respectively, are the materialized values of those sources, meaning that they can be obtained only if the stream is running. For example:
final ActorRef actor =
Source.actorRef(Integer.MAX_VALUE, OverflowStrategy.fail())
.to(Sink.foreach(m -> System.out.println(m)))
.run(materializer);
actor.tell("do something", ActorRef.noSender());
It's no different in Scala:
implicit val materializer = ActorMaterializer()
val actor =
Source.actorRef(Int.MaxValue, OverflowStrategy.fail)
.to(Sink.foreach(println))
.run()
actor ! "do something"

Netty-TCP-HTTP-MQTT design

I am trying to write a server-side java application that can accept tcp, http and mqtt communication (receive and send/ MongoDB as storage). From research, we decided that it could be a jar application based on Netty and paho for mqtt. We have 3 project using three of these protocols, therefore I am trying to unify the connection module. They each have different protocol style, for example:
-tcp: 0102330123456700
-http: HTTP POST /URL/count {"id":"02","count":"01234567"}
-mqtt: topic /02/count {"count":"01234567"}
Since we are a bit short of time, i am running them three in a silly but quick way---3 different thread listening to 3 different ports.
public class ServerLauncher {
public static void main(String[] args) {
NettyRestServer nettyRestServer = new NettyRestServer();
MqttServer mqttServer = new MqttServer();
EchoServer echoServer = new EchoServer();
new Thread(nettyRestServer,"HTTP Listening").start();
new Thread(mqttServer,"Mqtt Listening").start();
new Thread(echoServer,"socket Listening").start();
}
}
My questions are:
Since they are all based on tcp, is there a good way to manage them all together without wasting thread resource? maybe running just one thread for listening one port. I only find examples of single protocol.
For data storage, is it an okey design to push all the incoming messages to a concurrentHashMap across all thread/channels. Finally with a another thread running scheduled task, storage this concurrentHashMap into MongoDB and reset. or maybe use queue instead of concurrentHashMap

If you use Netty for all of these you can share the same EventLoopGroup for all servers which means all will share the same Threads.

You don't have to use three threads to start Server. You can do all of these in only one ServerBootstrap. And put the logic in ChannelHandler.
Netty's ChannelPipeline can dynamicly change ChannelHandler when getting the connecttion.
ctx.pipeline().addBefore(...)
ctx.pipeline().addAfter(...)
ctx.pipeline().remove(...)

Using EventLoopGroup in Netty with Multiple Channels

I am new to Netty but unfortunately there seems to be no detailed documentation/tutorial for a beginner.
I have multiple threads, each creating separate clients to connect to separate channels, using NettyChannelBuilder. The idea is that each channel will send & receive different kind of messages to/from different hosts. E.g. it looks like this:
class MyServiceClass{
void executeTasks() {
...
//here multiple tasks are executed in a for loop
executorService.execute(new Task(new Client());
...
}
}
class Client {
..
void connect() {
channel = NettyChannelBuilder.forAddress(host, port).build();
}
}
In this case, each task has its own client and clients are building their own channels to receive messages.
Should i create a single EventLoopGroup at executeTasks and give it to the Clients to be used while building their channel.
If this is the case, what is the advantage of using EventLoopGroup? What is it exactly doing at the background?

I'm not sure what you're asking. EventLoopGroups are just a grouping of threads used for netty. Using netty your clients will be on an EventLoopGroup and will be assigned to threads in a round robin matter so some may be on the same thread.
Personally I find the docs to be great but it's definitely not a framework designed for beginners.

Creating a server, techniques to keep a service running (or blocked?)

When creating a standalone server in Java (not using a container like tomcat/jetty), what are the various techniques to keep the service running and not ending?
I have seen where people use a ServerSocket (since you will be communicating with the service presumably), and they use ServerSocket.accept() which blocks until it receives a message. And this is usually done in a while loop:
while(...) {
serverSocket.accept();
}
(http://docs.oracle.com/javase/6/docs/api/java/net/ServerSocket.html#accept())
Is this the only way? If not, what other ways are there and any pros/cons with it?
Are there any libraries that help with building your own service, or its pretty much roll your own.

There are various libraries that help you roll your own Windows/Unix service in Java:
Apache Commons Daemon
Akuma
Java Service Wrapper
How you keep the application running depends on the actual needs of your application. If your application is a server, you would normally want to listen for incoming connections which usually involves some sort of blocking/polling. How you do that again depends on the type of server you want to build. Of the generic solutions there's the ServerSocket class and its method accept() that you already mentioned. Another possibility is to use java.nio and implement a reactor which provides a single-threaded server that can handle multiple connections at once (see http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf for details), but may be a bit hard to grasp and debug.

What you probably want is a multi-threaded server.
Each time the server accepts a connection, the server creates a thread to handle sending/reciving to that client. If you do not use threads in your server, it will only be able to handle one connection at a time.
So, as you meantioned, the server loops infinitly and listens for incomming connections:
while(true){
serverSocket.accept();
ClientHandler c = new ClientHandler(serverSocket);
A instance of the class ClientHandler will be created each time a connection is accepted. This class implements Runnable, and loops for incomming messages using getInputStream and getOutputStream on that socket:
public class ClientHandler implements Runnable{
DataInputStream in;
DataOutputStream out;
//ClientHandler constructor
public ClientHandler(Socket s) throws IOException{
in= new DataInputStream(socket.getInputStream());
out=new DataOutputStream(socket.getOutputStream());
thread.start();
}
The run method:
public void run() {
while(true){
String temp="";
while ((temp = (String) in.readUTF()) != null){ // Read from the input stream each iteration. When temp is not null a message is recived
System.out.println(temp);
Please that the above code does not take into account different exceptions that might occur and is very basic. But it should give you a basic idea on how a server using Sockets can be implemented.

For a quick solution (in a testing environment only!) you can go for something often dubbed as "Enterprise Loop" (because it is too often found in production systems):
while (true)
try {
// do something
} catch (Throwable t) {
// maybe log
}
However, this is not good style in the production code.
(see [1] for a parody of that idiom)
To create a service, you want one of the libraries from this answer.
If you "just need multithreading", have a look into the Java concurrency framework. I stronly suggest reading Java Concurrency in Practice, as multi-threading is much more that just starting another thread and errors are hard to debug.
[1] http://blog.antiblau.de/2016/01/26/java-enterprise-loop/

In-process ActiveMQ producer/consumer example?

I'm investigating using ActiveMQ as an embedded in-process message queue in my
application, but I'm a bit stuck on how I go about starting such an application
up. I envision it like so (pseudocode, of course):
configureBroker ()
broker.start ()
createProducer (broker)
producer.start ()
for each desired consumer
createConsumer (broker)
consumer.start ()
waitForSignal ()
signalProducerShutdown ()
waitForEmptyQueues ()
signalConsumerShutdown ()
broker.stop ()
I've tried to assemble a simple version of this, but I'm stuck on how to write
the producers and consumers in such a way as to have them work forever, or
until told to quit. What is the best way to do this? I'm speaking specifically about the threading aspect; what do I need/want to spawn off in its own thread, etc...
I'm completely new to message queue based applications, so please be verbose with your examples.

When you specify the ActiveMQConnectionFactory, you can specify "vm://" where name is the intra-vm specific name of your broker and it will start the broker within the VM.
For example,
String broker = "vm://stackOverflowTest";
ActiveMQConnectionFactory connectionFactory =
new ActiveMQConnectionFactory(broker);
Connection amqcon = connectionFactory.createConnection();
amqcon.start();
From there you can create your producers or consumers the same as if it was over the network. As long as you use the same name for the broker, you can have multiple threads / code open / talk to the same VM instance.
This solution only allows communication with the VM, it does not open any external ports. I'm assuming this is what you were looking for since you said you wanted embedded, in-process queues.