Develop Reference

Resend messages after timeout

I have a list of objects that I put in Spring AMQP. Objects come from the controller. There is a service that processes these objects. And this service may crash with an OutOfMemoryException. Therefore, I run several instances of the application.
There is a problem: when the service crashes, I lose the received messages. I read about NACK. And could use it in case of Exception or RuntimeException. But my service crashes in Error. Therefore, I cannot send NACK. Is it possible to set a timeout in AMQP, after which I would be sent a message again if I had not confirmed the messages that had arrived earlier?
Here is the code I wrote:
public class Exchanges {
public static final String EXC_RENDER_NAME = "render.exchange.topic";
public static final TopicExchange EXC_RENDER = new TopicExchange(EXC_RENDER_NAME, true, false);
}
public class Queues {
public static final String RENDER_NAME = "render.queue.topic";
public static final Queue RENDER = new Queue(RENDER_NAME);
}
#RequiredArgsConstructor
#Service
public class RenderRabbitEventListener extends RabbitEventListener {
private final ApplicationEventPublisher eventPublisher;
#RabbitListener(bindings = #QueueBinding(value = #Queue(Queues.RENDER_NAME),
exchange = #Exchange(value = Exchanges.EXC_RENDER_NAME, type = "topic"),
key = "render.#")
)
public void onMessage(Message message, Channel channel) {
String routingKey = parseRoutingKey(message);
log.debug(String.format("Event %s", routingKey));
RenderQueueObject queueObject = parseRender(message, RenderQueueObject.class);
handleMessage(queueObject);
}
public void handleMessage(RenderQueueObject render) {
GenericSpringEvent<RenderQueueObject> springEvent = new GenericSpringEvent<>(render);
springEvent.setRender(true);
eventPublisher.publishEvent(springEvent);
}
}
And this is the method that sends messages:
    #Async ("threadPoolTaskExecutor")
    #EventListener (condition = "# event.queue")
    public void start (GenericSpringEvent <RenderQueueObject> event) {
        RenderQueueObject renderQueueObject = event.getWhat ();
        send (RENDER_NAME, renderQueueObject);
}
private void send(String routingKey, Object queue) {
try {
rabbitTemplate.convertAndSend(routingKey, objectMapper.writeValueAsString(queue));
} catch (JsonProcessingException e) {
log.warn("Can't send event!", e);
}
}

You need to close the connection to get the message re-queued.
It's best to terminate the application after an OOME (which, of course, will close the connection).

akka websocket with java, counting clients number, sending message to client

I'm following the akka java websocket tutorial in attempt to create a websocket server. I want to implement 2 extra features:
Being able to display the number of connected clients, but the result
is always 0 or 1 , even when I know I have 100's concurrently
connected clients.
Websocket communication is biDirectional. Currently the server only respond with a message when client sends a message. How do I initiate sending a message from server to client?
Here's original akka java server example code with minimum modification of my client counting implementation:
public class websocketServer {
private static AtomicInteger connections = new AtomicInteger(0);//connected clients count.
public static class MyTimerTask extends TimerTask {
//called every second to display number of connected clients.
#Override
public void run() {
System.out.println("Conncurrent connections: " + connections);
}
}
//#websocket-handling
public static HttpResponse handleRequest(HttpRequest request) {
HttpResponse result;
connections.incrementAndGet();
if (request.getUri().path().equals("/greeter")) {
final Flow<Message, Message, NotUsed> greeterFlow = greeter();
result = WebSocket.handleWebSocketRequestWith(request, greeterFlow);
} else {
result = HttpResponse.create().withStatus(413);
}
connections.decrementAndGet();
return result;
}
public static void main(String[] args) throws Exception {
ActorSystem system = ActorSystem.create();
TimerTask timerTask = new MyTimerTask();
Timer timer = new Timer(true);
timer.scheduleAtFixedRate(timerTask, 0, 1000);
try {
final Materializer materializer = ActorMaterializer.create(system);
final Function<HttpRequest, HttpResponse> handler = request -> handleRequest(request);
CompletionStage<ServerBinding> serverBindingFuture =
Http.get(system).bindAndHandleSync(
handler, ConnectHttp.toHost("****", 1183), materializer);
// will throw if binding fails
serverBindingFuture.toCompletableFuture().get(1, TimeUnit.SECONDS);
System.out.println("Press ENTER to stop.");
new BufferedReader(new InputStreamReader(System.in)).readLine();
timer.cancel();
} catch (Exception e){
e.printStackTrace();
}
finally {
system.terminate();
}
}
//#websocket-handler
/**
* A handler that treats incoming messages as a name,
* and responds with a greeting to that name
*/
public static Flow<Message, Message, NotUsed> greeter() {
return
Flow.<Message>create()
.collect(new JavaPartialFunction<Message, Message>() {
#Override
public Message apply(Message msg, boolean isCheck) throws Exception {
if (isCheck) {
if (msg.isText()) {
return null;
} else {
throw noMatch();
}
} else {
return handleTextMessage(msg.asTextMessage());
}
}
});
}
public static TextMessage handleTextMessage(TextMessage msg) {
if (msg.isStrict()) // optimization that directly creates a simple response...
{
return TextMessage.create("Hello " + msg.getStrictText());
} else // ... this would suffice to handle all text messages in a streaming fashion
{
return TextMessage.create(Source.single("Hello ").concat(msg.getStreamedText()));
}
}
//#websocket-handler
}

Addressing your 2 bullet points below:
1 - you need to attach your metrics to the Message flow - and not to the HttpRequest flow - to effectively count the active connections. You can do this by using watchTermination. Code example for the handleRequest method below
public static HttpResponse handleRequest(HttpRequest request) {
HttpResponse result;
if (request.getUri().path().equals("/greeter")) {
final Flow<Message, Message, NotUsed> greeterFlow = greeter().watchTermination((nu, cd) -> {
connections.incrementAndGet();
cd.whenComplete((done, throwable) -> connections.decrementAndGet());
return nu;
});
result = WebSocket.handleWebSocketRequestWith(request, greeterFlow);
} else {
result = HttpResponse.create().withStatus(413);
}
return result;
}
2 - for the server to independently send messages you could create its Message Flow using Flow.fromSinkAndSource. Example below (this will only send one message):
public static Flow<Message, Message, NotUsed> greeter() {
return Flow.fromSinkAndSource(Sink.ignore(),
Source.single(new akka.http.scaladsl.model.ws.TextMessage.Strict("Hello!"))
);
}

In the handleRequest method you increment and then decrement the counter connections, so at the end the value is always 0.
public static HttpResponse handleRequest(HttpRequest request) {
...
connections.incrementAndGet();
...
connections.decrementAndGet();
return result;
}

How Future works in Vert.x?

I send a message using EventBus and i want to get the reply message into a variable then will return it.this is the code block.
public class MessageExecute {
private static final Logger logger = LoggerFactory.getLogger(MessageExecute.class);
public static <T> T sendMessage(Vertx vertx,String address,T message){
Future<Message<T>> future = Future.future();
vertx.eventBus().send(address, message, future.completer());
future.setHandler(new Handler<AsyncResult<Message<T>>>() {
#Override
public void handle(AsyncResult<Message<T>> event) {
logger.info("received reply message | thread - " + Thread.currentThread().getName());
}
});
boolean notFound = true;
while(notFound){
try{
if(future.result()!= null){
notFound = false;
}
}catch(Exception e){
}
}
return message;
}
}
Actually this is working fine.But some times While block never exit.Its mean future.result() not getting the value ,even after the reply message is received.I don't know this the correct way and I don't have clear idea about how the Futures work in vert.x .Is there any other way to implement these kind of scenario.

I recommend you to read about the Vertx-Sync project - http://vertx.io/docs/vertx-sync/java/
In examples, have the follow example that appears very similar to you case:
EventBus eb = vertx.eventBus();
HandlerReceiverAdaptor<Message<String>> adaptor = streamAdaptor();
eb.<String>consumer("some-address").handler(adaptor);
// Receive 10 messages from the consumer:
for (int i = 0; i < 10; i++) {
Message<String> received1 = adaptor.receive();
System.out.println("got message: " + received1.body());
}

How can i decouple my networkmanager using events?

i'm writing a program that connects with various TCP network devices. The GUI is made using JavaFX. The whole connection part is in its own package "Network". Roughly described, it looks like this: (I don't know much about UML, no blaming plaese :/ - i just needed a way to quickly describe how my program structure looks). http://i.stack.imgur.com/PSdsH.jpg
okay thats how it is:
The TCP classes are stored in a synchronized List in "NetworkManager". These classes hold information about the connection (how much data received yet, ip, mac etc.). The Rcv-Thread constantly tries to receive data.
well, this is what i want:
As soon as the Rcv-Thread receives a specific message, the controller should be invoked to do something (GUI refresh or whatever). Also the controller should stay decoupled from the "Network" module-> it is reused in another project. I want to achieve this behaviour through an custom event. In short: TCP-Rcv-Thread needs to be able to give information to the Controller. But i dont really know how to get it all to work. Lets see where i am:
I have an event class in the "Network" module.
import java.util.EventObject;
public class XEvent extends EventObject{
String message;
public XEvent(Object source, String message) {
super(source);
this.message = message;
}
public String getMessage() {
return message;
}
}
I have a listener class in the "Network" module.
import java.util.EventListener;
public interface XListener extends EventListener{
void handlerMethod1(XEvent event);
void handlerMethod2(XEvent event);
}
I tried to prepare my Rcv-Thread for firing the event:
import javax.swing.event.EventListenerList;
import java.io.IOException;
public class ReceiveDataThread implements Runnable {
protected EventListenerList listenerList = new EventListenerList();
}
protected void addXListener(XListener xListener) {
listenerList.add(XListener.class, xListener);
}
protected void removeListener(XListener xListener) {
listenerList.remove(XListener.class, xListener);
}
protected void fireHandlerMethod1(String message) {
XEvent event = null;
Object[] list = listenerList.getListenerList();
for (int i = 0; i < list.length; i += 2) {
if (list[i] == XListener.class) {
if (event == null) event = new XEvent(this, message);
XListener l = (XListener) list[i + 1];
l.handlerMethod1(event);
}
}
}
protected void fireHandlerMethod2(String message) {
XEvent event = null;
Object[] list = listenerList.getListenerList();
for (int i = 0; i < list.length; i += 2) {
if (list[i] == XListener.class) {
if (event == null) event = new XEvent(this, message);
XListener l = (XListener) list[i + 1];
l.handlerMethod2(event);
}
}
}
#Override
public void run() {
String s;
while (!stopThread) {
s = receiveData();
System.out.println("test");
fireHandlerMethod1(s);
}
}
The Controller (this class should react on the custom events) implements the Listener:
public class Controller implements Initializable, XListener {
#Override
public void handlerMethod1(XEvent event) {
System.out.println("Event1: " + event.getMessage());
}
#Override
public void handlerMethod2(XEvent event) {
}
}
And from there on i'm not really shure how to get it work that my events (fired from my Rcv-Thread) are noticed by my controller class. I think i have to add a listener to every Rcv-Thread object via the controller class (just like when i use a ButtonListener, ...) . The problem is: from my TCP Class i can't access the Rcv-Thread-object's addXListener method - even when set to public (but i can access the Rcv-Thread-Classes from the list). I tried to read as much as i can about the problem but cant figure out how to get this to work. What am i missing?
edit1: TCP class:
public class TCPClass{
private Thread receiveDataThread;
private String MAC;
private InetAddress IP;
private Socket socket = new Socket();
private int tcpSendPort;
private int timeOut = 10;
private ObjectOutputStream objectOutputStream;
private BufferedReader bufferedReader;
private String connectionStatus = "offline";
public TCPClass(DatagramPacket datagramPacket) {
IP = datagramPacket.getAddress();
setConnectionStatusOnline();
tcpSendPort = 50000 + NetworkManager.getNumberOfConnections();
MAC = extractMac(datagramPacket);
}
public void connect(int tcpPort) {
try {
socket = new Socket(IP, tcpPort, null, tcpSendPort);
bufferedReader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
receiveDataThread = new Thread(new ReceiveDataThread(this));
receiveDataThread.start();
InputStreamReader(socket.getInputStream()));
} catch (IOException e) {
e.printStackTrace();
System.out.println("on MAC: " + getMAC() + "\non Device:" + toString());
}
if (socket.isConnected()) {
setConnectionStatusConnected();
}
}
}
The NetworkManager creates an object of TCPClass and calls the connect() method.

Ok so after days i figured it out myself.
The main problem was that i was not able to call the addXListener() method of Rcv-Thread from the Controller. I took the Custom Event stuff out of the Rcv-Thread and moved it to the TCP-Class. Now i'm able to add the Listener to these classes. If i want to fire an event from the Rcv-Thread i simply call fireHandlerMethod() from its superclass (TCP-Class) - and everything works as expected.

Read an AMF object with flex socket

I'm currently trying to communicate between java and flex by using sockets and AMF serialized objects.
On the java side I use Amf3Input and Amf3Output from BlazeDS (flex-messaging-common.jar and flex-messaging-core.jar).
The connection is correctly established, and if i try to send object from flex to java, i can easily read objects :
FLEX side :
protected function button2_clickHandler(event:MouseEvent):void
{
var tmp:FlexAck = new FlexAck;
tmp.id="123456789123456789123456789";
tmp.name="A";
tmp.source="Aaaaaa";
tmp.ackGroup=false;
s.writeObject(tmp);
s.flush();
}
JAVA side :
ServerSocket servSoc = new ServerSocket(8888);
Socket s = servSoc.accept();
Amf3Output amf3Output = new Amf3Output(SerializationContext.getSerializationContext());
amf3Output.setOutputStream(s.getOutputStream());
Amf3Input amf3Input = new Amf3Input(SerializationContext.getSerializationContext());
amf3Input.setInputStream(s.getInputStream());
while(true)
{
try
{
Object obj = amf3Input.readObject();
if(obj!=null){
if (obj instanceof AckOrder){
System.out.println(((AckOrder)obj).getId());
}
}
}
catch (Exception e)
{
e.printStackTrace();
break;
}
}
amf3Output.close();
amf3Input.close();
servSoc.close();
In this way it works perfectly, but the problem is to read objects sent from the java side.
The code I use in java is :
for(int i=0;i<10;i++){
ack = new AckOrder(i,"A","B", true);
amf3Output.writeObject(ack);
amf3Output.writeObjectEnd();
amf3Output.flush();
}
I have an handler on ProgressEvent.SOCKET_DATA :
trace((s.readObject() as FlexAck).id);
But I have errors such as :
Error #2030: End of File detected
Error #2006: Index Out of bound
If i add manipulations on ByteArrays, i manage to read the first object, but not the following.
s.readBytes(tmp,tmp.length);
content = clone(tmp);
(content.readObject());
trace("########################## OK OBJECT RECEIVED");
var ack:FlexAck = (tmp.readObject() as FlexAck);
trace("**********************> id = "+ack.id);
I've spent many our trying to find something in several forums etc, but nothing helped.
So if someone could help me it would be great.
Thanks
Sylvain
EDIT :
Here is an example that I thought should work, but doesn't I hope that it's better illustrate what I aim to do (permanent connection with socket and an exchange of messages).
Java class :
import java.net.ServerSocket;
import java.net.Socket;
import awl.oscare.protocol.AckOrder;
import flex.messaging.io.SerializationContext;
import flex.messaging.io.amf.Amf3Input;
import flex.messaging.io.amf.Amf3Output;
public class Main {
public static void main(String[] args) {
while(true)
{
try {
ServerSocket servSoc = new ServerSocket(8888);
Socket s = servSoc.accept();
System.out.println("connection accepted");
Amf3Output amf3Output = new Amf3Output(SerializationContext.getSerializationContext());
amf3Output.setOutputStream(s.getOutputStream());
Amf3Input amf3Input = new Amf3Input(SerializationContext.getSerializationContext());
amf3Input.setInputStream(s.getInputStream());
while(true)
{
try
{
System.out.println("Reading object");
Object obj = amf3Input.readObject();
if(obj!=null)
{
System.out.println(obj.getClass());
if (obj instanceof AckOrder)
{
AckOrder order = new AckOrder();
order.setId(((AckOrder)obj).getId());
order.setName(((AckOrder)obj).getName());
order.setSource(((AckOrder)obj).getSource());
order.setAckGroup(((AckOrder)obj).isAckGroup());
System.out.println(((AckOrder)obj).getId());
amf3Output.writeObject(order);
amf3Output.writeObjectEnd();
amf3Output.flush();
}
}
}
catch (Exception e)
{
e.printStackTrace();
break;
}
}
amf3Output.close();
amf3Input.close();
servSoc.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Java Serializable object :
package protocol;
import java.io.Serializable;
public class AckOrder implements Serializable {
private static final long serialVersionUID = 5106528318894546695L;
private String id;
private String name;
private String source;
private boolean ackGroup = false;
public String getId() {
return this.id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return this.name;
}
public void setName(String name) {
this.name = name;
}
public void setSource(String source) {
this.source = source;
}
public String getSource() {
return this.source;
}
public void setAckGroup(boolean ackGroup) {
this.ackGroup = ackGroup;
}
public boolean isAckGroup() {
return this.ackGroup;
}
public AckOrder()
{
super();
}
}
Flex Side :
Main flex code :
<fx:Script>
<![CDATA[
import mx.collections.ArrayCollection;
import mx.controls.Alert;
import mx.events.FlexEvent;
import mx.utils.object_proxy;
private var _socket:Socket = new Socket();;
private function onCreationComplete():void
{
this._socket.connect("localhost",8888);
this._socket.addEventListener(ProgressEvent.SOCKET_DATA, onData);
}
private function onData(e:ProgressEvent):void
{
if(this._socket.bytesAvailable)
{
this._socket.endian = Endian.LITTLE_ENDIAN;
var objects:Array = [];
try{
while(this._socket.bytesAvailable > 0)
{
objects.push(this._socket.readObject());
}
}catch(e:Error){trace(e.message);}
trace("|"+(objects)+"|");
}
}
protected function sendButton_clickHandler(event:MouseEvent):void
{
var tmp:FlexAck = new FlexAck;
tmp.id="1";
tmp.name="A";
tmp.source="B";
tmp.ackGroup=false;
this._socket.writeObject(tmp);
this._socket.flush();
}
]]>
</fx:Script>
<s:Button x="0" y="0" name="send" label="Send" click="sendButton_clickHandler(event)"/>
Flex serializable object :
package
{
[Bindable]
[RemoteClass(alias="protocol.AckOrder")]
public class FlexAck
{
public function FlexAck()
{
}
public var id:String;
public var name:String;
public var source:String;
public var ackGroup:Boolean;
}
}
Edit 25/05/2011 :
I've added those listeners in my flex code :
this._socket.addEventListener(Event.ACTIVATE,onActivate);
this._socket.addEventListener(Event.CLOSE,onClose);
this._socket.addEventListener(Event.CONNECT,onConnect);
this._socket.addEventListener(Event.DEACTIVATE,onDeactivate);
this._socket.addEventListener(IOErrorEvent.IO_ERROR,onIOerror);
this._socket.addEventListener(SecurityErrorEvent.SECURITY_ERROR,onSecurityError);
But There's no errors and I still don't manage to receive objects correctly.

You have to send the AMF data as ByteArray on the server:
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
amf3Output.setOutputStream(baos);
amf3Output.writeObject(order);
amf3Output.flush();
amf3Output.close();
s.getOutputStream().write(baos.toByteArray());
Then
this._socket.readObject()
works as expected !

Hi the problem is caused by the following:
An AMF stream is stateful. When it serializes objects, it compresses them relative to objects that it have already been written.
Compression is achieved by referencing previously sent class descriptions, string values and objects using indexes (so for example, if the first string you sent was "heloWorld", when you later send that string, the AMF stream will sent string index 0).
Unfortunately, ByteArray and Socket do not maintain reference tables between readObject calls. Thus, even if you keep appending your newly read objects to the end of the same ByteArray object, each call to readObject instantiates new reference tables, discarding previously created ones (this means it should work for repeated references to the same string within an object tree)
In your example, you are always writing the same string values to properties. Thus when you send the second object, its string properties are not serialized as strings, but as references to the strings in the previously written object.
The solution, is to create a new AMF stream for each object you send.
This is complete rubbish of course(!) It means we can't really utilize the compression in custom protocols. It would be much better if our protocols could decide when to reset the these reference tables, perhaps when they got too big.
For example, if you have an RPC protocol, it would be nice to have an AMF stream pass the remote method names as references rather than strings for speed...
I haven't checked but I think this sort of thing is done by RTMP. The reason it probably wouldn't have been made available in developer objects like ByteArray and Socket (sigh, I hope this isn't true) is because Adobe wants to push us towards LCDS...
Addendum/edit: just found this, which provides a solution http://code.google.com/p/cvlib/

After looking at the code, I think what you want to do on the Java end is this:
for(int i=0;i<10;i++){
ack = new AckOrder(i,"A","B", true);
amf3Output.writeObject(ack);
}
amf3Output.flush();
When you do 'flush', you're sending information over the socket so you only had one object being sent at a time. On the Flex end, you should always try to see what's the length of the object and make sure you're not going over it which would cause this error.
EDIT:
private var _socket:Socket = new Socket();
private function onCreationComplete():void
{
// Add connection socket info here
this._socket.addEventListener(ProgressEvent.SOCKET_DATA, onData);
}
// This gets called every time we get new info, as in after the server flushes
private function onData(e:ProgressEvent):void
{
if(this._socket.bytesAvailable)
{
this._socket.endian = Endian.LITTLE_ENDIAN; // Might not be needed, but often is
// Try to get objects
var objects:Array = [];
try{
while(this._socket.bytesAvailable > 0)
{
objects.push(this._socket.readObject());
}
}catch(e:Error){}
// Do something with objects array
}
}
The onData function is called continually (every time the server sends info) since everything is asynchronous.