Develop Reference

Problem with concatenated messages in JSMPP

I have a problem sending concatenated messages to my SMSC, here, I will do my best to describe the problem.
When sending concatenated messages, my SMSC waits to receive all the segments of the message and then processes it to the provider and sends back the response, this means that when using the JSMPP java lib, whenever I receive a long message (more that 160 chars for GSM7) I will split the message into segments and submit each segment using the session.submitShortMessage method.
The problem is that this call is synchronous and waits for the SMSC to respond with submit_sm_resp. Therefor I need to use some sort of async mode to submit the messages.
I have no idea on how to do this with the current implementation of the library.
Do you know any way of doing this?
Thanks in advance

The way you have described SMSC is working in your case seems suspicious to me. I don't know of any SMSC that would only respond back once all segments have been submitted (here I am assuming the communication via smpp protocol). Considering your SMSC implements smpp specification version 3.4, you might need to talk with the team responsible for maintaining this SMSC.
Suppose your (jsmpp) client splits a long text message (more than 140 bytes) into two segments. For each segment, your client sends submit_sm and receives synchronously submit_sm_resp. This is the correct client side behaviour and can be seen one of the examples (SubmitLongMessageExample.java) from jsmpp library.
The example uses optional parameters when submitting the message to the SMSC, providing sequence number, total segments and most importantly, a unique message reference number.
The SMSC, should principally, synchronous respond back for each submitted segment and when it determines that for a unique message reference all segments has been received, it should then process the message and deliver it to the end user (mobile phone).
This behaviour/information can also be found in the specification at https://smpp.org/. If your SMSC does not behave like this, then the standard implementation by jsmpp library cannot help you (and for that matter anyone else)

You need to use the transciever bind-mode and use an implementation of the MessageReceiverListener onAcceptDeliverSm.

Synchronous/Asynchronous Serial Port communication With JSSC

I am trying to communicate between two computers using the Serial port and I am new to this area. I need to send requests from one computer(say A) to the other(say B) and receive responses for the requests sent. I am updating a Java Swing user interface with the responses.
I am using the jSSC library to do this. I have looked at the SerialPortReader examples and following is my understanding.
I will have to implement the SerialPortEventListener on both computers.A will use the writeBytes method to send the requests. B will listen to the commands sent using the SerialPortEventListener and will use the writeBytes method to send the response. A will listen to the data using its implemntation of SerialPortEventListener and when data is recieved, will update the UI. The following are my questions.
a) Is my above observation correct? Is there a different way to do this(for example is it possible that a writeBytes method that will return the response exists within the protocol?)
b) I read in several paces that serial port communication can be either synchronous or asynchronous. But from the examples, I can't understand if that code has implemented an asynchronous or synchronous communication. How would one go about implementing synchronous/asynchronous communication using jSSC? I am not asking for an implementation. Just some guidelines and what methods can be used.
c) It's possible that messages will be partly delivered. For example, if I send the command as a String "get variableThreeValue", it is possible that only the "get" or something like "get varia" will be received.(this can result in messages like "get get" etc..) How can I handle this kind of scenario? Again, I am not asking for an implementation. Just some guidelines and what methods can be used.

A classic "serial port" is something very "low level". Parameters like baud rate, start and stop bits and flow control must be set, and then byte sequences are read and written. The Java library uses a listener to receive events which are directly derived from what the serial driver senses (you find terms like "line" in the javadoc). How to react, depends on "the other side".
Flow control is what you use to avoid overrunning or overfilling the receiver. The lines of an RS-232 contain C(lear)T(o)S(end) and R(equest)TS, so "hardware handshake" is one option. Alternatively, US-ASCII defined control characters XON and XOFF which could be embedded in the data stream if it is not binary data. Flow control should not be an issue if the sending end doesn't send at full blast or computers differ significantly in speed.
As I understand, you'll connect some radio device to the port, and its documentation should specify all the parameters, and also the higher level protocol, i.e., how to transmit and receive data. The device may have special requirements, e.g., that you must pass on some setup data before you transmit actual data. (If you connect two computers with just a cable, then everything is up to you.)
Once you can basically send and receive, you'll have to think about safe transmission. You need to design a protocol:
message format, e.g. containing a length, a sequence number, data bytes, a CRC.
message sequencing, i.e., who may send what and when, e.g., a message A to B, an acknowledge from B to A, repeat. Or a NAK from B, and A must resend.
Perhaps you need a "session protocol", i.e., a login (as in ftp) and a logout
Timeout: what if either side doesn't receive another message within T?
Do you need a heartbeat, i.e., a message sent when the channel is idle to learn that the other side is still "alive".
A WLAN connection should make most of this unnecessary. Real "radio" (some short wave?) I've never heard of, but I'm not a radio expert.

XON/XOFF is software flow control. Assume bi directional serial bytes between producer a and b. If either sends a XON byte it means whoa, stop sending me bytes until I send you a XOFF byte.
For hardware replace XON byte with CTS and XOFF byte with RTS.

How to check if a message is really delivered in Netty using websocket?

I'm developing a websocket application by using Netty. I'd like to know if a message is really delivered from a source to a destination. In particular, let's assume that a client and a server have an open channel and exchange some messages for a while. At a certain point, the client goes down, but the channel is still active in Netty. I tried to use isReachable() before sending the message, but this method seems to be buggy in some scenarios (e.g. a machine with Win7 is up, but isReachable() returns false). Now, my idea is to implement a mechanism using ACKs, namely the server sends the message and the client sends back an ack. To do that, I need a timeout to see if, after a certain interval, the corresponding ack does not arrive. Is there something similar in Netty?

Regarding isReachable() - it's only a best effort API. The documentation points out that it tries to send an ICMP echo request or create a TCP connection to port 7 on the destination host, both of which are highly likely to be blocked by a firewall. Is this happening in your case?
As for the acknowledgement, there's nothing in Netty that provides this as standard, but it shouldn't be too difficult to implement. Firstly each message needs to be uniquely identifible by some sort of identifier, possibly a sequence number but a globally unique identifier means you can potentially recover across disconnections. Then you want to create a combined handler that implements both ChannelInboundHandler and ChannelOutboundHandler (assuming Netty 4). When a message is sent
add the message to a map indexed by its id
create a timer associated with the message id. Add it to another map indexed by message id
forward the message
When the ACK is received cancel the timer and remove the timer and message from their respective maps. If the timer fires use the associated id to decide what to do with the timer and message (possibly retransmit and reset the timer).
Netty provides a HashedWheelTimer for efficiently managing lots of timers with a resolution suitable for this kind of activity.
You may also want to consider putting a limit on the number of retries so you can stop and raise an error rather than continually indefinitely.

Is this a more effective way to optimize my networking model?

To begin I will explain my networking model:
Networking in my game consists of pairing objects on the remote server and the client. To give a short description, say there are multiple characters in the server world that need to be synchronized with a client (i'll consider just one to simplify things)
Each time a character on the server-side is created, the server will instantiate a ServerRpgCharacter - this class wraps the RpgCharacter and registers observers etc to monitor the character and broadcast relevant mutations to the character. The server then requests a pair object for ServerRpgCharacter (that is, it requests the client to instantiate a pair for this object that will communicate with it.) The pair can be any class, but any messages dispatched by ServerRpgCharacter on the remote end will be received by its respective pair on the client end.
It gets a little more involved with multiple clients but it ends up working out nicely.
Anyway, I have been thinking of multiple ways to optimize this model. The way it works now is that when an object dispatches a message to its pair, it is queued up into a 'snapshot.' Whenever any paired entity dispatches a messages it is thrown into the same snapshot. The snapshot is then compressed and dispatched at intervals of 200ms.
The problem is that I am using the TCP/IP protocol to transmit these snapshot. I'm not sure exactly how the TCP protocol works, but I assume that if a snapshot's packet is dropped, the entire snapshot would have to be re-sent.
Thus I am wondering If it would be more optimal if I discarded TCP and instead implemented a custom layer overtop of UDP where instead of dispatching one whole snapshot for all messages sent by all pairs, I have it so that these individual pairs maintain their own packet ordering and buffering. This way, if a packet for pair A is dropped, pair B can ignore the fact that pair A had one if it's packets dropped.
I then need to consider that compressing this data is less efficient since less is being transmitted.

Compression with TCP is more efficient as you can compress using the context of the entire stream. You cant do that with UDP as you have to compress each packet individually.
The benefit of UDP is you can drop packets and not resend them because you assume a later packet will update the information quickly enough.
The interval needs to be shorter as 200 ms will be noticeable to users. 50 ms might be a better option.
instead of the normal listener pattern, you can use a spacial lookup. When an event occurs it is noticable to things on the same level and for a distance of X squares. This will save you have to maintain lots of lists as players and monsters move around.

RabbitMQ grouping messages as one message ie coalescing messages

I'm trying to understand the best way to coalesce or chunk incoming messages in RabbitMQ (using Spring AMQP or the Java client directly).
In other words I would like to take say 100 incoming messages and combine them as 1 and resend it to another queue in a reliable (correctly ACKed way). I believe this is called the aggregator pattern in EIP.
I know Spring Integration provides an aggregator solution but the implementation looks like its not fail safe (that is it looks like it has to ack and consume messages to build the coalesced message thus if you shutdown it down while its doing this you will loose messages?).

I can't comment directly on the Spring Integration library, so I'll speak generally in terms of RabbitMQ.
If you're not 100% convinced by the Spring Integration implementation of the Aggregator and are going to try to implement it yourself then I would recommend avoiding using tx which uses transactions under the hood in RabbitMQ.
Transactions in RabbitMQ are slow and you will definitely suffer performance problems if you're building a high traffic/throughput system.
Rather I would suggest you take a look at Publisher Confirms which is an extension to AMQP implemented in RabbitMQ. Here is an introduction to it when it was new http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/.
You will need to tweak the prefetch setting to get the performance right, take a look at http://www.rabbitmq.com/blog/2012/05/11/some-queuing-theory-throughput-latency-and-bandwidth/ for some details.
All the above gives you some background to help solve your problem. The implementation is rather straightforward.
When creating your consumer you will need to ensure you set it so that ACK is required.
Dequeue n messages, as you dequeue you will need to make note of the DeliveryTag for each message (this is used to ACK the message)
Aggregate the messages into a new message
Publish the new message
ACK each dequeued message
One thing to note is that if your consumer dies after 3 and before 4 has completed then those messages that weren't ACK'd will be reprocessed when it comes back to life

If you set the <amqp-inbound-channel-adapter/> tx-size attribute to 100, the container will ack every 100 messages so this should prevent message loss.
However, you might want to make the send of the aggregated message (on the 100th receive) transactional so you can confirm the broker has the message before the ack for the inbound messages.