Develop Reference

Netty: How to limit websocket channel messages per second?

I need to limit messages received on websocket channel per second for netty server.
Could'n find any ideas how to do that.
Any ideas would be appreciated
Thank you

You need to add simple ChannelInboundHandlerAdapter handler to your pipeline and add the simple counter to channelRead(ChannelHandlerContext ctx, Object msg) method. I would recommend you to use some of CodaHale Metrics Class for that purpose.
Pseudo code:
private final QuotaLimitChecker limitChecker;
public MessageDecoder() {
this.limitChecker = new QuotaLimitChecker();
}
#Override
public void channelRead(ChannelHandlerContext ctx, Object msg) {
if (limitChecker.quotaReached(100)) { //assume limit is 100 req per sec
return;
}
}
Where QuotaLimitChecker is a class that increments counter and checks if the limit is reached.
public class QuotaLimitChecker {
private final static Logger log = LogManager.getLogger(QuotaLimitChecker.class);
private final int userQuotaLimit;
//here is specific implementation of Meter for your needs
private final InstanceLoadMeter quotaMeter;
public QuotaLimitChecker(int userQuotaLimit) {
this.userQuotaLimit = userQuotaLimit;
this.quotaMeter = new InstanceLoadMeter();
}
public boolean quotaReached() {
if (quotaMeter.getOneMinuteRate() > userQuotaLimit) {
log.debug("User has exceeded message quota limit.");
return true;
}
quotaMeter.mark();
return false;
}
}
Here is my implementation of QuotaLimitChecker that uses the simplified version Meter class of CodaHale Metrics library.

Strategy for multiple subscribers and flow of data using RxJava 2

I cannot decide how to implement this task correctly using RxJava2.
The problem is following. I am recording audio using AuidoRecord.
Currently I have implemented the custom Flowable class like that
private class StreamAudioRecordRunnable extends Flowable<short[]> implements Runnable {
private int mShortBufferSize;
private List<Subscriber<? super short[]>> mSubscribers = new ArrayList<>();
private short[] mAudioShortBuffer;
private void removeAllNullableSubscribers() {
mSubscribers.removeAll(Collections.singleton(null));
}
private void notifyAllSubscribers(short[] audioBuffer) {
removeAllNullableSubscribers();
for (Subscriber<? super short[]> subscriber : mSubscribers) {
subscriber.onNext(audioBuffer);
}
}
#Override
protected void subscribeActual(Subscriber<? super short[]> newSubscriber) {
mSubscribers.add(newSubscriber);
}
private void notifyAllSubscribersAboutError(Throwable error) {
for (Subscriber<? super short[]> subscriber : mSubscribers) {
subscriber.onError(error);
}
}
#Override
public void run() {
// Init stuff
while (mIsRecording.get()) {
int ret;
ret = mAudioRecord.read(mAudioShortBuffer, 0, mShortBufferSize);
notifyAllSubscribers(mAudioShortBuffer);
}
mAudioRecord.release();
}
}
As you can see I am manually adding subscribers to the list. Then when I get new buffer all subscribers are notified.
I am guessing that this is not the most performant way to do this.
What I need
As far as this flowable running in a service. It should run until the service is alive, even if there are no subscribers.
Subscribers are not constant, they may subscribe and then unsubscribe, but the Flowable/Observable should still be running.
As the data emitted by the Flowable is the stream, subscribers should not be notified about already emitted items, they should only get current streaming data. Fire and forget.
The Flowable should run even all subscribers are gone.
Please suggest the right strategy to implement this.
I would be grateful for any help.

Something like
public class StreamAudioRecordRunnable {
private int mShortBufferSize;
private short[] mAudioShortBuffer;
private ConnectedFlowable<short[]> audioFlowable();
public StreamAudioRecordRunnable() {
audioFlowable = Flowable.create(new ObservableOnSubscribe<short[]>() {
#Override
public void subscribe(FlowableEmitter<short[]> emitter) throws Exception {
try {
while (mIsRecording.get()) {
int ret;
ret = mAudioRecord.read(mAudioShortBuffer, 0, mShortBufferSize);
emitter.onNext(mAudioShortBuffer);
}
emitter.onComplete();
mAudioRecord.release();
} catch (Exception e) {
emitter.onError(e);
mAudioRecord.release();
}
}
}).subscribeOn(Schedulers.io()).publish();
}
public Flowable<short[]> getFlowable() {
return audioFlowable.hide();
}
#Override
public void start() {
audioObservable.connect();
}
}
would be my preference.

Callbacks in Java, similar to Javascript

I'm trying to write a callback system in Java that works similar to that of Javascripts, what I'm doing is I'm sending information across the network that has a "callback id" attached to it. When the client receives this data back from the server, it should locate the callback for that id form a collection and call it with the retrieved data.
Here's the current system I've written up while trying to achieve this:
public class NetworkCallback {
private int id;
private Callable callback;
public NetworkCallback(Callable callback) {
this.callback = callback;
}
public NetworkCallback setId(int id) {
this.id = id;
return this;
}
public int getId() {
return id;
}
public boolean execute(JSONObject data) {
try {
callback.call(); // data?
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
}
These were stored in a special container a created that would retain index, it's basically just an array with some helper classes. When the client gets information back it will search for the callback and then execute it.
void onMessageReceived(byte[] data) {
JSONObject json = JSONHelper.fromByteArray(data);
int callbackId = json.getInt("cbid");
if(callbackId != 0) {
callbacks.fetch(callbackId).execute(json);
}
}
The issue with this I noticed before even attempting to run the code, pondered for awhile, and ran out of things to think about. The callable class doesn't accept parameters. So, for example, say I wanted to pass a method as a callback like the following:
psuedo
method(param JSONObject data) {
print data
}
Granted this method isn't going to be the same every time it's called, so it will be created on the fly. An example in javascript of what I'm trying to achieve can be found below:
Javascript example of what I want
(function caller() {
called(function(data) {
console.log("Data: " + data);
});
})();
function called(callback) {
callback(Math.random());
}

You will want to use a Consumer for this. A consumer basically is an object on which you can call accept(data), which executes the callback.
An example:
public class Test {
public static void main(String[] args) {
Consumer consumer = new Consumer() {
#Override
public void accept(Object o) {
System.out.println(o.toString());
}
};
new Test().doSomething("Test", consumer);
}
public void doSomething(Object data, Consumer<Object> cb) {
cb.accept(data);
}
}
This prints "Test" in the console.

Encoding and Decoding multiple different types with Netty 5

I am trying to implement a network protocol with multiple different packet types. The problem I am facing is the most "proper" way of implementing this with Netty. I'll post some classes first, and then describe what I want to accomplish.
public class ItemStack {
public ItemStack(final int item, final int amount) {
if (item < 0) {
throw new IllegalArgumentException("item must be non-negative integer: " + item);
}
if (amount < 1) {
throw new IllegalArgumentException("amount must be positive integer: " + amount);
}
this.item = item;
this.amount = amount;
}
public int getItem() {
return item;
}
public int getAmount() {
return amount;
}
private final int item;
private final int amount;
}
public class ChatMessage {
public ChatMessage(final String playerName, final String message) {
if (playerName == null) {
throw new IllegalArgumentException("playerName must not be null");
}
if (message == null) {
throw new IllegalArgumentException("message must not be null");
}
this.playerName = playerName;
this.message = message;
}
public String getPlayerName() {
return playerName;
}
public String getMessage() {
return message;
}
private final int playerName;
private final int message;
}
Now, all POJO that are transmitted across the network will have a packet identifier. This will be a 1-byte code that will let the decoder know what type of packet it is, and how to decode it.
What would be the most appropriate way of handling this case? Would it be better (read more conventional) to have one PacketDecoder class that extends the ByteToMessageDecoder which reads one byte, determines the type, and then in that same class, decode the packet with the appropriate method like so:
public class PacketDecoder extends ByteToMessageDecoder {
protected void decode(
final ChannelHandlerContext context, final ByteBuf buf, List<Object> objects) throws Exception {
if (buf.readableBytes < 1) {
return;
}
final int opcode = buf.readByte() & 0xff;
final Packet packet = decodePacket(opcode);
objects.add(packet);
}
private Packet decodePacket(final int opcode, final ByteBuf buf) {
if (buf == null) {
throw new IllegalArgumentException("buf must not be null");
}
Packet packet = null;
switch (opcode) {
case 0:
packet = decodeItemStack(buf);
break;
case 1:
packet = decodeChatMessage(buf);
break;
// ...
}
return packet;
}
}
Or would it be better to just add every type of decoder to the pipeline?

I did this in my own program and I used a single decoder because it was more straight forward thing to do. The only reason I could see for wanting multiple decoders would be if need to extend or dynamically changing the protocol your server understands. For example, maybe certain aspects of your server are free and others are paid for extensions that license key turns on then I could see this architecture being important. Or you load extensions to the protocol dynamically maybe. I think you need a real reason to segment decoding into several decoders besides it being architecturally pure.
In this case you can add multiple decoders to the pipeline, but each decoder needs to play nice and forward packets not meant for it along to the next decoder in the pipeline. You also have to be careful not to pull bytes off that down stream decoders might need.
Here is what I wouldn't do. A decoder per message architecture. That will be cumbersome to write and maintain. Since there is overhead with each decoder written to play nice and forward packets I wouldn't go through that exercise each time I write a decoder. You could overcome that with a nice base class to extend, but why go through all that hassle when you can just parse the first byte and do a simple if ladder?

java events and listeners, bad implementation?

I am currently implementing custom events and listeners according to the code posted below. I have been told that this is a very dirty implementation and that this needs to be changed. However, i am very new to java and android and do not see what is wrong with the current implementation. The way i have it below works and seems to be doing everything i needed it too. I was wondering if some people could please take a look at my code and make some suggestions on what i should change and what i am doing wrong. Taking my example and modifying it so that i can see what your talking about would be greatly appreciated.
Thanks in advance!
/* SmartApp.java */
public class SmartApp extends Activity
{
private ConnectDevice cD = new ConnectDevice();
private DataRobot dR = new DataRobot();
private DataBuilder dB = new DataBuilder();
private DataSender dS = new DataSender();
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.intro);
cD.addDataReceivedListener(new DataReceivedListener() {
#Override
public void dataReceivedReceived(DataReceivedEvent event) {
// TODO Auto-generated method stub
dR.analyzeData(event.getData());
}
});
dR.addDataAnalyzedListener(new DataAnalyzedListener() {
#Override
public void dataAnalyzedReceived(DataAnalyzedEvent event) {
// TODO Auto-generated method stub
dB.submitData(event.getData());
}
});
dB.addDataBuilderListener(new DataBuilderListener() {
#Override
public void dataBuilderReceived(DataBuilderEvent event) {
// TODO Auto-generated method stub
dS.sendData(event.getData());
}
});
}
}
/* ConnectDevice.java
* This class is implementing runnable because i have a thread running that is checking
* the contents of a socket. Irrelevant to events. */
public class ConnectDevice implements Runnable {
private List _listeners = new ArrayList();
private String data;
/* Constructor */
public ConnectDevice() {// does some socket stuff here, irrelevant to the events}
public void run() {// does some socket stuff here, irrelevant to the events}
public synchronized void addDataReceivedListener(DataReceivedListener listener) {
_listeners.add(listener);
}
public synchronized void removeDataReceivedListener(DataReceivedListener listener) {
_listeners.remove(listener);
}
private synchronized void fireDataReceivedEvent(String temp) {
DataReceivedEvent dRE = new DataReceivedEvent(this, temp);
Iterator listeners = _listeners.iterator();
while(listeners.hasNext()) {
((DataReceivedListener)listeners.next()).dataReceivedReceived(dRE);
}
}
public interface DataReceivedListener {
public void dataReceivedReceived(DataReceivedEvent event);
}
}
/* DataRobot.java */
public class DataRobot {
/* This class is for analyzing the data */
private List _listeners = new ArrayList();
private String data;
public boolean analyzeData(String temp) {
/* Analyze the data
* This function analyzes the data, as explained in the OP
* This function fires the analyzed data event when finished
* analyzing the data.
*/
data = temp;
fireDataAnalyzedEvent(data); // this fires the dataanalyzedevent
return true; //for now this will always return true
}
public synchronized void addDataAnalyzedListener(DataAnalyzedListener listener) {
_listeners.add(listener);
}
public synchronized void removeDataAnalyzedListener(DataAnalyzedListener listener) {
_listeners.remove(listener);
}
private synchronized void fireDataAnalyzedEvent(String temp) {
DataAnalyzedEvent dRE = new DataAnalyzedEvent(this, temp);
Iterator listeners = _listeners.iterator();
while(listeners.hasNext()) {
((DataAnalyzedListener)listeners.next()).dataAnalyzedReceived(dRE);
}
}
public interface DataAnalyzedListener {
public void dataAnalyzedReceived(DataAnalyzedEvent event);
}
}
/* DataBuilder.java */
public class DataBuilder {
private List _listeners = new ArrayList();
private String data;
public boolean submitData(String temp) {
/* Builds the data
* This function builds the data, as explained in the OP
* This function fires the databuilder data event when finished
* building the data.
*/
data = temp;
fireDataBuilderEvent(data); //firing the databuilder event when finished
return true;
}
public synchronized void addDataBuilderListener(DataBuilderListener listener) {
_listeners.add(listener);
}
public synchronized void removeDataBuilderListener(DataBuilderListener listener) {
_listeners.remove(listener);
}
private synchronized void fireDataBuilderEvent(String temp) {
DataBuilderEvent dRE = new DataBuilderEvent(this, temp);
Iterator listeners = _listeners.iterator();
while(listeners.hasNext()) {
((DataBuilderListener)listeners.next()).dataBuilderReceived(dRE);
}
}
public interface DataBuilderListener {
public void dataBuilderReceived(DataBuilderEvent event);
}
}
/* DataSender.java */
/* this class has no event, because it is done firing events at this point */
public class DataSender {
private String data;
public boolean sendData(String temp) {
data = temp;
return true;
}
}
Below here are the event objects for each event. I Have each of this defined in a separate file, not sure if that is good procedure or not.
/* DataReceivedEvent.java */
public class DataReceivedEvent extends EventObject{
private String data;
public DataReceivedEvent(Object source, String temp) {
super(source);
// TODO Auto-generated constructor stub
data = temp;
}
public String getData() {
// this function is just an accessor function
return data;
}
}
/* DataAnalyzedEvent.java */
public class DataAnalyzedEvent extends EventObject{
private String data;
public DataAnalyzedEvent(Object source, String temp) {
super(source);
// TODO Auto-generated constructor stub
data = temp;
}
public String getData() {
// this function is just an accessor function
return data;
}
}
/* DataBuilderEvent.java */
public class DataBuilderEvent extends EventObject {
private String data;
public DataBuilderEvent(Object source, String temp) {
super(source);
// TODO Auto-generated constructor stub
data = temp;
}
public String getData() {
// this function is just an accessor function
return data;
}
}

I would not say it is a "very dirty implementation". Using callbacks/observers/listeners is a good practice in my opinion.
When I write Android applications I like to layer it such that the "application" is plain old Java with no Android imports and could theoretically be used in a Swing app, a Java EE-based web site, etc. The "Android" part is strictly user interface.
What I use callbacks for is to allow the Android code to register interest in events that take place in the application. For example, in a Blackjack game, an Activity might call game.getDealer().playHand() to tell the application to perform the dealer hand play logic. As that logic executes in the application, events are fired like cardDrawn(card), cardFlipped(card), handTotalChanged(handTotal), etc. The Android part of the app listens to these and redraws things on the screen accordingly (but it knows nothing about Blackjack).
I actually just have my activities implement interfaces like CardListener, HandListener, etc. so they can receive the event directly (unlike how you do it), but your style isn't necessarily a bad way.

I agree with #SingleShot in theory, for the parts of your Android application that can be Android-agnostic, and so long as the overhead introduced by all the indirection layers does not slow the app down too much. IMHO, in many apps, there is relatively little that fits this description.
In another post, you proposed your above solution for one activity to communicate to another activity. In Android, activities aren't just some Java objects you can toss around willy-nilly. They are managed by the OS and have particular lifecycles. While the observer/observable pattern is quite delightful in some places, it is unsuitable where the observer/observable connection will create garbage collection problems. In particular, one activity cannot, and should not, be trying to hold some sort of listener interface on another activity.
Similarly, a clean observer/observable pattern may break down in the face of databases, threads, services, and other bits of Android reality.
So, in pure Java code, isolated from Android, what you have is probably OK. However, do not go around recommending it as solutions for Android-specific problems unless you know it will work for those Android-specific problems. And, when you start trying to make your code work in an Android app, please do not be shocked if you run into problems trying to make your textbook pattern implementation work within the constraints placed upon Android apps.