Develop Reference

parallel stream with kafka consumer records

I have kafka records:
ConsumerRecords<String, Events> records = kafkaConsumer.poll(POLL_TIMEOUT);
I want to run the below code using parallel streams, not multithreading.
records.forEach((record) -> {
Event event = record.value();
HTTPSend.send(event);
});
I tried with mlutithreading but I want to try parallelstream:
for (ConsumerRecord<String, Event> record : records) {
executor.execute(new Runnable() {
#Override
public void run() {
HTTPSend.send(Event);
}
});
}
Actually I'm facing issue with HTTP.send with multithreading (even with a thread pool of 1 thread). I'm getting
"Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target".
This is a request over https. This error comes only for the first time the request is made. Afterwards, the exception vanishes. poof!
For multithreading i'm using:
int threadCOunt=1;
BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(threadCOunt, true);
RejectedExecutionHandler handler = new ThreadPoolExecutor.CallerRunsPolicy();
ExecutorService executor = new ThreadPoolExecutor(threadCOunt, threadCOunt, 0L, TimeUnit.MILLISECONDS, queue, handler);
HTTPSend.send() is:
long sizeSend = 0;
SSLContext sc = null;
try {
sc = SSLContext.getInstance("TLS");
sc.init(null, TRUST_ALL_CERTS, new SecureRandom());
} catch (NoSuchAlgorithmException | KeyManagementException e) {
LOGGER.error("Failed to create SSL context", e);
}
// Ignore differences between given hostname and certificate hostname
HostnameVerifier hv = (hostname, session) -> true;
// Create the REST client and configure it to connect meta
Client client = ClientBuilder.newBuilder()
.hostnameVerifier(hv)
.sslContext(sc).build();
WebTarget baseTarget = client.target(getURL()).path(HTTP_PATH);
Response jsonResponse = null;
try {
StringBuilder eventsBatchString = new StringBuilder();
eventsBatchString.append(this.getEvent(event));
Entity<String> entity = Entity.entity(eventsBatchString.toString(), MediaType.APPLICATION_JSON_TYPE);
builder = baseTarget.request();
LOGGER.debug("about to send the event {} and URL {}", entity, getURL());
jsonResponse = builder.header(HTTP_ACK_CHANNEL, guid.toString())
.header("Content-type", MediaType.APPLICATION_JSON)
.header("Authorization", String.format("Meta %s", eventsModuleConfig.getSecretKey()))
.post(entity);

I see what you want to do, and I'm not sure that's the best idea (I'm also not sure it's not).
The poll / commit model of Kafka allows simple backpressure and retention of the last item processed if you crashed. By returning to your poll loop "immediately" you are telling Kafka "I am ready for more", and committing the offset (manually or automatically) tells Kafka that you have successfully read up to that point.
What you seem to want to do is read off Kafka as fast as possible, committing offsets, then putting the Kafka records into an executor queue then you balance your requests per second etc from that.
I'm not 100% sure that's a good idea: what happens if your app crashes? You may have committed some Kafka messages that actually didn't make it upstream. If you do really want to do this, I would suggest manually committing the offset (via commitSync) upon completion of the Runnable, instead of letting the high level consumer do it for you.
Why might you want to use a thread executor: I think these can be accomplished with Kafka too.
You may want to post multiple messages to the web server at the same time. A well paritioned Kafka topic will let multiple consumers / consumer groups consumer multiple partitions, thus - assuming a perfectly scaling HTTP server - would let you parallelize the posting of messages to your server. Yay for process based concurrency!
Maybe the web server is not perfectly scalable, or slow for this request (say each request takes 1 second): you need to limit the number of requests per second the web server takes, if you have a queue you might have a couple threads posting while not backing up Kafka.
In this case you can set max.poll.records to a scalable value that your web server requires. There's probably a better way to do this too, although it's escaping me at the moment.
If your web server takes a long time to respond you may get errors related to failing heartbeats. In that case I direct you to this SO answer on the timeout / heartbeat topic.
Instead of using a thread executioner, thus making synchronous HTTP requests appear to be async, I would use an evented HTTP client like Netty, thus achieving parallelism without thread based concurrency.

For solving a "slow consumer" use case where you're doing I/O processing, you should use something like Parallel Consumer (PC) to avoid the "head of line blocking" problem you're describing.
By using PC, you can processing all your keys in parallel, regardless of how long it takes to do your I/O.
It also comes with a non blocking Vert.x module which more efficiently uses the CPU.
PC directly solves for this, by sub partitioning the input partitions by key and processing each key in parallel.
It also tracks per record acknowledgement. Check out Parallel Consumer on GitHub (it's open source BTW, and I'm the author).

How can I use an EventExecutorGroup with an NioDatagramChannel in Netty?

I'm new to Netty, but I've worked with NIO a bit recently.
I'm building a UDP server, and it seems from the examples I've tried that unlike the NioServerSocketChannel.class, when you use the NioDatagramChannel.class you cannot assign an EventLoopGroup workerGroup to handle the Datagrams once they've been accepted by the socket. My code, based on the Netty QuoteOfTheMoment server example looks like this:
public class PositionServer {
int port;
public PositionServer(int port) {
this.port = port;
System.out.println("Port set to " + this.port);
}
public void run() {
EventLoopGroup bossGroup = new NioEventLoopGroup();
try {
Bootstrap bootstrap = new Bootstrap();
bootstrap.group(bossGroup).channel(NioDatagramChannel.class)
.handler(new PositionServerHandler());
System.out.println("Binding to port " + port);
bootstrap.bind(port).sync().channel().closeFuture().await();
} catch (InterruptedException e) {
bossGroup.shutdownGracefully();
e.printStackTrace();
}
}
public static void main(String[] args) {
new PositionServer(4000).run();
}
}
This works perfectly fine, right now the PositionServerHandler simply outputs the messages to the console.
I want to push the messages into a database, they're JSON strings, so I'd like to convert them to JSON first. I don't want to do the in the bossGroup as it'll block with the database accessing and JSON processing. But I can't see an obvious way to add a workerGroup.
Googling didn't reveal much either, except that my workerGroup in this case should probably be an EventExecutorGroup and not an EventLoopGroup, but I'm not even 100% certain about that.
In NIO, I would have two thread pools, and I'd use some sort of queue, one pool of threads would push Strings to the queue, the other would take Strings from the queue, convert them to JSON objects and push them to the database.
Can I do something similar in Netty?
In which case my PositionServerHandler.class would accept a queue in the constructor, and push either whole DatagramPackets to it, or the message contents, then have another EventLoopGroup/EventExecutorGroup to take those messages and push them to the database.
It seems that ServerBootstrap doesn't handle NioDatagramChannels for some reason, that doesn't make sense to me.
Is this feasible? Am I missing some obvious solution to this?

Can I do something similar in Netty?
Yes. But first a clarification of terminology. In typical TCP server samples, the "boss group" is the event loop group that does TCP accept. The "worker group" is the event loop group that does TCP receive. Since UDP sockets do not "accept", but they do receive, the distinction between boss group and worker group is meaningless, and in the cited example, the variable is just called "group". It performs the same function as "worker group" in a typical TCP server, namely, it processes the incoming data.
Keep in mind that a single NioEventLoopGroup contains a thread pool which can be configured to have any number of threads.
In NIO, I would have two thread pools, and I'd use some sort of queue...
You can use the same design. You are still using NIO after all - you just have Netty to help you. The NioEventLoop "worker" group is your "first" thread pool which pushes Strings to the queue. Create your second thread pool any way you want, either using the standard Java APIs or using a second netty event loop group which is not attached to any channel. Everything else remains the same.

Why does async-http-client does not throttle my requests?

I have an Akka actor that owns an AsyncHttpClient. This actor must handles a lot of asynchronous requests. Because my system cannot handle thousands of requests simultaneously, I need to limit the number of concurrent requests.
Right now, I'm doing this :
AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder().setAllowPoolingConnection(true)
.addRequestFilter(new ThrottleRequestFilter(32))
.setMaximumConnectionsPerHost(16)
.setMaxRequestRetry(5)
.build();
final AsyncHttpClient httpClient = new AsyncHttpClient(new NettyAsyncHttpProvider(config));
When my actor receives a message, I use the client like this :
Future<Integer> f = httpClient.prepareGet(url).execute(
new AsyncCompletionHandler<Integer>() {
#Override
public Integer onCompleted(Response response) throws Exception {
// handle successful request
}
#Override
public void onThrowable(Throwable t){
// handle failed request
}
}
);
The problem is that requests are never put in the client queue and are all processed like the configuration doesn't matter. Why doesn't this work as it should?

From the maintainer:
setMaxConnectionsPerHost only caps the number of connections that can be open to a given host. There's no built-in queuing mechanism for requests that might need a connection while there's none available.
So basically, it's a hard limit. Also, in versions of the library prior to, I believe, 1.9.10, the maximumConnectionsPerHost field was not being properly utilized by the code to limit the number of concurrent connections per host. Instead, there was a bug where the client only looked at the maximumConnectionsTotal field.
Link to issue referenced on GitHub

Netty: how do I reduce delay between consecutive messages from the server?

I'm on the dev team for a socket server which uses Netty. When a client sends a request, and the server sends a single response, the round trip time is quite fast. (GOOD) We recently noticed that if the request from the client triggers two messages from the server, even though the server writes both messages to the client at about the same time, there is a delay of more than 200ms between the first and second message arriving on the remote client. When using a local client the two messages arrive at the same time. If the remote client sends another request before the second message from the server arrives, that second message is sent immediately, but then the two messages from the new request are both sent with the delay of over 200ms.
Since it was noticed while using Netty 3.3.1, I tried upgrading to Netty 3.6.5 but I still see the same behavior. We are using NIO, not OIO, because we need to be able to support large numbers of concurrent clients.
Is there a setting that we need to configure that will reduce that 200+ ms delay?
editing to add a code snippet. I hope these are the most relevant parts.
#Override
public boolean openListener(final Protocol protocol,
InetSocketAddress inetSocketAddress) throws Exception {
ChannelFactory factory = new NioServerSocketChannelFactory(
Executors.newCachedThreadPool(),
Executors.newCachedThreadPool(),
threadingConfiguration.getProcessorThreadCount());
ServerBootstrap bootstrap = new ServerBootstrap(factory);
final ChannelGroup channelGroup = new DefaultChannelGroup();
bootstrap.setPipelineFactory(new ChannelPipelineFactory() {
.... lots of pipeline setup snipped ......
});
Channel channel = bootstrap.bind(inetSocketAddress);
channelGroup.add(channel);
channelGroups.add(channelGroup);
bootstraps.add(bootstrap);
return true;
}
The writer factory uses ChannelBuffers.dynamicBuffer(defaultMessageSize) for the buffer, and when we write a message it's Channels.write(channel, msg).
What else would be useful? The developer who migrated the code to Netty is not currently available, and I'm trying to fill in.

200ms strikes me as the magic number of the Nagle's algorithm. Try setting the TcpNoDelay to true on both sides.
This is how you set the option for the server side.
serverBootstrap.setOption("child.tcpNoDelay", true);
This is for the client side.
clientBootStrap.setOption("tcpNoDelay", true);
Further reading: http://www.stuartcheshire.org/papers/NagleDelayedAck/

How does the Netty threading model work in the case of many client connections?

I intend to use Netty in an upcoming project. This project will act as both client and server. Especially it will establish and maintain many connections to various servers while at the same time serving its own clients.
Now, the documentation for NioServerSocketChannelFactory fairly specifies the threading model for the server side of things fairly well - each bound listen port will require a dedicated boss thread throughout the process, while connected clients will be handled in a non-blocking fashion on worker threads. Specifically, one worker thread will be able to handle multiple connected clients.
However, the documentation for NioClientSocketChannelFactory is less specific. This also seems to utilize both boss and worker threads. However, the documentation states:
One NioClientSocketChannelFactory has one boss thread. It makes a connection attempt on request. Once a connection attempt succeeds, the boss thread passes the connected Channel to one of the worker threads that the NioClientSocketChannelFactory manages.
Worker threads seem to function in the same way as for the server case too.
My question is, does this mean that there will be one dedicated boss thread for each connection from my program to an external server? How will this scale if I establish hundreds, or thousands of such connections?
As a side note. Are there any adverse side effects for re-using a single Executor (cached thread pool) as both the bossExecutor and workerExecutor for a ChannelFactory? What about also re-using between different client and/or server ChannelFactory instances? This is somewhat discussed here, but I do not find those answers specific enough. Could anyone elaborate on this?

This is not a real answer to your question regarding how the Netty client thread model works. But you can use the same NioClientSocketChannelFactory to create single ClientBootstrap with multiple ChannelPipelineFactorys , and in turn make a large number of connections. Take a look at the example below.
public static void main(String[] args)
{
String host = "localhost";
int port = 8090;
ChannelFactory factory = new NioClientSocketChannelFactory(Executors
.newCachedThreadPool(), Executors.newCachedThreadPool());
MyHandler handler1 = new MyHandler();
PipelineFactory factory1 = new PipelineFactory(handler1);
AnotherHandler handler2 = new AnotherHandler();
PipelineFactory factory2 = new PipelineFactory(handler2);
ClientBootstrap bootstrap = new ClientBootstrap(factory);
// At client side option is tcpNoDelay and at server child.tcpNoDelay
bootstrap.setOption("tcpNoDelay", true);
bootstrap.setOption("keepAlive", true);
for (int i = 1; i<=50;i++){
if(i%2==0){
bootstrap.setPipelineFactory(factory1);
}else{
bootstrap.setPipelineFactory(factory2);
}
ChannelFuture future = bootstrap.connect(new InetSocketAddress(host,
port));
future.addListener(new ChannelFutureListener()
{
#Override
public void operationComplete(ChannelFuture future) throws Exception
{
future.getChannel().write("SUCCESS");
}
});
}
}
It also shows how different pipeline factories can be set for different connections, so based on the connection you make you can tweak your encoders/decoders in the channel pipeline.

I am not sure your question has been answer. Here's my answer: there's a single Boss thread that is managing simultaneously all the pending CONNECTs in your app. It uses nio to process all the current connects in a single (Boss) thread, and then hands each successfully connected channel off to one of the workers.

Your question mainly concerns performance. Single threads scale very well on the client.
Oh, and nabble has been closed. You can still browse the archive there.