I'm using hazel cast IMGD for my app. I have used queues for internal communication. I added an item listener to queue and it works great. Whenever a queue gets a message, listener wakes up and needed processing is done.
Problem is its single threaded. Sometimes, a message takes 30 seconds to process and messages in queue just have to wait until previous message is done processing. I'm told to use Java executor service to have a pool of threads and add an item listener to every thread so that multiple messages can be processed at same time.
Is there any better way to do it ? may be configure some kind of MDB or make the processing asynchronous so that my listener can process the messages faster
#PostConstruct
public void init() {
logger.info(LogFormatter.format(BG_GUID, "Starting up GridMapper Queue reader"));
HazelcastInstance hazelcastInstance = dc.getInstance();
queue = hazelcastInstance.getQueue(FactoryConstants.QUEUE_GRIDMAPPER);
queue.addItemListener(new Listener(), true);
}
class Listener implements ItemListener<QueueMessage> {
#Override
public void itemAdded(ItemEvent<QueueMessage> item) {
try {
QueueMessage message = queue.take();
processor.process(message.getJobId());
} catch (Exception ex) {
logger.error(LogFormatter.format(BG_GUID, ex));
}
}
#Override
public void itemRemoved(ItemEvent<QueueMessage> item) {
logger.info("Item removed: " + item.getItem().getJobId());
}
}
Hazelcast IQueue does not support asynchronous interface. Anyway, asynchronous access would not be faster. MDB requires JMS, which is pure overhead.
What you really need is multithreaded executor. You can use default executor:
private final ExecutorService execService = ForkJoinPool.commonPool();
Related
I have Kafka Produce which sends the message to kafka .And i log the message in database in the both onsucess and onFailure with the help stored procedure . As shown in the code i am using asynchronous
should i mark my callStoredProcedure method in the repository as synchronised to avoid deadlocks? i believe synchronised is not needed as callback will be executed sequentially in a single thread.
from the below link
https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html
Note that callbacks will generally execute in the I/O thread of the
producer and so should be reasonably fast or they will delay the
sending of messages from other threads. If you want to execute
blocking or computationally expensive callbacks it is recommended to
use your own Executor in the callback body to parallelize processing.
Should i execute callbacks in other thread ?
And can u share the code snippet how to excute callback in other thread. like parallelise callback in 3 threads
My code snippet
#Autowired
private Myrepository myrepository;
public void sendMessageToKafka(List<String> message) {
for (String s : message) {
future = kafkaTemplate.send(topicName, message);
future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
#Override
public void onSuccess(SendResult<String, String> result) {
System.out.println("Message Sent " + result.getRecordMetadata().timestamp());
myrepository.callStoredProcedure(result,"SUCCESS");
}
#Override
public void onFailure(Throwable ex) {
System.out.println(" sending failed ");
myrepository.callStoredProcedure(result,"FAILED");
}
});
}
private final ExecutorService exec = Executors.newSingleThreadExecutor();
...
this.exec.submit(() -> myrepository.callStoredProcedure(result,"SUCCESS"));
The tasks will still be run on a single thread (but not the Kafka IO thread).
If it can't keep up with your publishing rate, you might need to use a different executor such as a cached thread pool executor or Spring's ThreadPoolTaskExecutor.
I am familiar with Netty basics and have used it to build a typical application server running on TCP designed to serve many clients/connections. However, I recently have a requirement to build a server which is designed to handle handful of clients or only one client most of the times. But the client is the gateway to many devices and therefore generate substantial traffic to the server I am trying to design.
My questions are:
Is it possible / recommended at all to use Netty for this use case? I have seen the discussion here.
Is it possible to use multithreaded EventExecutor to the channel handlers in the pipeline so that instead of channel EventLoop, the concurrency is achieved by the EventExecutor thread pool? Will it ensure that one message from the client will be handled by one thread through all handlers, while the next message by another thread?
Is there any example implementation available?
According to the documentation of io.netty.channel.oio you can use it if you don't have lot's of client. In this case, every connection will be handled in a separate thread and use Java old blocking IO under the hood. Take a look at OioByteStreamChannel::activate:
/**
* Activate this instance. After this call {#link #isActive()} will return {#code true}.
*/
protected final void activate(InputStream is, OutputStream os) {
if (this.is != null) {
throw new IllegalStateException("input was set already");
}
if (this.os != null) {
throw new IllegalStateException("output was set already");
}
if (is == null) {
throw new NullPointerException("is");
}
if (os == null) {
throw new NullPointerException("os");
}
this.is = is;
this.os = os;
}
As you can see, the oio Streams will be used there.
According to your comment. You can Specify EventExecutorGroup while adding handler to a pipeline as this:
new ChannelInitializer<Channel> {
public void initChannel(Channel ch) {
ch.pipeline().addLast(new YourHandler());
}
}
Let's take a look at the AbstractChannelHandlerContext:
#Override
public EventExecutor executor() {
if (executor == null) {
return channel().eventLoop();
} else {
return executor;
}
}
Here we see that if you don't register your EventExecutor it will use the child event group you specified while creating the ServerBootstrap.
new ServerBootstrap()
.group(new OioEventLoopGroup(), new OioEventLoopGroup())
//acceptor group //child group
Here is how reading from channel is invoked AbstractChannelHandlerContext::invokeChannelRead:
static void invokeChannelRead(final AbstractChannelHandlerContext next, Object msg) {
final Object m = next.pipeline.touch(ObjectUtil.checkNotNull(msg, "msg"), next);
EventExecutor executor = next.executor();
if (executor.inEventLoop()) {
next.invokeChannelRead(m);
} else {
executor.execute(new Runnable() { //Invoked by the EventExecutor you specified
#Override
public void run() {
next.invokeChannelRead(m);
}
});
}
}
Even for a few connections I would go with NioEventLoopGroup.
Regarding your question:
Is it possible to use multithreaded EventExecutor to the channel
handlers in the pipeline so that instead of channel EventLoop, the
concurrency is achieved by the EventExecutor thread pool? Will it
ensure that one message from the client will be handled by one thread
through all handlers, while the next message by another thread?
Netty's Channel guarantees that every processing for an inbound or an outbound message will occur in the same thread. You don't have to hack an EventExecutor of your own to handle this. If serving inbound messages doesn't require long-lasting processings your code will look like basic usage of ServerBootstrap. You might find useful to tune the number of threads in the pool.
I need to send a email during registration process , so for this reason i am using Java Mail API , this is working fine , but observed that
the email process is taking nearly 6 seconds (which is too long ) so Ajax call making the user wait too long for response
so for this reason i have decided to use background thread for sending email so the user need not wait for the Ajax call response (Jersey REST Web Service call)
My question is it a good practice to creating threads in a webapplication for every request ??
#Path("/insertOrUpdateUser")
public class InsertOrUpdateUser {
final static Logger logger = Logger.getLogger(InsertOrUpdateUser.class);
#GET
#Consumes("application/text")
#Produces("application/json")
public String getSalesUserData(#QueryParam(value = "empId") String empId
)
throws JSONException, SQLException {
JSONObject final_jsonobject = new JSONObject();
ExecutorService executorService = Executors.newFixedThreadPool(10);
executorService.execute(new Runnable() {
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed",e);
}
}
});
}
} catch (SQLException e) {
} catch (Exception e) {
}
finally {
}
return response;
}
}
And this is my Utility class for sending email
public class SendEmailUtility
{
public static String sendmail(String sendto)
throws IOException
{
String result = "fail";
Properties props_load = getProperties();
final String username = props_load.getProperty("username");
final String password = props_load.getProperty("password");
Properties props_send = new Properties();
props_send.put("mail.smtp.auth", "true");
props_send.put("mail.smtp.starttls.enable", "true");
props_send.put("mail.smtp.host", props_load.getProperty("mail.smtp.host"));
props_send.put("mail.smtp.port", props_load.getProperty("mail.smtp.port"));
Session session = Session.getInstance(props_send,
new javax.mail.Authenticator() {
#Override
protected PasswordAuthentication getPasswordAuthentication()
{
return new PasswordAuthentication(username, password);
}
});
try {
Message message = new MimeMessage(session);
message.setFrom(new InternetAddress(props_load.getProperty("setFrom")));
message.setRecipients(Message.RecipientType.TO, InternetAddress.parse(sendto));
message.setText("Some Text to be send in mail");
Transport.send(message);
result = "success";
} catch (MessagingException e) {
result = "fail";
logger.error("Exception Occured - sendto: " + sendto, e);
}
return result;
}
}
Could you please let me know if this is best practice to do in a web application ??
There are host of ways you can handle it, so it all depends on whether your application server has that much resources (memory, threads etc.) to handle your implementation, so it makes you best person to decide on which approach to go.
As such it is not bad practice to spawn parallel threads for doing something if it is justified by design, but typically you should go with controlled threads.
Please note that whether you use newSingleThreadExecutor() or newFixedThreadPool(nThreads), under-the-hoods there will always be a ThreadPoolExecutor object created.
My recommendation will be to use seconds option in below list i.e. "Controlled number of threads", and in that specify max thread count as you see fir.
One thread for each request
In this approach one thread will be created for each incoming request from GUI, so if you are getting 10 requests for inserting/updating user then 10 threads will be spawned which will send emails.
Downside of this approach is that there is no control on number of threads so you can end with StackOverflowException or may be memory issue.
Please make sure to shutdown your executor service else you will end up wasting JVM resources.
// inside your getSalesUserData() method
ExecutorService emailExecutor = Executors.newSingleThreadExecutor();
emailExecutor.execute(new Runnable() {
#Override
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed", e);
}
}
});
emailExecutor.shutdown(); // it is very important to shutdown your non-singleton ExecutorService.
Controlled number of threads
In this approach, some pre-defined number of threads will be present and those will process your email sending requirement. In below example I am starting a thread pool with max of 10 threads, then I am using a LinkedBlockingQueue implementation so this will ensure that if there are more than 10 requests and currently all my 10 threads are busy then excess of requests will be queued and not lost, this is the advantage you get with LinkedBlockingQueue implementation of Queue.
You can initialize you singleton ThreadPoolExecutor upon application server start, if there are no requests then no threads will be present so it is safe to do so. In fact I use similar configuration for my prod application.
I am using time to live seconds as 1 seconds so if a thread is ideal in JVM for more than 1 seconds then it will die.
Please note that since same thread pool is used for processing all you requests, so it should be singleton and do not shutdown this thread pool else your tasks will never be executed.
// creating a thread pool with 10 threads, max alive time is 1 seconds, and linked blocking queue for unlimited queuing of requests.
// if you want to process with 100 threads then replace both instances of 10 with 100, rest can remain same...
// this should be a singleton
ThreadPoolExecutor executor = new ThreadPoolExecutor(10, 10, 1, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>());
// inside your getSalesUserData() method
executor.execute(new Runnable() {
#Override
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed", e);
}
}
});
Java's default cached thread pool
This approach is much like above, only that Java will initialize the ThreadPoolExecutor for you as ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue<Runnable>());
Here max number of threads will be Integer.MAX_VALUE, so threads will be created as needed and time to live will be 60 seconds.
If you want to use this way then below is the way.
// this should be a singleton
ExecutorService emailExecutor = Executors.newCachedThreadPool();
// from you getSalesUserData() method
emailExecutor.execute(new Runnable() {
#Override
public void run() {
try {
SendEmailUtility.sendmail(emaildummy);
} catch (IOException e) {
logger.error("failed", e);
}
}
});
Manually creating of ExecutorService on java web serer is bad idea. In your implementation for each request you create 10 threads.
Better solution is to use ManagedExecutorService (example) if you work with JEE7 or ThreadPoolTaskExecutor if you work with Spring(docs).
If you work with Tomcat you should read this thread.
The best practice is to use a single ExecutorService to provide a thread pool for all requests. You probably want to configure the ExecutorService with a non-zero, but limited, number of threads.
The idea here is that you will have some threads that are reused throughout the lifetime of the application. You get the added benefit that if there is a temporary slowdown (or halt) in sending emails, you don't end up with a growing number of threads Instead, you end up with a growing number of pieces of work (emails to send) to be executed, which is much less resource intensive than extra threads.
I am using Java EmailSender class.
I simply started a new thread to send mail because it was blocking the main thread and I was getting Time out an exception.
String link = "http://localhost:PORT/api/v1/registration/confirm?token=" +token;
//Sending mail in thread beacause it block main thread
new Thread(
() -> emailSender.sendMail(request.getEmail(),buildEmail(request.getFirstName(),
link))).start();
I have a Spring JMS Application that is using ActiveMQ version 5.10. I am performing a simple test to concurrency. I am using Spring Boot, current version and annotations to configure JMSListener and message producers.
The message producer just throws messsages on a queue as fast as it can. The message listener is pulling messages off the queue, but sleeping for 1 second after getting the message -- simulating some work that the message listener would need to do after getting a message.
I have the JMSListener set to 100-1000 concurrent threads. If I start the message producer and consumer at the same time (both run in their own JVM) the consumer never gets above the minimum configured threads, even though the max range is set 1000.
If I let the producer start first and place a few thousand messages on the queue, then start 1 or more instances of the consumer, it will raise the threads steadily, starting at 100 then 20 or so threads each second until it gets to a state where there is about 20-30 messages in the queue that are in-flight. It never catches the producer -- there is always some messages in queue even though the consumer is no where near its maxConcurrency count.
Why doesn't the message consumer burst into a bunch of additional threads to empty the queue instead of letting the queue have the 20-30 messages in it? Isn't there a way for the consumer continue to add threads faster in order to catch up with the messages in queue?
Here are the relevant parts of the code.
Message Producer
#Component
public class ClientServiceImpl implements ClientService {
private static final String QUEUE="message.test.queue";
#Autowired
private JmsTemplate jmsTemplate;
#Override
public void submitMessage(ImportantMessage importantMessage) {
System.out.println("*** Sending " + importantMessage);
jmsTemplate.convertAndSend(QUEUE, importantMessage);
}
}
Message Consumer
#SpringBootApplication
#EnableJms
public class AmqConsumerApplication {
public static void main(String[] args) {
SpringApplication.run(AmqConsumerApplication.class, args);
}
#Value("${JMSHost}")
private String JMS_BROKER_URL;
#Autowired
static Command command;
#Bean
public ConnectionFactory connectionFactory() {
ConnectionFactory factory= new ActiveMQConnectionFactory(JMS_BROKER_URL);
((ActiveMQConnectionFactory)factory).setTrustAllPackages(true);
((ActiveMQConnectionFactory)factory).setOptimizeAcknowledge(true);
((ActiveMQConnectionFactory)factory).setAlwaysSessionAsync(false);
return factory;
}
}
With the listener configured as such...
#Component
public class TransformationListener {
private static final String QUEUE="message.test.queue?consumer.prefetchSize=10";
#JmsListener(destination=QUEUE, concurrency = "100-1000")
public void handleRequest(ImportantMessage importantMessage) {
System.out.println("*** Recieved message: " + importantMessage + " on thread" + Thread.currentThread().getId());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Are you still facing this behavior ?
Did you read this advice "Pooled Consumers and prefetch" on http://activemq.apache.org/what-is-the-prefetch-limit-for.html
Did you tried with prefetchSize=0 or 1 ? I think 1 can resolve your problem.
If prefetchSize is > 1 maybe you need to decrease the AbortSlowAckConsumerStrategy to lower than default 30s.
To have more than 100 threads consuming messages in your case you need more than 1000 messages not consumed and not prefetched in the queue because the prefetchSize is to 10.
I'm reading 'Java Concurrency in Practice', one example sort of makes me confused, it's about producer-consumer logging service:
public class LogWriter {
private final BlockingQueue<String> queue;
private final LoggerThread logger;
private boolean shutdownRequested = false;
public LogWriter(Writer writer) {
this.queue = new LinkedBlockingQueue<String>(CAPACITY);
this.logger = new LoggerThread(writer);
}
public void start() { logger.start(); }
public void shutdownlog() { shutdownRequested = true; }
public void log(String msg) throws InterruptedException {
if (!shutdownRequested)
queue.put(msg);
else
throw new IllegalStateException("logger is shut down");
}
private class LoggerThread extends Thread {
private final PrintWriter writer;
...
public void run() {
try {
while (true)
writer.println(queue.take());
} catch(InterruptedException ignored) {
} finally {
writer.close();
}
}
}
}
From the book, it's unreliable if we would shutdown it. It wrote:
Another approach to shutting down LogWriter would be to set a "shutdown requested" flag to prevent further messages from being submitted,as shown in Listing7.14.The consumer could then drain the queue upon being notified that shutdown has been requested, writing out any pending messages and unblocking any producers blocked in log. However, this approach has race conditions that make it unreliable. The implementation of log is a checkthenact sequence: producers could observe that the service has not yet been shut down but still queue messages after the shutdown,again with the risk that the producer might get blocked in log and never become unblocked. There are tricks that reduce the likelihood of this (like having the consumer wait several seconds before declaring the queue drained), but these do not change the fundamental problem, merely the likelihood that it will cause a failure.
I don't quite understand it. does it mean that another thread happens to run into queue.put(msg) right after the shutdownflag set to true?
Thanks, guys.
The loophole is between the producer checking the flag by checking shutdownRequested and then putting the message in the queue. If shutdown happens and the worker stops performing in this tiny timespan, this happens.
Although its unlikely, it might happen that you enqueue messages when flag is already set.
But I cant see that the producer getting blocked, because the worker simply ignores the shutdown flag.
If the producer tries to enqueue a message while the queue is full it blocks, but becomes unblocked when the worker takes elements from the queue.