I'm not really understanding the dyno and worker process model of Heroku as it relates to a single process but multi-threaded Java-based server.
For example: How do I know (for a single dyno) how many processors are available for my background threads? Do I need to use something like RabbitMQ and create a separate process (app) for each background processing task and communicate between the server and these? Seems a little overkill for some Scheduled Tasks using Thread Cached Executors. Should all Futures be changed to inter-process Futures?
I guess it comes down to this question. Can I no longer write a multi-threaded server and scale the processors available to my server process in order to accommodate my thread activity? Or do I need to refactor my architecture to use separate processes for concurrency? If the former, do I need workers or just multiple dynos?
Thanks.
Heroku supports multiple concurrency models, so it's really up to you how you would like to architect your application. You have access to the full Java stack, so if something makes more sense to just be run as multiple threads in your web processes, you can definitely do that, or you can always enqueue jobs on something like RabbitMQ or Redis and process them on separate worker dynos. Multithreading is simpler and makes sense if the amount of work is light and proportional to your web requests because it will be scaled along with the web dynos; however, if the work is large, not proportional, and/or needs to be scaled independently, then breaking it out into a separate process would be better.
Heroku was originally just a Ruby platform, which does not have the same threading capabilities as Java, so the use of separate worker dynos is more important for Ruby and this is reflected in some of the documentation and examples out there, which might have led to your confusion. Luckily, with Java you have more options available to you and can use what's best for the job at hand.
Related
Should parallel programming be used in the development of microservices in case the microservices are scalable and, for instance, deployed as ECS on AWS?
If yes, what are the benefits of consuming more resources by one instance vs the same resources by N instances?
How does parallel programming match https://12factor.net/
P.S. to be more specific - should I conceptually use parallel streams rather than simple streams?
Basically the link that you provided also provides answer to your question already
This does not exclude individual processes from handling their own internal multiplexing, via threads inside the runtime VM, or the async/evented model found in tools such as EventMachine, Twisted, or Node.js. But an individual VM can only grow so large (vertical scale), so the application must also be able to span multiple processes running on multiple physical machines.
https://12factor.net/concurrency
Sure, imagine a microservice that needs to execute multiple independent calls to a dB or to other microservice and aggregate the results. As the calls are independent, they can be executed in parallel so that the total time is at most the time it takes to execute the slowest call.
Parallel streams must be used when the tasks at hand are mutually exclusive and can be done in parallel. However, parallel programming comes with an overhead of using a little more resources. So depending on the tasks at hand, you need to make a decision with trade-offs, which would be the best for you.
What I understood from Vert.x documentation (and a little bit of coding in it) is that Vert.x is single threaded and executes events in the event pool. It doesn't wait for I/O or any network operation(s) rather than giving time to another event (which was not before in any Java multi-threaded framework).
But I couldn't understand following:
How single thread is better than multi-threaded? What if there are millions of incoming HTTP requests? Won't it be slower than other multi-threaded frameworks?
Verticles depend on CPU cores. As many CPU cores you have, you can have that many verticles running in parallel. How come a language that works on a virtual machine can make use of CPU as needed? As far as I know, the Java VM (JVM) is an application that uses just another OS process for (here my understanding is less about OS and JVM hence my question might be naive).
If a single threaded, non-blocking concept is so effective then why can't we have the same non-blocking concept in a multi-threaded environemnt? Won't it be faster? Or again, is it because CPU can execute one thread at a time?
What I understood from Vert.x documentation (and a little bit of coding in it) is that Vert.x is single threaded and executes events in the event pool.
It is event-driven, callback-based. It isn't single-threaded:
Instead of a single event loop, each Vertx instance maintains several event loops. By default we choose the number based on the number of available cores on the machine, but this can be overridden.
It doesn't wait for I/O or any network operation(s)
It uses non-blocking or asynchronous I/O, it isn't clear which. Use of the Reactor pattern suggests non-blocking, but it may not be.
rather than giving time to another event (which was not before in any Java multi-threaded framework).
This is meaningless.
How single thread is better than multi-threaded?
It isn't.
What if there are millions of incoming HTTP requests? Won't it be slower than other multi-threaded frameworks?
Yes.
Verticles depend on CPU cores. As many CPU cores you have, you can have that many verticles running in parallel. How come a language that works on a virtual machine can make use of CPU as needed? As far as I know, the Java VM (JVM) is an application that uses just another OS process for (here my understanding is less about OS and JVM hence my question might be naive).
It uses a thread per core, as per the quotation above, or whatever you choose by overriding that.
If a single threaded, non-blocking concept is so effective then why can't we have the same non-blocking concept in a multi-threaded environemnt?
You can.
Won't it be faster?
Yes.
Or again, is it because CPU can execute one thread at a time?
A multi-core CPU can execute more than one thread at a time. I don't know what 'it' in 'is it because' refers to.
First of all, Vertx isn't single threaded by any means. It just doesn't spawn more threads that it needs.
Second, and this is not related to Vertx at all, JVM maps threads to native OS threads.
Third, we can have non-blocking behavior in multithreaded environment. It's not one thread per CPU, but one thread per core.
But then the question is: "what are those threads doing?". Because usually, to be useful, they need other resources. Network, DB, filesystem, memory. And here it becomes tricky. When you're single threaded, you don't have race conditions. The only one accessing the memory at any point of time is you. But if you're multi threaded, you need to concern yourself with mutexes, or any other way to keep you data consistent.
Q:
How single thread is better than multi-threaded? What if there are millions of incoming HTTP requests? Won't it be slower than other multi-threaded frameworks?
A:
Vert.x isn't a single threaded framework, it does make sure that a "verticle" which is something you deploy within you application and register with vert.x is mostly single threaded.
The reason for this is that concurrency with multiple threads over complicates concurrency with locks synchronisation and other concept that need to be taken care of with multi threaded communication.
While verticles are single threaded the do use something called an event loop which is the true power behind this paradigm called the reactor pattern or multi reactor pattern in Vert.x's case. Multiple verticles can be registered within one application, communication between these verticles run through an eventbus which empowers verticles to use an event based transfer protocol internally but this can also be distributed using some other technology to manage the clustering.
event loops handle events coming in on one thread but everything is async so computation gets handled by the loop and when it's done a signal notifies that a result can be used.
So all computation is either callback based or uses something like Reactive.X / fibers / coroutines / channels and the lot.
Due to the simpler communication model for concurrency and other nice features of Vert.x it can actually be faster than a lot of the Blocking and pure multi threaded models out there.
the numbers
Q:
If a single threaded, non-blocking concept is so effective then why can't we have the same non-blocking concept in a multi-threaded environemnt? Won't it be faster? Or again, is it because CPU can execute one thread at a time?
A:
Like a said with the first question it's not really single threaded. Actually when you know something is blocking you'll have to register computation with a method called executeBlocking which wil make it run multithreaded on an ExecutorService managed by Vert.x
The reason why Vert.x's model is mostly faster is also here because event loops make better use of cpu computation features and constraints. This is mostly powered by the Netty project.
the overhead of multi threading with it's locks and syncs imposes to much strain to outdo Vert.x with it's multi reactor pattern.
I had written a code using executor service in java. Here I am creating 10 worker threads to process database fetched rows. Each thread will be assigned with one resultant row. This approach will work fine when the application is deployed and running on single instance/node.
Can anyone suggest how this will behave when my application is deployed in multiple nodes/cluster?
Do I have to take care of any part of code before deploying into cluster?
04/12/15: Any more suggestions?
You should consider the overhead of each task. Unless the task is of moderate size, you might want to batch them.
In a distributed context the overhead if much higher so you are more likely to need to batch the work.
You will need to a framework, so the considerations will depend on the framework you chose.
I'm working in a Java EE application and I want that some WebServices are executed in parallel.
I would like to know the pros and cons of 2 different approaches:
Use JMS queues and MDBs, so each message I put in the queue would be executed in parallel. This way the application part that put the message into the queue would have a while, that waits the MDBs to response in a RS Queue.
Use the java concurrent API (Future / Callable).
ADDED
This is what the application needs to do:
The application already does it via an MDB, but I was thinking about a refactoring.
TODAY'S SCENARIO:
//CALLER CLASS
FOREACH INTEGRATION
PUT MESSAGE INTO A QUEUE AND STORE AN ARRAY OF CORRELATION_IDs
END
THREAD.SLEEP(X) // SOMETIME FOR INTEGRATION TO FINISH
WHILE (true){
GET RESPONSE FROM THE RESPONSE QUEUE FOR EACH INTEGRATION USING THE CORRELATION PREVIOUSLY STORED
}
//MDB CLASS
HAS A HUGE SWITCH CASE THAT PROCESS EACH INTEGRATION
RETURN THE RESULT INTO THE RESPONSE QUEUE;
Questions:
Is it ok to use the concurrent API in java? In my opinion using the concurrent API will eliminate a layer of failure (JMS).
My deployment environment is Websphere. Is it a good practice to create your own threads with the concurrent java API.
Thanks in advance
Whatever solution you go with, you will eventually need to cope with a burst of traffic. The JMS/MDB the burst is controlled by the queue effectively. Also a point to consider is that the queue can be made persistence, so it will survive a server restart. Also a queue can be distributed across many servers, giving you horizontal scalability.
The thread approach is of course quicker to develop, test and deploy. However, I would consider using a BlockingQueue so that your threads do not run amock.
jms pros: you can have persistence, you can connect to existing infrastructure
jms cons: seems to heavy to be used only as a dispatcher
manual concurrency cons: well, it's manual. and parallel programming is difficult. some webservers (especially clouds) may forbid to create your own threads
not sure what exactly you want to do but webserver by default processes requests in parallel, so maybe you don't need anything else?
I was just comparing the performance of scala actors vs java threads.
I was amazed to see the difference, I observed that with my system I was able to spawn maximum ~2000 threads (live at a time) only But with the same system I was able to spawn ~500,000 actors of scala.
Both programs used around 81MB of Heap memory of JVM.
Can you explain How java thread are this much heavy weight than scala / akka actors?
What is the key factor which made scala-actor this much light weight?
If I want to achieve best scalability, Should I go for actor based web server instead of java based traditional web/app server like JBoss or Tomcat?
Thanks.
Scala actors (including the Akka variety) use Java threads. There's no magic: more than a few thousand threads running simultaneously is a problem for most desktop machines.
The Actor model allows for awake-on-demand actors which do not occupy a thread unless they have work to do. Some problems can be modeled effectively as lots of sleeping agents waiting to get some work, who will do it relatively quickly and then go back to sleep. In that case, actors are a very efficient way to use Java threading to get your work done, especially if you have a library like Akka where performance has been a high priority.
The Akka docs explain the basics pretty well.
All reasonably scalable web servers have to solve this sort of problem one way or another; you probably ought not be basing your decision for web server primarily on whether actors are used under the hood, and regardless of what you use you can always add actors yourself.
An Akka actor is not equivalent to a thread. It is more like a Callable that is executed on a threadpool.
When a message is dispatched to an actor, that actor is placed on a threadpool to process the message. When it is done, the pooled thread can be used to execute other actors.