I have below code which formats the files from one form to another. Process takes some time so we have wait time of 5 mins in case file takes time to process we destroy it.
The application flow is a http call is made from browser which hits spring boot #Controller class and lastly executes below code which is into #Service class of springboot application.
On load test I see in task manager many formatter.exe present even after springboot application is closed. My question is it correct way to implement in multi user concurrent environment. Also can some help me how can I increase performance executing "exe" when multiple request are made simultaneously
process = Runtime.getRuntime().exec(runtime.exec("c:\\modifier\\formatter.exe"););
if (!process.waitFor(5, TimeUnit.MINUTES)) {
process.destroyForcibly();
process = null;
}
This is not a good practice to wait for http request to complete in 5 minutes as well as waiting for separate process to complete. I assume, your endpoint is synchronized (not async request mapping) since you have not provided mapping details.
If you start a separate process and until you explicitly shutdown or kill it, that certain process will be running (if that process hanged). Refer this question to get an understand how to terminate a process and refer this document as well.
As I said, keeping a http request waiting for 5 minutes is not a good practice. I suggest a different approach for this solution as you use Spring Boot. You can simply make your endpoint asynchronous using #Async annotation for not to wait the request till process to complete. (How To Do #Async in Spring is a good document to see in this case)
Now you can change the controller implementation to use a message broker (RabbitMQ, ActiveMQ, JMS and so forth) queue requests and immediately respond back to the client (Messaging with RabbitMQ). So, your client (browser) see the response immediately even without process is started. You can handle the response at client side as you want then.
Now you can write a different program to dequeue messages from the broker and start the separate process. It doesn't matter how long that process takes as we have already given the response to the client and you don't need to destroy the process until that process is completed (if it hanged, just kill the process and re-queue message to the queue. This way we can ensure that every request will be processed).
After the process is done, you can notify the client by a push notification or Websocket implementation with resulted data.
I know this could be overdoing a simple task. But if you want to keep your feature's reliability and usability, this is worth doing. Also, you can use this in Microservice architecture. In fact this is a concept of Microserivces. Here is a good read to learn some about this approach.
Related
please give me some advice about the best pattern of the task solution. My task is this:
User makes a request to the Camunda processor through own rest
controller
BPMN schema on a backend side consists of a chain of
several asynchronous services
Data will be ready for response to
the User only when one final service on BPMN makes it.
Every chain works not greater than 10-15 secs. And users sessions count is less than 500 an hour.
How to organize the work of the rest controller? Is it acceptable to force controller waiting of result in the same call? Where a bottleneck?
Can you use some server push technology? If it were just a couple of seconds, I'd say go for waiting in the rest controller.
Being 15 seconds and thinking about scalability, I'd say to follow some kind of asynchronous pattern with the client to.
Client sends a request to do something
The controller delegates the work to some external process and returns to the client ok
The process ends and a response is ready.
If the other side is a browser, use some kind of server push technology to notify it. If it is an application use some kind of rpc, polling or any other inter process mechanism to communicate.
Note that depending on the hosting technology, there are different limits on concurrent connections. Check Spring Boot - Limit on number of connections created for tomcat.
Is making REST based web service (POST) asynchronous is the best way to handle thousands of requests at one time (Keeping in mind that I have only single instance of server serving the request)?
Edited:
Jersey is wrongly tagged.
For eg: I have a rest based web service, which is supposed to be consumed by 100 thousand clients within a very short span of time (~60 seconds). I understand that if I am allowed to deploy multiple instance of the server, then I can use a load balancer to handle all my incoming request and delegate them accordingly. But I am restricted to use only single instance. What design could I opt within this restriction?
I could think of making the request asynchronous( which will not respond to client immediately ) in order to be able to let the server be free from this load and handle the requests at it's own pace.
For now we can ignore memory limitations.
Please let me know if this clarifies your doubt?
The term asynchronous could have different meanings in different places. For a web application code, it could refer to a Nonblocking I/O server such as Node or Netty/Akka which is a way for HTTP Requests to time multiplex on the same worker threads. If you're writing callbacks or using async or future constructs, it probably is non-blocking I/O which people sometimes refer to as asynchronous.
However, I could have REST API running on Node which implements non-blocking I/O, but the API or the overall architecture is still fully synchronous. For example, let's say I have an API endpoint POST /photos, which takes in a photo, creates image thumbnails, stores the URLs of the photo in a SQL Db and then stores the images in S3. The REST API could still block from the initial POST until after the image is processed and stored.
A second way is for the server to accept the photo process as a job and return immediately. Then the server could store the photo in a in memory or network based queue to be processed later by some other worker thread. In fact, I could even implement this async architecture even with a blocking server like some good old Java 7 and Jetty.
I need to be able to achieve two threading tasks in a container and need to understand the best practices for doing this. Here are the two classes of task I need to accomplish:
During a web services call, I need to start a thread that continues processing after the response has been sent. No message is required back to the original sender when processing is complete.
A web services call may need to spawn multiple threads that need to run in parallel to each other. The response to the original request should be blocked until all the workers have completed. Elements of the response will be drawn from pieces of each of the thread's results.
Of course, I could create my own instance of a java.util.concurrent.Executor and use it, but I suspect containers might be smart enough to provide one that they manage.
FWIW - I'm using WebSphere 6.1 on JDK 1.5.0 (I know, ancient...but it is what it is). I am running web services developed using Apache CXF, so I'm in the servlet container, but configured with Spring.
For 1) you might want to look at Asynchronous Beans. Alternatively use a Message Driven Bean which picks up and actions a message you send to a Queue. There's the Quartz stuff from Spring you might want to look at too. I think with Servlet 3 (no chance on WAS 6.1!) you might get the async support without the Async Work Manager or JMS approach, but until then I don't know a better way than these patterns.
For 2) generally blocking the request is a risky business (what if you hit the timeout). However, you're in the servlet container so you're ok using something from java.util.concurrent e.g. ExecutorService as you mentioned. Alternatively use messaging to send the work off somewhere else and block until it completes.
Generally, I wouldn't start threads from inside a container because there is the chance that j2ee full compliance is turned on and your app would die. Under full compliance threads are not allowed to be created. What you want to do is set up a JMS queue that you submit your "work to be done" to. You can then have an MDB listening to the queue which performs that actual operation that your thread would have done.
Can someone point me to a tutorial or similar code where JMS is used by a web app to execute a long running background process? (instead of using threads), I'm fairly familiar with the concepts of JMS messaging, but never used any JMS API or brokers (i'm looking at learning Apache ActiveMQ)
I'd like to be able to:
submit a message to the queue to run a process
check the status (progress) on that process at arbitrary times
Thanks!
The real point of using JMS in your context is to start tasks asynchronously. This is called fire and forget in middleware lingo. JMS has guaranteed delivery semantics, meaning that once the message has been put on the queue it is guaranteed to get there ... eventually.
The idea is you do any tasks you need to do and if you have any tasks in the process that can be done at a later time, then you put a message on a queue and later it will execute. This allows you to cut down processing by a significant amount while somebody is waiting for a response.
Another benefit of JMS is that the different parts of the system do not need to be running at the same time. The part that consumes messages can be down for maintenance while your front end still works.
The previous post is accurate in terms of a model to put orders or requests into a queue asynchronously and then have them be picked up later. However, it doesn't really address the question of long running processes.
In terms of queues and topics, the benefit of persistent queues is that if there are no consumers on the queue then messages will be waiting for consumption until there is a subscriber. In a topic, you need to create a durable subscription in order to make sure a consumer that is not connected will receive messages that are sent in its absence once it reconnects.
So, how are you defining a long-running-process? For a multi-step process you would typically use something like a workflow engine. There are options like a BPM tool or something like "OS Workflow". You can also do a home-grown solution that could look like the example below
1) There would need to be some sort of workflow definition that defines the steps in the process. This could be a properties file or an XML file.
2) Web App puts a message on a queue or topic (pub/sub) with an indication of the process to be executed (or you can have specific destinations for different processes)
3) A Dispatcher MDB picks the 'order' up off the queue with a status of 'NEW' and starts processing the first step.
4) Once the step is complete, the MDB puts a new message on the queue indicating the process being executed and either the next step to be executed, or the last step that was executed (depending on how deterministic you want the process to be)
5) The MDB picks up the message and sees that the process is 'IN_PROGRESS'. It either determines the next step to be executed or reads the step to be executed next from the message (either a JMS header value or within the message, perhaps in an XML format)
6) Steps 4 & 5 are repeated until the process instance is complete
In this case you will need an external representation of the order and process instance information. This will allow you to check the status of a request from your WebApp. Your order would need to be read and persisted with an updated status after each step in the process such that the WebApp could access the status information.
The key component of this architecture is the dispatcher MDB that listens for messages and executes the next step of the process. When I worked with OS Workflow that was one key piece that was missing. In this manner, you can control the number of threads that are executing process steps by controlling the number of MDB's in the pool and consumers on the queue. In this architecture I would recommend a queue over a topic for the workflow steps. However after each process step you could publish a message to a topic for subscribers to get updated status information.
With the Java EE6 technologies including JPA you could easily create an XSD, generate domain data model POJO's with JAXB and use JPA for persistence. We did a webcast earlier this year that covered the JEE6 technologies that are currently supported in WebLogic. Here are the replays: http://www.oracle.com/technetwork/middleware/weblogic/learnmore/weblogic-javaee6-webcasts-358613.html.
I'm also still interested to speak with you about your JBoss migration :) jeffrey.west#oracle.com
I'm having a hard time figuring out how to architect the final piece of my system. Currently I'm running a Tomcat server that has a servlet that responds to client requests. Each request in turn adds a processing message to an asynchronous queue (I'll probably be using JMS via Spring or more likely Amazon SQS).
The sequence of events is this:
Sending side:
1. Take a client request
2. Add some data into a DB related to this request with a unique ID
3. Add a message object representing this request to the message queue
Receiving side:
1. Pull a new message object from the queue
2. Unwrap the object and grab some information from a web site based on information contained in the msg object.
3. Send an email alert
4. update my DB row (same unique ID) with the information that operation was completed for this request.
I'm having a hard figuring out how to properly deal with the receiving side. On one hand I can probably create a simple java program that I kick off from the command line that picks each item in the queue and processes it. Is that safe? Does it make more sense to have that program running as another thread inside the Tomcat container? I will not want to do this serially, meaning the receiving end should be able to process several objects at a time -- using multiple threads. I want this to be always running, 24 hours a day.
What are some options for building the receiving side?
"On one hand I can probably create a simple java program that I kick off from the command line that picks each item in the queue and processes it. Is that safe?"
What's unsafe about it? It works great.
"Does it make more sense to have that program running as another thread inside the Tomcat container?"
Only if Tomcat has a lot of free time to handle background processing. Often, this is the case -- you have free time to do this kind of processing.
However, threads aren't optimal. Threads share common I/O resources, and your background thread may slow down the front-end.
Better is to have a JMS queue between the "port 80" front-end, and a separate backend process. The back-end process starts, connects to the queue, fetches and executes the requests. The backend process can (if necessary) be multi-threaded.
If you are using JMS, why are you placing the tasks into a DB?
You can use a durable Queue in JMS. This would keep tasks, even if the JMS broker dies, until they have been acknowledged. You can have redundant brokers so that if one broker dies, the second automatically takes over. This could be more reliable than using a single DB.
If you are already using Spring, check out DefaultMessageListenerContainer. It allows you to create a POJO message driven bean. This can be used from within an existing application container (your WAR file) or as a separate process.
I've done this sort of thing by hosting the receiver in an app server, weblogic in my case, but tomcat works fine, too. Don't poll the queue, use an event-based model. This could be hand-coded or it could be a message-driven web service. If the database update is idempotent, you could update the database and send the email, then issue the commit on the queue. It's not a problem to have several threads that all read from the same queue.
I've use various JMS solutions, including tibco, activemq (before apache subsumed it) and joram. Joram was the more reliable opensource solution, but that may have changed now that it's part of apache.