I'm working in an Spring application that downloads data from different APIs. For that purpose I need a class Fetcher that interacts with an API to fetch the needed data. One of the requirements of this class is that it has to have a method to start the fetching and a method to stop it. Also, it must download all asynchronously because users must be able to interact with a dashboard while fetching data.
Which is the best way to accomplish this? I've been reading about task executors and the different annotations of Spring to schedule tasks and execute them asynchronously but this solutions don't seem to solve my problem.
Asynchronous task execution is what you're after and since Spring 3.0 you can achieve this using annotations too directly on the method you want to run asyncrhonously.
There are two ways of implementing this depending whether you are interested in getting a result from the async process:
#Async
public Future<ReturnPOJO> asyncTaskWithReturn(){
//..
return new AsyncResult<ReturnPOJO>(yourReturnPOJOInstance);
}
or not:
#Async
public void asyncTaskNoReturn() {
//..
}
In the former method the result of your computation conveyed by yourReturnPOJOInstance object instance, is stored in an instance of org.springframework.scheduling.annotation.AsyncResult<V> which in return implements the java.util.concurrent.Future<V> that the caller can use to retrieve the result of the computation later on.
To activate the above functionality in Spring you have to add in your XML config file:
<task: annotation-driven />
along with the needed task namespace.
The simplest way to do this is to use the Thread class. You supply a Runnable object that performs the fetching functionality in the run() method and when the Thread is started, it invokes the run method in a separate thread of execution.
So something like this:
public class Fetcher implements Runnable{
public void run(){
//do fetching stuff
}
}
//in your code
Thread fetchThread = new Thread(new Fetcher());
fetchThread.start();
Now, if you want to be able to cancel, you can do that a couple of ways. The easiest (albeit most violent and nonadvisable way to do it is to interrupt the thread:
fetchThread.interrupt();
The correct way to do it would be to implement logic in your Fetcher class that periodically checks a variable to see whether it should stop doing whatever it's doing or not.
Edit To your question about getting Spring to run it automatically, if you wanted it to run periodically, you'll need to use a scheduling framework like Quartz. However, if you just want it to run once what you could do is use the #PostConstruct annotation. The method annotated with #PostConstruct will be executed after the bean is created. So you could do something like this
#Service
public class Fetcher implements Runnable{
public void run(){
//do stuff
}
#PostConstruct
public void goDoIt(){
Thread trd = new Thread(this);
trd.start();
}
}
Edit 2 I actually didn't know about this, but check out the #Async discussion in the Spring documentation if you haven't already. Might also be what you want to do.
You might only need certain methods to run on a separate thread rather than the entire class. If so, the #Async annotation is so simple and easy to use.
Simply add it to any method you want to run asynchronously, you can also use it on methods with return types thanks to Java's Future library.
Check out this page: http://www.baeldung.com/spring-async
Related
I'm very new to Hazelcast, and it might very well be that I am missing something glaringly obvious, but here goes.
I have a Java Application that runs distributed, each containing its own Hazelcast Instance. I need Hazelcast to schedule a job that will run at a fixed rate, but never simultaneously on several instances. To achieve this I plan to use the IScheduledExecutorService and create a job that implements Runnable and NamedTask.
My problem is that the job needs to call methods on the application. My understanding is that the job is serialized and deserialized by hazelcast, which means that I can't just create a Runnable and feed it the objects it needs through its constructor. So how do I "Get back" to the application objects from the Hazelcast job?
For example, say I had a plain old java Runnable that i would like to execute in a Hazelcast Executor like this:
public class DoStuffJob implements Runnable, NamedTask {
private MyResource resource;
public DoStuffJob (MyResource resource){
this.resource = resource;
}
#Override
public String getName() {
return "Do stuff";
}
#Override
public void run() {
resource.doAllTheStuff();
}
}
How would I create a Runnable I can execute on Hazelcast, that can still access MyResource on the instance it executes on?
The only option I have found is to make the job HazelcastInstanceAware, and use the HazelcastInstance.getUserContext() to keep the object, but I am hoping it is somehow possible to "get back" to the executing application.
Thank you in advance.
You could have your Runnable task put the derived data into a distributed data-structure - probably an IMap. It would then be accessible from any of your JVMs. Would that handle your requirements?
I am new to Spring Boot and just implemented a normal Spring Boot application with HTTP where endpoints receive data and put in a database. Now I want some data to put in both databases and a class with data structure. Since there will be continuous operations with this data I need to operate with it as a separate process.
#Service
public class RulesManager {
private HashMap<Integer, Rule> rules = new HashMap<Integer, Rule>();
public void addRule(Rule rule) {
// Add rule to the database
}
// should be running in the background
public void updateRules(){
// Continuous check of rules and update of this.rules HashMap
}
}
#SpringBootApplication
public class RulesApplication {
public static void main(String... args) {
SpringApplication.run(RulesApplication.class, args);
// How do I call RulesManager.updateRules() to run in the background and make changes to rules hashmap???
}
}
So while listening to HTTP requests I want my application to run background process which will never stop and repeat itself. I am not sure how to call that class from the main RulesApplication class so that both http requests and background process were able to make changes to this.rules HashMap. Will be grateful for any tip or advice.
If you are just looking to start a always on process when app starts ( even better when RuleManager gets initialized ), then you should simply create a new thread in the constructor of RuleManager :
methodCalledByConstructor()
{
new Thread(()->{
// loop start
// access and check the hashmap
// do what is necessary
// sleep for a sometime
// loop end
}).start();
}
But if the work is only required when some event occurs, then use observer pattern for more elegant solution.
Try to define a new Thread for example "LocalRulesHandling" and annotate it with #Component and inside this thread add your implementations regarding the rules hashmap.
In the RulesApplication class try to get the spring context and the get the execution thread bean and then start this thread.
ApplicationContext conttext = SpringApplication.run(RulesApplication.class, args);
LocalRulesHandling handling = context.getBean(LocalRulesHandling.class);
handling.start();
I'm currently checking out the following guide: https://developer.android.com/topic/libraries/architecture/guide.html
The networkBoundResource class:
// ResultType: Type for the Resource data
// RequestType: Type for the API response
public abstract class NetworkBoundResource<ResultType, RequestType> {
// Called to save the result of the API response into the database
#WorkerThread
protected abstract void saveCallResult(#NonNull RequestType item);
// Called with the data in the database to decide whether it should be
// fetched from the network.
#MainThread
protected abstract boolean shouldFetch(#Nullable ResultType data);
// Called to get the cached data from the database
#NonNull #MainThread
protected abstract LiveData<ResultType> loadFromDb();
// Called to create the API call.
#NonNull #MainThread
protected abstract LiveData<ApiResponse<RequestType>> createCall();
// Called when the fetch fails. The child class may want to reset components
// like rate limiter.
#MainThread
protected void onFetchFailed() {
}
// returns a LiveData that represents the resource
public final LiveData<Resource<ResultType>> getAsLiveData() {
return result;
}
}
I'm a bit confused here about the use of threads.
Why is #MainThread applied here for networkIO?
Also, for saving into the db, #WorkerThread is applied, whereas #MainThread for retrieving results.
Is it bad practise to use a worker thread by default for NetworkIO and local db interaction?
I'm also checking out the following demo (GithubBrowserSample): https://github.com/googlesamples/android-architecture-components
This confuses me from a threading point of view.
The demo uses executors framework, and defines a fixed pool with 3 threads for networkIO, however in the demo only a worker task is defined for one call, i.e. the FetchNextSearchPageTask. All other network requests seem to be executed on the main thread.
Can someone clarify the rationale?
It seems you have a few misconceptions.
Generally it is never OK to call network from the Main (UI) thread but unless you have a lot of data it might be OK to fetch data from DB in the Main thread. And this is what Google example does.
1.
The demo uses executors framework, and defines a fixed pool with 3 threads for networkIO, however in the demo only a worker task is defined for one call, i.e. the FetchNextSearchPageTask.
First of all, since Java 8 you can create simple implementation of some interfaces (so called "functional interfaces") using lambda syntax. This is what happens in the NetworkBoundResource:
appExecutors.diskIO().execute(() -> {
saveCallResult(processResponse(response));
appExecutors.mainThread().execute(() ->
// we specially request a new live data,
// otherwise we will get immediately last cached value,
// which may not be updated with latest results received from network.
result.addSource(loadFromDb(),
newData -> result.setValue(Resource.success(newData)))
);
});
at first task (processResponse and saveCallResult) is scheduled on a thread provided by the diskIO Executor and then from that thread the rest of the work is scheduled back to the Main thread.
2.
Why is #MainThread applied here for networkIO?
and
All other network requests seem to be executed on the main thread.
This is not so. Only result wrapper i.e. LiveData<ApiResponse<RequestType>> is created on the main thread. The network request is done on a different thread. This is not easy to see because Retrofit library is used to do all the network-related heavy lifting and it nicely hides such implementation details. Still, if you look at the LiveDataCallAdapter that wraps Retrofit into a LiveData, you can see that Call.enqueue is used which is actually an asynchronous call (scheduled internally by Retrofit).
Actually if not for "pagination" feature, the example would not need networkIO Executor at all. "Pagination" is a complicated feature and thus it is implemented using explicit FetchNextSearchPageTask and this is a place where I think Google example is done not very well: FetchNextSearchPageTask doesn't re-use request parsing logic (i.e. processResponse) from RepoRepository but just assumes that it is trivial (which it is now, but who knows about the future...). Also there is no scheduling of the merging job onto the diskIO Executor which is also inconsistent with the rest of the response processing.
I have looked around and around for this answer, but I have not been able to find a good answer. I would like to create a system based on Quartz that allows people to schedule their own tasks. I will use a pseudo example.
Let's say my main method for my Quartz program is called quartz.java.
Then I have a file called sweep.java that implements the Quartz "job" interface.
So in my quartz.java, I schedule my sweep.java to run every hour. I run quartz.java, and it works fine. GREAT; however, now I want to add a dust.java to the quartz scheduler; however, since this is a production service, I don't want to have to stop my quartz.java file, add in my dust.java, and recompile and run quartz.java again. This downtime would be unacceptable.
Does anyone have any ideas on how I could accomplish this? It seems impossible because how could you ever feed another java file into the program without recompiling, linking, etc.
I hope that this example is clear. Please let me know if I need to clarify any part of it.
Partial answer: it is possible to compile, and then instantiate, a class, programatically.
Here are links to example code:
how to compile from a String;
CompilerOutput;
CompilerOutputDirectory.
The extracted class is grabbed in the third source file (see method getGeneratedClass, which returns a Class<?> object).
HOWEVER: keep in mind that this is potentially dangerous to do so. One problem, which can be quite serious if you are not careful, is that when you dynamically instantiate a class, its static initialization blocks are executed. And these can potentially wreak havoc on your application. So, in addition, you'll have to create an appropriate SecurityContext.
In the code above, I actually only ever get the Class<?> object and never instantiate it in any way, so no code is executed. But your usage scenario is quite different.
I have not tried any of these but are worth trying .
1) Consider using Quartz camel endpoint .
If my understanding is right, Apache Camel lets you create the camel routes on the fly.
It just needs to deploy the camel-context.xml into a container taking into consideration that the required classes would be already available on classpath of container.
2) Quartz lets you create a job declaratively i.e. with xml configuration of job and trigger.
You can find more information here.
3) Now this requires some efforts ;-)
Create an interface which has a method which you will execute as a part of job. Lets say this will have a method called
public interface MyDynamicJob
{
public void executeThisAsPartOfJob();
}
Create your instances of Job methods.
public EmailJob implements MyDynamicJob
{
#Override
public void executeThisAsPartOfJob()
{
System.out.println("Sending Email");
}
}
Now in your main scheduler engine, use the Observer pattern to store/initiate the job dynamically.
Something like,
HashMap jobs=new HashMap<String,MyDynamicJob>();
// call this method to add the job dynamically.
// If you add a job after the scheduler engine started , find a way here how to reiterate over this map without shutting down the scheduler :-).
public void addJob(String someJobName,MyDynamicJob job)
{
jobs.add(someJobName,job);
}
public void initiateScheduler()
{
// Iterate over the jobs map to get all registered jobs. Create
// Create JobDetail instances dynamically for each job Entry. add your custom job class as a part of job data map.
Job jd1=JobBuilder.newJob(GenericJob.class)
.withIdentity("FirstJob", "First Group").build();
Map jobDataMap=jd1.getJobDataMap();
jobDataMap.put("dynamicjob", jobs.get("dynamicjob1"));
}
public class GenericJob implements Job {
public void execute(JobExecutionContext arg0) throws JobExecutionException {
System.out.println("Executing job");
Map jdm=arg0.getJobDetail().getJobDataMap();
MyDynamicJob mdj=jdm.get("dynamicjob");
// Now execute your custom job method here.
mdj.executeThisAsPartOfJob();
System.out.println("Job Execution complete");
}
}
Because of all the problems we can meet when trying to use Hibernate in a multithreaded application (1st clue, 2nd clue, 3rd clue, etc.), I was thinking of another solution: implementing the logical part within a classic Controller, and simply call it from my thread using URL.openConnection().
In other words, instead of doing something like this:
MyThread.java
public class MyThread implements Runnable {
#Override
public void run() {
// do some great stuff with Hibernate
}
}
Anywhere.java
new Thread(new MyThread()).start();
I would like to try something like that:
MyController.java
#Controller
public class MyController {
#RequestMapping(value = "myUrl", method = RequestMethod.GET)
public void myMethod() {
// do some great stuff with Hibernate
}
}
MyThread.java
public class MyThread implements Runnable {
#Override
public void run() {
// simple call the above mapped url
}
}
Anywhere.java
new Thread(new MyThread()).start();
What do you think about it? Good or bad? I haven't tried yet, but I think such a solution will prevent the common errors we can meet using Hibernate in multithreading, because the server will execute the logical part as if someone were requesting the fake page.
PS: I know there are some solutions to use Hibernate in multithreaded applications, but each time I try one, another appears, and that until the I'm-fed-up-with-it point of no return.
PS2: I'm aware that such a solution need to be secured (e.g. UID as a token).
I don't really see what problem you're trying to solve here. Hibernate is almost always used in a multi-threaded environment. In webapps, for example, concurrent requests are handled by multiple concurrent threads, and each thread uses its own Hibernate session. And that doesn't cause any problem.
You will have problem if you share the same session among threads, or if you share a given entity among threads.
If you start your own thread, and this thread uses its own session and entities, I don't see why you would have any problem.