I have a Spring Batch job which takes parameters, and the parameters are usually the same every time the job is run. By default, Spring Batch doesn't let you re-use the same parameters like that... so I created a simple incrementer and added it to my job like this:
http://numberformat.wordpress.com/2010/02/07/multiple-batch-runs-with-spring-batch/
When using the standard CommandLineJobRunner to run my job, I have to pass the -next parameter in order for my incrementer to be used.
However, when I run an end-to-end job test from within a JUnit class, using JobLauncherTestUtils.launchJob( JobParameters )... I can't find a way to declare that my incrementer should be used. The job is just quietly skipped, presumably because it has already been run with those parameters (see note below).
The JobParameters class is meant to hold a collection of name-value pairs... but the -next parameter is different. It starts with a dash, and has no corresponding value. I tried various experiments, but trying to add something to the JobParameters collection doesn't seem to be the ticket.
Does anyone know the JUnit equivalent to passing -next to CommandLineJobRunner?
NOTE: I presume that the the issue is my incrementer being ignored, because:
The job works the first time, and it works if I wipe out the job repository database. It only fails on retries.
The job works fine, retries and all, when I hardcode the variables and remove the parameters altogether.
JobLauncherTestUtils class contains a getUniqueJobParameters method which serves exactly the same need.
/**
* #return a new JobParameters object containing only a parameter for the
* current timestamp, to ensure that the job instance will be unique.
*/
public JobParameters getUniqueJobParameters() {Map<String, JobParameter>
parameters = new HashMap<String, JobParameter>();
parameters.put("random", new JobParameter((long) (Math.random() * JOB_PARAMETER_MAXIMUM)));
return new JobParameters(parameters);
}
Sample usage would be,
JobParameters params = new JobParametersBuilder (jobLauncherTestUtils.getUniqueJobParameters()).toJobParameters();
//extra parameters to be added
JobExecution jobExecution = jobLauncherTestUtils.launchJob(params);
Related
I wanted to pass the collection of String as a step parameter.
Since I didn't find a way to construct JobParameter for collection, I decided to pass it as a string with comma-separated values.
My code to execute the job:
#Autowired
private JobLauncher jobLauncher;
#Autowired
private Job myJob;
public void execute() {
List<String> myCollection = getMyCollection();
jobLauncher.run(myJob, new JobParameters(ImmutableMap.<String, JobParameter> builder()
.put("myCollection", new JobParameter(String.join(",", myCollection)))
.build())
...
}
I define the Step as follows:
#Bean
#StepScope
public Step myStep(#Value("#{jobParameters['myCollection']}") String myCollectionString) {
List<String> myCollection = ArrayUtil.asList(lisReferencesString.split(","));
...
}
But when execution is started I'm getting the error:
org.postgresql.util.PSQLException: ERROR: value too long for type character varying(250)
Since the job params are stored as a column value, I can't pass too long strings as a param.
Could you suggest how I could overcome it?
The default length of job parameters of type String is 250, see BATCH_JOB_EXECUTION_PARAMS. The scripts provided by Spring Batch are just a starting point, you can update them as needed. So in your case, you need to increase the length of BATCH_JOB_EXECUTION_PARAMS#STRING_VAL as required.
I think there is no way you can pass the collection as job parameter. You can probably
Split the string with 250 characters each and send them in multiple parameters
Save the parameters somewhere in temp table or file and read in the job wherever required.
Please check these threads
how to send a custom object as Job Parameter in Spring Batch?
ArrayList cannot be cast to org.springframework.batch.core.JobParameter
My project needs to build a file containing logs and load it to S3.
To do this, whenever a Spring Batch job is ran, I create a file like this:
String startTime = new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss").format(new Date());
new File(startTime + "_error_logs");
I then add it to a set of JobParameters which get passed to my JobLauncher like this:
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
JobParameters param = new JobParametersBuilder()
.addString("startTime", startTime)
.toJobParameters();
jobLauncher.run(getJob(), param);
Then, throughout the project, I want to be able to access this job parameter. For example, I can get it in my JobCompletionNotificationListener class which in the afterJob function, takes JobExecution as a parameter, meaning that I can do this:
String startTime = jobExecution.getJobParameters().getString("startTime");
However, in the GlobalControllerExceptionHandler class, I don't have access to this JobExecution object meaning that I cannot get hold of the startTime parameter.
Is there anyway I can pass data to it? Or is there a better approach to this problem? I know that I will have the same issue in other classes. Other approaches I have thought of won't work either, for example, storing the String in a file won't work because then if another job is ran in parallel, it will become confused.
You can inject a JobExplorer in your GlobalControllerExceptionHandler and query for the jobExecution you are interested in. Once you get a handle to the execution you want, you can get the parameter as you mentioned:
String startTime = jobExecution.getJobParameters().getString("startTime");
I'm running a scheduled task in my Spring application that runs a job. The job itself is fetched at the beginning of the task. After that a loop takes place that modifies the job in each iteration (++ a counter). After the loop I merge my instance using the entity manager. It works fairly well, but I'm facing an issue trying to modify the instance from another place. Since the instance has a 'paused' flag, I'm trying to set it. But whenever I do it's quickly reset again, due to the scheduled task unsetting it again (as far as I can tell).
Here's some code:
// This method is called using the #Scheduled annotation to be looping
// constantly with one second delay between invocations.
#Transactional
public void performActions() {
Job job = jobRepository.findFirstByPausedAtIsNull();
// Skip if no unpaused job exists
if(job == null) return;
// Iterate through batch of job actions
for(Action action : job.nextActions()) {
action.perform();
job.increaseActionsPerformedCount();
// Merge the action into the persistence context
entityManager.merge(action);
}
// Merge the job into the persistence context
entityManager.merge(job);
}
Now I'm trying to be able to pause the job at any time from the outside. I use a controller endpoint to call a pause method on the jobService. This method looks like this:
public Job pause(long id) throws JobNotFoundException, JobStatusException {
Job job = this.show(id);
if(job.getPausedAt() != null) throw new JobStatusException("The job is already paused");
job.pause(); // This sets the flag on the instance, same as job.setPausedAt(new Date())
return jobRepository.save(campaign); // Uses CrudRepository
}
Now calling the method works fine and it actually returns the Job with pausedAt set. But the value is reset quickly after.
I've tried just straight up fetching a fresh instance from the database at the end of performAction and setting the modified instance pausedAt to the freshly fetched one's value.
Any idea how this could be achieved properly?
As far as I understand , You need to stop the job when the pause flag is set ... you can achieve this by applying optimistic lock ... add a #Version field to Job .... apply LockModeType.OPTIMISTIC to the job that you retrieved in performAction() -either by adding it to the find() method or call a refresh() after retrieval -the first is better- ..... now if the other endpoint changes the pause flag the version field will be incremented and you will get OptimisticLockException at persisting .... this has some implications :
1- whatever state changes in the Job , the same behavior will happen (not only the pause field)
2- You will need to handle the Exception from inside the persistence context (i.e. inside performActions()) because after returning it might be mapped to any other exception type ... this is the idea I have now, may be there is something better that gives you more control (track only the pause attribute)
I'm new in spring. Please help me to understand what I have to use (TaskExecutor, #Sceduled,Quarts Sceduler,...) to implement this problem:
I have an Order object and Contacts (connected with 1:N relationship. One Order can have many Contacts). So
When Order is created, application have to send email to all connected Contacts.
When new Contact lately is created and connected to the already created Order, this Contact also have to get an email.
When Order will expire , 2 days later Contact have to get an email.
Step1:
When Order is created, application have to send email to all connected Contacts.
Add 2 new columns into Contacts Table ( or similar table).
is_Send_Email -> boolean type
Email_Send_Time -> timestamp/date type
While inserting a new row ( new order is created), set is_Send_Email=true and Email_Send_Time = current time. for all related Contacts.
2.When new Contact lately is created and connected to the already created Order, this Contact also have to get an email.
When adding a contact to the Order,set is_Send_Email=true and Email_Send_Time = Current time (while inserting) for newly added Contacts.
3.When Order will expire , 2 days later Contact have to get an email.
Set is_Send_Email=true for all contacts in that expiring order and Email_Send_Time = Current time+2 days.
Step2:
Enable scheduling using #EnableScheduling in your configuration class.
#Configuration
#EnableScheduling
public class AppConfig {
#Bean
public MyBean bean() {
return new MyBean();
}
}
Step3:
Use #Scheduled annotation to call your mail sending method at specific intervals.
As per Spring documentation..
34.4.2 The #Scheduled Annotation
The #Scheduled annotation can be added to a method along with trigger
metadata.
For example, the following method would be invoked every 5
seconds with a fixed delay, meaning that the period will be measured
from the completion time of each preceding invocation.
#Scheduled(fixedDelay=5000) public void doSomething() {
// something that should execute periodically
}
If a fixed rate execution is desired, simply change the property name
specified within the annotation. The following would be executed every
5 seconds measured between the successive start times of each
invocation.
#Scheduled(fixedRate=5000) public void doSomething() {
// something that should execute periodically
}
For fixed-delay and fixed-rate tasks, an initial delay may be
specified indicating the number of milliseconds to wait before the
first execution of the method.
#Scheduled(initialDelay=1000, fixedRate=5000) public void
doSomething() {
// something that should execute periodically
}
If simple periodic scheduling is not expressive enough, then a cron
expression may be provided. For example, the following will only
execute on weekdays.
#Scheduled(cron="*/5 * * * * MON-FRI") public void doSomething() {
// something that should execute on weekdays only
}
[Tip] You can additionally use the zone attribute to specify the time
zone in which
the cron expression will be resolved. Notice that the methods to be
scheduled must have void returns and must not expect any arguments. If
the method needs to interact with other objects from the Application
Context, then those would typically have been provided through
dependency injection.
Step4:
Check each record in Order table, if is_Send_Email=true for an record, then trigger email for that Order/Contacts whatever.
How to send email using Spring , You can refer this article.
Happy Learning :-)
I have looked around and around for this answer, but I have not been able to find a good answer. I would like to create a system based on Quartz that allows people to schedule their own tasks. I will use a pseudo example.
Let's say my main method for my Quartz program is called quartz.java.
Then I have a file called sweep.java that implements the Quartz "job" interface.
So in my quartz.java, I schedule my sweep.java to run every hour. I run quartz.java, and it works fine. GREAT; however, now I want to add a dust.java to the quartz scheduler; however, since this is a production service, I don't want to have to stop my quartz.java file, add in my dust.java, and recompile and run quartz.java again. This downtime would be unacceptable.
Does anyone have any ideas on how I could accomplish this? It seems impossible because how could you ever feed another java file into the program without recompiling, linking, etc.
I hope that this example is clear. Please let me know if I need to clarify any part of it.
Partial answer: it is possible to compile, and then instantiate, a class, programatically.
Here are links to example code:
how to compile from a String;
CompilerOutput;
CompilerOutputDirectory.
The extracted class is grabbed in the third source file (see method getGeneratedClass, which returns a Class<?> object).
HOWEVER: keep in mind that this is potentially dangerous to do so. One problem, which can be quite serious if you are not careful, is that when you dynamically instantiate a class, its static initialization blocks are executed. And these can potentially wreak havoc on your application. So, in addition, you'll have to create an appropriate SecurityContext.
In the code above, I actually only ever get the Class<?> object and never instantiate it in any way, so no code is executed. But your usage scenario is quite different.
I have not tried any of these but are worth trying .
1) Consider using Quartz camel endpoint .
If my understanding is right, Apache Camel lets you create the camel routes on the fly.
It just needs to deploy the camel-context.xml into a container taking into consideration that the required classes would be already available on classpath of container.
2) Quartz lets you create a job declaratively i.e. with xml configuration of job and trigger.
You can find more information here.
3) Now this requires some efforts ;-)
Create an interface which has a method which you will execute as a part of job. Lets say this will have a method called
public interface MyDynamicJob
{
public void executeThisAsPartOfJob();
}
Create your instances of Job methods.
public EmailJob implements MyDynamicJob
{
#Override
public void executeThisAsPartOfJob()
{
System.out.println("Sending Email");
}
}
Now in your main scheduler engine, use the Observer pattern to store/initiate the job dynamically.
Something like,
HashMap jobs=new HashMap<String,MyDynamicJob>();
// call this method to add the job dynamically.
// If you add a job after the scheduler engine started , find a way here how to reiterate over this map without shutting down the scheduler :-).
public void addJob(String someJobName,MyDynamicJob job)
{
jobs.add(someJobName,job);
}
public void initiateScheduler()
{
// Iterate over the jobs map to get all registered jobs. Create
// Create JobDetail instances dynamically for each job Entry. add your custom job class as a part of job data map.
Job jd1=JobBuilder.newJob(GenericJob.class)
.withIdentity("FirstJob", "First Group").build();
Map jobDataMap=jd1.getJobDataMap();
jobDataMap.put("dynamicjob", jobs.get("dynamicjob1"));
}
public class GenericJob implements Job {
public void execute(JobExecutionContext arg0) throws JobExecutionException {
System.out.println("Executing job");
Map jdm=arg0.getJobDetail().getJobDataMap();
MyDynamicJob mdj=jdm.get("dynamicjob");
// Now execute your custom job method here.
mdj.executeThisAsPartOfJob();
System.out.println("Job Execution complete");
}
}