Develop Reference

Junit Spring avoid to load twice application context datasource

I have this configuration classes:
#ComponentScan(
basePackages = {
"mypackage.controller",
"mypackage.service",
"mypackage.repository"
}
)
#TestPropertySource(locations="classpath:configuration.properties")
#Import({
H2Configuration.class
})
public class TestConfiguration {
}
#Configuration
public class H2Configuration {
#Bean
public DataSource dataSource() throws SQLException {
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
EmbeddedDatabase db = builder
.setType(EmbeddedDatabaseType.H2)
.addScript("h2/create.sql")
.addScript("h2/insert.sql")
.build();
db.getConnection().setAutoCommit(false);
return db;
}
}
And I have this two class tests:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(loader=AnnotationConfigContextLoader.class, classes = { TestConfiguration.class })
public class FirstRepositoryTest {
#Autowired
MyFirstRepositoryImpl repository;
#Before
public void initTest() {
}
#Test(expected = NullPointerException.class)
public void testNullRecords() {
repository.foo(null, null);
}
}
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(loader=AnnotationConfigContextLoader.class, classes = { TestConfiguration.class })
public class SecondRepositoryTest {
#Autowired
MySecondRepositoryImpl repository;
#Before
public void initTest() {
}
#Test(expected = NullPointerException.class)
public void testSomethingNullRecords() {
repository.something(null, null);
}
}
If I run junit test once for each class, all goes well.
In clean install phase tests fails because the application context is initialized twice.
For example it try to create the h2 tables twice and do the insert.sql script twice.
What I have to do for initialize the h2 database and so application context only once?
Thanks

I think you could start looking at the Spring documentation about Integration Testing.
It can also be a good practice to use transactional tests for integration tests (#Transactional), which rollback at the end of each test : see Transaction Management.
To avoid the cost of recreating the ApplicationContext for each test class, the cache may be used as explained here : Context Caching.
For integration testing with Embedded Database, you can also find documentation : Testing Data Access Logic with an Embedded Database.
A note from the previous link, matching your use case :
However, if you wish to create an embedded database that is shared
within a test suite, consider using the Spring TestContext Framework
and configuring the embedded database as a bean in the Spring
ApplicationContext as described in Creating an Embedded Database by
Using Spring XML and Creating an Embedded Database Programmatically.
I hope you will find some useful references.

Another good tip I found from Spring Boot documentation from Embedded Database Support :
They say :
If you are using this feature in your tests, you may notice that the
same database is reused by your whole test suite regardless of the
number of application contexts that you use. If you want to make sure
that each context has a separate embedded database, you should set
spring.datasource.generate-unique-name to true.
So to make each EmbeddedDatabase unique, you may try to create them with :
EmbeddedDatabase db = new EmbeddedDatabaseBuilder()
.generateUniqueName(true)
...
.build();

In unit testing you must garantee that every test is repeatible hance context independent. Due to this is not good idea to load the context only once. Is better to reset after the execution. For this you can use #DirtiesContext(classMode = ClassMode.AFTER_CLASS) in your test classes
So you will force your context to restart when the next junit class is launched

So the reason that this is failing is that the database (H2) is resident in memory when you run the tests as part of clean/install. The create/insert scripts have already executed after the first test is run. Any subsequent test execution after this point will result in a re-execution of the same script(s) and the error will occur.
Update your create script with a DROP TABLE IF EXISTS <table name>;. This will ensure that the table is dropped then recreated.
NOTE: I'm not sure why you've specified AnnotationConfigContextLoader explicitly. I think, without that, the runner SpringJUnit4ClassRunner will cache contexts that have not been changed. I don't know specifically if that is the case here though.

How to make a Spring Batch step depends on previous step?

I am using Spring Batch to read some data from CSV files and put it in a database.
My Batch job must be compound of 2 steps :
Check files (names, extension, content ..)
Read lines from CSV and save them in DB (ItemReader, ItemProcessor,
ItemWriter..)
Step 2 must not be executed if Step 1 generated an error (files are not conform, files doesn't exist ...)
FYI, I am using Spring Batch without XML configuration ! Only annotations :
Here's what my job config class looks like :
#Configuration
#EnableBatchProcessing
public class ProductionOutConfig {
#Autowired
private StepBuilderFactory steps;
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private ProductionOutTasklet productionOutTasklet;
#Autowired
private CheckFilesForProdTasklet checkFilesForProdTasklet;
#Bean
public Job productionOutJob(#Qualifier("productionOut")Step productionOutStep,
#Qualifier("checkFilesForProd") Step checkFilesForProd){
return jobBuilderFactory.get("productionOutJob").start(checkFilesForProd).next(productionOutStep).build();
}
#Bean(name="productionOut")
public Step productionOutStep(){
return steps.get("productionOut").
tasklet(productionOutTasklet)
.build();}
#Bean(name = "checkFilesForProd")
public Step checkFilesForProd(){
return steps.get("checkFilesForProd")
.tasklet(checkFilesForProdTasklet)
.build();
}
}

What you are looking for is already the default behavior of Spring Batch i.e. next step wouldn't be executed if previous step has failed. To mark current step as failed step, you need to throw a run time exception which is not caught.
If exception is not handled, spring batch will mark that step as failed and next step wouldn't be executed. So all you need to do is to throw an exception on your failed scenarios.
For complicated job flows , you might like to use - JobExecutionDecider , Programmatic Flow Decisions

As the documentation specifies you can use the method "on" which starts a transition to a new state if the exit status from the previous state matches the given pattern.
Your code could be similar to something like this :
return jobBuilderFactory.get("productionOutJob")
.start(checkFilesForProd)
.on(ExitStatus.FAILED.getExitCode()).end()
.from(checkFilesForProd)
.on("*")
.to(productionOutStep)
.build();

How to create and launch spring batch jobs at runtime

We have a requirement to carry out data movement from 1 database to other and exploring spring batch for the same. User of our application selects source and target datasource along with the list of tables for which the data needs to be moved.
Need help with following:
The information necessary to build a job comes at runtime from our web application - that includes datasource details and list of table names. We would like to create a new job by sending these details to the job builder module and launch it using JobLauncher. How do we write this job builder module?
We may have multiple users raising data movement requests in parallel, so need a way to create multiple jobs and run them in suitable order.
We have used the Java based configuration to create a job and launch it from a web container. The configuration is as follows
#Bean
public Job loadDataJob(JobCompletionNotificationListener listener) {
RunIdIncrementer inc = new RunIdIncrementer();
inc.setKey(new Date().toString());
JobBuilder builder = jobBuilderFactory.get("loadDataJob")
.incrementer(inc)
.listener(listener);
SimpleJobBuilder simpleBuilder = builder.start(preExecute());
for(String s : getTables()){
simpleBuilder.next(etlTable(s));
}
simpleBuilder.next(postExecute());
return simpleBuilder.build();
}
#Bean
#Scope("prototype")
public Step etlTable(String tableName) {
return stepBuilderFactory.get(tableName)
.<Map<String,Object>, Map<String,Object>> chunk(1000)
.reader(dbDataReader(tableName))
.processor(processor())
.writer(dbDataWriter(tableName))
.build();
}
Currently we have hardcoded the source and target datasource details into respective beans. The getTables() returns a list of tables (hardcoded) for which the data needs to be moved.
RestController that launches the job
#RestController
public class MyController {
#Autowired
JobLauncher jobLauncher;
#Autowired
Job job;
#RequestMapping("/launchjob")
public String handle() throws Exception {
try {
JobParameters jobParameters = new JobParametersBuilder().addLong("time", new Date().getTime()).toJobParameters();
jobLauncher.run(job, jobParameters);
} catch (Exception e) {
}
return "Done";
}
}

Concerning your first question, you definitely have to use JavaConfiguration. Moreover, you shouldn't define your steps as spring beans, if you want to create a job with a dynamic number of steps (for instance a step per table you have to copy).
I've written a couple of answers to questions about how to create jobs dynamically. Have a look at them, they might be helpful
Spring batch execute dynamically generated steps in a tasklet
Spring batch repeat step ending up in never ending loop
Spring Batch - How to generate parallel steps based on params created in a previous step
Spring Batch - Looping a reader/processor/writer step
Edited
Some remarks concerning your second question:
Firstly, you are using a normal JobLauncher and I assume your instantiate the SimpleJobLauncher. This means, you can provide a job with jobparameters, as you have shown in your code above. However, the provided "job" does not have to be a "SpringBean"-instance, so you don't have to Autowire it and therefore, you can use create-methodes as I suggested in the answers to the questions mentioned above.
Secondly, if you create your Job instance for every request dynamically, there is no need to pass the whole configuration as jobparameters, since you can pass the "configuration properties" like datasource and tables to be copied directly as parameters to your "createJob" method. You could even create your DataSource-instances "on the fly", if you don't know all possible datasources in advance.
Thirdly, I would consider every request as a "single run", which cannot be "restarted". Hence, I'd just but some "meta information" into the jobparameters like user, date/time, datasource names (urls) and a list of tables to be copied. I would use this kind of information just as a kind of logging/auditing which requests where issued, but I wouldn't use the jobparameter-instances as controlparameters inside the job itself (again, you can pass the values of these parameters during the construction time of the job and steps by passing them to your create-Methods, so the structure of your job is created according to your parameters and hence, during runtime - when you could access your jobparameters - there is nothing to do based on the jobparameters).
Finally, if a request fails (meaning the jobs exits with an error) simply a new request has to be executed in order to retry, but this request would be a complete new request and not a restart of an already executed job launch (since I would add the request time to my jobparameters, every launch would be a unique launch).
Edited 2:
Not creating the Job as a Bean doesn't mean to not use Autowiring. Here is an example, aus I would structure my Beans.
#Component
#EnableBatchProcessing
#Import() // list with imports as neede
public class JobCreatorComponent {
#Autowire
private StepBuilderFactory stepBuilder;
#Autowire
private JobBuilderFactory jobBuilder;
public Job createJob(all the parameters you need) {
return jobBuilder.get(). ....
}
}
#RestController
#Import(JobCreatorComponent.class)
public class MyController {
#Autowired
JobLauncher jobLauncher;
#Autowired
JobCreatorComponent jobCreator;
#RequestMapping("/launchjob")
public String handle() throws Exception {
try {
Job job = jobCreator.createJob(... params ...);
JobParameters jobParameters = new JobParametersBuilder().addLong("time", new Date().getTime()).toJobParameters();
jobLauncher.run(job, jobParameters);
} catch (Exception e) {
}
return "Done";
}
}

by using #JobScope on itemreader no need to do things manually at run time just have to annoted your respective reader with #Jobscope, on each interaction with controller you will get fresh record processing.
This is type of job on demand where you can execute the job for goals like do the db migration or get the specific reporting like that.

JUnit Test Cases for Config

I have a javaconfig file that looks like this:
#Configuration
public class ServiceConfig {
#Autowired
FooBean someBean;
#Bean
#Scope(value="session", proxyMode = ScopedProxyMode.TARGET_CLASS)
public FooService fooService() {
return new FooServiceImpl(someBean, fooB());
}
private Foo fooB() {
return new FooB();
}
}
And I have created a junit test file like so based on this stack answer :
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = ServiceConfig.class)
public class ServiceConfigTest {
}
But I have a couple questions:
Should I test all my config files with this one junit test file? I have 4 config files in total including the ServiceConfig file, so in the ContextConfiguration should I just list them all out or have a junit test for each one indivually?
What am I supposed to test here? I tried reading this spring guide but I'm not really understand what behavior I should test here....just if something gets autowired successfully?

Should I test all my config files with this one junit test file? I
have 4 config files in total including the ServiceConfig file, so in
the ContextConfiguration should I just list them all out or have a
junit test for each one indivually?
In your test class, #ContextConfiguration must be positioned on the root of the class.
So for testing each configuration, you have to create a test class by Configuration.
What am I supposed to test here? I tried reading this spring guide but
I'm not really understand what behavior I should test here....just if
something gets autowired successfully?
Testing if autowired is successful seems not very useful. It would be as unit test that Spring feature works. If you want to unit test these classes, you should test your own processing. For the moment, you have not really. So, I am not sure that testing them has great value.

Test the intended behaviour of your system or individual units of functionally. As part of this the configs will be tested.
You don't need to make tests to make sure something is wired correctly. That should be implied on the basis that the functionality you are testing works.
You can aggregate multiple configurations into one configuration class using #Import.

Spring Boot and unit testing of controllers, can I read the value of #RequestMapping at runtime?

In a Java Spring Boot application, I define the controllers path in a dedicated properties file. e.g.
#PropertySource("classpath:/my.properties")
#RequestMapping("${controller1.path}")
public class Controller{
#RequestMapping("/dosomething")
public void doSomethingREST(){
}
}
where my.properties looks like:
controller1.path=rest/path
The path of the REST service will be then http://localhost:8080/rest/path/dosomething
How can I read the path in the unit test class?
Should I necessarily write it manually?
Suppose that I change it, doesn't seem to be a very flexible approach.
Suggestions to make it more dynamic?

As far as I know, there is no way to unit-test the value of an annotation, unless you would use reflection - but in that case you would in fact have written a "change detector", which is probably not what you want.
You could write an integration test however. The "Testing" chapter from the Spring Boot Reference Guide provides an introduction how you could integration test your application.
Basically, the procedure is as follows:
#RunWith(SpringJUnit4ClassRunner.class)
#WebIntegrationTest
#SpringApplicationConfiguration(classes = SampleDataJpaApplication.class)
public class MyTest {
#Value("${local.server.port}")
int port;
RestTemplate template = new TestRestTemplate();
#Test
public void testRequest() throws Exception {
template.getForEntity("http://localhost:" + port + "/rest/path/dosomething", String.class);
// Somehow verify that your application did the right thing, e.g. because some mocked component was called or the system is in a certain state.
}
}