Develop Reference

Spring boot with testcontainers - how to prevent DB initialization on context reload

Context
I have a suite of Integration tests in a Spring boot application. The test context uses a MSSQL docker container for it's database using the testcontainers framework.
Some of my tests use Mockito with SpyBean which, apparently by design, will restart the Spring context since the spied beans cannot be shared between tests.
Since I am using a non-embedded database that lives for the duration of all my tests, the database is provisioned by executing my schema.sql and data.sql at the start by using:-
spring.datasource.initialization-mode=always
The problem is that when the Spring context is restarted, my database is re-initialized again which triggers errors such as unique constraint issues, table already exists etc.
My parent test class is as follows if it's of any help:-
#ActiveProfiles(Profiles.PROFILE_TEST)
#Testcontainers
#SpringJUnitWebConfig
#AutoConfigureMockMvc
#SpringBootTest(classes = Application.class)
#ContextConfiguration(initializers = {IntegrationTest.Initializer.class})
public abstract class IntegrationTest {
private static final MSSQLServerContainer<?> mssqlContainer;
static {
mssqlContainer = new MSSQLServerContainer<>()
.withInitScript("setup.sql"); //Creates users/permissions etc
mssqlContainer.start();
}
static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(final ConfigurableApplicationContext configurableApplicationContext) {
TestPropertyValues.of("spring.datasource.url=" + mssqlContainer.getJdbcUrl())
.applyTo(configurableApplicationContext.getEnvironment());
}
}
}
Each integration test extends this so that the context (for non-spied tests) is shared and setup occurs just once.
What I want
I would like to be able to execute the startup scripts just one time on startup and never again despite any number of context reloads. If the Spring test framework could remember that I already have a provisioned DB, that would be ideal.
I am wondering if there are any existing configurations or hooks that may help me
If something like the following existed, it'd be perfect.
spring.datasource.initialization-mode=always-once
But, as far as I can tell, it doesn't :(
Possible, but incomplete, solutions
Test container init script
new MSSQLServerContainer<>().withInitScript("setup.sql");
This works and ensures I can run a startup script the first time only since the container is started up just once. However withInitScript only takes a single argument rather than an array. As such, I would need to concatenate all my scripts into one file which means I'd have to maintain two sets of scripts.
If you only had one script, this would work fine.
Continue on error
spring.datasource.continue-on-error=true
This works in the sense that startup errors in the schema are ignored. But.. I want it to fail on startup if someone put some dodgy SQL in the scripts.
Spring event hooks
I couldn't get this to work. My idea was that I could listen for the ContextRefreshedEvent and then inject a new value for spring.datasource.initialization-mode=never.
It's a bit of a hack but I tried something like the following
#Component
public static class EventListener implements ApplicationListener<ApplicationEvent> {
#Autowired
private ConfigurableEnvironment environment;
#Override
public void onApplicationEvent(final ApplicationEvent event) {
log.info(event.getClass().getSimpleName());
if (event instanceof ContextRefreshedEvent) {
TestPropertyValues.of("spring.datasource.initialization-mode=never")
.applyTo(this.environment);
}
}
}
My guess is when the context restarts, it will also reload all my original property sources again which has mode=always. I would need an event right after the properties are loaded and right before the schema creation occurs.
So with that, does anyone have any suggestions?

So I ended up finding a workaround for this. Feels hacky but unless someone else is able to suggest a more appropriate and less obscure fix, then this is what I'll go with.
The solution uses a combination of #tsarenkotxt suggestion of AtomicBoolean and my #3 partial solution.
#ActiveProfiles(Profiles.PROFILE_TEST)
#Testcontainers
#SpringJUnitWebConfig
#AutoConfigureMockMvc
#SpringBootTest(classes = Application.class)
#ContextConfiguration(initializers = {IntegrationTest.Initializer.class})
public abstract class IntegrationTest {
private static final MSSQLServerContainer mssqlContainer;
//added this
private static final AtomicBoolean initDB = new AtomicBoolean(true);
static {
mssqlContainer = new MSSQLServerContainer()
.withInitScript("setup.sql"); //Creates users/permissions etc
mssqlContainer.start();
}
static class Initializer implements ApplicationContextInitializer {
#Override
public void initialize(final ConfigurableApplicationContext configurableApplicationContext) {
TestPropertyValues.of(
"spring.datasource.url=" + mssqlContainer.getJdbcUrl(),
//added this
"spring.datasource.initialization-mode=" + (initDB.get() ? "always" : "never"))
.applyTo(configurableApplicationContext.getEnvironment());
//added this
initDB.set(false);
}
}
}
Basically I set spring.datasource.initialization-mode to be always on the very first startup, since the db hasn't been setup yet, and then reset it to never for every context initialization thereafter. As such, Spring won't attempt to execute the startup scripts after the first run.
Works great but I don't like having to hide this configuration here so still hoping someone else will come up with something better and more "by design"

Skipping integration tests in spring boot on VPN DNS availability

I have a moderately heavy springboot service, it takes 10-15 seconds to boot on a happy flow, and (1-2) minutes to fail on a retry/failover flow. This is ok for my business flows, and is how I expect a healthy service to behave.
I have integration tests (that run some end-to-end flows in my service), that can only test the actual integration status while the test machine (or dev machine) is connected to a specific VPN.
I want to auto skip integration tests if I'm not connected to VPN.
consider the following code
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {Server.class}, // auto scans a bunch of components
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) // slow loading context
public class IntegrationTest {
#BeforeClass
public static void beforeClass() {
Assume.assumeTrue(DnsTool.vpnConnected()); // fast failing test
}
#Test
public void testIntegration() {
// some test logic
}
}
When the assumptions pass, my tests run, and all is good.
When the assumptions fail, my tests get skipped, but only after trying to load my expensive context.
How can I avoid the long running time for my test suite?
Things I tried:
Subclassing SpringJUnit4ClassRunner, and overriding isTestMethodIgnored.
Adding a TestExecutionListener, and throwing the assumption exception in beforeTestClass
These made no impression on Spring, and the context got loaded any way.
Things I didn't try:
Lazy init comes with 2.2.X next stable release of spring I think.
Lazy init potentially makes my problem go away, but I feel like there should be some easy spring-test/junit fix that I'm missing.
Thanks in advance for the help.

To me, this sounds like something that you shouldn't do in tests at all.
Tests (at least IMHO), are supposed to check the business cases and assume that the environment is set up and ready.
Maybe it worth to delegate this functionality to build tool and CI.
Example:
Define a profile in maven (or whatever build tool you use) that will run integration tests that require VPN. Define profile that will run all the rest of integration tests as well.
Activate the profile if some system property is available.
In CI tool (like Jenkins) as a part of CI even before you run maven, run the script that will check the VPN connection. Based on the results set the system properties and run maven with these properties. The required profiles will be loaded and all the tests / only tests that do not require VPN will be run.
Update
If you need to make it work from Spring (and it looks like you prefer this way),
Spring has a special annotation called #IfProfileValue
By default, it matches against system properties and if the value doesn't match the test gets ignored.
It looks something like this (and note that you can put this annotation on class as well, then it will work for all test methods in the class):
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyTestClass {
#IfProfileValue(name = "os.name", values = {"Linux"})
#Test
public void testMe() {
// will run only in linux, otherwise
// won't even try to load an
// application context
....
}
}
This covers the case when you resolve the VPN connectivity externally and run the tests with a property. However, if you want to implement the VPN connectivity check in java, this annotation along not enough because it can work only with Java system properties, so in order to work with custom logic you need to implement org.springframework.test.annotation.ProfileValueSource:
public class VPNConnectivityProfileValueSource implements ProfileValueSource {
private String vpnEnabled = "true";
public VPNConnectivityProfileValueSource () {
// no spring context is available here
ClassPathResource resource = new ClassPathResource("vpn-config.properties");
if (resource.exists()) {
// read the VPN address,
//
//this.testProps = PropertiesLoaderUtils.loadProperties(resource);
// invoke your utility, check the connectivity, etc.
this.vpnEnabled = ...
}
}
#Override
public String get(String key) {
// this is important method,
if(key.equals("vpn.enabled") {
return this.vpnEnabled;
}
else return System.getProperty(key);
}
}
The last thing is to make the test aware of the ProfileValueSource:
For this there is another special annotation that you put on the test:
#ProfileValueSourceConfiguration(VPNConnectivityProfileValueSource.class)
All in all it the test can look like this:
#RunWith(SpringRunner.class)
#SpringBootTest
#ProfileValueSourceConfiguration(VPNConnectivityProfileValueSource.class)
#IfProfileValue(name = "vpn.enabled", value = "true")
public class MyTestClass {
#Test
public void testMe() {
....
}
}
All the classes/annotations I've mentioned reside in package org.springframework.test.annotation

Junit Spring avoid to load twice application context datasource

I have this configuration classes:
#ComponentScan(
basePackages = {
"mypackage.controller",
"mypackage.service",
"mypackage.repository"
}
)
#TestPropertySource(locations="classpath:configuration.properties")
#Import({
H2Configuration.class
})
public class TestConfiguration {
}
#Configuration
public class H2Configuration {
#Bean
public DataSource dataSource() throws SQLException {
EmbeddedDatabaseBuilder builder = new EmbeddedDatabaseBuilder();
EmbeddedDatabase db = builder
.setType(EmbeddedDatabaseType.H2)
.addScript("h2/create.sql")
.addScript("h2/insert.sql")
.build();
db.getConnection().setAutoCommit(false);
return db;
}
}
And I have this two class tests:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(loader=AnnotationConfigContextLoader.class, classes = { TestConfiguration.class })
public class FirstRepositoryTest {
#Autowired
MyFirstRepositoryImpl repository;
#Before
public void initTest() {
}
#Test(expected = NullPointerException.class)
public void testNullRecords() {
repository.foo(null, null);
}
}
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(loader=AnnotationConfigContextLoader.class, classes = { TestConfiguration.class })
public class SecondRepositoryTest {
#Autowired
MySecondRepositoryImpl repository;
#Before
public void initTest() {
}
#Test(expected = NullPointerException.class)
public void testSomethingNullRecords() {
repository.something(null, null);
}
}
If I run junit test once for each class, all goes well.
In clean install phase tests fails because the application context is initialized twice.
For example it try to create the h2 tables twice and do the insert.sql script twice.
What I have to do for initialize the h2 database and so application context only once?
Thanks

I think you could start looking at the Spring documentation about Integration Testing.
It can also be a good practice to use transactional tests for integration tests (#Transactional), which rollback at the end of each test : see Transaction Management.
To avoid the cost of recreating the ApplicationContext for each test class, the cache may be used as explained here : Context Caching.
For integration testing with Embedded Database, you can also find documentation : Testing Data Access Logic with an Embedded Database.
A note from the previous link, matching your use case :
However, if you wish to create an embedded database that is shared
within a test suite, consider using the Spring TestContext Framework
and configuring the embedded database as a bean in the Spring
ApplicationContext as described in Creating an Embedded Database by
Using Spring XML and Creating an Embedded Database Programmatically.
I hope you will find some useful references.

Another good tip I found from Spring Boot documentation from Embedded Database Support :
They say :
If you are using this feature in your tests, you may notice that the
same database is reused by your whole test suite regardless of the
number of application contexts that you use. If you want to make sure
that each context has a separate embedded database, you should set
spring.datasource.generate-unique-name to true.
So to make each EmbeddedDatabase unique, you may try to create them with :
EmbeddedDatabase db = new EmbeddedDatabaseBuilder()
.generateUniqueName(true)
...
.build();

In unit testing you must garantee that every test is repeatible hance context independent. Due to this is not good idea to load the context only once. Is better to reset after the execution. For this you can use #DirtiesContext(classMode = ClassMode.AFTER_CLASS) in your test classes
So you will force your context to restart when the next junit class is launched

So the reason that this is failing is that the database (H2) is resident in memory when you run the tests as part of clean/install. The create/insert scripts have already executed after the first test is run. Any subsequent test execution after this point will result in a re-execution of the same script(s) and the error will occur.
Update your create script with a DROP TABLE IF EXISTS <table name>;. This will ensure that the table is dropped then recreated.
NOTE: I'm not sure why you've specified AnnotationConfigContextLoader explicitly. I think, without that, the runner SpringJUnit4ClassRunner will cache contexts that have not been changed. I don't know specifically if that is the case here though.

How to test Rest-APIs that rely on persistence with arquillian

I would like to test a class that provides a rest endpoint via JAX-RS. This class depends on a JPA EntityManager an thus on a database which needs to be populated prior to test execution. I saw solutions for database population like dbunit, but I want to populate the data directly from my test class (or delegated via object mother pattern). But when testing rest endpoints I need to use the annotation option #Deployment(testable = false) which refuses me to inject the EntityManager into my test class.
So how can I solve this situation?
Or are there any better best practices? (maybe mocking, but that's also not possible for black box tests)

You could create a bean to generate your test data:
#Startup
#Singleton
public class TestDataGenerator {
#PersistenceContext
private EntityManager em;
#PostConstruct
private void generateTestData() {
// Generate your test data
}
}
The TestDataGenerator class defined above is annotated with #Singleton (ensuring there will be only one instance of the class) and #Startup (for eager initialization during the application startup sequence).
Add the TestDataGenerator class to your Arquillian deployment:
#RunWith(Arquillian.class)
public class MyArquillianTest {
private Client client = ClientBuilder.newClient();
#Deployment
#RunAsClient
public static WebArchive createDeployment() {
return ShrinkWrap.create(WebArchive.class)
.addClasses(TestDataGenerator.class, ...)
.addAsResource("test-persistence.xml", "META-INF/persistence.xml")
.addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml");
}
#Test
public void testAPI(#ArquillianResource URL deploymentUrl) {
// Test your REST service
WebTarget target = client.target(deploymentUrl.toURI()).path("api");
}
}
Observe that #RunAsClient is equivalent to #Deployment(testable = false).
The #ArquillianResource annotation allows you to inject the URL to test your web application.

For tests, I usually try and separate black box and unit testing completely (I suppose it's a preference on how you do it).
For example, my REST Api could rely on whatever it wants, but usually it doesn't do much but call my database layer or some sort of facade accessing my DB layer. The objects are injected, yes, but usually I make the fields package private, which meant that you can set them from the same package (which works with Junit as well).
For example:
public class Facade1 {
#Inject
Facade facade;
public void doSomething() { ... }
}
This class represents my REST API. I could test doSomething by simply adding a mock object as facade. Mind you, this is a quite useless test, but you get the idea. Unit tests should happen in isolation with as much mocking as possible.
Now testing the actual Rest API I usually resort to a python integration tester. Python has nice http libraries that allow you to make request very easily.
For that, I set up a staging environment for my Rest Server. The environment is a live-like representation for testing. Everything there needs to works and needs to be on the same version as the production deployment.
I then use python to poke my REST Api and verify the responses. Since I've set up my staging environment I have complete control over database content. Therefore it is easy for me to test that all responses are correct.
My typical process then is:
Compile
Build
Deploy
Integration test
I hope that helps. If you want clearer examples, you might want to post a bit more code as it's a bit hard to imagine for me what it is exactly you'd like to do :)

How to call datasource in Junit

I have a maven WAR package with JSF pages. Usually I use
#Resource(name = "jdbc/Oracle")
private DataSource ds;
to call database when I deploy the WAR package on Glassfish server. But in case of JUnit tests when I build the package on my laptop with netbeans I cannot use this datasource.
How I can solve this problem? I want to run the JUnit tests with database tables right after I build the package but I don't have a datasource.
What are the possible solutions?

Do you actually want to run your unit tests against the database? Personally I would try to avoid this, since it usually ties the test too closely to the state of the database and often prevents you from actually testing the "unit" and all the possible states you might like to handle. It will also probably make your unit tests take some time to run, which is not ideal.
An alternative would be to create a mock DataSource, using for example EasyMock or Mockito. Or you could create your own mock implementation of the DataSource interface, if you know you would like to define some common behaviour for DataSources across many tests.
If you really want to use the database, you would have to look at manually instantiating whatever implementation of DataSource you are using (e.g. OracleDataSource) and then using this in your class.
In either case, you will probably have to switch to using constructor or method injection to make it a bit easier to set the DataSource on the instance you are testing. (Otherwise you will have to use reflection to set the private variable.)
For example, your class might look like this:
public class DatabaseOperations {
private DataSource dataSource;
#Resource(name = "jdbc/Oracle")
public void setDataSource(DataSource dataSource) {
this.dataSource = dataSource;
}
}
And then your test might look like this:
public class DatabaseOperationsTest {
public void testSomeOperation() {
DatabaseOperations databaseOperations = new DatabaseOperations();
databaseOperations.setDataSource(new MockDataSource());
}
}

If you really do need to run tests using an injected DataSource, you could consider using Arquillian which will create a deployment unit for you, deploy it to either an embedeed or remote Glassfish container, together with a configured DataSource specifically for testing if you wish. They have a guide for this specific scenario.
The advantage is that you will have a full-blown container with CDI. You control what gets package so you can provide test stubs for CDI classes. You can also control the deployment configuration (test vs. production configuration). It is non-invasive.