I have a moderately heavy springboot service, it takes 10-15 seconds to boot on a happy flow, and (1-2) minutes to fail on a retry/failover flow. This is ok for my business flows, and is how I expect a healthy service to behave.
I have integration tests (that run some end-to-end flows in my service), that can only test the actual integration status while the test machine (or dev machine) is connected to a specific VPN.
I want to auto skip integration tests if I'm not connected to VPN.
consider the following code
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {Server.class}, // auto scans a bunch of components
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) // slow loading context
public class IntegrationTest {
#BeforeClass
public static void beforeClass() {
Assume.assumeTrue(DnsTool.vpnConnected()); // fast failing test
}
#Test
public void testIntegration() {
// some test logic
}
}
When the assumptions pass, my tests run, and all is good.
When the assumptions fail, my tests get skipped, but only after trying to load my expensive context.
How can I avoid the long running time for my test suite?
Things I tried:
Subclassing SpringJUnit4ClassRunner, and overriding isTestMethodIgnored.
Adding a TestExecutionListener, and throwing the assumption exception in beforeTestClass
These made no impression on Spring, and the context got loaded any way.
Things I didn't try:
Lazy init comes with 2.2.X next stable release of spring I think.
Lazy init potentially makes my problem go away, but I feel like there should be some easy spring-test/junit fix that I'm missing.
Thanks in advance for the help.
To me, this sounds like something that you shouldn't do in tests at all.
Tests (at least IMHO), are supposed to check the business cases and assume that the environment is set up and ready.
Maybe it worth to delegate this functionality to build tool and CI.
Example:
Define a profile in maven (or whatever build tool you use) that will run integration tests that require VPN. Define profile that will run all the rest of integration tests as well.
Activate the profile if some system property is available.
In CI tool (like Jenkins) as a part of CI even before you run maven, run the script that will check the VPN connection. Based on the results set the system properties and run maven with these properties. The required profiles will be loaded and all the tests / only tests that do not require VPN will be run.
Update
If you need to make it work from Spring (and it looks like you prefer this way),
Spring has a special annotation called #IfProfileValue
By default, it matches against system properties and if the value doesn't match the test gets ignored.
It looks something like this (and note that you can put this annotation on class as well, then it will work for all test methods in the class):
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyTestClass {
#IfProfileValue(name = "os.name", values = {"Linux"})
#Test
public void testMe() {
// will run only in linux, otherwise
// won't even try to load an
// application context
....
}
}
This covers the case when you resolve the VPN connectivity externally and run the tests with a property. However, if you want to implement the VPN connectivity check in java, this annotation along not enough because it can work only with Java system properties, so in order to work with custom logic you need to implement org.springframework.test.annotation.ProfileValueSource:
public class VPNConnectivityProfileValueSource implements ProfileValueSource {
private String vpnEnabled = "true";
public VPNConnectivityProfileValueSource () {
// no spring context is available here
ClassPathResource resource = new ClassPathResource("vpn-config.properties");
if (resource.exists()) {
// read the VPN address,
//
//this.testProps = PropertiesLoaderUtils.loadProperties(resource);
// invoke your utility, check the connectivity, etc.
this.vpnEnabled = ...
}
}
#Override
public String get(String key) {
// this is important method,
if(key.equals("vpn.enabled") {
return this.vpnEnabled;
}
else return System.getProperty(key);
}
}
The last thing is to make the test aware of the ProfileValueSource:
For this there is another special annotation that you put on the test:
#ProfileValueSourceConfiguration(VPNConnectivityProfileValueSource.class)
All in all it the test can look like this:
#RunWith(SpringRunner.class)
#SpringBootTest
#ProfileValueSourceConfiguration(VPNConnectivityProfileValueSource.class)
#IfProfileValue(name = "vpn.enabled", value = "true")
public class MyTestClass {
#Test
public void testMe() {
....
}
}
All the classes/annotations I've mentioned reside in package org.springframework.test.annotation
Related
I'm writing a #Repository/#Service integration test that leverages an embedded database. In my test class, I would like to preload my database with some data.
I'm currently using #BeforeEach to load in my sample data, however, this code is run upon each test in my class.
Is there any way that I can load in my test data after Spring application context has loaded, but before any test has been run?
My current approach:
#BeforeEach
public void before() {
repository.save(...); // -> prepopulates repository with sample data
}
#Test
public void testService() {
service.get(...); // -> gathers existing record
}
#Test
public void deleteById() {
service.delete(...); // -> deletes existing record
}
However... with this, I am required to flush out the records after every test. Otherwise any unique constraints can easily be violated.
Rather than using #BeforeEach which is required to run before every test... is it possible to load this in in a #BeforeAll kind of fashion that happens after the spring application context has been loaded?
Is there any way that I can load in my test data after Spring application context has loaded
Basically yes, I think you can do that:
The idea is to load the SQL data when the application context is started or in the process of being started.
For example, spring boot integration with Flyway works this way (the bean of Flyway is created and loaded). So, in theory, you could merely use Flyway with test migrations that will contain all the relevant SQL scripts of test data generation.
How can you do this technically?
Here is one way:
Create a special bean (just like the way it works with Flyway) that would depend on your repository and in post construct save the data:
#Component
public class SqlGenerationBean {
#Autowired
private MyRepository repo;
#PostConstruct
public void init() {
repo.save();
}
}
Another way of doing is to create a listener that will be called upon the application context started and again will call the same repo.save().
In both cases the bean/listener code should not be accessible from production (it's only for tests): so put it somewhere under src/test/java for example
Now once the application context is started you can use a neat trick:
Mark your tests with #Transactional annotation. Spring will wrap the code in an artificial transaction that will be rolled back automatically (even if the test succeeds) so that all the data that you'll modify during the test will be rolled back and basically before each test, you'll have the same state (that is identical to the state of the database when/after the application context starts). Of course, if you use DDL in the test, some databases can't make it a part of transaction but it depends on the database really.
Another interesting point here is that the application context can be cached even between the test cases (created only once), so keep this in mind.
In this case I would just create a constructor for the test class. It will be triggered before everything.
#BeforeEach runs before each tests but after all initialisations .
you can also just use Mockito and mock the result without need to clean and overcomplicate
Just add following snippet to your code. This is just like you can do to detect that Spring application is really started.
#Configuration
public class AppConfig implements ApplicationListener<ApplicationReadyEvent> {
/**
* This is to indicate in the logs when the application has actually started and everything is loaded.
*/
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
ApplicationContext context = event.getApplicationContext();
Environment env = context.getEnvironment();
// do what you want on application start
}
}
P.S. For database manipulation in test #Sql is the best candidate as was mentioned in comment.
Context
I have a suite of Integration tests in a Spring boot application. The test context uses a MSSQL docker container for it's database using the testcontainers framework.
Some of my tests use Mockito with SpyBean which, apparently by design, will restart the Spring context since the spied beans cannot be shared between tests.
Since I am using a non-embedded database that lives for the duration of all my tests, the database is provisioned by executing my schema.sql and data.sql at the start by using:-
spring.datasource.initialization-mode=always
The problem is that when the Spring context is restarted, my database is re-initialized again which triggers errors such as unique constraint issues, table already exists etc.
My parent test class is as follows if it's of any help:-
#ActiveProfiles(Profiles.PROFILE_TEST)
#Testcontainers
#SpringJUnitWebConfig
#AutoConfigureMockMvc
#SpringBootTest(classes = Application.class)
#ContextConfiguration(initializers = {IntegrationTest.Initializer.class})
public abstract class IntegrationTest {
private static final MSSQLServerContainer<?> mssqlContainer;
static {
mssqlContainer = new MSSQLServerContainer<>()
.withInitScript("setup.sql"); //Creates users/permissions etc
mssqlContainer.start();
}
static class Initializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
#Override
public void initialize(final ConfigurableApplicationContext configurableApplicationContext) {
TestPropertyValues.of("spring.datasource.url=" + mssqlContainer.getJdbcUrl())
.applyTo(configurableApplicationContext.getEnvironment());
}
}
}
Each integration test extends this so that the context (for non-spied tests) is shared and setup occurs just once.
What I want
I would like to be able to execute the startup scripts just one time on startup and never again despite any number of context reloads. If the Spring test framework could remember that I already have a provisioned DB, that would be ideal.
I am wondering if there are any existing configurations or hooks that may help me
If something like the following existed, it'd be perfect.
spring.datasource.initialization-mode=always-once
But, as far as I can tell, it doesn't :(
Possible, but incomplete, solutions
Test container init script
new MSSQLServerContainer<>().withInitScript("setup.sql");
This works and ensures I can run a startup script the first time only since the container is started up just once. However withInitScript only takes a single argument rather than an array. As such, I would need to concatenate all my scripts into one file which means I'd have to maintain two sets of scripts.
If you only had one script, this would work fine.
Continue on error
spring.datasource.continue-on-error=true
This works in the sense that startup errors in the schema are ignored. But.. I want it to fail on startup if someone put some dodgy SQL in the scripts.
Spring event hooks
I couldn't get this to work. My idea was that I could listen for the ContextRefreshedEvent and then inject a new value for spring.datasource.initialization-mode=never.
It's a bit of a hack but I tried something like the following
#Component
public static class EventListener implements ApplicationListener<ApplicationEvent> {
#Autowired
private ConfigurableEnvironment environment;
#Override
public void onApplicationEvent(final ApplicationEvent event) {
log.info(event.getClass().getSimpleName());
if (event instanceof ContextRefreshedEvent) {
TestPropertyValues.of("spring.datasource.initialization-mode=never")
.applyTo(this.environment);
}
}
}
My guess is when the context restarts, it will also reload all my original property sources again which has mode=always. I would need an event right after the properties are loaded and right before the schema creation occurs.
So with that, does anyone have any suggestions?
So I ended up finding a workaround for this. Feels hacky but unless someone else is able to suggest a more appropriate and less obscure fix, then this is what I'll go with.
The solution uses a combination of #tsarenkotxt suggestion of AtomicBoolean and my #3 partial solution.
#ActiveProfiles(Profiles.PROFILE_TEST)
#Testcontainers
#SpringJUnitWebConfig
#AutoConfigureMockMvc
#SpringBootTest(classes = Application.class)
#ContextConfiguration(initializers = {IntegrationTest.Initializer.class})
public abstract class IntegrationTest {
private static final MSSQLServerContainer mssqlContainer;
//added this
private static final AtomicBoolean initDB = new AtomicBoolean(true);
static {
mssqlContainer = new MSSQLServerContainer()
.withInitScript("setup.sql"); //Creates users/permissions etc
mssqlContainer.start();
}
static class Initializer implements ApplicationContextInitializer {
#Override
public void initialize(final ConfigurableApplicationContext configurableApplicationContext) {
TestPropertyValues.of(
"spring.datasource.url=" + mssqlContainer.getJdbcUrl(),
//added this
"spring.datasource.initialization-mode=" + (initDB.get() ? "always" : "never"))
.applyTo(configurableApplicationContext.getEnvironment());
//added this
initDB.set(false);
}
}
}
Basically I set spring.datasource.initialization-mode to be always on the very first startup, since the db hasn't been setup yet, and then reset it to never for every context initialization thereafter. As such, Spring won't attempt to execute the startup scripts after the first run.
Works great but I don't like having to hide this configuration here so still hoping someone else will come up with something better and more "by design"
I have a number of very complex integration tests written with use of Spring Test framework. What I would like to do is to fetch last n lines from server log file in case test is failed.
For this purposes I created test:
#TestExecutionListeners({LogFetchingTestListener.class})
public MyComplexTestClass extends AbstractTestNGSpringContextTests {
#Test
public void myComplexTest() {
// Here goes the test logic...
}
}
and test listener:
public class LogFetchingTestListener extends ABstractTestExecutionListener {
#Override
public void afterTestExecution(TestContext context) {
// Use some spring beans to get entities to connect to servers
}
}
Everything is fine but I was unable to find properties in TestContext or somewhere else which will let me know if my test has failed or not.
It is pretty critical as I have a big number of tests and it is completely senseless to fetch logs for test cases that were run successfully.
Do Spring provide this information in test listeners? If no, is there any workarounds for this?
Thank you.
I'd recommend overriding afterTestMethod(TestContext) instead of afterTestExecution(TestContext), since the former will be executed after #AfterMethod lifecycle methods in the test class.
To obtain the exception (i.e., the cause of a failed test), you just need to access testContext.getTestException().
I would like to test a class that provides a rest endpoint via JAX-RS. This class depends on a JPA EntityManager an thus on a database which needs to be populated prior to test execution. I saw solutions for database population like dbunit, but I want to populate the data directly from my test class (or delegated via object mother pattern). But when testing rest endpoints I need to use the annotation option #Deployment(testable = false) which refuses me to inject the EntityManager into my test class.
So how can I solve this situation?
Or are there any better best practices? (maybe mocking, but that's also not possible for black box tests)
You could create a bean to generate your test data:
#Startup
#Singleton
public class TestDataGenerator {
#PersistenceContext
private EntityManager em;
#PostConstruct
private void generateTestData() {
// Generate your test data
}
}
The TestDataGenerator class defined above is annotated with #Singleton (ensuring there will be only one instance of the class) and #Startup (for eager initialization during the application startup sequence).
Add the TestDataGenerator class to your Arquillian deployment:
#RunWith(Arquillian.class)
public class MyArquillianTest {
private Client client = ClientBuilder.newClient();
#Deployment
#RunAsClient
public static WebArchive createDeployment() {
return ShrinkWrap.create(WebArchive.class)
.addClasses(TestDataGenerator.class, ...)
.addAsResource("test-persistence.xml", "META-INF/persistence.xml")
.addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml");
}
#Test
public void testAPI(#ArquillianResource URL deploymentUrl) {
// Test your REST service
WebTarget target = client.target(deploymentUrl.toURI()).path("api");
}
}
Observe that #RunAsClient is equivalent to #Deployment(testable = false).
The #ArquillianResource annotation allows you to inject the URL to test your web application.
For tests, I usually try and separate black box and unit testing completely (I suppose it's a preference on how you do it).
For example, my REST Api could rely on whatever it wants, but usually it doesn't do much but call my database layer or some sort of facade accessing my DB layer. The objects are injected, yes, but usually I make the fields package private, which meant that you can set them from the same package (which works with Junit as well).
For example:
public class Facade1 {
#Inject
Facade facade;
public void doSomething() { ... }
}
This class represents my REST API. I could test doSomething by simply adding a mock object as facade. Mind you, this is a quite useless test, but you get the idea. Unit tests should happen in isolation with as much mocking as possible.
Now testing the actual Rest API I usually resort to a python integration tester. Python has nice http libraries that allow you to make request very easily.
For that, I set up a staging environment for my Rest Server. The environment is a live-like representation for testing. Everything there needs to works and needs to be on the same version as the production deployment.
I then use python to poke my REST Api and verify the responses. Since I've set up my staging environment I have complete control over database content. Therefore it is easy for me to test that all responses are correct.
My typical process then is:
Compile
Build
Deploy
Integration test
I hope that helps. If you want clearer examples, you might want to post a bit more code as it's a bit hard to imagine for me what it is exactly you'd like to do :)
I have managed to get alternate descriptors to work with my unit-tests running on OpenEJB using stubs for dependant EJB components, when each test is executed on their own. But once I introduce a test suite, it seems that deployment descriptor is taken from the first test added to the suite.
Some code to explain it better. Beans under test are something like
#Stateless
#Local(A.class)
public class ABean implements A {
// Bean implementation, no dependencies
}
#Stateless
#Local(B.class)
public class BBean implements B {
#EJB
A aBean; // Dependency to ABean
// Rest of the implementation
}
And testcase for B (testcase for A is similar, except it does not set the property for using alternate descriptor)
public class BBeanTest {
private B bean;
#Before
public void bootContainer() throws Exception {
Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY,
"org.apache.openejb.client.LocalInitialContextFactory");
props.put("openejb.altdd.prefix", "test"); // Use stubs
System.out.println("boot B: " + props);
context = new InitialContext(props);
bean = (B) context.lookup("BBeanLocal");
}
}
And as said, this all works just fine when executed alone. The alternate descriptor injects a stub implementation of A interface.
When using the following test suite, things start to fall apart.
#RunWith(Suite.class)
#Suite.SuiteClasses({
ABeanTest.class,
BBeanTest.class
})
public class MySuite {
// Empty on purpose, annotations do the trick
}
When running this suite, the alternate descriptor for testing B is not taken into use. Although, the output shows that at least the property is set before each test
boot A: {java.naming.factory.initial=org.apache.openejb.client.LocalInitialContextFactory}
boot A: {java.naming.factory.initial=org.apache.openejb.client.LocalInitialContextFactory}
boot A: {java.naming.factory.initial=org.apache.openejb.client.LocalInitialContextFactory}
boot B: {java.naming.factory.initial=org.apache.openejb.client.LocalInitialContextFactory, openejb.altdd.prefix=test}
boot B: {java.naming.factory.initial=org.apache.openejb.client.LocalInitialContextFactory, openejb.altdd.prefix=test}
If I reverse the order of loading tests to suite, i.e. add BBeanTest.class before ABeanTest.class, it'll use the alternate descriptor. As the ABean has no dependencies, this'll work fine in this case, but probably causes problems with bigger setups with multiple alternate descriptors.
Any pointers?
Thanks in advance.
EDIT Based on the log output, the container is actually booted only once for the first test as it takes approx. 2,5 seconds to execute while the others take around 0,001 seconds.
EDIT2 OpenEJB version is Apache OpenEJB 3.1.4 build: 20101112-03:32
Based on the log output, the container is actually booted only once for the first test as it takes approx. 2,5 seconds to execute while the others take around 0,001 seconds.
As you rightly noticed, the initialization happens only once.
#RunWith(Suite.class)
#Suite.SuiteClasses({
ABeanTest.class,
BBeanTest.class
})
Hence in this case, both ABeanTest and BBeanTest ran within the same container instance, with the same initial context properties as set by ABeanTest.
In your case, since you need different settings for the two test classes, I think dumping the container instance at ABeanTest #AfterClass and using a new one in BBeanTest should do it.
This blog post shows how