Dynamically creating schema.sql before Spring context - java

I'm writing integration tests for project and I want to merge all database migration scripts into schema.sql before Spring picks it up to populate the database.
For this I use a small class that searches the project for sql files and merges them into one.
I've created a Suite like so:
#RunWith(Suite.class)
#Suite.SuiteClasses({MyTests.class})
public class SuiteTest {
#BeforeClass
public static void setUp() throws IOException {
RunMigrations.mergeMigrations();//this one merges all sqls into one file, called schema.sql
}
}
Then, here's my test:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(classes = App.class)
#ActiveProfiles(resolver = CustomActiveProfileResolver.class)
#ContextConfiguration(classes = App.class)
public class MyTests extends AbstractTransactionalJUnit4SpringContextTests {
#PostConstruct
public void before() {
mvc = MockMvcBuilders.webAppContextSetup(context).addFilter(springSecurityFilterChain).build();
}
#Test
#Transactional
public void Test1(){ //do stuff }
}
But this doesn't work as I thought it would. It looks like Spring tries to run schema.sql faster than I'm creating it and fails like this:
Caused by: org.hibernate.tool.schema.spi.SchemaManagementException: Schema-validation: missing table [application]
If I just turn off my code that generates schema.sql and let Spring run with already created schema then all is good. But if I delete schema.sql and let my class to generate it, then it fails as described.
I've tried to override run(RunNotifier notifier) method in SpringJUnit4ClassRunner and put my migration merger in there, before it calls super.run(notifier) method, but that still doesn't work.
Is there a way to generate that schema.sql before Spring gets its hands on it?
P.S.
I cannot use flyway for production environment. Perhaps it is possible using it just for tests?
UPD:
After some experimenting, I've set this in test.yml:
spring.jpa.hibernate.ddl-auto: none
Now it loads context, performs one test which just gets Oauth2 token and fails with other tests that perform POST and GET requests because it cannot execute #sql annotations that put additional data before test methods. The database seems to be untouched, that is without any tables whatsoever.


You can enable flyway just for tests either by using #TestPropertySource(properties = {"spring.flyway.enabled=true"}) annotation or by creating a test Spring profile with it's own property file. The latter would look like:
#RunWith(SpringRunner.class)
#SpringBootTest
#ActiveProfiles("test")
public MyTest {
with src/test/resources/application-test.yml file:
spring:
flyway:
enabled: true
and flyway-core as a test scoped dependency.
Do note that Flyway properties in Spring Boot were renamed in Spring Boot 2.0.


Perhaps someone may find this useful.
I managed to solve this by using exec-maven-plugin with failsafe plugin.
The test class setup remains the same, I only removed #BeforeClass annotation from Suite.
Here's the POM:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${org.apache.maven.plugins.maven-surefire-plugin-version}</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${org.apache.maven.plugins.maven-failsafe-plugin-version}</version>
<executions>
<execution>
<id>integration-test-for-postgres</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify-for-postgres</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>${org.codehaus.mojo.exec-maven-plugin-version}</version>
<executions>
<execution>
<id>build-test-environment</id>
<phase>generate-test-resources</phase>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<!--This class prepares the schema.sql file which is fed to Spring to init DB before tests.-->
<mainClass>...GenerateTestDBSchema</mainClass>
<arguments>
<argument>...</argument><!--the migration folder-->
<argument>...</argument><!--The path where to put schema sql-->
</arguments>
</configuration>
</plugin>
</plugins>
</build>
The GenerateTestDBSchema class has main method and uses args array to accept paths where to find migrations and where to put schema.sql.
public static void main(String[] args) {
try {
mergeMigrations(args[0], args[1]);
} catch (IOException e) {
LOG.error(e.getMessage());
}
}
The mergeMigrations() method is straightforward: just take all files from directory, merge them and write to output path. This way Spring has it's schema.sql before context is launched and it itself decides where to run migrations.
Thanks to #ActiveProfiles(resolver = CustomActiveProfileResolver.class) in integration test, spring resolves profiles and picks up application-{profileName}.yml and sets database address automatically.

Related

Java Maven OpenApi (3.0) Codegen is generating unwanted test file, how to remove that?

I am using OpenApi(3.0) for api definition and the openapi-generator-maven-plugin which generates files for me (api objects + endpoints).
Its however generating a test file in the build folder that I do not want. Its called 'OpenApiGeneratorApplicationTests'. It always blocks my compilation bc in the 'target' (=build) folder I do not have the right Spring Boot setup.
How can I avoid the generation of this test file?
This is my maven config:
<plugin>
<groupId>org.openapitools</groupId>
<artifactId>openapi-generator-maven-plugin</artifactId>
<version>6.0.1</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>${project.basedir}/src/main/resources/ApiDef.yaml</inputSpec>
<generatorName>spring</generatorName>
<modelPackage>${clientPackage}.model</modelPackage>
<invokerPackage>${clientPackage}.invoker</invokerPackage>
<apiPackage>${clientPackage}.api</apiPackage>
<generateApis>true</generateApis>
<generateApiTests>false</generateApiTests>
<generateModelTests>false</generateModelTests>
<configOptions>
<delegatePattern>true</delegatePattern>
</configOptions>
</configuration>
</execution>
</executions>
</plugin>

I've encountered today the same issue. So long story short:
By default the generator maven plugin for Spring will generate the 'invoker' - which by implementation is actually a #SpringBootApplication starter class.
#SpringBootApplication
#ComponentScan(basePackages = {"com.mypackage.invoker", "com.mypackage.api" , "org.openapitools.configuration"})
public class OpenApiGeneratorApplication {
public static void main(String[] args) {
SpringApplication.run(OpenApiGeneratorApplication.class, args);
}
#Bean
public Module jsonNullableModule() {
return new JsonNullableModule();
}
}
and will also generate the associated test class OpenApiGeneratorApplicationTests
#SpringBootTest
class OpenApiGeneratorApplicationTests {
#Test
void contextLoads() {
}
}
The problem here is that the src/test/com/mypackage/invoker/OpenApiGeneratorApplicationTests.java, and actually the whole package under test is marked as source, not as test-source and in my case this will result in compile time errors (since the test-scoped dependencies are not found).
I managed to get around it by configuring the maven plugin with
<configOptions>
<interfaceOnly>true</interfaceOnly>
...
</configOptions>
which will skip the generation of the SpringBootApplication and the associated Tests.
I've also submitted a ticket to the OpenApi Generator project's github for marking the

Configuring connection and read timeouts for a generated api rest client

I am using swagger-codegen-maven-plugin to generate an api rest client. I am using the resttemplate library and my pom.xml configuration looks like the following.
<build>
<plugins>
<plugin>
<groupId>io.swagger</groupId>
<artifactId>swagger-codegen-maven-plugin</artifactId>
<version>2.3.1</version>
<executions>
<execution>
<goals>
<goal>generate</goal>
</goals>
<configuration>
<inputSpec>${project.basedir}/../swagger.yml</inputSpec>
<language>java</language>
<generateApis>true</generateApis>
<generateApiTests>false</generateApiTests>
<generateSupportingFiles>true</generateSupportingFiles>
<generateModelDocumentation>false</generateModelDocumentation>
<modelPackage>my.base.package.here</modelPackage>
<apiPackage>my.base.api.package.here</apiPackage>
<modelNamePrefix>MyApiPrefix</modelNamePrefix>
<configOptions>
<library>resttemplate</library>
<sourceFolder>src/gen/java</sourceFolder>
<java8>true</java8>
<dateLibrary>java8</dateLibrary>
<hideGenerationTimestamp>true</hideGenerationTimestamp>
</configOptions>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
The client generation works sucessfully. Now I am using this client in a spring boot app and I need to configure the connection timeout and the read timeout values.
I was surprised to find no setters for these two properties on the generated ApiClient. So I had to come out with a workaround that uses the RestTemplateBuilder to do that. It looks like this:
#Configuration
public class MyApiClientConfiguration {
#Autowired
public MyApiClientConfiguration(
final RestTemplate restTemplate,
final RestTemplateBuilder restTemplateBuilder
) {
restTemplateBuilder
.setConnectTimeout(connectTimeoutValue)
.setReadTimeout(readTimeoutValue)
.configure(restTemplate);
ApiClient apiClient = new ApiClient(restTemplate);
MyApiPrefixApi.setApiClient(apiClient);
}
}
The doc for the configure method does not say much:
configure
public T configure(T restTemplate)
Configure the provided RestTemplate instance using this builder.
Is this the standard approach to configure timeouts when using the resttemplate library? UPDATE: My actual implementation does not work (it seems the ResttemplateBuilder own configurations are interfering in a wrong way with the autowired RestTemplate (that I am trying to further configure through the builder)) configuration. The api client generated by other libraries (such as jersey for example) provide a setter to directly configure the timeout using:
apiClient.setConnectTimeout(timeoutValue);

Though the use of the configure method does not work, it is possible to configure the timeout values by creating a RestTemplate bean using the RestTemplateBuilder's build method.
#Bean
public RestTemplate restTemplate(
RestTemplateBuilder restTemplateBuilder,
final int serviceConnectionTimeout,
final int serviceReadTimeout
) {
return restTemplateBuilder
.setConnectTimeout(serviceConnectionTimeout)
.setReadTimeout(serviceReadTimeout)
.build();
}
Ref: Spring RestTemplate timeout

maven-failsafe-plugin: Groups are getting ignored when setUp parameterized Test

I have configured the maven-failsafe-plugin for excluding/including some testcategories:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<groups>my.company.SomeImportantCategory</groups>
</configuration>
</plugin>
</plugins>
Now I have a parameterized test which is NOT annotated with SomeImportantCategory:
#RunWith(Parameterized.class)
#Category(AnotherCategory.class)
public class SomeTestIT {
#Parameters(name = "{0}")
public static Collection<TestData[]> setUp() throws Exception {
// Loading Some Excel-Sheets. I'm using github.com/fhm84/jexunit
return doSomethingVeryTimeConsuming("with/this/excel-file.xls");
}
}
Now I run the integration-tests with this profile. Maven does execute the setUp-Method for collecting test-cases.
Do you know how to skip this?
It would be ok for me to access the setUp-Method and do some Java-magic like reading out the included/excluded Groups (how?!?) and skipping doSomethingVeryTimeConsuming using reflection.

Rerunning failed cucumber tests using cucumber-jvm

I have a Cucumber-JVM, JUnit, Selenium setup. I initiate the run by running RunSmokeTests.java using JUnit within Eclipse. I have also set up a maven profile to run the tests from command line, and possibly Jenkins in the future.
When the tests are run then some of them may fail sometimes, mainly due to the application taking longer than expected. I would then have to re-run these scenarios. At the moment I run them by manually attaching #rerun tag to the ones that failed and then running RunReruns.java, which is similar to RunSmokeTest.java but with #rerun tag.
With the increasing number of automated tests it is time consuming to tag the tests and start the run and clear the tags. Is there a automated way with Cucumber-JVM to re-run failed tests?
RunSmokeTests.java
package testGlueClasses;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
#Cucumber.Options(features = "src/test/java", strict = true, format = {
"html:target/CucumberReport", "json:target/JSON/Cucumber.json",
"FrameworkCore.CustomTestReporter" }, tags = { "#SmokeTest" }, glue = {
"FrameworkCore", "MyApp.Utils", "MyApp.StepDefinitions" })
public class RunSmokeTests {
}
Maven snippet:
<profile>
<id>smoke</id>
<properties>
<include.tests>
**/RunSmokeTests.java
</include.tests>
</properties>
</profile>

I came up with another solution to rerun just failed test using maven & cucumber.
1) Record test failures using a RunNotifier
public class RerunningCucumber extends Cucumber {
private final String className;
#SuppressWarnings("rawtypes")
public RerunningCucumber(Class clazz) throws InitializationError, IOException {
super(clazz);
className = clazz.getSimpleName();
}
#Override
public void run(RunNotifier notifier) {
notifier.addListener(new RunListener(){
public void testFailure(Failure failure) throws Exception {
Throwable error = failure.getException();
if (error instanceof AssertionError){
//Nothing. This is a normal failure. Continue
return;
}
//No! A wild exception has appeared!
//Let's run this test again.
RerunningCucumber.addFile(className);
}
});
super.run(notifier);
}
private static final String filename = "target/rerun.properties";
private static final Set<String> addedClasses = new HashSet<String>();
public static synchronized void addFile(String className) throws IOException{
//First find the file
if (addedClasses.contains(className)){
return;
}
File file = new File(filename);
if (!file.exists()){
//Need to create the file
PrintWriter writer = new PrintWriter(file, "UTF-8");
writer.print("retryclasses=**/"+className+".class");
writer.close();
}
else {
PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter(file, true)));
out.print(",**/"+className+".class");
out.close();
}
addedClasses.add(className);
}
}
2) Use custom class as a runner for the cucumber tests.
This will run the tests, and whenever there is a failure, output the failed class to a file. Trick is to keep features short and create a lot of test classes to avoid repeating tests.
#RunWith(RerunningCucumber.class)
#CucumberOptions(features = {"classpath:features/testFeature.feature}, format = {
"html:target/cucumber-html-report/testFeature.html",
"json:target/cucumber-json-report/testFeature.json"},
tags = {"#testFeature"})
public class RunTestFeature {
}
3) Add a Rerun profile to maven.
This does three things: 1) it loads the failed classes into memory, 2) cleans JUST the failed classes properties file, and 3) reruns ONLY the failed tests as loaded from the properties file:
<profile>
<id>retry</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<!-- Associate the read-project-properties goal with the initialize
phase, to read the properties file. -->
<execution>
<phase>pre-clean</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>target/rerun.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-clean-plugin</artifactId>
<version>2.6.1</version>
<configuration>
<filesets>
<fileset>
<directory>target</directory>
<includes>
<include>rerun.properties</include>
</includes>
</fileset>
</filesets>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<echo>Retrying the following classes: "${retryclasses}"</echo>
</target>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<includes>
<include>${retryclasses}</include>
</includes>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
4) Usage
First test run:
mvn clean test
Next test runs:
mvn clean test -Pretry
mvn clean test -Pretry
mvn clean test -Pretry
...
You can repeat as many times as you want until there are no errors.

I don't have an executable example at hand, but you can do this also on the jvm. There is a RerunFormatter that writes a text file listing the file and line numbers of failed scenarios:
#CucumberOptions(format = {"rerun:target/rerun.txt"})
You should be able to specify this file as input for another test class by prefixing it with #:
#CucumberOptions(features = {"#target/rerun.txt"})

You can pass cucumber options to mvn as below
mvn clean verify -Dcucumber.options="#rerun.txt"
Note there is a tricky part here. If you are using the same test runner for both first run and rerun (and I believe that's what you want), then the test runner would contains something like
#CucumberOptions(plugin = { "rerun:target/rerun.txt"})
If you fire your rerun with maven using the same rerun file name as below
mvn clean verify -Dcucumber.options="#target/rerun.txt"
then cucumber will complain it could not find the rerun file. Why? Because the plugin "rerun:target/rerun.txt" will delete the file first with this test runner.
Workaround is copy/rename the file first, then kick off the mvn run like
mv target/rerun.txt rerun.txt && mvn clean verify -Dcucumber.options="#rerun.txt"
And this is actually what you want. Because say if there are 5 failed scenarios in file target/rerun.txt. And with the rerun after some fix, 2 of them passed. Now the target/rerun.txt will contain the remaining 3 failed scenarios only, which would be your new start point along the debugging way.

For cucumber + java on maven i found this command:
mvn clean test -Dsurefire.rerunFailingTestsCount=2
You must have an actual version of surefire plugin, mine is 3.0.0-M5.
And nothing else special u even need.
Found solution here Surefire rerun failing tests not working

1)
With junit4 (cucumber-junit engine) it can be done easily with rerun plugin
and features cucumber option. Add another maven profile for runner for failed scenarios, for example RerunCucumber.class.
Run your initial build with main test runner and pass rerun plugin:
#RunWith(Cucumber.class)
#CucumberOptions(tags = "#wip",
monochrome = true,
plugin = {"html:target/cucumber", "json:target/wip.json", "rerun:target/rerun_wip.txt"})
public class RunCucumber {
}
After build is finished, failed scenarios will be written to target/rerun_wip.txt.
Then failed scenarios can be executed via rerunner:
#RunWith(Cucumber.class)
#CucumberOptions(features = {"#features = {"#target/rerun_wip.txt"}"},
monochrome = true,
plugin = {"html:target/rerun/failed_tests", "json:target/rerun/failed_tests.json"})
public class RerunCucumber {
}
Will be executed tests from target/rerun_wip.txt.
2)
With junit5 (cucumber-junit-platform-engine) there is no such approach (no features cucumber option). More read about rerun failed scenarios with junit5: https://github.com/cucumber/cucumber-jvm/tree/main/junit-platform-engine, in 'Rerunning failed scenarios' section

You can use cucumber-jvm-parallel-plugin contributed code as a workaround until it goes live. Hit commands as shown below.
git clone -b tagwiseOutlinewiseIssueRerun https://github.com/sugatmankar/cucumber-jvm-parallel-plugin.git
mvn clean install.
Now edit you project pom file and use as stated here.
Example for using this plugin is here.

Injecting Maven project information into Swing Application Framework resources?

I have a Maven project using the Swing Application Framework and would like to inject project information from the pom.xml into my application's global resources to avoid duplication.
The base application (provided via netbeans) uses Application.title, Application.version, Application.vendor, Application.description resources etc for Window titles and about box configuration but I can't find a way to set these values programatically at run time and I'm not a maven maven so don't have the skills to inject them at build time.
Anyone have any recommendations on how best to achieve the desired result?

You could try using filtered resources. If you create a property file, say src/main/resources/com/myapp/app.properties that looks like this:
version=${project.version}
name=${project.name}
id=${project.artifactId}
Them you need to enable filtering in your pom.xml:
<build>
<resources>
<resource>src/main/resources</resource>
<filtering>true</filtering>
</resources>
</build>
Now when maven builds your project, it'll expand the property file, and place it on the classpath. Then you can just call getResourceAsStream("/com/myapp/app.properties") to read it into your app.
Whist maven does automatically create a file /META-INF/maven/$groupId/$artifactId/pom.properties, this may not have all the information you need.

You can keep those in separte property file and read it from both pom.xml and your application.
Another option is to read pom.xml file from classpath (mvn will put it in META-INF folder) and parse it from there as plain xml file.
I would go with first option.

I would try using the maven-antrun-plugin. Pass the necessary maven properties to ant and create an ant build script which modifies an application properties file or the spring context configuration directly.

Another way would be to generate a separate properties file with the properties-maven-plugin and then add this properties file to the application bundle names:
For the pom.xml to write application.properties file:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<phase>process-resources</phase>
<goals>
<goal>write-project-properties</goal>
</goals>
</execution>
</executions>
<configuration>
<outputFile>${project.build.outputDirectory}/application.properties</outputFile>
</configuration>
</plugin>
...
</plugins>
...
</build>
For including the application.properties into your application:
public class MyApplication extends SingleFrameApplication
public MyApplication() {
super();
addGeneratedApplicationProperties();
}
private void addGeneratedApplicationProperties() {
ResourceManager resourceManager = getContext().getResourceManager();
getContext().setApplicationClass(MyApplication.class);
List<String> bundleNames = new LinkedList<String>(resourceManager.getApplicationBundleNames());
bundleNames.add(0, "application");
resourceManager.setApplicationBundleNames(bundleNames);
}
...
}
However, I find the maven-filter-solution way more elegant.

Categories

Resources