I have a project that has unit and integration tests. I have two test suites:
#RunWith(ClasspathSuite.class)
#ClasspathSuite.ClassnameFilters({ ".*Test", "!.*IntegrationTest", "!.*ResourceTest", "!.*DAOTest" })
#ClasspathSuite.SuiteTypes({ SuiteType.JUNIT38_TEST_CLASSES, SuiteType.TEST_CLASSES, SuiteType.RUN_WITH_CLASSES })
public class AutoTestSuite {
}
which runs 70 tests in about 20 seconds
#RunWith(ClasspathSuite.class)
#ClasspathSuite.SuiteTypes({ SuiteType.JUNIT38_TEST_CLASSES, SuiteType.TEST_CLASSES, SuiteType.RUN_WITH_CLASSES })
#ClasspathSuite.ClassnameFilters({ ".*IntegrationTest" })
public class IntegrationTestSuite {
}
which runs 99 integration tests in about 80 seconds.
This is fine, but when I try to run my maven build locally and inside Jenkins the test numbers are between 12-18 minutes minutes to run 348 tests. Clearly my pom.xml, even accounting for jacoco to inspect. Can anyone see what is wrong with the following that I invoke with "mvn clean package:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.6.0.201210061924</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jacoco.argLine.unit</propertyName>
<destFile>${jacoco.destFile.unit}</destFile>
</configuration>
</execution>
<execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<propertyName>jacoco.argLine.it</propertyName>
<destFile>${jacoco.destFile.it}</destFile>
</configuration>
</execution>
</executions>
</plugin>
<!-- UNIT tests only with mvn clean test -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${version.surefire}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${version.surefire}</version>
</dependency>
</dependencies>
<configuration>
<argLine>${jacoco.argLine.unit}
-Dfile.encoding=${project.build.sourceEncoding} -Xmx512m
</argLine>
<forkMode>always</forkMode>
<parallel>classes</parallel>
<includes>
<include>**/*.class</include>
</includes>
<excludes>
<exclude>**/*.IntegrationTest.java</exclude>
</excludes>
</configuration>
</plugin>
<!-- INTEGRATION tests that are run with "mvn clean verify" -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${version.surefire}</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${version.surefire}</version>
</dependency>
</dependencies>
<configuration>
<forkMode>pertest</forkMode>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<argLine>${jacoco.argLine.it}
-Dfile.encoding=${project.build.sourceEncoding} -Xmx512m
</argLine>
<includes>
<include>**/*.class</include>
</includes>
</configuration>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Is there a reason the tests appear to be duplicated and taking so long?
I'm not a maven or surefire expert, but I think the surefire plugin executes every test file it finds in under your test directory. This means the default configuration doesn't need test suites. Surefire finding and then running your test suite PLUS your other tests could be the cause of the 2x the number of tests to be run. But I don't know why it would take more than twice as long to run.
If this is the case, you can tell surefire to only run your test suite:
<configuration>
...
<includes>
<include>AutoTestSuite.java</include>
</includes>
</configuration>
instead of **/*.class
EDIT: Because Jacoco has already instrumented the classes you need to use
<configuration>
...
<includes>
<include>AutoTestSuite.class</include>
</includes>
</configuration>
instead otherwise you don't run the instrumented version.
Related
I have a problem with running groovy test with java tests. I have java test in directory /src/test/java and groovy tests in directory /src/test/groovy and maven plugin config like it:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M4</version>
<configuration>
<useFile>false</useFile>
<includes>
<include>**/*Test.java</include>
<include>**/*Spec.java</include>
</includes>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.12.1</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
But with this config only groovy spock test works without java junit tests.
What am I doing wrong?
Thanks in advance for help
I had exactly same issue and adding dependency to junit-vintage-engine, solved it, e.g.:
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M4</version>
<configuration>
<useFile>false</useFile>
<includes>
<include>**/*Test.java</include>
<include>**/*Spec.java</include>
</includes>
</configuration>
<dependencies>
<dependency>
<groupId>org.junit.vintage</groupId>
<artifactId>junit-vintage-engine</artifactId>
<version>5.8.1</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>1.12.1</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
I have a a unit test (ProductDaoTest.java) and an integration test (ProductDaoIT.java) in my maven application.
I would like to execute only the integration test during the mvn verify command call but the unit test also gets executed even after excluding it using the <exclude> tag in the maven-failsafe-plugin configuration.
How can I fix this problem?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Updated POM (with solution):
<!-- For skipping unit tests execution during execution of IT's -->
<profiles>
<profile>
<id>integration-test</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<!-- Skips UTs -->
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<!-- Binding the verify goal with IT -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<configuration>
<port>5000</port>
<path>${project.artifactId}</path>
</configuration>
<executions>
<execution>
<id>start-tomcat</id>
<phase>pre-integration-test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<fork>true</fork>
</configuration>
</execution>
<execution>
<id>stop-tomcat</id>
<phase>post-integration-test</phase>
<goals>
<goal>shutdown</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
mvn clean install - Runs only unit tests by default
mvn clean install -Pintegration-test - Runs only integration tests by default
In Maven, test step is before verify step in the lifecycle.
So it you don't skip this step, it is bound to execute.
If you want to skip test , either use -Dmaven.test.skip=true as khmarbaise suggested, either create a dedicated Maven profile for IT where you will ignore unit-tests in this way :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
Generally, you create a Maven profile for integration tests, so if it is the case, gathering all the configuration in a place is better that scattering it.
I am using TestNG and maven failsafe for my integration tests.
The tests are executed and they pass however there are absolutely no details printed out.
I even created a dummy test and have System.out.println in that and none of that is printed out.
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
<properties>
<property>
<name>surefire.testng.verbose</name>
<value>10</value>
</property>
</properties>
</configuration>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<skip>false</skip>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I want the testNG summary to be printed and logging statements to be printed ...
I am also using the groovy-eclipse-compiler plugin .. not sure if that makes a difference though
When you use a testing framework, you need to use the maven surefire plugin to have a summary of your results. I dont think that anything is executed the way you do it right now, so no output is expectable.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${basedir}/src/main/resources/testng.xml</suiteXmlFile>
</suiteXmlFiles>
<skipTests>${skipTests}</skipTests>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>
When trying to get failsafe bound to the lifecycle, nothing is executed at all. I have read this guide and this related question, and according to this information, it should be possible to make maven execute an the goal integration-test of failsafe in the integration-test, when I specify it in the build/pluginManagement/plugins-section in the pom.xml like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<configuration>
<includes>
<include>**/*IT</include>
</includes>
</configuration>
<executions>
<execution>
<id>failsafe-integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>failsafe-verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Unfortunately, this does not force maven to run failsafe:integration-test at all (neither with mvn integration-test nor mvn verify)
But if I try to use failsafe with the plugin-specification like this (from here with added configuration):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<includes>
<include>**/*IT</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
At least maven compile failsafe:integration-test runs. But unfortunately, this does not call pre- and post-integration-test. I am struggeling for this for a while now, and have no clue - it should be bound as it is.
Does anybody know why this happens, or how I can fix it?
The thing you did is to define it only in pluginManagement but you have to run it really like this. The definition in pluginManagement is good practice to pin the version of the plugin.
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
Apart from that it's not necessary to give include rules for the maven-failsafe-plugin cause it has already defaults defined so no need for that.
I am trying to get a JBehave story to execute in Maven it is completely ignoring the JBehave plugin. I've spent several hours using different configurations but it looks like the plugin isn't being executed at all. Any recommendations/tips would be appreciated!
All my JBehave classes live in:
src/at/java
Relevant parts of my pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.5</version>
<executions>
<execution>
<id>add-test-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/at/java</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jbehave</groupId>
<artifactId>jbehave-maven-plugin</artifactId>
<executions>
<execution>
<id>run-stories-as-embeddables</id>
<phase>integration-test</phase>
<configuration>
<includes>
<include>**/*.java</include>
</includes>
</configuration>
<goals>
<goal>run-stories-as-embeddables</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<executions>
<execution>
<id>integration-test</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<include>**/*.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
Best is to change the location of your test classes to src/test/java and change the name of the stories based on the documentation of JBehave.
JBehave running with maven follow the Maven rules for location of code and text artifacts.
For test scope you must put them in src/test/java and src/test/resources. For compile scopes is src/main/java and src/main/resources.
With JBehave with maven you could use two scopes (test or compile), you just need to set which one you want in the plugin configuration, so you choose where to put your artifacts. it defaults to compile.
In your case you are adding a new test source so you must set the scope to test:
see detail here.
Maybe the jbehave-maven-plugin could not find the compiled test classes (scenarios) because it looks in the wrong classpath.
Please look at your target directory and search the embeddable classes -> target/classes or target/test-classes?
To solve the problem i must set the scope of jbehave-maven-plugin to test in the configuration of my project pom.xml.
here is a example
<plugin>
<groupId>org.jbehave</groupId>
<artifactId>jbehave-maven-plugin</artifactId>
<executions>
<execution>
<id>run-stories-as-embeddables</id>
<phase>integration-test</phase>
<goals>
<goal>run-stories-as-embeddables</goal>
</goals>
<configuration>
<scope>test</scope>
<includes>
<include>**/*Scenarios.java</include>
</includes>
<ignoreFailureInStories>true</ignoreFailureInStories>
<ignoreFailureInView>false</ignoreFailureInView>
</configuration>
</execution>