Develop Reference

Unit tests getting executed during integration test phase even after excluding them - java

I have a a unit test (ProductDaoTest.java) and an integration test (ProductDaoIT.java) in my maven application.
I would like to execute only the integration test during the mvn verify command call but the unit test also gets executed even after excluding it using the <exclude> tag in the maven-failsafe-plugin configuration.
How can I fix this problem?
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<excludes>
<exclude>**/*Test.java</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Updated POM (with solution):
<!-- For skipping unit tests execution during execution of IT's -->
<profiles>
<profile>
<id>integration-test</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<!-- Skips UTs -->
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
<!-- Binding the verify goal with IT -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.19.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.2</version>
<configuration>
<port>5000</port>
<path>${project.artifactId}</path>
</configuration>
<executions>
<execution>
<id>start-tomcat</id>
<phase>pre-integration-test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<fork>true</fork>
</configuration>
</execution>
<execution>
<id>stop-tomcat</id>
<phase>post-integration-test</phase>
<goals>
<goal>shutdown</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
mvn clean install - Runs only unit tests by default
mvn clean install -Pintegration-test - Runs only integration tests by default

In Maven, test step is before verify step in the lifecycle.
So it you don't skip this step, it is bound to execute.
If you want to skip test , either use -Dmaven.test.skip=tr‌​ue as khmarbaise suggested, either create a dedicated Maven profile for IT where you will ignore unit-tests in this way :
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skipTests>true</skipTests>
</configuration>
</plugin>
Generally, you create a Maven profile for integration tests, so if it is the case, gathering all the configuration in a place is better that scattering it.

Related

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/POM/4.0.0"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.idexx</groupId>
<artifactId>qe-lynxx-automation</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<serenity.version>2.0.27</serenity.version>
<lean.ft.version>14.50.0</lean.ft.version>
<test.directory>${project.build.testSourceDirectory}/tests</test.directory>
<tags></tags>
</properties>
<dependencies>
I have all the required dependencies but not including here.
I am unable to run test parallelly with forkCount or methods. It works fine would the sureFire plugin but I cannot use the surefire plugin to generate serenity reports.
I have tried a combination of forkCount and parallel that didn't work either.
I was able to fork multiple Java VMs using the sureFire.
I am trying to run test parallelly in multiple virtual machines. Our application is swing based java application and we are using LeanFt to automate the testing process.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>3.0.0-M3</version>
<configuration>
<forkCount>2</forkCount>
<reuseForks>false</reuseForks>
<includes>
<!-- Run every java class in the 'tests' package -->
<include>${test.directory}/*.java</include>
</includes>
<systemPropertyVariables>
<tags>${tags}</tags>
</systemPropertyVariables>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
<version>${serenity.version}</version>
<executions>
<execution>
<id>serenity-reports</id>
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>

I figured out how it actually works. All you have to do is add Surefire plugin with Failsafe plugin when you use both of them. It works like a charm.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<skip>false</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.22.1</version>
<configuration>
<includes>
<!-- Run every java class in the 'tests' package -->
<include>${test.directory}/*.java</include>
</includes>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>net.serenity-bdd.maven.plugins</groupId>
<artifactId>serenity-maven-plugin</artifactId>
<version>${serenity.maven.version}</version>
<configuration>
<tags>${tags}</tags>
</configuration>
<executions>
<execution>
<id>serenity-reports</id>enter code here
<phase>post-integration-test</phase>
<goals>
<goal>aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

I am developing a java web application (No Spring).I wanted to use separate db for production and testing.I have two files in src/main/resources - env.properties and env.test.properties.
I have defined the profile in pom.xml as mentioned in https://maven.apache.org/guides/mini/guide-building-for-different-environments.html.
<profiles>
<profile>
<id>test</id>
<build>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<delete file="${project.build.outputDirectory}/environment.properties"/>
<copy file="src/main/resources/environment.test.properties"
tofile="${project.build.outputDirectory}/environment.properties"/>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
<classifier>test</classifier>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
However, when I run test by maven test -Ptest, I see that my test is getting executed with the db from env.properties and then after completion of test , the profile switching happens.
I am also having a jebkins pipeline which builds tests and deploys.
Am I missing something here ? What is the correct way to read the properties from env.test.properties(activate the profile and run test) ?
Thanks a lot.

You're doing this the hard way.
Get rid of the profiles and move the file from src/main/resources/environment.test.properties to src/test/resources/environment.properties
Resources in src/test/resources/ will be found and loaded before those in src/main/resources when unit tests are being executed.

I am using TestNG and maven failsafe for my integration tests.
The tests are executed and they pass however there are absolutely no details printed out.
I even created a dummy test and have System.out.println in that and none of that is printed out.
<plugin>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
<configuration>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
<properties>
<property>
<name>surefire.testng.verbose</name>
<value>10</value>
</property>
</properties>
</configuration>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
<configuration>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
<skip>false</skip>
<excludes>
<exclude>none</exclude>
</excludes>
<includes>
<include>**/*IntegrationTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
I want the testNG summary to be printed and logging statements to be printed ...
I am also using the groovy-eclipse-compiler plugin .. not sure if that makes a difference though

When you use a testing framework, you need to use the maven surefire plugin to have a summary of your results. I dont think that anything is executed the way you do it right now, so no output is expectable.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${basedir}/src/main/resources/testng.xml</suiteXmlFile>
</suiteXmlFiles>
<skipTests>${skipTests}</skipTests>
<testFailureIgnore>true</testFailureIgnore>
</configuration>
</plugin>

I have a project for which the code coverage numbers are pretty low. After doing a deep-dive I found out the jacoco code-coverage stats are depending on target directoy instead of src.
target/generated-sources/delombok/com/
If I exclude the target directory the coverage goes down 0%. So isnt code-coverage supposed to be measured against the code (src) rather than the target folder. Below is the screenshot from SonarQube.
So the question is how do i make sure the code coverage is measured for src instead of target?
Following is the pom without dependencies:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>removed</groupId>
<artifactId>xxxx</artifactId>
<version>0.0.11-SNAPSHOT</version>
<packaging>war</packaging>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>cassandra-maven-plugin</artifactId>
<version>2.1.7-1</version>
<configuration>
<script>${basedir}/src/main/cassandra/local_run.cql</script>
<cqlVersion>3.2.0</cqlVersion>
<startNativeTransport>true</startNativeTransport>
<nativeTransportPort>9142</nativeTransportPort>
<jmxPort>17199</jmxPort>
<storagePort>17000</storagePort>
<loadFailureIgnore>false</loadFailureIgnore>
<cuLoadFailureIgnore>false</cuLoadFailureIgnore>
</configuration>
<executions>
<execution>
<id>start</id>
<goals>
<goal>start</goal>
<goal>load</goal>
</goals>
<phase>process-classes</phase> <!--Bind cassandra start and load to process-classes which is the phase that is called up before tomcat is run -->
</execution>
<execution>
<id>stop</id>
<goals>
<goal>stop</goal>
</goals>
<!--Bind cassandra stop to generate-test-sources which is the phase that is called up before
tests are run. This is because our tests start their own cassandra -->
<phase>integration-test</phase>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.7</version>
</dependency>
</dependencies>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.projectlombok</groupId>
<artifactId>lombok-maven-plugin</artifactId>
</plugin>
<!--Needed to add this so that src/main/java is recognized as a source folder -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
</plugin>
<!-- Unit Test -->
<!-- Integration Test -->
<plugin>
<artifactId>maven-enforcer-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
<configuration>
<warName>activationloginedge</warName>
</configuration>
</plugin>
<!-- Tomcat Integration Test -->
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<configuration>
<path>/activationloginedge</path>
</configuration>
<executions>
<execution>
<id>tomcat-run</id>
<goals>
<goal>run</goal>
</goals>
<phase>pre-integration-test</phase>
<configuration>
<systemProperties>
<!-- We want test configuration for running integration tests. -->
<archaius.deployment.environment>test</archaius.deployment.environment>
<logback-lib.env>filesystem</logback-lib.env>
</systemProperties>
<fork>true</fork>
</configuration>
</execution>
<execution>
<id>tomcat-shutdown</id>
<goals>
<goal>shutdown</goal>
</goals>
<phase>post-integration-test</phase>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.4.201502262128</version>
<executions>
<execution>
<id>pre-unit-test</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/jacoco-ut.exec</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent.-->
<propertyName>surefireArgLine</propertyName>
</configuration>
</execution>
<!--Ensures that the code coverage report for unit tests is created after unit tests have been run.-->
<execution>
<id>post-unit-test</id>
<phase>test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/jacoco-ut.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
</configuration>
</execution> <execution>
<id>pre-integration-test</id>
<phase>pre-integration-test</phase>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<destFile>${project.build.directory}/jacoco-it.exec</destFile>
<!-- Sets the name of the property containing the settings for JaCoCo runtime agent.-->
<propertyName>failsafeArgLine</propertyName>
<
/configuration>
</execution>
<!-- Ensures that the code coverage report for integration tests after integration tests have been run.-->
<execution>
<id>post-integration-test</id>
<phase>post-integration-test</phase>
<goals>
<goal>report</goal>
</goals>
<configuration>
<!-- Sets the path to the file which contains the execution data. -->
<dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
<!-- Sets the output directory for the code coverage report. -->
<outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.12.4</version>
<configuration>
<!-- Sets the VM argument line used when unit tests are run. -->
<argLine>${surefireArgLine}</argLine>
<!-- Skips unit tests if the value of skip.unit.tests property is true -->
<skipTests>${skip.unit.tests}</skipTests>
<!-- Excludes integration tests when unit tests are run. -->
<excludes>
<exclude>**/IT*.java</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.15</version> <executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<!-- Sets the VM argument line used when integration tests are run-->
<argLine>${failsafeArgLine}</argLine>
<!-- Skips integration tests if the value of skip.integration.tests property is true-->
<skipTests>${skip.integration.tests}</skipTests> </configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

OK, the Lombok Maven plugin modifies the list of source folders during the Maven build, this is why the target/generated-sources/delombok folder is recognized by SonarQube as a source folder.
In order to force SonarQube to consider only your own source files, you should add the following property to your POM:
<properties>
<sonar.sources>src/main/java</sonar.sources>
</properties>
This will solve your problem.

When trying to get failsafe bound to the lifecycle, nothing is executed at all. I have read this guide and this related question, and according to this information, it should be possible to make maven execute an the goal integration-test of failsafe in the integration-test, when I specify it in the build/pluginManagement/plugins-section in the pom.xml like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<configuration>
<includes>
<include>**/*IT</include>
</includes>
</configuration>
<executions>
<execution>
<id>failsafe-integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>integration-test</goal>
</goals>
</execution>
<execution>
<id>failsafe-verify</id>
<phase>verify</phase>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
Unfortunately, this does not force maven to run failsafe:integration-test at all (neither with mvn integration-test nor mvn verify)
But if I try to use failsafe with the plugin-specification like this (from here with added configuration):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.17</version>
<executions>
<execution>
<id>integration-test</id>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<includes>
<include>**/*IT</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
At least maven compile failsafe:integration-test runs. But unfortunately, this does not call pre- and post-integration-test. I am struggeling for this for a while now, and have no clue - it should be bound as it is.
Does anybody know why this happens, or how I can fix it?

The thing you did is to define it only in pluginManagement but you have to run it really like this. The definition in pluginManagement is good practice to pin the version of the plugin.
<project>
[...]
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
[...]
</project>
Apart from that it's not necessary to give include rules for the maven-failsafe-plugin cause it has already defaults defined so no need for that.