I have been working with cucumber/Java and JUnit4 (CucumberOptions) for years without trouble running the tests in both IntelliJ and maven command line.
Recently, i have been trying to make the move to JUnit5 and i was able to have all tests running in IntelliJ (only, unfortunately)
My POC project has the following structure:
junit5
-Features (folder with feature files)
-resources (folder with files used in tests)
-src
--test
---java
----stepdefs
-----SetupEnvHook
-----StepDefs
----AllTest (testrunner wip)
----JU4Test (testrunner JUnit4)
----JU5Test (testrunner Junit5)
---resources (test resources)
-junit5.iml
-pom.xml
The JU5Test.java file :
import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.SelectDirectories;
import org.junit.platform.suite.api.Suite;
import stepdefs.SetupEnvHook;
import io.cucumber.java.Before;
import static io.cucumber.core.options.Constants.*;
#Suite
#SelectDirectories("Features")
//#ConfigurationParameter(key = PARALLEL_EXECUTION_ENABLED_PROPERTY_NAME, value = "true")
#ConfigurationParameter(key = PLUGIN_PUBLISH_ENABLED_PROPERTY_NAME, value = "false")
#ConfigurationParameter(key = PLUGIN_PUBLISH_QUIET_PROPERTY_NAME, value = "true")
#ConfigurationParameter(key = PLUGIN_PROPERTY_NAME, value = "json:target/cucumber-reports/cucumber.json")
#ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "stepdefs, my.external.steps.stepdefinition")
public class JU5Test {
#Before
public static void beforeSuite() {
SetupEnvHook.setEnvironment("QA");
}
}
The beforeSuite() method is also used in the JU4Test.
When i set a breakpoint in SetupEnvHook.setEnvironment("QA"); it is completely ignored due to the fact that the Before Annotation is not working, while another breakpoint inside the same
#io.cucumber.java.BeforeAll(order = 9999)
Annotation in SetupEnvHook class is triggered correctly.
My pom file is as follows :
<dependencies>
<dependency>
<groupId>my.external</groupId>
<artifactId>steps</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.9.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>8</source>
<target>8</target>
</configuration>
</plugin>
</plugins>
</build>
Please ignore the my external dependancy. This dependancy is related to the stepdefinitions in the test runner file glue property.
I know that group and version values are also missing but these are all fed from the same dependancy in red so as to have more control on the versions everyone uses.
This is all done in Java 8 using
org.apache.maven.plugins:maven-compiler-plugin:3.10.1
org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M7
io.cucumber:cucumber-java:7.8.1
io.cucumber:cucumber-junit:7.8.1
io.cucumber:cucumber-junit-platform-engine:7.8.1
org.junit.jupiter:junit-jupiter-api:5.9.1
org.junit.jupiter:junit-jupiter-engine:5.9.1
org.junit.platform:junit-platform-suite-api:1.9.1
org.junit.platform:junit-platform-suite-engine:1.9.1
I already tried using different Annotations not only from io.cucumber.java but also from org.junit (which is basically JUnit4) and org.junit.jupiter.api with no success obviously.
Running through maven command line ends up with :
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test (default-test) on project junit5: No tests were executed!
It does not however state that 'no tests were found', had that issue initially and got it solved.
From looking at the error i suspect i may have something missing from the pom.xml surefire plugin but i cannot figure out what. (this pom is the same used to run the JU4Test without issues)
Anyone else have any thoughts on what i can try next? or better yet, the solution for this xD
Edit: remove images
It does not however state that 'no tests were found', had that issue initially and got it solved.
From looking at the error i suspect i may have something missing from the pom.xml surefire plugin but i cannot figure out what. (this pom is the same used to run the JU4Test without issues)
From your description it is impossible to say what is wrong with your project. Your list of depencies includes dependencies not included in your POM.
You may want to consider starting your project from scratch. You can use the https://github.com/cucumber/cucumber-java-skeleton for that.
When i set a breakpoint in SetupEnvHook.setEnvironment("QA"); it is completely ignored due to the fact that the Before Annotation is not working
The reason the #Before annotation is ignored is because you are using a Cucumber annotation on a class that is not part of the glue path.
Though I suspect you are trying to find a mapping for JUnit 4s #BeforeClass. Currently there is not such thing in JUnit 5s Suite Engine. If you need it, you should consider making a pull requests.
Alternatively you could create a package with a single class for each environment and use Cucumbers #BeforeAll hooks to set the environment. Then for each #Suite you configure a different glue path to include those hooks.
Though I think it would be even better to read the target environment from an environment variable and have it default to something sane. You can then use different CI jobs for each environment.
Related
This is my test (maven-plugin-testing-harness 3.3.0, junit 5.6.2):
import java.io.File;
import org.apache.maven.plugin.testing.AbstractMojoTestCase;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
public final class MyMojoTest extends AbstractMojoTestCase {
#BeforeEach
public void setup() throws Exception {
this.setUp();
}
#Test
public void executeIt() throws Exception {
final File pom = new File("src/test/resources/my-test-pom.xml");
final MyMojo mojo = MyMojo.class.cast(
this.lookupMojo("mygoal", pom)
);
mojo.execute();
}
}
This is what I have in MyMojo (maven-plugin-api 3.8.4):
import org.apache.maven.plugin.AbstractMojo;
import org.apache.maven.plugins.annotations.Mojo;
import org.apache.maven.plugins.annotations.Parameter;
#Mojo(name = "my", defaultPhase = LifecyclePhase.COMPILE)
public final class MyMojo extends AbstractMojo {
#Parameter(defaultValue = "${project}", readonly = true)
private MavenProject project;
}
The problem is that mojo returned by lookupMojo() doesn't have the project attribute set (it's null).
Some solution was proposed here, but I'm not sure how it can work with JUnit 5.
I tried with the same configurations as mentioned above. The plugin works fine but none of the tests having lookupMojo() seems to be working.
A similar test example can be referred here. There is a difference in the setUp method from your class MyMojoTest and example provided in the link.
super.setUp(); should be called instead of this.setUp() so as to initialize all the objects in AbstractMojoTestCase class.
The possible reason that the test case with maven-plugin-testing-harness 3.3.0 and junit 5.6.2 will not work because they are not compatible.
The reasons are
maven-plugin-testing-harness was built to be compatible with Junit4. Latest update was a long time ago i.e. Dec 17, 2014. Junit 4 and Junit 5 are not compatible. We have to make use of Junit-Vintage-Engine to make it work.
maven-plugin-testing-harness was develop using JDk-7 and minimum requirements for Junit 5 is Jdk-8.
Information from the harness plugin Manifest file
Implementation-Vendor-Id: org.apache.maven.plugin-testing
Built-By: igor
Build-Jdk: 1.7.0_55
Specification-Vendor: The Apache Software
Foundation Specification-Title: Maven Plugin Testing Mechanism
Maven version supported is also different for both the jars.
link
Few other links confirm the same.
There are very few libraries and informational link available for plugin testing with Junit5. I could find only a handful of them, although I haven't tried them yet.
Library:
<dependency>
<groupId>com.soebes.itf.jupiter.extension</groupId>
<artifactId>itf-assertj</artifactId>
<version>0.11.0</version>
<scope>test</scope>
</dependency>
Few more Jupiter extension libraries in this link
Examples related to it.
Example 1
Example 2
Example 3
Possible solutions
Solution #1: Use AbstractMojoTestCase.lookupConfiguredMojo() method
Please, consider the implementation of the test class as an example: maven-plugin-testing/ParametersMojoTest.java at maven-plugin-testing-3.3.0 · apache/maven-plugin-testing.
Considering this example, please, note the Mojo instantiation approach:
The readMavenProject() method.
The Mojo instantiation uses the readMavenProject() and lookupConfiguredMojo() methods:
MavenProject project = readMavenProject( new File( "src/test/projects/default" ) );
ParametersMojo mojo = (ParametersMojo) lookupConfiguredMojo( project, "parameters" );
This Mojo instantiation approach provides the instantiated Mojo with the correct MavenProject parameter value.
Some additional references
Related Stack Overflow answer.
Solution #2: Test pom.xml for plugin: Use project stub
It is necessary to update the test pom.xml file by introducing the project element (of the configuration element) with the stub.
For example:
<project>
<…>
<build>
<plugins>
<plugin>
<artifactId>touch-maven-plugin</artifactId>
<configuration>
<project implementation="org.apache.maven.plugin.testing.stubs.MavenProjectStub">
<groupId implementation="java.lang.String">test-group-id</groupId>
<artifactId implementation="java.lang.String">test-artifact-id</artifactId>
<version implementation="java.lang.String">1.0.0-SNAPSHOT</version>
</project>
<…>
</configuration>
</plugin>
</plugins>
</build>
</project>
Some additional references
An example. maven-jar-plugin/pom.xml at maven-jar-plugin-3.2.2 · apache/maven-jar-plugin.
Related Stack Overflow answer.
Using a Spring maven context, I would like to run specific tests based on a maven profile. I would like to have an easy way of tagging the test groups. If possible I would like to use the annotations.
Which options are there, like maven command line parameters, maven profiles specification, etc.
Say I have the following tests:
Example:
// annotation("integration")
public class GeopointFormatterTest {
#Test
public void testIntegration1() { ... }
#Test
public void testIntegration2() { ... }
Annotations like #Profile (which is for creating beans) and #ActiveProfile (which is for selecting specific profiles for creating beans) cannot be used for selecting tests, of course. All tests just run for statements like:
mvn clean install -Pdevelopment
mvn clean install -Pdevelopment -Dspring.profiles.active=acceptance
mvn clean install -Pdevelopment -Dspring.profiles.active=integration
As suggested, I used also #IfProfileValue. This is a good way for selecting tests based on system property values. System property values can be overruled by a CustomProfileValueSource class, like in: #ProfileValueSourceConfiguration(CustomProfileValueSource.class)
EDIT and ALTERNATIVE
The GREAT answers below focus on JUnit's #Category mechanism. Thanks to all!
A different approach is via these steps: [1] set a property within a maven profile and [2] use the property to skip tests via the of the standard surefire test plugin.
[1] Setting the properties via a profile:
<profiles>
<profile>
<id>integrationtests</id>
<properties>
<integration.skip>false</integration.skip>
<acceptance.skip>true</acceptance.skip>
</properties>
</profile>
... other profiles
[2] Using the properties in the surefire test plugin to skip tests.
<build>
<plugins>
<plugin>
<!-- Run the integration test-->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.plugin.version}</version>
<configuration>
<skipTests>${acceptance.skip}</skipTests>
Start in maven: mvn clean install –Pintegrationtests
Take a look at junit categories.
You'd tag your tests with specific category annotations
public interface FastTests { /* category marker */ }
public interface SlowTests { /* category marker */ }
#Category(SlowTests.class)
public class A {
#Test public void a() {}
}
then form a suite like
#RunWith(Categories.class)
#IncludeCategory({FastTests.class})
#SuiteClasses({A.class, B.class})
public static class FastTestSuite {
//
}
And then run it with
mvn -Dtest=FastTestSuite test
Note also that if you don't want to manually specify your unit test case classes in the suite class, you can also use the help of ClasspathSuite and then just limit based on categories.
You will probably need to categorize your tests using the #Category annotation. A complete example has been provided in the Surefire documentation provided here - search for the string Using JUnit Categories.
Assuming that you have categorized your tests accordingly, you will now be able to setup one or more profiles in your maven build which will trigger these tests as per the category
<profiles>
<profile>
<id>slow-tests</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<groups>com.mycompany.SlowTests</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>fast-tests</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.11</version>
<configuration>
<groups>com.mycompany.FastTests</groups>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
You can specify one or more profiles on the command line when running the tests.
mvn test -Pslow-tests,fast-tests
You can specify the profile with this flag:
mvn test -Dspring.profiles.active=acceptance
In my latest project I have an "integration" profile that I use to run the integration tests against an embedded H2 database.
We did solved categorization of junit in the following steps.
I did create a project for you in github.
https://github.com/djaganathan/unit-test-samples
Caveat:- Junit Categorization packages still says as experimental.
1) Created a category of interfaces
/**
* This interface used to categories Junit Test
* those Tests will be executed during bamboo build run
*/
public interface ReleaseTest {
}
2) Tagged the unit testcases with the category you want
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.junit.internal.runners.JUnit4ClassRunner;
import org.junit.runner.RunWith;
import com.github.djaganathan.unit.test.category.ReleaseTest;
#RunWith(JUnit4ClassRunner.class)
public class GeneralTest {
#Test
#Category(value={ReleaseTest.class})
public void doTestForRelease(){}
#Test
public void doTestForDev(){}
}
3) Create profile in maven and attach to it
4) Run the command as mvn test -PreleaseTest
Based on the answers of a number of posts, I created a simple test project that demonstrates a number of code quality & testing features:
Performing either unit tests OR integration tests with plugins Surefire and Failsafe.
Improving code quality via the plugin findbugs.
Determine test coverage stats with plugin Jacoco.
Enjoy!
I have a JUnit4 test suite and I want to execute it under the JUnit Plug-in Test run configuration.
It passes successfully when running through the JUnit Test configuration, but for plug-in conf it fails in different ways.
For example if I use JUnit4TestAdapter it fails with ClassCastException, and if I trying to run it only through the #RunWith annotation it wrotes "No methods found" error. For both implementations I use JUnit4 Test Runner setting inside run configuration.
I use
Eclipse Neon.1a Release (4.6.1)
Jdk 1.8
linking JUnit 4.1 lib to the plugin.
For first case it seems that Eclipse proceed to use the JUnit3 version when executing the suite.
Here it is:
#RunWith(Suite.class)
#SuiteClasses({ DndTest.class })
public class JSTestSuite {
public static Test suite() {
return new JUnit4TestAdapter(JSTestSuite.class);
}
}
And exception is:
java.lang.ClassCastException: junit.framework.JUnit4TestAdapter cannot be cast to junit.framework.Test
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.getTest(RemoteTestRunner.java:403)
While starting the test I have a strange log message in console:
!ENTRY org.eclipse.update.configurator 2017-05-04 17:58:57.279
!MESSAGE Could not install bundle ../../platform/eclipse/plugins/org.eclipse.jdt.junit4.runtime_1.1.600.v20160505-0715.jar No match is available for the required execution environment: J2SE-1.5
I see this lib is on the place, but I can't understand why it failing to be loaded. For JUnit3 Test Runner setting junit3 lib is loaded ok.
There are some bugs related to such issues (like this) but it is really hard to understand what can I do in this case.
For second case I just try to execute simple JUnit4 case without using the JUnit4TestAdapter, but it can't find any methods.
Reloading of eclipse and renaming of the methods didn't help.
What can I do in this case?
I found that there are couple of issues. First I consider one by one to solve the issue.
Issue#1: No match is available for the required execution environment: J2SE-1.5
Issue#2: java.lang.ClassCastException: junit.framework.JUnit4TestAdapter cannot be cast to
junit.framework.Test
Solution for Issue#1:
First solution:
Right-click on your project
Click Properties
Click the "Java Compiler" option on the left menu
Under JDK compliance section on the right, change it to "1.8"
Second solution:
If you use maven in your project, then you can change pom.xml file as below:
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
Resource Link: Maven "build path specifies execution environment J2SE-1.5", even though I changed it to 1.7
Solution for Issue#2:
JUnit runner has a dependency on JUnit 3.8. It won't be used but without it, the whole platform can't be initialized.
So you need 2 versions like the following
Check you have below plugins
org.eclipse.xtext.xbase.junit
org.junit (3.8.2)
org.junit (4.8.2)
How to check for JUnit?
In eclipse, please check in 2 sections.
Check your Project Properties->Java Build Path->Libraries (tab)
Check you Project's Run Configurations->JUnit->Classpath (tab)
How to fix?
To fix the error make sure you have org.junit 3.8 in the target platform!
All credit goes to A Paul.
Resource Link:
https://stackoverflow.com/a/21334162/2293534
Aaron Digulla has commented like below:
Despite the fact that org.eclipse.xtext.junit4 imports org.junit
4.5.0, org.eclipse.xtext.junit (note the missing "4" at the end) seems to have a dependency to JUnit 3.8. After adding the old, outdated
JUnit bundle, the plugin tests started.
I'm running some service testing using restassured and cucumber and they work fine locally just using Maven test.
The issue is if I run Maven clean, then I must run Maven update or it will not work (Says it can't find my Cucumber feature files). For reference it says:
No features found at [classpath:classpath/classpath]
This wouldn't be a huge issue except I need to have this running through Bamboo where I can't call Maven update.
So I either need to figure out what is wrong with my POM to begin with to cause this issue, or how I can run Maven update through the goals/environment variables.
The POM is fairly simple, only having the needed dependencies/reporting stuff.
The build part of the POM is as follows:
<build>
<finalName>Test</finalName>
<directory>target</directory>
<outputDirectory>target/classes</outputDirectory>
<testOutputDirectory>target/test-classes</testOutputDirectory>
<sourceDirectory>src/main/java</sourceDirectory>
<testSourceDirectory>src/test/java</testSourceDirectory>
<resources>
<resource>
<directory>src/test/resources</directory>
</resource>
</resources>
</build>
This is all in Java 8 using Eclipse as the IDE.
I would avoid specifying anything in the build section in my pom and instead use the default values.
That is, I would keep my feature files in the same package as the runner or a sub package.
The runner could for example live in the package se/thinkcode/tage
As in the directory:
./test/java/se/thinkcode/tage
This means that the feature files should live in the directory:
./test/resources/se/thinkcode/tage
This would allow me to minimize the configuration in the runner. I typically use runners that looks like this:
package se.thinkcode.tage;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
public class RunCukesTest {
}
This is the smallest configuration possible if you want to run Cucumber using JUnit from Maven.
It is even smaller that the example supplied by the Cucumber team: https://github.com/cucumber/cucumber-java-skeleton
Looks like defining the features/glue in the cucumber options fixed this.
I do believe there is a better option though.
I added the following cucumber options:
features ="src/test/java",
glue = "packagename",
I have a maven project (github) that uses the Maven integration test verifier extensively.
The tests refer to the project I'm testing. For example, this pom does:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.4.1</version>
<dependencies>
<dependency>
<!-- My project which customizes the plugin. -->
<groupId>com.google.security</groupId>
<artifactId>fences-maven-enforcer-rule</artifactId>
<version>1.2-beta-SNAPSHOT</version>
</dependency>
</dependencies>
...
Where that dependency is to relative path ../../../../../pom.xml.
Then my junit test uses a Verifier to run the integration test.
// testProjectName is the basename of the directory
// containing the POM above.
File testDir = ResourceExtractor.simpleExtractResources(
getClass(), "/" + testProjectName);
Verifier verifier = new Verifier(
testDir.getAbsolutePath(),
null, debug == Debug.VERBOSE, true /* forkJvm */);
// Clean up after previous runs.
verifier.deleteArtifacts("test");
Result goalResult = Result.PASS;
try {
verifier.executeGoal("verify");
} catch (#SuppressWarnings("unused") VerificationException ex) {
goalResult = Result.FAIL;
}
I can test this by doing mvn install -DskipTests=true && mvn test but that is less than ideal, because if I ever change code without reinstalling, I end up running tests against an out-of-date version, and because someone downloading the project for the first time can't just do mvn test.
Is there some way to tweak the POM or the junit TestCase so that the dependency is on the classes that were just compiled to target/classes?
You need to either build and install the depenendy before you built the dependend project or both have to be part of the same reactor build. I think it does not work if it is in the same module (at least not with an artifact dependency, the test classes are automatically on the test class path - but the plugin might not load from there).