TestNG nested tests or test steps - java

I'm wondering is it possible to make TestNG show in it's report something like nested tests or test steps.
The thing is that I have pretty big integration test cases and it would be nice if I could decouple it on some steps, say, with something like allure's #Step annotation.
For now the test case is shown as one huge separate entry in intellij IDEA report with a lot of logs which is very difficult to go through and analyze.

You can use createNode method of ExtentTest class. it will creating node under main test.

I've never tried to use Inner TestNG classes but that might be possible and I do believe IntelliJ would render the nested view. You could make a simple little project to try it out. There is a guy who seems to have tried inner classes here: http://makeseleniumeasy.com/2018/05/13/testng-tutorials-12-how-to-run-inner-testng-class-from-testng-xml/ . Let me know if it works. Won't know if IntelliJ renders the nesting until you try.
Bonus: If that works, would be interesting to execute your TestNG using Gradle testng plugin and see if HTML report also renders the nesting.

Inner classes can be used to form groups. For one level of nesting the inner classes must be made public.
Try the following:
public class foo {
public class bar1 {
#Test
public void test11() {System.out.println("test11");}
#Test
public void test12() {System.out.println("test12");}
}
public class bar2 {
#Test
public void test21() {System.out.println("test21");}
#Test
public void test22() {System.out.println("test22");}
}
}

JUnit does have #Nested to group them and you can use the shortcut cmd+R etc. to run them, especially if you have lots of test cases
For TestNG, either split them into different (inner) classes, or use structure view in IntelliJ (https://www.jetbrains.com/help/idea/viewing-structure-of-a-source-file.html)
Right-click all those tests that you want to run

Now you can use groups feature of TestNG E.g.:
class RecordsAccessorTest {
#BeforeMethod(groups = "NewRecordGroup")
public void setUp() {
// some set up for only new record creation is tested
}
#Test(groups = "NewRecordGroup")
public void testNewRecordCreation_happyPath() {...}
#Test(groups = "NewRecordGroup")
public void testNewRecordCreation_errorPath() {...}
#Test
public void someOtherTestNotInGroup() {...}
}

Related

How to add test method's comment sections from IntelliJ automatically?

I am creating test classes from the IntelliJ automatically like following
This gives me the test class as following in the related module accordingly:
public class MyClassTest {
#Test
public void myMethod() {
}
}
What I am looking for is that
Does IntelliJ can automatically insert Given When Then comment sections into test methods?
I am searching for something like following:
public class MyClassTest {
#Test
public void myMethod() {
// Given
// When
// Then
}
}
These sections are useful for the reader coming after some times passed but usually missed while writing test methods. I am looking for a solution to add this behavior to IDE.
Of course. Navigate to Settings - Editor - File and Code templates - JUnit Test Method

JUnit: is it possible to create a Test Suite that executes all test of classes that share a naming convention?

I am aware that I can make a TestSuite enumerating all the classes that I want, for example:
#RunWith(Suite.class)
#SuiteClasses({SQLServerTests1.class, SQLServerTest2.class, ... })
public class AllSQLServerTests {}
However I have almost 100+ classes and I don't want to have to remember to include any new one in the #SuiteClasses annotation.
As my classes have a naming convention (starting with "SQLServer" for example) I am searching for a way to do something like this:
#RunWith(Suite.class)
#SuiteClasses(prefix="SQLServer")
public class AllSQLServerTests {}
is it possible with plain JUnit? with spring or any other framework?
Tag'em
You can add many tags to each test or test class:
#Test
#Tag("red")
#Tag("production")
public void testWithColour() {...}
#RunWith(JUnitPlatform.class)
#IncludeTags("red & !production")
public class JUnit5Example {
//...
}
You can also use #ExcludeTags but it cannot co-exist with #IncludeTags
Run all in test package
#RunWith(JUnitPlatform.class)
#SelectPackages("com.acme.megaproduct.slowtests")
public class JUnit5Example {
//...
}
Write custom Test Runner
Perhaps none of the above can acommodate your needs, in which case you can add custom filtering by writing your own runner.
See here for step by step how to do it.
Then you just use it like:
#RunWith(MyCustomRunner.class)
public class CustomTestSuite {
//...
}

How do I make a junit white list of tests to run

I'm working on writing unit tests for a class that I'm developing. Another developer is developing other tests for the same class for methods that he's developing. So our tests find themselves in the same JUnit test class.
So what I wanted to do was to set up a test suite to run just my tests while I'm developing as a temporary measure. I created a Category for my tests and have marked them as such. I then created a class to be my test suite. I told it to include tests that belong to this category. When I run it, it still runs everything. There are a lot of tests, so it would be tedious to mark all the tests I don't want ran with #Ignore. Is there a way to say, run only the tests in a category but none else?
You can write a wrapper test class which method calls the main test class (only your method), then run Junit tests on the wrapper class.
public class MainTestClass {
#Test
public void yourFirstTest() {
...
}
#Test
public void yourSecondTest() {
...
}
#Test
public void otherFirstTest() {
...
}
}
public class WrapperTestClass {
#Test
public void yourFirstTest() {
new MainTestClass().yourFirstTest();
}
#Test
public void yourSecondTest() {
new MainTestClass().yourSecondTest();
}
}
I think you can implement your own 'org.junit.runner.RunWith' and then annotate your test class to use it as necessary.
#RunWith(MyRunnerClass.class)
Note: The correct solution here is in the above comments regarding code branches etc.

#BeforeClass runs multiple times for the same class in eclipse

I am writing some junit tests in eclipse and I need to do some time consuming setup before the tests. Appeared that #BeforeClass should be the way to do this. I currently tested this on a class that has 2 #Test functions.
When I right click on a class in eclipse and chose "Run As" -> "JUnit Test" I can see that the #BeforeClass is executed before both functions.
I even tried to change #BeforeClass to #Before and stored in a boolean whether we had already executed this function, but it seems that eclipse created two class objects from the same class, one for each test to run so that did not help either.
So what should I do to have a setup function run only one time even if I have many tests ? Or am I just using eclipse incorrectly when trying to run the tests ?
The setup is something like this:
public class SuperClass {
#BeforeClass
public void { // do timeconsuming setup }
}
public class TestClass extends SuperClass {
#Test
public void test1() { // perform first test }
#Test
public void test2() { // perform second test }
}
Making static the method annotated with BeforeClass may be the solution:
#BeforeClass
public static void
#BeforeClass methods should be static in order to be executed only once.

Grouping JUnit tests

Is there any way to group tests in JUnit, so that I can run only some groups?
Or is it possible to annotate some tests and then globally disable them?
I'm using JUnit 4, I can't use TestNG.
edit: #RunWith and #SuiteClasses works great. But is it possible to annotate like this only some tests in test class? Or do I have to annotate whole test class?
JUnit 4.8 supports grouping:
public interface SlowTests {}
public interface IntegrationTests extends SlowTests {}
public interface PerformanceTests extends SlowTests {}
And then...
public class AccountTest {
#Test
#Category(IntegrationTests.class)
public void thisTestWillTakeSomeTime() {
...
}
#Test
#Category(IntegrationTests.class)
public void thisTestWillTakeEvenLonger() {
...
}
#Test
public void thisOneIsRealFast() {
...
}
}
And lastly,
#RunWith(Categories.class)
#ExcludeCategory(SlowTests.class)
#SuiteClasses( { AccountTest.class, ClientTest.class })
public class UnitTestSuite {}
Taken from here: https://community.oracle.com/blogs/johnsmart/2010/04/25/grouping-tests-using-junit-categories-0
Also, Arquillian itself supports grouping:
https://github.com/weld/core/blob/master/tests-arquillian/src/test/java/org/jboss/weld/tests/Categories.java
Do you want to group tests inside a test class or do you want to group test classes? I am going to assume the latter.
It depends on how you are running your tests. If you run them by Maven, it is possible to specify exactly what tests you want to include. See the Maven surefire documentation for this.
More generally, though, what I do is that I have a tree of test suites. A test suite in JUnit 4 looks something like:
#RunWith(Suite.class)
#SuiteClasses({SomeUnitTest1.class, SomeUnitTest2.class})
public class UnitTestsSuite {
}
So, maybe I have a FunctionTestsSuite and a UnitTestsSuite, and then an AllTestsSuite which includes the other two. If you run them in Eclipse you get a very nice hierarchical view.
The problem with this approach is that it's kind of tedious if you want to slice tests in more than one different way. But it's still possible (you can for example have one set of suites that slice based on module, then another slicing on the type of test).
To handle the globally disabling them, JUnit (4.5+) has two ways One is to use the new method assumeThat. If you put that in the #BeforeClass (or the #Before) of a test class, and if the condition fails, it will ignore the test. In the condition you can put a system property or something else that can be globally set on or off.
The other alternative is to create a custom runner which understands the global property and delegates to the appropriate runner. This approach is a lot more brittle (since the JUnit4 internal runners are unstable and can be changed from release to release), but it has the advantage of being able to be inherited down a class hierarchy and be overridden in a subclass. It is also the only realistic way to do this if you have to support legacy JUnit38 classes.
Here is some code to do the custom Runner. Regarding what getAppropriateRunnerForClass might do, the way I implemented it was to have a separate annotation that tells the custom runner what to run with. The only alternative was some very brittle copy paste from the JUnit code.
private class CustomRunner implements Runner
private Runner runner;
public CustomRunner(Class<?> klass, RunnerBuilder builder) throws Throwable {
if (!isRunCustomTests()) {
runner = new IgnoredClassRunner(klass);
} else {
runner = getAppropriateRunnerForClass(klass, builder);
}
public Description getDescription() {
return runner.getDescription();
}
public void run(RunNotifier notifier) {
runner.run(notifier);
}
}
EDIT: The #RunWith tag only works for a whole class. One way to work around that limiation is to move the test methods into a static inner class and annotate that. That way you have the advantage of the annotation with the organization of the class. But, doing that won't help with any #Before or #BeforeClass tags, you will have to recreate those in the inner class. It can call the outer class's method, but it would have to have its own method as a hook.
In JUnit 5 you can declare #Tag for filtering tests, either at the class or method level; analogous to test groups in TestNG or Categories in JUnit 4
From the javadoc :
tags are used to filter which tests are executed for a given test
plan. For example, a development team may tag tests with values such
as "fast", "slow", "ci-server", etc. and then supply a list of tags to
be used for the current test plan, potentially dependent on the
current environment.
For example you could declare a test class with a "slow" #Tag that will be inherited for all methods and override it for some methods if required :
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
#Tag("slow")
public class FooTest{
//
#Test
void loadManyThings(){
...
}
#Test
void loadManyManyThings(){
...
}
#Test
#Tag("fast")
void loadFewThings(){
...
}
}
You could apply the same logic for other test classes.
In this way test classes (and methods too) belongs to a specific tag.
As a good practice instead of copying and pasting #Tag("fast") and #Tag("slow") throughout the test classes, you can create custom composed annotations.
For example :
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import org.junit.jupiter.api.Tag;
#Target({ ElementType.TYPE, ElementType.METHOD })
#Retention(RetentionPolicy.RUNTIME)
#Tag("slow")
public #interface Slow {
}
and use it as :
#Test
#Slow
void slowProcessing(){
...
}
To enable or disable test marked with a specific tag during the text execution you can rely on the maven-surefire-plugin documentation :
To include tags or tag expressions, use groups.
To exclude tags or tag expressions, use either excludedGroups.
Just configure in your pom.xml the plugin according to your requirement (example of the doc) :
<build>
<plugins>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<groups>acceptance | !feature-a</groups>
<excludedGroups>integration, regression</excludedGroups>
</configuration>
</plugin>
</plugins>
</build>
For information the test goal documentation is not updated.
Try JUnit Test Groups. From documentation :
#TestGroup("integration")
public class MyIntegrationTest {
#ClassRule
public static TestGroupRule rule = new TestGroupRule();
...
}
Execute a simple test group: -Dtestgroup=integration
Execute multiple test groups: -Dtestgroup=group1,group2
Execute all test groups: -Dtestgroup=all
You can create test Suite objects that contain groups of tests. Alternatively, your IDE (like Eclipse) may have support for running all the tests contained in a given package.
You can Use Test Suite(http://qaautomated.blogspot.in/2016/09/junit-test-suits-and-test-execution.html) or you can Junit Categories(http://qaautomated.blogspot.in/2016/09/junit-categories.html) for grouping your test cases effectively.

Categories

Resources