I am trying to figure out how to work with test setup, body and teardown in the execution summary of an allure report generated from junit5. I am using aspectjweaver as jvm agent as this seams to be necessary to display the execution summary whatsoever.
I figured the junit5 #BeforeEach and #AfterEach methods should be displayed in test setup and teardown. Unfortunately for me they are always displayed inside the test body and setup/teardown do not show up at all.
In following example
#BeforeEach
public void before() {
before_step();
}
#Step("before step")
public void before_step() {
assertTrue(true);
}
the allure report will show the method in the test body like this.
Does someone have an idea why this is? Is this just not implemented correctly, do i need to implement something differently or do I maybe have a completely wrong idea as how this should be displayed?
Thanks for any help!
In order to enable test fixtures you need to use allure-junit5 dependency.
Then you can use AllureJunit5 extension on your tests - that will enable fixtures reporting.
The global configuration is also possible. To enable AllureJunit5 extension for all
your tests you need to start your JVM with -Djunit.jupiter.extensions.autodetection.enabled=true system property.
For more details on automatic extension registration please follow https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic
Related
I have two methods in my test class:
#Test
#Stories( "story1")
public void test01(){
}
#Test
#Stories( "story2")
public void test02(){
}
#Test
#Stories( "story1")
public void test03(){
}
To run tests Im using:
mvn clean test site
It will execute all test. But my question is, how to execute tests when I want to execute only tests with specific user story (ie. story1)
I know in python it can be done by
py.test my_tests/ --allure_stories=story1
But I don't know how to do it in java using maven
In Java there is no need for Allure to do such sort of things, because you can do it using your test runner, e.g. TestNG.
Just create Listener or BeforSuite which will check your environment variable e.g. -DallureStories and match it with ITestContext to disable tests not in your stories list.
For our usecase, as an example - we need to run a JUnit test, even if it is added multiple times within a Test Suite, without being skipped.
Currently we notice that JUnit test runner skips a Test with the same name, if it finds the test somewhere else within a Test Suite. Here is an example screenshot to show test "Case_A" within "Procedure_A" being skipped within a Test Suite -
Could this behaviour be overriden, if so could someone point us in the right direction?
I did some research arround this problem.
Simple setting - one Test "TestCase_A" and one Suite "TestProcedure_A" that runs the TestCase_A twice :
public class TestCase_A {
#Test
public void test() throws Exception {
System.out.println("Case_A RUN");
Assert.assertTrue(true);
}
}
#RunWith(Suite.class)
#Suite.SuiteClasses({ TestCase_A.class, TestCase_A.class })
#SuppressWarnings("all")
public class TestProcedure_A {
}
I run the test Suite using Eclipse and maven.
Finding: The sysout statement actually shows, that the TestCase_A runs twice!
Therefore, the Eclispe View is misleading. Test are run multiple times - the tree also reflects this. However, the status of the actual single calls is not displayed properly in the Eclispe Junit View.
I presume the view is based on the junit.runner.TestRunListener. It probably worth looking into that.
I've managed to get my Android project transitioned over to JUnit4, and of course the main reason I wanted to do it isn't working. Would love any help if anyone's got ideas here.
The problem I'm trying to solve is that I want to automatically skip certain tests if the build is not pointed at the staging server. I've got this set up with a BUILD_TYPE which is using gradle to inject the base URL.
I set up an assumeThat clause in my setup which correctly identifies when the build is not staging, but instead of halting and ignoring the rest of the test, it throws an exception and fails.
Here's my base class for my live API tests - I've annotated descending from this with #RunWith(AndroidJUnit4.class), so in theory this should always be run with the JUnit4 runner:
package com.[my package].nonuitests.liveservertests;
import android.support.test.runner.AndroidJUnit4;
import com.[my package].nonuitests.BaseAndroidTest;
import org.junit.Test;
import org.junit.runner.RunWith;
/**
* Tests against the live API. All tests descending from this class will
* be ignored if the BUILD_TYPE is not staging.
*/
#RunWith(AndroidJUnit4.class)
public class BaseLiveServerTests extends BaseAndroidTest {
private static final String STAGE = "staging";
/******************
* SETUP/TEARDOWN *
******************/
#Override
public void setUp() throws Exception {
super.setUp();
//TODO: Y U NO WORK?!
//This should cause the rest of the test to be skipped if it fails,
//but is instead throwing an AssumptionViolatedException.
assumeTrue(STAGE.equals(BuildConfig.BUILD_TYPE));
}
}
So, my questions:
Is there a better way to do this? In theory this could be done with flavors, but I was trying that earlier and it made everything else way more complicated.
My research indicates there's some kind of thing that Google's not implementing in their runner that's causing this to bomb out, but I'm having a hell of a time figuring out what I can/should do to fix this. Any suggestions of things I should subclass to get this to work as expected?
Any other thoughts would be most appreciated. Thanks!
Edit (1/26/15 11:40am CST): Per Grzesuav's suggestion, I took a stab at implementing this as an #Rule, but at the moment it's still not working. This seems a promising path, but it ain't working at the moment.
Edit 2 (1/26/15 12:15pm CST): OK, now it's working.
https://github.com/junit-team/junit/wiki/Assumptions-with-assume
ad 2) Custom runners could differently treat assume statement. To fix it you should write own version of Android runner and implement a way of dealing with assumes as native JUnit runner does or make a bug for android test runner.
ad 1) Suggested by me : try use JUnit Rules :
http://www.codeaffine.com/2013/11/18/a-junit-rule-to-conditionally-ignore-tests/
http://cwd.dhemery.com/2010/12/junit-rules/
OK, finally got it working with #Rules per Grzesuav's suggestion, although with significant changes since MethodRule has been deprecated. Here's a gist of what it turned out to be - I'll try to keep that updated as I refine it.
Some important notes:
You have to instantiate your #Rule in your test class, or you'll never actually hit any of your checks.
As of right now, this will not mark the test as ignored on Android, it'll just pass it without actually testing anything.
In Junit 4.12 it cannot handle tearDown
If you have tearDown you have to add an if statement with your condition rather than assumeTrue. I think the owners of Junit say it isn't supposed to work with #After
#After
override fun tearDown() {
if (junit == worksAgain()) {
I want to create custom html report for test run in JUnit. Problem I have is releasing resources and closing tags after all tests are done.
I keep one FileChannel opened for writing to report. Since it should be table with row for each test and there are hundreds of them, I don't want to open and close the channel for each test. Problem that appears here is tests organization - I have nested suites, so testRunFinished is not an option (refers to single suite, not all tests, and I saw this question). TestWatcher also will not help me, since it refers to single test only.
Tools used: maven 3.0.5, ff webdriver, junit 4.11.
I was considering two options:
1) opening and closing channel on each test run
2) overwriting finalize() to make it close the channel
None of them seems pretty... I've searched through many pages, but nobody seems to have the same problem I have.
Any prettier solutions?
Yes, see here (Before and After Suite execution hook in jUnit 4.x):
#RunWith(Suite.class)
#SuiteClasses({Test1.class, Test2.class})
public class TestSuite {
#BeforeClass
public static void setUp() {
System.out.println("setting up");
}
#AfterClass
public static void tearDown() {
System.out.println("tearing down");
}
}
When writing code that interacts with external resources (such as using a web service or other network operation), I often structure the classes so that it can also be "stubbed" using a file or some other input method. So then I end up using the stubbed implementation to test other parts of the system and then one or two tests that specifically test calling the web service.
The problem is I don't want to be calling these external services either from Jenkins or when I run all of the tests for my project (e.g. "gradle test"). Some of the services have side effects, or may not be accessible to all developers.
Right now I just uncomment and then re-comment the #Test annotation on these particular test methods to enable and disable them. Enable it, run it manually to check it, then remember to comment it out again.
// Uncomment to test external service manually
//#Test
public void testSomethingExternal() {
Is there is a better way of doing this?
EDIT: For manual unit testing, I use Eclipse and am able to just right-click on the test method and do Run As -> JUnit test. But that doesn't work without the (uncommented) annotation.
I recommend using junit categories. See this blog for details : https://community.oracle.com/blogs/johnsmart/2010/04/25/grouping-tests-using-junit-categories-0.
Basically, you can annotate some tests as being in a special category and then you can set up a two test suites : one that runs the tests of that category and one that ignores tests in that category (but runs everything else)
#Category(IntegrationTests.class)
public class AccountIntegrationTest {
#Test
public void thisTestWillTakeSomeTime() {
...
}
#Test
public void thisTestWillTakeEvenLonger() {
....
}
}
you can even annotate individual tests"
public class AccountTest {
#Test
#Category(IntegrationTests.class)
public void thisTestWillTakeSomeTime() {
...
}
Anytime I see something manually getting turned on or off I cringe.
As far as I can see you use gradle and API for JUnit says that annotation #Ignore disables test. I will add gradle task which will add #Ignore for those tests.
If you're just wanting to disable tests for functionality that hasn't been written yet or otherwise manually disable some tests temporarily, you can use #Ignore; the tests will be skipped but still noted in the report.
If you are wanting something like Spring Profiles, where you can define rulesets for which tests get run when, you should either split up your tests into separate test cases or use a Filter.
You can use #Ignore annotation to prevent them from running automatically during test. If required, you may trigger such Ignored tests manually.
#Test
public void wantedTest() {
return checkMyFunction(10);
}
#Ignore
#Test
public void unwantedTest() {
return checkMyFunction(11);
}
In the above example, unwantedTest will be excluded.