When writing code that interacts with external resources (such as using a web service or other network operation), I often structure the classes so that it can also be "stubbed" using a file or some other input method. So then I end up using the stubbed implementation to test other parts of the system and then one or two tests that specifically test calling the web service.
The problem is I don't want to be calling these external services either from Jenkins or when I run all of the tests for my project (e.g. "gradle test"). Some of the services have side effects, or may not be accessible to all developers.
Right now I just uncomment and then re-comment the #Test annotation on these particular test methods to enable and disable them. Enable it, run it manually to check it, then remember to comment it out again.
// Uncomment to test external service manually
//#Test
public void testSomethingExternal() {
Is there is a better way of doing this?
EDIT: For manual unit testing, I use Eclipse and am able to just right-click on the test method and do Run As -> JUnit test. But that doesn't work without the (uncommented) annotation.
I recommend using junit categories. See this blog for details : https://community.oracle.com/blogs/johnsmart/2010/04/25/grouping-tests-using-junit-categories-0.
Basically, you can annotate some tests as being in a special category and then you can set up a two test suites : one that runs the tests of that category and one that ignores tests in that category (but runs everything else)
#Category(IntegrationTests.class)
public class AccountIntegrationTest {
#Test
public void thisTestWillTakeSomeTime() {
...
}
#Test
public void thisTestWillTakeEvenLonger() {
....
}
}
you can even annotate individual tests"
public class AccountTest {
#Test
#Category(IntegrationTests.class)
public void thisTestWillTakeSomeTime() {
...
}
Anytime I see something manually getting turned on or off I cringe.
As far as I can see you use gradle and API for JUnit says that annotation #Ignore disables test. I will add gradle task which will add #Ignore for those tests.
If you're just wanting to disable tests for functionality that hasn't been written yet or otherwise manually disable some tests temporarily, you can use #Ignore; the tests will be skipped but still noted in the report.
If you are wanting something like Spring Profiles, where you can define rulesets for which tests get run when, you should either split up your tests into separate test cases or use a Filter.
You can use #Ignore annotation to prevent them from running automatically during test. If required, you may trigger such Ignored tests manually.
#Test
public void wantedTest() {
return checkMyFunction(10);
}
#Ignore
#Test
public void unwantedTest() {
return checkMyFunction(11);
}
In the above example, unwantedTest will be excluded.
Related
I am trying to figure out how to work with test setup, body and teardown in the execution summary of an allure report generated from junit5. I am using aspectjweaver as jvm agent as this seams to be necessary to display the execution summary whatsoever.
I figured the junit5 #BeforeEach and #AfterEach methods should be displayed in test setup and teardown. Unfortunately for me they are always displayed inside the test body and setup/teardown do not show up at all.
In following example
#BeforeEach
public void before() {
before_step();
}
#Step("before step")
public void before_step() {
assertTrue(true);
}
the allure report will show the method in the test body like this.
Does someone have an idea why this is? Is this just not implemented correctly, do i need to implement something differently or do I maybe have a completely wrong idea as how this should be displayed?
Thanks for any help!
In order to enable test fixtures you need to use allure-junit5 dependency.
Then you can use AllureJunit5 extension on your tests - that will enable fixtures reporting.
The global configuration is also possible. To enable AllureJunit5 extension for all
your tests you need to start your JVM with -Djunit.jupiter.extensions.autodetection.enabled=true system property.
For more details on automatic extension registration please follow https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic
I have setup an embedded mongo via flapdoodle (de.flapdoodle.embed).
Quite a lot of mongo operations hence i would like to run all of them as a suite and setup the mongo just once in testsuite.
Now when i run the test cases via mvn install , it seems to run the test cases individually.
Is there a way to run test cases only from suite and not as a class.
baeldung.com describes the use of JUnit 5 Tags, which are very well suited for your case.
You can mark tests with two different tags:
#Test
#Tag("MyMongoTests")
public void testThatThisHappensWhenThatHappens() {
}
#Test
#Tag("MyTestsWithoutMongo")
public void testThatItDoesNotHappen() {
}
And execute either set in a suite, e.g.
#IncludeTags("MyMongoTests")
public class MyMongoTestSuite {
}
In your case, the tests could be categorized by whether Mongo is in the application context or not. So, theoretically, it might be possible to create a JUnit 5 Extension to add the tag. That would be the more complex solution though.
I have two methods in my test class:
#Test
#Stories( "story1")
public void test01(){
}
#Test
#Stories( "story2")
public void test02(){
}
#Test
#Stories( "story1")
public void test03(){
}
To run tests Im using:
mvn clean test site
It will execute all test. But my question is, how to execute tests when I want to execute only tests with specific user story (ie. story1)
I know in python it can be done by
py.test my_tests/ --allure_stories=story1
But I don't know how to do it in java using maven
In Java there is no need for Allure to do such sort of things, because you can do it using your test runner, e.g. TestNG.
Just create Listener or BeforSuite which will check your environment variable e.g. -DallureStories and match it with ITestContext to disable tests not in your stories list.
I am having a problem, one test class seems to be interfering with the other in my test suite.
I have a suite, which executes two classes, one is called MergeTestSuite.java (which is another Suite), and the other is called RecordTest.java.
RecordTest extends one class already tested by MergeTestSuite.java
I created another suite as follows:
#RunWith(Suite.class)
#Suite.SuiteClasses( {
MergeTestSuite.class,
RecordTest.class
})
public class CoreTestSuite {
#BeforeClass
public static void install() throws Throwable {
RegistryUtils.cleanupResources();
}
}
Both MergeTestSuite.class and RecordTest.class run fine individually. If I run CoreTestSuite, the second test will fail, unless I remove MergeTestSuite.class from the list.
A Junit TestSuite provides extra features for multiple tests. For instance, you can control the order in which the tests run and you can also combine multiple test suites into another suite. I think this older doc from Junit 3.1.8 describes it best.
In our project, we currently have a large number of (junit) tests that are split into three categories: unit, integration, wicket.
I now want to group these tests so I can run only one (or two) of those categories. The only thing I found are junit test suites and categories as described here: http://www.wakaleo.com/component/content/article/267
My problem is, I don't want to declare every single test in the Test Suits with #SuiteClasses.
Is there a way to add the suite classes with wildcards / patterns?
Assuming my understanding of the question is correct, it actually can be done using JUnit. The code below was used with JUnit 4.11 and allowed us to split all tests into 2 categories: "uncategorized" and Integration.
IntegrationTestSuite.java
/**
* A custom JUnit runner that executes all tests from the classpath that
* match the <code>ca.vtesc.portfolio.*Test</code> pattern
* and marked with <code>#Category(IntegrationTestCategory.class)</code>
* annotation.
*/
#RunWith(Categories.class)
#IncludeCategory(IntegrationTestCategory.class)
#Suite.SuiteClasses( { IntegrationTests.class })
public class IntegrationTestSuite {
}
#RunWith(ClasspathSuite.class)
#ClasspathSuite.ClassnameFilters({ "ca.vtesc.portfolio.*Test" })
class IntegrationTests {
}
UnitTestSuite.java
/**
* A custom JUnit runner that executes all tests from the classpath that match
* <code>ca.vtesc.portfolio.*Test</code> pattern.
* <p>
* Classes and methods that are annotated with the
* <code>#Category(IntegrationTestCategory.class)</code> category are
* <strong>excluded</strong>.
*/
#RunWith(Categories.class)
#ExcludeCategory(IntegrationTestCategory.class)
#Suite.SuiteClasses( { UnitTests.class })
public class UnitTestSuite {
}
#RunWith(ClasspathSuite.class)
#ClasspathSuite.ClassnameFilters({ "ca.vtesc.portfolio.*Test" })
class UnitTests {
}
IntegrationTestCategory.java
/**
* A marker interface for running integration tests.
*/
public interface IntegrationTestCategory {
}
The first sample test below is not annotated with any category so all its test methods will be included when running the UnitTestSuite and excluded when running IntegrationTestSuite.
public class OptionsServiceImplTest {
#Test
public void testOptionAssignment() {
// actual test code
}
}
Next sample is marked as Integration test on the class level which means both its test methods will be excluded when running the UnitTestSuite and included into IntegrationTestSuite:
#Category(IntegrationTestCategory.class)
public class PortfolioServiceImplTest {
#Test
public void testTransfer() {
// actual test code
}
#Test
public void testQuote() {
}
}
And the third sample demos a test class with one method not annotated and the other marked with the Integration category.
public class MarginServiceImplTest {
#Test
public void testPayment() {
}
#Test
#Category(IntegrationTestCategory.class)
public void testCall() {
}
}
Try using ClasspathSuite
I also had the same problem where I had more then 5500 jUnit tests. I categorised then into 3 groups and created 3 suites using the above jUnit extension. Its great.
You could put them in different packages. Most IDEs have a way to run all the tests in a given package. It's also pretty simple to find all the test classes in a package with a shell script for running tests as part of a build or whatever. I don't know if there's a way to do it with ant, but I would imagine so.
TestNG lets you tag tests as being in particular groups, then run those groups. That sound like exactly what you want, apart from the fact that it's not JUnit!
You could abuse JUnit's assumption mechanism to do what you want: have a system property for each group of tests, and then start each test by assuming that the appropriate property is set. Running all tests will run everything, but everything you don't want will be ignored.
Even if you use JUnit categories, you still won't be able to use wildcards/patterns since categories are annotations, which are Java types.
As pointed out by other commenters, this is exactly why TestNG uses strings to define groups instead of annotations:
#Test(groups = { "database.ACCOUNTS", "fast-test" })
public void newAccountsShouldBeCreated() { ... }
Once you have defined your groups this way, you can include and exclude groups using regular expressions (e.g. "database.*", "front-end.*", etc...).
TestNG is indeed not based on JUnit, but it's very easy to convert all your JUnit tests to TestNG. Here are two blog posts that give an overview of the process:
http://beust.com/weblog/2011/01/04/one-click-test-conversions/
http://beust.com/weblog/2011/02/07/are-your-unit-tests-talking-to-each-other-behind-your-back/
See Junit category or How to run all tests belonging to a certain Category in JUnit 4
have you considered using TestNG?
This is built on JUnit but a lot more powerfull: See comparison.
Grouping is easy.
Tranforming your tests from JUnit to TestNG should be straightforward.
Alternatively, you could create 3 ant scripts that will each run their unit tests but this is less flexible.