Ok, I have multiple classes in my testNG suite and all the methods inside those classes has the priority defined. When I run the single class, all the methods runs in a order defined as per the priority but when I run multiple classes, testNG doesn't run it in sequential order. First, it'll run single method from Class A then it'll run another method from Class B and then another method from Class C.
I want testNG to run all the methods from Class A and then move to other classes and run all the methods from those classes before moving to other class. Is there any way to achieve this ?
This post should cover how to retain the order specified for classes while you use priority.
Related
Currently I have one java class (Step Definitions.java) that contains all -
#Before, #After, #Given, #When, #Then
I want to split these annotations into three different classes. One class for the #Before / #After. One class for #Given; One class for #When; and One class for #Then.
All 4 classes will be under the same package. Can this be done? Do I have to change anything in the runner class? Any other references I have to make to these seperate classes? or should this just work like that, when I call the Gherkin in my feature file?!
It will work out off the box. As long as the code is mentioned in the package structure given to the glue option for cucumberoptions, it will be loaded and executed. All the stepdefinition and hook code are loaded for each scenario, so doesnt matter which class the code exists.
Though better would be to separate them according to parts of the application they deal with.
I have a question about parameterized tests in JUnit. I am running a test suite with all of my test classes, it is a requirement for my course to have a test suite to run all of my test classes, so I cannot modify that. The issue is that I have a bunch of Entry objects (let's just take this as an object with a unique id starting from 1 and incremented every time a new instance of it is created), and they are being pre-processed by JUnit. On compiling and running my program, I have 9 entries which are declared in the ParamTest class. Within another class (EntryTest) I have one Entry that I have created and it should have an ID of one. However, it has an ID of 10, meaning the 9 entries from the parameterized test class have been created before hand.
My question is, is there anyway to force the ParamTest class not to do any of the pre-processing before the EntryTest class is run or is this impossible. In the suite I have made sure to declare EntryTest before ParamTest. If it's impossible is there anyway I can get around this other than creating separate suites or running the tests separately? I was thinking a public static int to keep track of the ID from the pre-processed amounts but it sounds like an ugly solution.
I think your testing is going to get ugly, fast, unless you have a way of resetting your static class to a known state.
I would recommend you expose a package-private method that allows you to reset the ID value to something specific (e.g. 0).
Tests should be entirely independent of one another, even within the same test class.
I'm using Spring Test with TestNG to test our DAOs, and I wanted to run a specific text fixture script before certain methods, allowing the modifications to be rolled back after every method so that the tests are free to do anything with the fixture data.
Initially I thought that 'groups' would be fit for it, but I already realized they're not intended for that (see this question: TestNG BeforeMethod with groups ).
Is there any way to configure a #BeforeMethod method to run only before specific #Tests? The only ways I see are workarounds:
Define an ordinary setup method and call at the beginning of every #Test method;
Move the #BeforeMethod method to a new class (top level or inner class), along with all methods that depend on it.
Neither is ideal, I'd like to keep my tests naturally grouped and clean, not split due to lack of alternatives.
You could add a parameter your #BeforeMethod with the type 'java.lang.reflect.Method'. TestNG will then inject the reflection information for the current test method including the method name, which you could use for switching.
If you add another 'Object' parameter, you will also get the invocation parameters of the test method.
You'all find all on possible parameters for TestNG-annotated methods in chapter 5.18.1 of the TestNG documentation.
Tests are simply not designed to do this. Technically speaking, a single tests is supposed to handle being idempotent for itself meaning it sets up, tests, and takes down. That is a single test. However, a lot of tests sometimes have the same set-up and take down method, whereas other tests need one set-up before they all run. This is the purpose of the #Before type tags.
If you don't like set-up and tear-down inside your test, your more then welcome to architect your own system, but technically speaking, if certain methods require specific set-ups or tear-downs, then that really should be embodied IN the test, since it is a requirement for test to pass. It is ok to call a set-up method, but ultimately, it should be OBVIOUS that a test needs a specific set-up in order to pass. After all, if your using specific set-ups, aren’t you actually testing states rather than code?
I've written a number of tests, divided not only into separate classes but, depending on what area of my application they're testing, into separate sub-packages. So, my package structure looks a bit like this:
my.package.tests
my.package.tests.four
my.package.tests.one
my.package.tests.three
my.package.tests.two
In the my.package.tests package I have a parent class that all tests in the sub-packages one through four extend. Now, I would like to choose the order in which the tests are run; not within the classes (which seems to be possible with the FixMethodOrder annotation) but the order of the classes or sub-packages themselves (So those in sub-package one first, then those in two, ect.). Some of the test classes use the Parameterized runner, just in case that makes a difference. Choosing the order is not required for the tests to succeed, they are independent of each other; it would however be helpful to sort them to reflect the order in which the various parts of the program are normally used as it makes the analysis a bit easier.
Now, ideally I would like to have some configuration file which tells JUnit which order the tests should be executed in; I'm guessing however that it won't be so easy. Which options do I have here and what would be the easiest one? I'd also prefer to only have to list the sub-packages, not the various classes in the packages or even the test functions in the classes.
I'm using Java 1.6.0_26 (I don't have a choice here) and JUnit 4.11.
You can do this with test suites. (you can "nest" these, too)
#SuiteClasses({SuiteOne.class, SuiteTwo.class})
#RunWith(Suite.class)
public class TopLevelSuite {}
#SuiteClasses({Test1.class, Test2.class})
#RunWith(Suite.class)
public class SuiteOne {}
#SuiteClasses({Test4.class, Test3.class})
#RunWith(Suite.class)
public class SuiteTwo {}
... And so on. Use the "toplevelsuite" as an entry point, and the tests will execute in the order in which you define the #SuiteClasses array.
As for the methods within a class, you can specify the order they execute in with #FixMethodOrder (as you have already mentioned in your question.
I am fairly new to Java. I have constructed a single JUnit test class and inside this file are a number of test methods. When I run this class (in NetBeans) it runs each test method in the class in order.
Question 1: How can I run only a specific sub-set of the test methods in this class?
(Potential answer: Write #Ignore above #Test for the tests I wish to ignore. However, if I want to indicate which test methods I want to run rather than those I want to ignore, is there a more convenient way of doing this?)
Question 2: Is there an easy way to change the order in which the various test methods are run?
Thanks.
You should read about TestSuite's. They allow to group & order your unit test methods. Here's an extract form this article
"JUnit test classes can be rolled up
to run in a specific order by creating
a Test Suite.
EDIT: Here's an example showing how simple it is:
public static Test suite() {
TestSuite suite = new TestSuite("Sample Tests");
suite.addTest(new SampleTest("testmethod3"));
suite.addTest(new SampleTest("testmethod5"));
return suite;
}
This answer tells you how to do it. Randomizing the order the tests run is a good idea!
Like the comment from dom farr states, each unit test should be able to run in isolation. There should be no residuals and no given requirements after or before a test run. All your unit tests should pass run in any order, or any subset.
It's not a terrible idea to have or generate a map of Test Case --> List of Test and then randomly execute all the tests.
There are a number of approaches to this, but it depends on your specific needs. For example, you could split up each of your test methods into separate test classes, and then arrange them in different test suites (which would allow for overlap of methods in the suites if desired). Or, a simpler solution would be to make your test methods normal class methods with one test method in your class that calls them in your specific order. Do you want them to be dynamcially called?
I've not been using Java that long either, but as far as I've seen there isn't a convenient method of marking methods to execute rather than ignore. Instead I think this could be achieved using the IDE. When I want to do that in Eclipse I can use the junit view to run individual tests by clicking on them. I imagine there is something similar in Netbeans.
I don't know of an easy way to reorder the test execution. Eclipse has a button to rerun tests with the failing tests first, but it sounds like you want something more versatile.