I have two classes that each implement something differently but the tests are the same for both. Each class does some things that would affect the other class if they run in parallel, so they can't run in parallel. (that's the rationale behind the code below)
If you run the class Both, see below, in Eclipse, as a TestNG test, one would expect it to run tests test1 and test2 of class ClassAAA first and then the same test methods for ClassBBB, because ClassBBB's group annotations specify that it depends on ClassAAA's annotation.
However, what you find out is that, seemingly, TestNG has a different way of looking at it, and, "seemingly", it ignores the group order and runs the tests of the two clases in parallel.
class Both {
#Test(groups={"base"})
public static abstract class BothBase {
#Test public void test1() { System.out.println("test1"+name()); }
#Test public void test2() { System.out.println("test2"+name()); }
protected String name() {
String s = getClass().getName();
s = s.substring( 1 + s.lastIndexOf("$"));
return " - " + s;
}
}
#Test(groups={"gr1"})
public static class ClassAAA extends BothBase { }
#Test(groups={"gr2"},dependsOnGroups={"gr1"})
public static class ClassBBB extends BothBase { }
}
The output is:
test1 - ClassAAA
test1 - ClassBBB
test2 - ClassAAA
test2 - ClassBBB
One way, which i don't like, to try to "force" it to honor the desired group order, is to add a dummy test method to the leaf classes, as follows:
#Test(groups={"gr1"})
public static class ClassAAA extends BothBase {
#Test public void dummyTestMustBeInAllLeavesToEnforceGroupOrder() {
System.out.println("dummyTestMustBeInAllLeavesToEnforceGroupOrder"+name());
}
}
#Test(groups={"gr2"},dependsOnGroups={"gr1"})
public static class ClassBBB extends BothBase {
#Test public void dummyTestMustBeInAllLeavesToEnforceGroupOrder() {
System.out.println("dummyTestMustBeInAllLeavesToEnforceGroupOrder"+name());
}
}
This still doesn't completely do what one would expect. The output is:
test1 - ClassAAA
test2 - ClassAAA
test2 - ClassBBB
dummyTestMustBeInAllLeavesToEnforceGroupOrder - ClassAAA
test1 - ClassBBB
dummyTestMustBeInAllLeavesToEnforceGroupOrder - ClassBBB
This means that it started running the tests of ClassBBB before finishing the tests of ClassAAA.
I don't like the fact that i have to add a dummy/unrelated method to each, to get TestNG to understand that they cannot be run in parallel. In fact, i don't care which class runs first... And, i didn't really accomplish what i wanted because they are still running in parallel.
The stupidest way to do this, which would accomplish my goal is to move the tests from the base class to each of the leaf classes - is that how you are suppose to do these things in TestNG?
Is there another way of doing that? i'm sure someone is going to suggest priorities - but again, that does not convey the true intention - i don't have any priority - just don't want them to run in parallel. Also i don't want to write XML files...
This problem is coming due to static inner classes. Try below code you will get required output
import org.testng.annotations.Test;
#Test(groups={"base"})
public abstract class BothBase {
#Test public void test1() { System.out.println("test1"+name()); }
#Test public void test2() { System.out.println("test2"+name()); }
protected String name() {
String s = getClass().getName();
s = s.substring( 1 + s.lastIndexOf("$"));
return " - " + s;
}
}
#Test(groups={"gr1"})
class ClassAAA extends BothBase { }
#Test(groups={"gr2"},dependsOnGroups={"gr1"})
class ClassBBB extends BothBase { }
Many ways
Simplest - priority
Bit harder( groups)
Check testng manual(for ex. this http://www.mkyong.com/tutorials/testng-tutorials/).
Related
There are a lot of questions on the execution order of TestNG tests, but I haven't found one that specifically answers this question - unless I just cannot find it.
I know there's things to do that can achieve what I'm looking for, such as:
#Test above every test method within the class with the use of Priority
#Test above every test method within the class with the use of Depends on
The use of preserve-order in the XML file
However, what I want to know is:
If you declare #Test at the class level, not the method level, how to "preserve" the order in which the methods are declared? Can you even do this?
For example, if my class looks like this:
#Test
public class mySampleClass
{
public void test()
{
//Run first - First method in the class
}
public void run()
{
//Run second - Second method in the class
}
public void execute()
{
//Run last - Last method in the class
}
}
Can I ensure that when I run this that the methods are executed in the order they are within the class instead of alphabetically?
Not sure why you want your tests to run that way, Even though it is not possible to do it directly, it IS possible using a MethodInterceptor:
public class MyInterceptor implements IMethodInterceptor {
// keep your list of method names in order
private static final List<String> order = List.of("test", "run", "execute");
#Override
public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
methods.sort(Comparator.comparing(m -> order.indexOf(m.getMethod().getMethodName())));
return methods;
}
}
Now add the #Listeners(value = MyInterceptor.class) annotation to your class.
NOTE: List.of is available only in java 9+, if you are using a lower version, you could use Arrays.asList.
Is there any way that I can run a class 3 times or more in Selenium. So it runs in below order:
method1
method2
method3
method1
method2
method3
method1
method2
method3
import com.test
Class A{
#Test(priority =1)
public void method1(){`System.out.print('method1');`}
#Test(priority =2)
public void method2(){`System.out.print('method2');`}
#Test(priority =3)
public void method3(){`System.out.print('method3');`}
}
As far as I know, there is no easy way in TestNG to do so. Annotation parameter invocationCount only works on method level, not on class, so annotating your class with #Test(invocationCount = 3) doesn't work.
As you're mentioning Selenium, my guess is you are trying to automate some repeated actions on a webpage. If so, then I think ideologically your best bet is to extract code from these three methods and just write another test that calls those internals, like this:
#Test(priority = 1)
public void method1() {
stuff1();
}
#Test(priority = 2)
public void method2() {
stuff2();
}
#Test(priority = 3)
public void method3() {
stuff3();
}
#Test
public void complexTest() {
for (int i = 0; i < 3; i++) {
stuff1();
stuff2();
stuff3();
}
}
private void stuff1() {
System.out.print("method1");
}
private void stuff2() {
System.out.print("method2");
}
private void stuff3() {
System.out.print("method3");
}
It's a good practice to treat each test as atomic test that can either fail or pass, and if you want to test some scenario that does particular set of actions three times, better introduce new test for this and make it clearly and explicitly "tell a story" of test.
Currently I have a strange behavior of my test.
For my test I'm using Groovy and JUnit 4. Groovy has his own strang logic, so don't worry about my code :D
Example 1:
#Log
class Test {
private static def filledFormFields = [
a: 1,
b: 2
]
#Before
void check() {
filledFormFields.each { k, v ->
log.info("$k -> $v")
}
}
#Test
void testA() {
System.exit(0)
}
}
log.info prints all information of filledFormFields (even it is static, yes it has to be static in my case).
Example 2:
#Log
class Test extends Manager {
private static def filledFormFields = [
a: 1,
b: 2
]
#Test
void testA() {
System.exit(0)
}
}
#Log
abstract class Manager {
#Before
void check() {
filledFormFields.each { k, v ->
log.info("$k -> $v")
}
}
}
log.info prints nothing, but why?
Why is my filledFormFields static?
Currently I'm creating Selenium/Geb JUnit Tests with Groovy. Each test instantiates the filledFormFields variable new. In my case I don't want that, because of that it's static. Also it's private because I have multiple test classes that useses this variable just with other equally other information. That was the only solution that I found to keep the varibales only in a class and not instantiated on every test.
I'm creating TestNG.xml programatically and running tests in parallel.
The issue is:
- I need to run Test2 after Test1
I tried Using 'dependsOnGroup' by assigning Test1 to a group and then asking Test2 to dependOn Test1's group.
But when I run the test suite, only Test1 will get Executed, Test2 will be skipped.
No errors reported.
#Test(groups = {"FirstTest"})
public class Test1 {
public void hello(){
syso("Test1");
}
}
#Test(groups = {"SecondTest"}, dependsOnGroups= {"FirstTest"}, alwaysRun=true)
public class Test2 {
public void hi(){
syso("Test2");
}
}
I'm using TestNG.6.9.6.jar
Adding priority would do what you need. #Test(priority=1). The lower priority would execute first.
#Test(priority=1)
public class Test1 {
public void hello(){
syso("Test1");
}
}
#Test(priority=2)
public class Test2 {
public void hi(){
syso("Test2");
}
}
It will first run Test1 and then Test2. So whichever classes you put in your test suite, it would consider the priorities of all the test functions in all it's classes.
Should do the needful for you in a lesser complicated way.
I hope it helps. :)
You can also use the dependsOnMethods() method instead of dependsOnGroup().
I am running some JUnit tests programatically with JUnitCore, and I want to get some data out of the test class once it is finished (so #AfterClass). Here is a pseudocode example of the constraints I am working under:
public class A {
public static String testData;
public static void runTest() {
JUnitCore juc = new JUnitCore();
juc.run(B);
// This is where I would like to access testData for this
// particular run
}
public static void setTestData(String s) {
testData = s;
}
}
public class B {
// Some #Test methods and stuff omitted
#AfterClass
public static void done(String s) {
A.setTestData(someData);
}
}
My problem is that different threads might be calling runTest(), so testData might be wrong. How do I work around this? I'm so lost.
If you really need/want to go with this design, you can make testData a java.lang.ThreadLocal<String>. This will solve the multi-threading issue.