How to log the #Test method details in testreport using testNG - java

I have a class and It contains one #Test method(sampleTest1). In that method I have 4 #Test method calls from some other class. When I'm running my test cases through TestNG It logs only the main test method in output console and said that 1 test method passed.
How to log all the 4 test methods which defined in sub-class file?
Please can someone help me out.
Thanks,
Sasi

Testng logs only the #Test method it calls himself. But if you are calling by creating objects of the testclass. it it wont log that as a separate test.
So what you can do is you can let testng call them and maintain the flow of execution by using dependsongroups and dependsonmethods. But beware once you use depends on groups testng will not preserve your test class order anymore.

You cannot have parent child relationship between the test cases for TestNG, if you try to call them from parent test they will execute but they wont be part of the report and you may not have any clue if they fail.
Annotation Transformers Approach
If you want to run a method based on some condition then you might have to look at Annotation Transformers
You need to create a class by implementing the IAnnotationTransformer interface. Check for your condition in transform method set the Enable property to true or false to disable the #Test testcase.
public class MyTransformer implements IAnnotationTransformer {
private MyParentClass parent;
public MyTransformer(MyParentClass parent){
this.parent = parent;
}
public void transform(ITest annotation, Class testClass,
Constructor testConstructor, Method testMethod)
{
if (checkConditionForMethod(testMethod.getName())) {
annotation.setEnabled(false);
}
}
public boolean checkConditionForMethod(String methodName){
return parent.isValidSwitchForMethod(methodName);
}
}
You add the Annotation Transformer programmatically:
TestNG tng = new TestNG();
tng.setAnnotationTransformer(new MyTransformer(parentClassInstance));
Inner classes approach
Other way for nested test cases is creating inner class for your parent class and then define the test cases within the inner class but without any condition.
For example if you below class structure:
TestClassParent [Testcase1Parent] [Testcase2Parent]
|_ TestSubClass1 [Testcase1Sub1] [Testcase1Sub1]
|_ TestSubClass2 [Testcase1Sub2] [Testcase2Sub2]
Then the order of execution would like this: and you need to define your test cases according to the order to execution to achieve your goal.
Testcase1Sub1
Testcase2Sub1
Testcase1Sub2
Testcase2Sub2
Testcase1Parent
Testcase2Parent

Related

What use is #TestInstance annotation in JUnit 5?

Can you give a simple explanation of #TestInstance annotation and how it is useful in JUnit 5?
I think we can achieve the same effect probably by making our fields static.
I think the docs provide a useful summary:
If you would prefer that JUnit Jupiter execute all test methods on the same test instance, simply annotate your test class with #TestInstance(Lifecycle.PER_CLASS). When using this mode, a new test instance will be created once per test class. Thus, if your test methods rely on state stored in instance variables, you may need to reset that state in #BeforeEach or #AfterEach methods.
The "per-class" mode has some additional benefits over the default "per-method" mode. Specifically, with the "per-class" mode it becomes possible to declare #BeforeAll and #AfterAll on non-static methods as well as on interface default methods. The "per-class" mode therefore also makes it possible to use #BeforeAll and #AfterAll methods in #Nested test classes.
But you've probably read that already and you are correct in thinking that making a field static will have the same effect as declaring the field as an instance variable and using #TestInstance(Lifecycle.PER_CLASS).
So, perhaps the answer to the question "how it could be useful in JUnit 5" is that using a #TestInstance ...
Is explicit about your intentions. It could be assumed that use of the static keyword was accidental whereas use of #TestInstance is less likely to be accidental or a result of thoughless copy-n-paste.
Delegates the responsibility for managing scope and lifecycle and clean up to the framework rather than having to remember to manage that yourself.
This annotation was introduced to reduce the number of objects created when running your unit tests.
Adding #TestInstance(TestInstance.Lifecycle.PER_CLASS) to your test class will avoid that a new instance of your class is created for every test in the class.
This is particulary usefull when you have a lot of tests in the same test class and the instantiation of this class is expensive.
This annotation should be used with caution. All unit tests should be isolated and independent of eachother. If one of the tests changes the state od the test class then you should not use this feature.
Making your fields static to achieve the same effect is not a good idea. It will indeed reduce the number of objects created but they cannot be cleaned up when all tests in the test class are executed. This can cause problems when you have a giant test suite.
#TestInstance is used to configure the lifecycle of test instances for the annotated test class or test interface:
PER_CLASS: A new test instance will be created once per test class.
PER_METHOD: A new test instance will be created for each test method, test factory method, or test template method. This mode is analogous to the behavior found in JUnit versions 1 through 4.
If #TestInstance is not explicitly declared on a test class or on a test interface implemented by a test class, the lifecycle mode will implicitly default to PER_METHOD.
Setting the test instance lifecycle mode to PER_CLASS enables the following features:
Shared test instance state between test methods in a given test class as well as between non-static #BeforeAll and #AfterAll methods in the test class.
Declaration of #BeforeAll and #AfterAll methods in #Nested test classes.
Declaration of #BeforeAll and #AfterAll on interface default methods.
Simplified declaration of #BeforeAll and #AfterAll methods in test classes implemented with the Kotlin programming language.
See the test instance lifecycle documentation for further details.
since no one provide a proper coding example, I would like to give a simple code sample as below to understand the concept,
Per Method Sample - Default Option in Junit5
Note two methods are static, otherwise it will fire an exception because class instantiate in each method.
#TestInstance(Lifecycle.PER_METHOD)
public class MathUtilTestPerMethod {
MathUtil util;
#BeforeAll
static void beforeAllInit() {
System.out.println("running before all");
}
#AfterAll
static void afterAllCleanUp() {
System.out.println("running after all");
}
#BeforeEach
void init() {
util = new MathUtil();
System.out.println("running before each...");
}
#AfterEach
void cleanUp() {
System.out.println("running after each...");
}
#Test
void testSum() {
assertEquals(2, util.addtwoNumbers(1, 1));
}
}
Per Class Sample
Note that static is removed from the two methods and MathUtil object is created globally not in a method, because class instantiate only once.
#TestInstance(Lifecycle.PER_CLASS)
public class MathUtilTestPerClass {
MathUtil util = new MathUtil();
#BeforeAll
void beforeAllInit() {
System.out.println("running before all");
}
#AfterAll
void afterAllCleanUp() {
System.out.println("running after all");
}
#BeforeEach
void init() {
System.out.println("running before each...");
}
#AfterEach
void cleanUp() {
System.out.println("running after each...");
}
#Test
void testSum() {
assertEquals(2, util.addtwoNumbers(1, 1));
}
}
This is also useful when writing tests in Kotlin (because it doesn't have static methods).
So, instead of using a companion object with #JvmStatic funs in it for #BeforeAll or #AfterAll, make the lifecycle PER_CLASS and annotate regular methods with #BeforeAll or #AfterAll:
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
class MyTest {
#BeforeAll
fun setup() {
println("I am invoked only once")
}
}
When using this approach, be careful to reset your instance variables in #BeforeEach or #AfterEach funs if necessary.
Thanks to this article for its help.

What is the equivalent of TestName rule in JUnit 5?

How can I get name of the test method in JUnit 5?
Declare a parameter of type TestInfo in your test method and JUnit will automatically supply an instance of that for the method:
#Test
void getTestInfo(TestInfo testInfo) { // Automatically injected
System.out.println(testInfo.getDisplayName());
System.out.println(testInfo.getTestMethod());
System.out.println(testInfo.getTestClass());
System.out.println(testInfo.getTags());
}
You can get test method name (and more) from the TestInfo instance as shown above.
In addition to what is written about injecting TestInfo to test method it is also possible to inject TestInfo to methods annotated with #BeforeEach and #AfterEach which might be useful sometimes:
#BeforeEach
void setUp(TestInfo testInfo) {
log.info(String.format("test started: %s", testInfo.getDisplayName());
}
#AfterEach
void tearDown(TestInfo testInfo) {
log.info(String.format("test finished: %s", testInfo.getDisplayName());
}
An alternative for having the test name globally available as was possible in JUnit 4 is to shim the functionality yourself in a setup method using the TestInfo interface.
From the JUnit documentation on "Dependency Injection for Constructors and Methods":
The TestInfo can then be used to retrieve information about the current container or test such as the display name, the test class, the test method, and associated tags.
Here we leverage the fact that the built-in resolvers will supply an instance of TestInfo corresponding to the current container or test as the value for parameters of type TestInfo to methods annotated as lifecycle hooks (here we use #BeforeEach).
import org.junit.jupiter.api.TestInfo;
public class MyTestClass {
String displayName;
#BeforeEach
void setUp(TestInfo testInfo) {
displayName = testInfo.getDisplayName();
// ... the rest of your setup
}
}
This for example enables you to reference the current test name in other non-test methods (such as various utility methods) without having to include the test name as a parameter to each function call from the initial test method to that utility method.
You can do the same for other information about the current container or test.
Seems like the only disadvantages are:
the instance variable cannot be made final, as it is set dynamically
may pollute your setup code
For reference, here is how the TestName-Rule might be implemented in JUnit 4:
public class MyTestClass {
#Rule
public final TestName name = new TestName();
}

How to use bean in static method which is called by #BeforeClass test method

Let's suppose I have a test class A which extends from class B. This class A has one method with #BeforeClass annotation:
#BeforeClass
public static void setUp(){
createFakeData();
}
The method createFakeData() is in class B and its function is to create an object in database.
In order to do that, I have a bean in class B:
#Autowired
private DummyObjectsFactory dummyObjectsFactory;
And the content of the method createFakeData() could be something like that which returns a FakeData object:
public FakeData createFakeData() throws Exception
{
return dummyObjectsFactory.createFakeData();
}
The problem I'm facing is that the #BeforeClass method has to be static, that means that the createFakeData method has to be static too. But I cannot set that method to static because my bean dummyObjectsFactory will be always null.
How can I make my createFakeData method static avoiding my dummyObjectsFactory bean to be not null?
As far as I know, that is not permitted in JUnit. However you can do some things for arrange it.
First, you can use TestNG, which allows to do exactly what you want.
If that is not an option, instead of using #BefloreClass annotation, you can use #Before. The difference between both is that #BeforeClass executes the method before all the tests while #Before executes the method before each test. If you use #Before, I would annotate the class with #Transactional in order to delete the data saved in the database.
In the end, if you don't want to execute the method for each test, you can use #Before and flat the call:
#Before
public void init(){
if(!fakeDataCalled){
createFakeData();
fakeDataCalled=true;
}
}
I believe you want to annotate your test class (B) with #RunWith(SpringRunner.class) so that the autowiring happens correctly.
Take a look at section 15.4.4 in the Spring Reference.
You can also just browse to that page and search for "RunWith"
Caveats:
I don't like static methods. They tend to make unit testing difficult.
I don't like extending tests to make other tests. I find it easier to make each test standalone and use composition for any shared functionality (i.e. make utility classes for shared test functionality that reside in the test source tree).
I don't like my unit tests depending on autowiring. I mock all the dependencies and inject them with either reflection or the #InjectMocks annotation.

Defining order of execution of #BeforeGroups methods for a test belonging to same group in Java

I have a test method belonging to multiple groups, I want to ensure that one #BeforeGroups method (setupHomePage) runs before another (setupUniversalSearchPage). The order must be defined within java only as I am supposed to use any testng xml.
Although in similar case in another test class the order of execution is as desired.
public abstract class HomePageTest extends TestBase
{
#BeforeGroups("homePageTests")
public void setupHomePage()
{
loadHomePage();
}
}
public abstract class UniversalSearchPageTest extends HomePageTest
{
#BeforeGroups("universalPageTests")
public void setupUniversalSearchPage()
{
navigateToSearchPage();
}
}
public class UniversalSearchPageBasicTest extends UniversalPageTest
{
#Test(groups = {"homePageTests","universalPageTests")
public void searchVerificationTest()
{
//test code here
}
}
So far I have tried following :
1. reversing the order of groups in #Test annotation of searchVerificationTest
2. adding #Test(groups = "homePageTests") above declaration of UniversalSearchPageBasicTest class.
I would like to know how the order #BeforeGroups method is determined as it is different in one class than other.
If it is of any relation I am using maven in eclipse and Appium
I am new to testng , let me know if I am missing anything basic here
#BeforeMethod methods will run in inheritance order - the highest superclass first, then going down the inheritance chain. #AfterMethod methods run in reverse order (up the inheritance chain).
Note, however, that the ordering of multiple annotated methods within one class is not guaranteed (so it's best to avoid that).
Also there is a online documentation available, checkout http://testng.org/doc/documentation-main.html#annotations

How can I get all methods which are called by a testMethod? (Java, TestNG, IAnnotationTransformer)

I'm currently using IAnnotationTransformer to exclude all tests listed in an XML which contain a particular string in their name, for example:
#Override
#SuppressWarnings("rawtypes")
public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
if(testMethod.getName().contains("StringGoesHere")){
annotation.setEnabled(false);
}
This works perfectly well, however due to the naming convention of my tests, and the vast amount of tests which I need to exclude, the ideal solution would be to find a way to exclude the test if it calls upon a particular method. For example, I want to exclude all tests which call upon the method .clickUserSettings() shown below:
#Test
public void editUserDetailsOnStackOverflow() {
final StackOverflow so = stackOverflow.loginAsMe()
.clickUserSettings()
.changeUserEmail()
.clickSave()
.assertChangeSuccessful();
}
Does anyone know how I might be able to achieve this? I feel like I've explored all other avenues (classes, packages, testMethods, groups, xml, etc) And the reason it has to be done in this way is because I currently have 40,000 tests (15,000 of which will likely be excluded depending on the browserMode variable set by the tester) This is the general idea I was hoping to achieve:
#Override
#SuppressWarnings("rawtypes")
public void transform(ITestAnnotation annotation, Class testClass, Constructor testConstructor, Method testMethod) {
if(testMethod.getAllCalledMethods.contains(".clickUserSettings()")){
annotation.setEnabled(false);
}
If this can be done it will solve my issue,
Cheers
This is not possible with TestNG. TestNG has visibility only to the methods that are annotated with its annotations and its not aware of the code path that a TestNG method traverses during its course of execution. [ Here I am assuming that the method clickUserSettings() is a utility method that your #Test methods perhaps call into, as part of its execution ]
So you have a couple of options
Resort to using groups wherein all methods that call into clickUserSettings() as part of a particular group and then exclude that group from execution.
Create your own custom annotation, annotate all methods that invoke clickUserSettings() using this annotation, and then modify your current AnnotationTransformer to look for this annotation at either the method level or at the class level.

Categories

Resources