There are a lot of questions on the execution order of TestNG tests, but I haven't found one that specifically answers this question - unless I just cannot find it.
I know there's things to do that can achieve what I'm looking for, such as:
#Test above every test method within the class with the use of Priority
#Test above every test method within the class with the use of Depends on
The use of preserve-order in the XML file
However, what I want to know is:
If you declare #Test at the class level, not the method level, how to "preserve" the order in which the methods are declared? Can you even do this?
For example, if my class looks like this:
#Test
public class mySampleClass
{
public void test()
{
//Run first - First method in the class
}
public void run()
{
//Run second - Second method in the class
}
public void execute()
{
//Run last - Last method in the class
}
}
Can I ensure that when I run this that the methods are executed in the order they are within the class instead of alphabetically?
Not sure why you want your tests to run that way, Even though it is not possible to do it directly, it IS possible using a MethodInterceptor:
public class MyInterceptor implements IMethodInterceptor {
// keep your list of method names in order
private static final List<String> order = List.of("test", "run", "execute");
#Override
public List<IMethodInstance> intercept(List<IMethodInstance> methods, ITestContext context) {
methods.sort(Comparator.comparing(m -> order.indexOf(m.getMethod().getMethodName())));
return methods;
}
}
Now add the #Listeners(value = MyInterceptor.class) annotation to your class.
NOTE: List.of is available only in java 9+, if you are using a lower version, you could use Arrays.asList.
Related
I have a question. I have multiple classes in a package: Let's say package is
com.myPackage.first
And this package has the following classes:
firstGood
secondGood
thirdBad
fourthGood
Each of these classes have a method with the same name but different implementation. So say each have a one particular function called:
public void runMe(){
}
For now I want to come up with a way to given a class name, it'll go inside the class and run that particular method.
So conceptually, my method will look like those:
ArrayList<Class> classList ; // where classList is a list of classes I want to run
public void execute(){
for(Class c : classList){
// Go inside that class, (maybe create an intance of that class) and run the method called run me
}
}
or
public void execute(Class c, String methodToRun){
for(Class c : classList){
// Go inside that class, (maybe create an intance of that class) and run the method called run me
}
}
For now. what I have been able to do is get the name of the classes I want to run the
runMe()
method. So I have been able to come with a way to get the arraylist of classes I want to run. So what I need help with is coming up with a method such that it takes a class name and run the method I want it to. Any help is appreciated. Thanks
I suggest having a look at Class.forName ( ... ) to get the class object, Class.newInstance(); if your classes have a default constructor (or Class.getDeclaredConstructor(...) otherwise) to create a new instance and then Class.getDeclaredMethod( ... ) to find the method and invoke it.
All of this without any regard if your idea is really a good one, since I really didn't quite understand WHY you want to do what you want to do...
interface Me {
void runMe();
}
Then let all classes implement Me.
And have a list of Mes
List<Class<Me>> ...
Then
void test(Class<Me> cl) {
Me me = cl.newInstance();
me.runMe();
}
My adage is always use reflection to solve a problem - now you have two problems. In view of that have you considered a simple pattern like this:
interface Runner {
public void runMe();
}
static abstract class BaseRunner implements Runner {
public BaseRunner() {
// Automagically register all runners in the RunThem class.
RunThem.runners.add(this);
}
}
class FirstGood extends BaseRunner implements Runner {
#Override
public void runMe() {
System.out.println(this.getClass().getSimpleName() + ":runMe");
}
}
class SecondGood extends BaseRunner implements Runner {
#Override
public void runMe() {
System.out.println(this.getClass().getSimpleName() + ":runMe");
}
}
static class RunThem {
static final Set<Runner> runners = new HashSet<>();
static void runThem() {
for (Runner r : runners) {
r.runMe();
}
}
}
public void test() {
Runner f = new FirstGood();
Runner s = new SecondGood();
RunThem.runThem();
}
Here all of your runMe objects extend a base class whose constructor installs the object in a Set held by the class that calls their runMe methods.
inline
void execute() throws Exception{
for (Class<?> c : classesList)
{
//If you don't already have an instance then you need one
//note if the method is static no need for any existing instance.
Object obj = Class.forName(c.getName());
// name of the method and list of arguments to pass
Method m = c.getDeclaredMethod(methodName,null);
//method accessibility check
if(!m.isAccessible())
m.setAccessible(true);
//invoke method if method with arguements then pass them as new Object[]{arg0...} instead of null
//if method is static then m.innvoke(null,null)
m.invoke(obj, null);
}
}
I would recommend using an Interface that defines the runMe() method and then have all your classes implement that interface. Then you would have a list of this Interface:
List<MyInterface> classes = new ArrayList<MyInterface>();
Then you could easily iterate over it and invoke "runMe()" on all of them or if you only want to invoke it for instances of a certain class you could do it like this:
public void execute(Class classForWhichToExecute) {
for (MyInterface myInterface : classes) {
if (classForWhichToExecute.isAssignableForm(myInterface)) {
myInterface.runMe();
}
}
}
Of course this wouldn't work if your method is a static method - so adding more information from your side would help.
I would suggest to use an interface with a common method to override in each class. So that any class can be casted to interface and use its method to execute the method.
interface GoodAndBad{
public void runMe();
}
Implemented class
class FirstGood implements GoodAndBad{
#override
public void runMe(){
// Code to be executed
}
}
You can use execute() method as follows
public void execute(List<GoodAndBad> classList){
for(GoodAndBad c : classList){
c.runMe();
// Go inside that class, (maybe create an intance of that class) and
// run the method called run me
}
}
Change the Class to GoodAndBad interface to change the other method too.
This is loosely coupling objects to support favor over composition in Java Object Oriented Design Patterns.
Never use Strings of method names to execute a method at anytime. There are plenty of other cool solutions for that using design patterns.
So, for example I have a few classes implementing the List<T> interface. How to test them - whether they implement the methods correctly?
Now I only see one way to do so:
public class MyListImplementationsTest {
private Collection<List<Integer>> listImplementations;
#BeforeClass
public static void setUp() throws Exception {
listImplementations = Arrays.asList(
new QuickList<Integer>(), new EfficientMemoryList<Integer>()
);
}
#Test
public void testIsEmptyAfterCreationEmptyList() {
// Use forEachList(handler) in order to not iterate
// the lists manually every time.
// May be there is no need to do so,
// because using <<for (item : items)>> instead of
// iterating using index prevents from OutOfBounds errors
forEachList(new OnEachListHandler<Integer>() {
#Override
public void onEach(List<Integer> list) {
assertTrue(list.isEmpty());
}
});
}
private <T> void forEachList(OnEachListHandler<T> handler) {
for (List<T> each : listImplementations) {
handler.onEach(each);
}
}
private static interface OnEachListHandler<T> {
void onEach(List<T> each);
}
}
But in my opinion it's complicated to iterate lists in every test.
Is there more elegant way to test classes implementing the same interface in JUnit4?
You can create a base test which can test anything of type List<T> plus an abstract method which creates such a list.
Then implement a test per list type which extends the base test. JUnit will run the test cases from the base class plus any that you define in the extension.
abstract class AbstractListTest<T> {
protected abstract List<T> createList();
#Test
public void testIsEmpty() {
List<T> list = createList();
assertTrue(list.isEmpty());
}
...more tests...
}
class QuickListTest extends AbstractListTest<QuickList> {
protected QuickList createList() {
return new QuickList();
}
}
JUnit won't run the abstract base class but it will see the inherited tests and run all of them. You can also add new tests to QuickListTest or override ones from the base class.
Basically, JUnit will take the class, find all public #Test methods from the whole inheritance tree and run them.
I will consider breaking up the tests for different list implementations into their respective test cases, so that they pass or fail independently.
Using your .isEmpty() as an example, if QuickList.isEmpty() and EfficientMemoryList.isEmpty() have different implementations i.e. different meaning for the concept of empty, then it makes sense for them to be tested independently. Currently, your testIsEmptyAfterCreationEmptyList will fail if 1 list implementation failed, but the others passed.
Otherwise, if QuickList.isEmpty() and EfficientMemoryList.isEmpty() share the same implementation, then you can consider moving the implementation to a common base class, and write tests for that base class.
Just because classes share the same interface, doesn't mean their tests need to be lumped and coupled.
Create a test each implementation separatelly: QuickListTest and EfficientMemoryListTest.
QuickListTest.java
public class QuickListTest extends ListBase {
#Test
public void shouldBeEmpty() throws Exception {
assertThatIsEmpty(new QuickList<Integer>());
}
}
BaseList.java
public abstract class ListBase {
protected void assertThatIsEmpty(QuickList<Integer> actual) {
assertThat(actual).isEmpty();
}
}
Let's say I have two implementations of a search algorithm that return the same result for the same input. They both implement the same interface.
How can I use a single [TestClass] for testing both implementations, rather then create two test files with eventually the same logic ?
Can I tell MSUnit to launch one of the tests twice with different constructor parameter?
Perhaps I should (n)inject it somehow ?
Use an abstract test class:
[TestClass]
public abstract class SearchTests
{
private ISearcher _searcherUnderTest;
[TestSetup]
public void Setup()
{
_searcherUnderTest = CreateSearcher();
}
protected abstract ISearcher CreateSearcher();
[TestMethod]
public void Test1(){/*do stuff to _searcherUnderTest*/ }
// more tests...
[TestClass]
public class CoolSearcherTests : SearcherTests
{
protected override ISearcher CreateSearcher()
{
return new CoolSearcher();
}
}
[TestClass]
public class LameSearcherTests : SearcherTests
{
protected override ISearcher CreateSearcher()
{
return new LameSearcher();
}
}
}
You've tagged your question with NUnit, but you ask about MSTest. What you are asking about can be achieved with parameterized test fixtures in NUnit. I am not familiar enough with MSTest to suggest an equivalent approach there, and a quick search indicates that MSTest may not have this feature.
In NUnit you parameterize the test fixture by applying multiple [TestFixture(...)] attributes to the fixture class with different parameters. These parameters will be passed to the fixture constructor.
Since there are limits on the types of parameter that can be passed, you'll probably need to pass a string in specifying the algorithm, then in the constructor assign the delegate or object that provides the search algorithm to a member field which is used in the tests.
For example:
using System;
using System.Collections.Generic;
using NUnit.Framework;
namespace MyTests
{
public static class SearchAlgorithms
{
public static int DefaultSearch(int target, IList<int> data)
{
return data.IndexOf(target);
}
public static int BrokenSearch(int target, IList<int> data)
{
return 789;
}
}
[TestFixture("forward")]
[TestFixture("broken")]
public class SearchTests
{
private Func<int, IList<int>, int> searchMethod;
public SearchTests(string algorithmName)
{
if (algorithmName == "forward")
{
this.searchMethod = SearchAlgorithms.DefaultSearch;
return;
}
if (algorithmName == "broken")
{
this.searchMethod = SearchAlgorithms.BrokenSearch;
}
}
[Test]
public void SearchFindsCorrectIndex()
{
Assert.AreEqual(
1, this.searchMethod(2, new List<int> { 1, 2, 3 }));
}
[Test]
public void SearchReturnsMinusOneWhenTargetNotPresent()
{
Assert.AreEqual(
-1, this.searchMethod(4, new List<int> { 1, 2, 3 }));
}
}
}
I'd rather have two different [TestMethod] in one [TestClass] each testing only one implementation: this way a failing test will always correctly point you which implementation went wrong.
If you are using NUnit you can pass through a variable declared in an attribute
http://www.nunit.org/index.php?p=testCase&r=2.5.6
if you use something like:
[TestCase(1)]
[TestCase(2)]
public void Test(int algorithm)
{
//..dostuff
}
if will run once for 1, once for 2, uses the same setup/teardown too :)
There isn't an equivalent in MSTest however you can fudge it somewhat as explained here:
Does MSTest have an equivalent to NUnit's TestCase?
I can't say I'm extremely happy with this approach, but here's what I ended up doing. I then went to look for a better approach and found this question. This approach meets the criteria, 1) I'm using MS Test, 2) I write the test logic only 1 time, 3) I can tell which implementation failed (and double clicking on the test will take me to the right test class).
This approach uses a base class to contain all the actual test logic, and then a derived class for each implementation (I have 3) that sets the specific implementation on the base interface and overrides the base test methods.
[TestClass]
public abstract class SearchTestBase
{
protected ISearcher Searcher { get; set; }
[TestMethod]
public virtual void Find_Results_Correct()
{
// Arrange (code here)
// Act (single line here)
var actual = Searcher.Results(input);
// Assert
}
}
(different file...)
[TestClass]
public class FastSearcherTest : SearcherTestBase
{
[TestInitialize]
public void TestInitialize()
{
Searcher = new FastSearcher();
}
[TestMethod]
public override void Find_Results_Correct()
{
base.Find_Results_Correct();
}
}
(different file...)
[TestClass]
public class ThoroughSearcherTest : SearcherTestBase
{
[TestInitialize]
public void TestInitialize()
{
Searcher = new ThoroughSearcher();
}
[TestMethod]
public override void Find_Results_Correct()
{
base.Find_Results_Correct();
}
}
So what I don't like about this approach is that every time I want to add a test I need to go to each of the test files and override the new test method. What I do like are the 3 requirements you had. If I need to change a test, I change the logic in just one place.
The advantage I see to this solution over the similar one of having a single method called by two tests is that I don't have to repeat the code for setting up the right implementation. In this solution you have a single line that calls the base.TestName(), and not two lines, one to set the Searcher and another to call the test. The Visual Studio also makes writing this much faster... I just type, "override" and get a list of choices. Auto complete writes the rest for me.
Clarifications based on my testing.
The accepted answer (to use an abstract class) works as long as the abstract class and concrete classes are in the same assembly.
If you desire to have the abstract class and concrete classes in different assemblies, the approach mentioned by KarlZ, unfortunately seems to be necessary. Not sure why this is the case. In this scenario, the TestExplorer will not show TestMethod.
Also, the accepted answer uses concrete classes nested within the abstract class. This does not appear to be a requirement.
Test with MSTestV2 (1.1.17), VS2017.
Here are sample classes used.
Assembly 1
[TestClass]
public abstract class SampleExternal
{
[TestMethod]
public void SampleTest01()
{
Assert.IsTrue(false, this.GetType().Name);
}
}
Assembly 2
[TestClass]
public abstract class Sample
{
[TestMethod]
public void SampleTest01()
{
Assert.IsTrue(false, this.GetType().Name);
}
[TestClass]
public class SampleA : Sample
{
}
}
[TestClass]
public class SampleB : Sample
{
}
[TestClass]
public class SampleC : SampleExternal
{
}
[TestClass]
public class SampleD : SampleExternal
{
}
using these, the test for SampleA and SampleB will execute (and fail by design), but SampleC & SampleD will not.
I was wondering if anyone had a pattern that would help me achieve the following:
We have a JPA entity called Employee and on it there is a setLineManager method. We also have a separate updateLineStructureService, which is a Spring-managed service bean. We want to try and ensure that this setLineManager method can only be called from updateLineStructureService and not directly from any other class.
Is there a way to allow the service access to this method without exposing it to any other classes? I am aware that I could give the method package level access and put the service in the same package as Employee, but that will not fit our package structure so I would prefer not to do that. I am also aware that I could make the method private and just access it through reflection in this one place, but I do not like that solution at all.
Any ideas?
You can inspect the stacktrace (using Throwable#getStackTrace()) and see if it contains the allowed method on specified position.
In the following code snippet, System.PrivateEmployee is not visible outside the System class. Thus effectively privateMethod is private and can only be called from within the System class. Since System.PrivateEmployee extends System.PublicEmployee it can be used outside the System class as System.PublicEmployee
public class System
{
public static interface PublicEmployee { void publicMethod ( ) ; }
private static interface PrivateEmployee extends PublicEmployee { void privateMethod ( ) ; }
}
Use an inner class only available to the other service class:
public class Employee
{
static {
LineStructureService.registerEmployeeHelper(new EmployeeHelper() {
#Override
public void setLineManager(Employee emp, Object foo) {
emp.setLineManager(foo);
}
});
}
public static void init() {}
private void setLineManager(Object foo) { }
}
public class LineStructureService
{
private static volatile EmployeeHelper _helper;
static {
// ensure that Employee class is loaded and helper registered
Employee.init();
}
public static synchronized void registerEmployeeHelper(EmployeeHelper helper) {
_helper = helper;
}
public void doSomething(Employee emp)
{
// now this class can call setLineManager on Employee
_helper.setLineManager(emp, blah);
}
public interface EmployeeHelper {
public void setLineManager(Employee emp, Object foo);
}
}
The only way that a class can access private methods of other classes is with inner classes. If that is not an option, this can't be done.
One approach is to make two forms of Employee.
"BasicEmployee" has all the methods except setLineManager(). "ExtendedEmployee" extends BasicEmployee and adds a public void setLineManager(). (I'm assuming these are classes, but they could also be interfaces instead) Underneath the hood, everything is really a FullEmployee (for clarity, you could make BasicEmployee abstract). But, in the code, in all the classes except UpdateLineStructureService, you declare it as a BasicEmployee. Only in UpdateLineStructureService is it declared as a FullEmployee. So, only UpdateLineStructureService has easy access to setLineManager()
Now, a rogue coder could always cast their BasicEmployee to an ExtendedEmployee to access setLineManager(), so this isn't totally secure. But it's a reasonable pattern to limit access.
You could use AOP (e.g. AspectJ or CDI) to intercept the call to setLineManager(); if the caller is updateLineStructureService() call the method; if not do nothing, or raise an exception or whatever.
I found static block is running only once when I execute multiple JUnit tests. How can I force it to run for each test method? I am using latest JUnit 4.8.2
Also, according to xUnit design principle, every method should be totally independent on others. Why static block only be executed once?
#Test TestMethod1 () {
Accounts ac = new Accounts();
ac.method1(); //kill the thread inside
}
#Test TestMethod2 () {
Accounts ac = new Accounts();
ac.method2(); // the thread is no longer available!!
}
class Accounts {
static {
// initalize one thread to monitor something
}
}
This even happens when TestMethod1 and TestMethod2 are in the different Test Classes.
static blocks are only executed on class loading because that is what they are: class initializers. To have a static block run multiple times would require you to unload the class (not an easy thing to do...).
If you need to use static blocks, you can come up with ways to test them. Why not unwrap the block into a public (static) method? All you have to do in that world is test the method:
static {
staticInitMethod();
}
public static void staticInitMethod(){
//insert initialization code here
}
you also might be able to get away with just an ordinary initializer
{//not static
//insert initialization code here
}
Although, the truth is most code doesn't need to use initializers like this at all.
Edit: Turns out Oracle likes the static method approach http://download.oracle.com/javase/tutorial/java/javaOO/initial.html
Why static block only be executed once?
Because that is the whole point of static initializer blocks!
Or to put it another way, if you want some initialization code to execute multiple times, put it in a regular constructor or method, or (in a tiny number of cases) a non-static initializer block.
In the context of JUnit, the normal way to implement test startup and shutdown code using setUp() and tearDown() methods.
If you are trying to unit test the execution of static initialization in your own code, you are in for a rough road I think. But then, unit testing of code with static state (e.g. singletons) is always difficult ... and that's one of the reasons that people think that static state is a bad idea.
Consider using a Dependency Injection (aka Inversion of Control) framework instead of singletons.
Alternatively, consider modifying your singletons / static initialization code to make it easier to test. For instance, add a static method that allows a test to re-execute the initialization. (And before you say that this breaks the singleton pattern: yes I know. You need to choose between design / implementation "purity" and ease of testing.)
Is the static code for the tests for the class being tested?
If the code is static so the tests can share, then you need to move the code into its own class. Then either have the test class constructor instantiate a static instance or create a test suite that does the same thing.
If you want each test to stand alone, then move what you are doing in your static block into the setup()/teardown() methods, it's what they are there for.
Static block is executed only once when first time class is loaded into JVM. Junit provide #Before annotation which is basically used for required initialization fro test case. This can be used for executing static blocks of class. for example I have following class Car
public class Car implements Vehicle{
private String type = "lmv";
static {
VehicleFactoryWithoutRefl.getInstance().registerVehicle("car", new Car());
}
#Override
public void moveForward() {
}
#Override
public String getType() {
return type;
}
#Override
public Vehicle createVehicle() {
return new Car();
}
}
and I want to execute static block of this class in Junit before creating the car instance. I have to load this class in setUp() using class.forName("package.ClassName") Junit code.
public class TestFactory {
#Before
public void setUp() {
try {
Class.forName("com.cg.dp.factory.Car");
} catch (ClassNotFoundException e) {
//e.printStackTrace();
}
}
#Test
//#Ignore
public void testFactoryInstanceWithoutRefl() {
Vehicle v1 = VehicleFactoryWithoutRefl.getInstance().newVehicle("car");
assertTrue(v1 instanceof Car);
}
}
Um... make it non-static? You can have instance initializer blocks too (same as static blocks, just without the static keyword). But test setup code should actually go into an explicit setUp() or #Before method.