Develop Reference

Unable to run a cucumber feature within IntelliJ

I have a feature with a number of steps but the first one is the only working and the rest are working fine. Now when I add new steps to the same feature there not working either its very confusing. Its lis like half the feature is working but the other half is no working.
Please see test runner below:
import io.cucumber.junit.Cucumber;
import io.cucumber.junit.CucumberOptions;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
#CucumberOptions(features = "src/test/java/features", glue = {"stepDefinitions"})
public class TestRunner {
}
Folder structure:
Error message:
You can implement this step using the snippet(s) below:
#Given("Add Place Payload with {string} {string} {string}")
public void add_place_payload_with(String string, String string2, String string3) {
// Write code here that turns the phrase above into concrete actions
throw new io.cucumber.java.PendingException();
}
I am also getting a red exclamation mark on the feature:

Modify CucumberOptions tags when running Cucumber in Java

I was wondering if there was a way to modify CucumberOptions tags while Cucumber is running?
I'm not sure if this is possible or not but I was wondering if there was a way to modify tags while Cucumber is running. In my example code, I would like to add another tag "#Login" once Cucumber runs. I am trying to setup a configuration where I can select which feature I want to run without going into the Runner class.
Settings Class
String AddTags = "#Login";
set = new HashMap<String, String>(){
{put("Tags", AddTags);
Runner
import org.junit.runner.RunWith;
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
#RunWith(Cucumber.class)
#CucumberOptions (features="src/test/cucumber/features",
tags = "#Smoke", //For instance, once cucumber runs I want to add
tag "#Login". //So something like adding Settings.set.get("Tags");
plugin = {"pretty", "html:target/cucumber-
htmlreport","json:target/cucumber-report.json"}
)
public class Runner {
}
Not sure if this is possible with Cucumber but wanted to ask.

You can use Tag expressions to combine several tags, for example:
**Expression Description**
#fast Scenarios tagged with #fast
#wip and not #slow Scenarios tagged with #wip that aren’t also tagged with #slow
#smoke and #fast Scenarios tagged with both #smoke and #fast
#gui or #database Scenarios tagged with either #gui or #database

What about using a list of tags?
tags = "#Smoke,#Login"

Convert from java to TestNG

Thought that it will be an easy, but appears to be complicated (for me). Could someone, please, help to find correct solution for following:
Have 200 tests inside one package selenium.tests.postTip which looks like this one below:
package selenium.tests.postTip;
import java.util.List;
import selenium.BaseTest;
import selenium.DriverBase;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.testng.annotations.Test;
public class PostBadmintonOddEven extends DriverBase {
#Test
public void main() throws Exception {
WebDriver driver = BaseTest.login();
Thread.sleep(2000);
List<WebElement> allSports = driver.findElements(By.xpath("//ul[#class='be-sportpicker__list']//div[#class='be-sport-thumb__title be-typo-be']"));
driver.findElement(By.xpath("/html/body/be-root/main//be-scroller//be-sportpicker//ul[#class='be-sportpicker__list']/li/a[contains(#href,\"/badminton\")]")).click();
Thread.sleep(2000);
BaseTest.betTypeOddEvenFullEvent(driver);
}
}
Since that I have to run those tests as a single test (which already works perfectly) and as a Suite, my question is:
How can I achieve to create (I tried with one xml file without any success) TestNG suite with all tests in it. Tried to run all of them, but only first can run, rest of them not. What am I doing wrong?
Note: BaseTest method which I am calling on last line, doing data check and finish when loop (for loop) finishes.
I tried with following in above test without success to start second test (aomost same as shown above):
#AfterTest
driver.quit();
Thank you in advance.

What you are doing wrong is naming your test class and methods incorrectly. Rename your test class so that it should have Test label either in prefix or suffix. Also you have annotated main method with #Test annotation. Main method is defacto entry point from where JVM starts. Use custom names for methods , that describes the test method.

Instead of #AfterTest method, you can check with #AfterMethod
If that's not solve,
What do you use to get all tests, in TestNG ? #DataProvider with dynamic Array or Excel !

NullPointerException when using multiple stepdefinitions in Cucumber-jvm

I am currently building a framework to test a Rest-API endpoint. As I am planning to write a lot of test cases, I decided to organize the project to allow me to reuse common Step Definition methods.
The structure is as follows;
FunctionalTest
com.example.steps
-- AbstractEndpointSteps.java
-- SimpleSearchSteps.java
com.example.stepdefinitions
-- CommonStepDefinition.java
-- SimpleSearchStepDefinition.java`
However when I try to call SimpleSearchSteps.java methods I get a NullPointerException
CommonStepDefinition Code
package com.example.functionaltest.features.stepdefinitions;
import net.thucydides.core.annotations.Steps;
import com.example.functionaltest.steps.AbstractEndpointSteps;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
public class CommonStepDefinition {
#Steps
private AbstractEndpointSteps endpointSteps;
#Given("^a base uri \"([^\"]*)\" and base path \"([^\"]*)\"$")
public void aBaseUriAndBasePath(String baseURI, String basePath) {
endpointSteps.givenBasepath(baseURI, basePath);
}
#When("^country is \"([^\"]*)\"$")
public void countryIs(String country)
{
endpointSteps.whenCountry(country);
}
#Then("^the status code is (\\d+)$")
public void theStatusCodeIs(int statusCode) {
endpointSteps.executeRequest();
endpointSteps.thenTheStatusCodeIs200(statusCode);
}
}
SimpleSearchStepDefinition.java
package com.example.functionaltest.features.stepdefinitions;
import net.thucydides.core.annotations.Steps;
import com.example.functionaltest.steps.EndpointSteps;
import cucumber.api.java.en.When;
public class SimpleSearchStepDefinition {
#Steps
private SimpleSearchSteps searchSteps;
#When("^what is \"([^\"]*)\"$")
public void whatIs(String what) {
searchSteps.whenWhatIsGiven(what);
}
}

Looks like you are missing holder class for Cucumber annotation, something like this you should have so that cucumber knows and identified that steps and features of yours:
#RunWith(Cucumber.class)
#CucumberOptions(
glue = {"com.example.functionaltest.features.steps"},
features = {"classpath:functionaltest/features"}
)
public class FunctionalTest {
}
Note that, in your src/test/resources you should have functionaltest/features folder with your .feature files according to this sample, you can ofc, change it by your design

Can you take a look at Karate it is exactly what you are trying to build ! Since you are used to Cucumber, here are a few things that Karate provides as enhancements (being based on Cucumber-JVM)
built-in step-definitions, no need to write Java code
re-use *.feature files and call them from other scripts
dynamic data-driven testing
parallel-execution of tests
ability to run some routines only once per feature
Disclaimer: I am the dev.

I solved this issue by using a static instance of RequestSpecBuilder in the AbstractEndpointSteps instead of RequestSpecification.
Therefore, I was able to avoid duplication of StepDefinitions and NPE issues altogether

Ordering unit tests in Eclipse's JUnit view

The JUnit view in Eclipse seems to order the tests randomly. How can I order them by class name?

As Gary said in the comments:
it would be nice if Unit Runner could
be told to go ahead and order them by
class name. Hmm, maybe I should look
into the source code...
I did look but there's no hint of a functionality to sort these names. I would suggest a change request to the JUnit plugin, but I don't think, that there are lot of people using this thing, so: DIY.
I would like to see the solution if you modify the plugin code.

One thing that one might do is using the schema of JUnit 3.x. We used a test suite that was called AllTests where you add the tests to it in a specific order. And for every package we got another AllTests. Giving those test suites a name being the same as the package enables one to easily build a hierarchy that should be valued by the junit plugin.
I really dislike how it is even presenting the test methods inside the Junit viewer. It should be in the very same order as they are specified in the TestCase class. I order those methods in the way of importance and features. So the upmost failing method is to correct first and then the more special one in the later part of the test case.
That is really annoying that the test runner is scrambling those. I will take a look at it myself and if I find a solution I will update this answer.
Update:
My problem with the ordering of method names within a TestCase is related to this one:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7023180 (Thanks Oracle!).
So in the end oracle changed the ordering of the methods within a class.getMethods or class.getDeclaredMethods call. Now the methods are random and can change between different runs of the JVM. It seams to be related to optimizations of compare or even is an attempt to compress method name - who knows... .
So whats left. First one can use: #FixMethodOrder (from javacodegeeks.com):
#FixMethodOrder(MethodSorters.DEFAULT) – deterministic order based on an internal comparator
#FixMethodOrder(MethodSorters.NAME_ASCENDING) – ascending order of method names
#FixMethodOrder(MethodSorters.JVM) – pre 4.11 way of depending on reflection based order
Well that is stupid but explains why people start using test1TestName schema.
Update2:
I use ASM since Javassist also produces random sorted methods on getMethods(). They use Maps internally. With ASM I just use a Visitor.
package org.junit.runners.model;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;
import com.flirtbox.ioc.OrderTest;
/**
* #author Martin Kersten
*/
public class TestClassUtil {
public static class MyClassVisitor extends ClassVisitor {
private final List<String> names;
public MyClassVisitor(List<String> names) {
super(Opcodes.ASM4);
this.names = names;
}
#Override
public MethodVisitor visitMethod(int access, String name, String desc,
String signature, String[] exceptions) {
names.add(name);
return super.visitMethod(access, name, desc, signature, exceptions);
}
}
private static List<String> getMethodNamesInCorrectOrder(Class<?> clazz) throws IOException {
InputStream in = OrderTest.class.getResourceAsStream("/" + clazz.getName().replace('.', '/') + ".class");
ClassReader classReader=new ClassReader(in);
List<String> methodNames = new ArrayList<>();
classReader.accept(new MyClassVisitor(methodNames), 0);
return methodNames;
}
public static void sort(Class<?> fClass, List<FrameworkMethod> list) {
try {
final List<String> names = getMethodNamesInCorrectOrder(fClass);
Collections.sort(list, new Comparator<FrameworkMethod>() {
#Override
public int compare(FrameworkMethod methodA, FrameworkMethod methodB) {
int indexA = names.indexOf(methodA.getName());
int indexB = names.indexOf(methodB.getName());
if(indexA == -1)
indexA = names.size();
if(indexB == -1)
indexB = names.size();
return indexA - indexB;
}
});
} catch (IOException e) {
throw new RuntimeException("Could not optain the method names of " + fClass.getName() + " in correct order", e);
}
}
}
Just put this in your src/test/java folder in the package org.junit.runners.model. Now copy the org.junit.runners.model.TestClass of the junit 4.5 lib to the same package and alter its constructor by adding the sorting routine.
public TestClass(Class<?> klass) {
fClass= klass;
if (klass != null && klass.getConstructors().length > 1)
throw new IllegalArgumentException(
"Test class can only have one constructor");
for (Class<?> eachClass : getSuperClasses(fClass))
for (Method eachMethod : eachClass.getDeclaredMethods())
addToAnnotationLists(new FrameworkMethod(eachMethod));
//New Part
for(List<FrameworkMethod> list : fMethodsForAnnotations.values()) {
TestClassUtil.sort(fClass, list);
}
//Remove once you have verified the class is really picked up
System.out.println("New TestClass for " + klass.getName());
}
Here you go. Now you have nicely sorted methods in the order they are declared within the java file. If you wonder the class path is usually set that way that everything in your src (target or bin) folder is considered first by the classloader. So while defining the very same package and the same class you can 'override' every class / interface in any library you use. Thats the trick!
Update3
I was able to get a tree view of every package and every class in the right order to.
The idea is to subclass ParentRunner and then add all classes to it that you identify as being public and having methods annotated with test.
Add a getName() method returning only the package name of the class your suite runner is representing (so you see the tree as a package tree without the suite's class name).
Inspect subdirectories if you find a certain suite class (I use AllTests for all suite classes).
If you do not find a suite class in a subdirectory check all of its subdirectories, this way you dont miss a package containing tests if the parent directory is not containing a suite.
That was it. The suite class I add everywhere is:
#RunWith(MySuiteRunner.class)
public class AllTests {
}
That's it. It should you give enough to start and extend on this one. The suite runner is only using reflection but I sort the test classes and suits of the subdirectories alphabetically and suits of subdirectories (which represent the packages they are in) are sorted upmost.

If you really need hard dependency between your JUnit test, try JExample extension
JExample introduces producer-consumer relationships to unit-testing.
A producer is a test method that yields its unit under test as return value.
A consumer is a test method that depends on one or more producers and their return values.
You can install it in Eclipse, for Junit4.4 or 4.5.
import jexample.Depends;
#Test
#Depends("#testEmpty")
public Stack<Integer> testPush(Stack<Integer> $) {
$.push(42);
assertFalse($.isEmpty());
return $;
}
As mentioned in this IBM article "In pursuit of code quality: JUnit 4 vs. TestNG":
One thing the JUnit framework tries to achieve is test isolation.
On the downside, this makes it very difficult to specify an order for test-case execution, which is essential to any kind of dependent testing.
Developers have used different techniques to get around this, like specifying test cases in alphabetical order or relying heavily on fixtures (#Before #After) to properly set things up.
These workarounds are fine for tests that succeed, but for tests that fail, they have an inconvenient consequence: every subsequent dependent test also fails. In some situations, this can lead to large test suites reporting unnecessary failures
So beware: if you retain any solution for ordering your JUnit tests the way you want... you need to think if that solution support a "skip" feature in order to allow other tests to proceed even if one of them fails.

mark wrote:
it orders them base on execution time,
maybe you should sort your methods?
source/sort members
mark is right. But you cannot sort your unit test. It's not allowed to speculate about the order of execution.
Unit tests have to be built independently and it's random, how they are called by the UnitRunner.
In most cases, the test methods are sorted alphabetically. The classes are random. Try to use a TestSuite to order your tests.

Ordering tests in JUnit view has been filed as bug #386453 in Eclipse Bugzilla. Commenting and/or voting there may help to get more visibility to this problem.

11 years later, the JUnit view does have name ordering, and just got "Execution time" ordering as well.
See Eclipse 4.17 (2020-09)
Sort test results by execution time
JUnit view now provides the ability to sort results by execution time.
By default, results will be sorted by execution order.
Choosing Sort By > Execution Time from the JUnit View menu will reorder the results once all tests are complete.
While tests are still running, they will be shown in execution order.
Sorting by execution order results in:

I was also searching for a solution for this, and I found a kind of crack from the below URL. I don't know whether it works for you or not, but it worked for me in Spring Tool Suite 2.5.2.
http://osdir.com/ml/java.junit.user/2002-10/msg00077.html