Develop Reference

Gradle + TestNG: How to list which classes were touched by a unit test?

I'd like to list what classes are touched during the execution of each unit test. I want to discover which tests have an overly large scope and should use a smaller unit instead. Measuring the code coverage via IntelliJ or JaCoCo doesn't help me as I cannot drill down to a single test. Has anyone managed to do something similar? I found a similar question which was asked ten years ago, but never answered. I currently use Java 8 with Gradle and TestNG. I feel like I have to build some Gradle plugin or modify TestNG in some way, but I have no idea where to start.
Any help is greatly appreciated.

OK, I've discovered that Openclover can do this. I've setup a dummy maven project and when I execute the tests via this command chain:
mvn clean clover:setup test clover:aggregate clover:clover
Then clover generates a report in the "target\site\clover" subdirectory of my project. In that directory is an index.html which you can use to browse through the results. You need to navigate to a test and to a certain test method to see the breakdown. It looks like this:
We can see that the method testCalculateBalance of BalanceCalculatorTest covers BalanceCalculator, PriorServiceBalanceProvider and TimeAccount, but we don't see TimeAccountProvider, because it is mocked in the test. I'll paste the code snippets, so you get the full picture. Bottom line is, it works, but it is not very user friendly.
public class BalanceCalculatorTest {
#Test
public void testCalculateBalance() {
TimeAccountProvider timeAccountProvider = Mockito.mock(TimeAccountProvider.class);
Mockito.doReturn(Collections.singleton(new TimeAccount())).when(timeAccountProvider).getTimeAccounts();
BalanceCalculator balanceCalculator = new BalanceCalculator(timeAccountProvider);
Assert.assertEquals(balanceCalculator.calculateBalance(), 2);
}
}
public class BalanceCalculator {
private final TimeAccountProvider timeAccountProvider;
public BalanceCalculator(TimeAccountProvider timeAccountProvider) {
this.timeAccountProvider = timeAccountProvider;
}
public int calculateBalance() {
Set<TimeAccount> timeAccounts = timeAccountProvider.getTimeAccounts();
int sum = 0;
for (TimeAccount timeAccount : timeAccounts) {
Set<TimeAccountDetail> bookings = timeAccount.getTimeAccountDetails();
sum += bookings.stream()
.mapToInt(TimeAccountDetail::getAmount)
.sum();
}
sum += new PriorServiceBalanceProvider().getPriorBalance();
return sum;
}
}

How to add multiple feature files to Cucumber Runner Class [duplicate]

This question already has answers here:
How to order feature files in Cucumber test suite?
(6 answers)
Closed 3 years ago.
I have 3 separate feature files and one CucumberRunnerClass. As per the sequence need to execute those are listed below:
Feature files : Login.feature, NavigateCusMngt.feature, AddCustomer.feature
However, when executing it goes to first execute the AddCustomer.feature, then Login.feature and finally
NavigateCusMngt.feature.
Therefore, I observed AddCustomer.feature - skipped, system logged in then NavigateCusMngt.feature -gives errors.
#CucumberOptions(
features = {"src/test/resources/features/Login.feature", "src/test/resources/features/NavigateCusMngt.feature", "src/test/resources/features/AddCustomer.feature"},
glue = {"phptravelstestcases"},
tags = {"~#Ignore"},
format = {
"pretty",
"html:target/cucumber-reports/cucumber-pretty/mercury-tours-RegisterUserTest",
"json:target/cucumber-reports/json-reports/mercury-tours-RegisterUserTest.json",
"rerun:target/cucumber-reports/rerun-reports/mercury-tours-RegisterUserTest.txt"
}
)
please give me a solution.

The feature files are parsed alphabetically. I named mine with a starting letter in the right order, e.g.
A-Login.feature
B-NavigateCusMngt.feature
C-AddCustomer.feature
It's not ideal in the long run, but it is a workable solution.

Reporting on multiple Selenium Webdriver job ran from Jenkins pipeline job

I have a pipeline job using Groovy script set up to run multiple tests in "parallel", but I am curious as to how to get the report(s) unified.
I am coding my Selenium tests in Java and using TestNG and Maven.
When I look at the report in target/surefire-reports, the only thing there is the "last" test ran of "suite".
How can I get a report that combines all of the tests within the Pipeline parallel job?
Example Groovy code:
node() {
try {
parallel 'exampleScripts':{
node(){
stage('ExampleScripts') {
def mvnHome
mvnHome = tool 'MAVEN_HOME'
env.JAVA_HOME = tool 'JDK-1.8'
bat(/"${mvnHome}\bin\mvn" -f "C:\workspace\Company\pom.xml" test -DsuiteXmlFile=ExampleScripts.xml -DenvironmentParam="$ENVIRONMENTPARAM" -DbrowserParam="$BROWSERPARAM" -DdebugParam="false"/)
} // end stage
} // end node
}, // end parallel
'exampleScripts2':{
node(){
stage('ExampleScripts2') {
def mvnHome
mvnHome = tool 'MAVEN_HOME'
env.JAVA_HOME = tool 'JDK-1.8'
bat(/"${mvnHome}\bin\mvn" -f "C:\workspace\Company\pom.xml" test -DsuiteXmlFile=ExampleScripts2.xml -DenvironmentParam="$ENVIRONMENTPARAM" -DbrowserParam="$BROWSERPARAM" -DdebugParam="false"/)
} // end stage
} // end node
step([$class: 'Publisher', reportFilenamePattern: 'C:/workspace/Company/target/surefire-reports/testng-results.xml'])
} // end parallel
There is a little more to this code after this in terms of emailing the test runner the result of the test and such.
This works great, other than the reporting aspect.

I prefer to use ExtentReports because it has a ExtentX server that allows to you report on multiple different test reports.
I used to use ReportNG but development on that stalled and so I don't recommend it any more. It doesn't allow you combine reports anyway.
Other than that, you could use CouchBase or similar JSON database to store test results and then generate your own report from that information.

How do I get the result of a Cucumber feature

I'm trying to run Cucumber features in JUnit 5 Jupiter. I've lifted some code from the Cucumber-jvm source and adapted it for JUnit 5's TestFactory. It is working: I see my features running when I run all JUnit tests (this is Kotlin code, but the same applies to Java):
#CucumberOptions(
plugin = arrayOf("pretty"),
features = arrayOf("classpath:features")
)
class Behaviours {
#TestFactory
fun loadCucumberTests() : Collection<DynamicTest> {
val options = RuntimeOptionsFactory(Behaviours::class.java).create()
val classLoader = Behaviours::class.java.classLoader
val resourceLoader = MultiLoader(classLoader)
val classFinder = ResourceLoaderClassFinder(resourceLoader, classLoader)
val runtime = Runtime(resourceLoader, classFinder, classLoader, options)
val cucumberFeatures = options.cucumberFeatures(resourceLoader)
return cucumberFeatures.map<CucumberFeature, DynamicTest> { feature ->
dynamicTest(feature.gherkinFeature.name) {
var reporter = options.reporter(classLoader)
feature.run(options.formatter(classLoader), reporter, runtime)
}
}
}
}
However, JUnit reports that every feature was successful, whether or not it actually was. When features fail, the results are correctly pretty-printed, but the generated DynamicTest passes. Neither gradle test nor Intellij notice the error: I have to inspect the text output.
I think I have to figure out, in the Executable passed as the second parameter to dynamicTest, what the result of the feature was, and raise an assertion when appropriate. How do I determine the result of feature or feature.gherkinFeature at that point?
And is there a way to get at the results for each scenario in the feature? Or better, is there a way to run a specific scenario, so that I can create a DynamicTest for each scenario, giving me better reporting granularity in JUnit?

To record the result of a Cucumber scenario as a JUnit5, I found it easiest to implement a JunitLambdaReporter, which is essentially a simpler version of the existing JunitReporter. Once you have a reporter that remembers what the current scenario is, then you can create a #TestFactory that uses this logic:
return dynamicTest(currentScenario.getName(), () -> {
featureElement.run(formatter, reporter, runtime);
Result result = reporter.getResult(currentScenario);
// If the scenario is skipped, then the test is aborted (neither passes nor fails).
Assumptions.assumeFalse(Result.SKIPPED == result);
Throwable error = result.getError();
if (error != null) {
throw error;
}
});

Java - having to test many inputs manually

I am using eclipse and need to test many files for my application. This mean, I have to go to: `run -> run configurations -> arguments', change them and re-run, for about 30 different test files.
Is there a quicker way to do this?
I have googled java automated testing. Just need some guidance, I am abit confused.
thanks
daniel

You should setup a Maven project or an ant build file to perform a suite of tests in one click rather than going one by one as you currently do.
Otherwise you can simply put all the tests you want to run in a specific package or folder then select : "Run all tests in the selected project, package or source folder" in the JUnit Run/debug configuration :
Another way with Eclipse is to create a test suite :
Open the New wizard
Select Java > JUnit > JUnit Test Suite and click Next.
Enter a name for your test suite class
Select the classes that should be included in the suite.

if it's just command line variations, you can sort it out adding a simple class like this (wrote this on the fly without a javac, may have errors)
public class PropertyRunner {
private static String commands [] = {"TEST_1", "TEST_2", "TEST_3" };
public static void main(String[] args) throws FileNotFoundException, IOException
{
Properties config = new Properties();
config.load(new FileInputStream("config.props"));
// config.props contains all my tests in the format:
// TEST_1=-a|-k|ccccc
// TEST_2=-b|-k|ccccd
// TEST_3=-c|-k|FEFEF
// now run test cases:
for (String key : commands) {
String cmdLine = config.getProperty(key);
// cmdLine is in format "-a|-b|ccccc"
String childArgs[] = cmdLine.split("\\|");
// Exec your main class passing args directly or via threads
// YourApp.main(childArgs);
}
System.exit(0);
}
}
It's pretty simple, you store all your command lines in a property file, and then iterate on every key executing your real main class and passing the arguments read from the property file.