We have a large number of complex integration tests that run for several hours.
How do I receive TestNG XML reports while the tests are running, not after the run?
You can build a TestNG listener which extends org.testng.TestListenerAdapter and override its org.testng.TestListenerAdapter#onFinish wherein you can build the logic to push the results of a <test> tag after its run to a data source of your own. You can also try making it more real-time by building an implementation of the listener interface org.testng.IInvokedMethodListener and within org.testng.IInvokedMethodListener#afterInvocation check if a method is a test method and if yes, start recording the results to a data source of your choice.
Related
I have a Java project, which consists of loads of maven modules and a considerable amount of unit/integration tests. The project is configured to create test reports via the surefire plugin. Now this plugin basically creates an XML-report per test class and is scheduled to run once a day and executed on Jenkins.
What I want to do is, send the those XML-reports to a test management system (XRAY) in order to make them more visible and manageable. My (naive) approach would be to just just add a post build script on Jenkins and send those reports via curl to the test managements REST-API. This API offers a way to send a single report file at a time. This report file can either be single or nested, i.e. I can basically send both of the following and it works:
Single report
<testsuite>
...
</testsuite>
Aggregated report
<testsuites>
<testsuite ... />
<testsuite ... />
</testsuites>
The REST-API can handle both, that is the IBM JUnit schema and the standard surefire schema
Now to the problem; I obviously want to combine those reports into one to avoid having to make a billion requests to the REST-API. However I can't seem to find an automated way. What I've tried so far is
play around with the surefire plugin to merge the XML reports, but no appropriate option seems to exist
Organise Tests into a (JUnit) test suite, but the output remains an xml report per test
finding alternative plugins/tools that address this issue, no luck
The only other way I can think of is to write a "merge script" myself, possibly using some sort of XSLT-transformation. But I'd rather not. Any help is much appreciated, thanks!
The solution would be to use an external utility for that as surefire seems to not support it.
I've used successfully junit-merge utility, which is an NPM package, as you can see for example in this tutorial.
The usage is pretty straightforward; you just need to specify the output file and the input folder containing the multiple JUnit XML based reports.
junit-merge -o results.xml -d target/surefire-reports/
I have around 200 testNG test cases can be executed through maven by and suite.xml file. But I want to convert these test cases into web service or any other similar webservice so that anybody can call any test case from there machine and will be able to know whether that particular functionality is working fine at that moment.
But what if no one calls the test webservices for longer time? You won't know the state of your application, if you have any failures/regressions.
Instead, you can use
continuous integration to run the tests automatically on every code push; see Jenkins for a more complete solution; or, more hacky, you can create your own cron job/demon/git hook on a server to run your tests automatically
a maven plugin that displays the results of the last execution of the automated tests; see Surefire for a html report on the state of the last execution of each test
I am making an framework that internally user JUnit and REST Assured. This framework will have the 4 #Test methods for CRUD operations. Whenever the user want to do any operation, he will call only that particular Test method. But at the end of the each operation(say GET or DELETE or any other), it should generate the report.
I tried using surefire-report plugin. As I have read, that will generate report only when we build the project(running all the Test methods).
Is there any mechanism that fulfills my requirement of generation report for individual run also?
Execution will be like : final output will be the jar with individual CRUD facility.API.execute(GET, end_point_name);API.execute(POST, end_point_name,data);Test method get and post is called respectively for the above calls. Report should be generated for both the test cases for normal run as java application.
There are 3 solutions to your problem :
Either you write your logger statement and do proper logging of the events. You can either store it in DEBUG, INFO etc mode for better understanding and more control.
ExtentReports is another way to go :
http://www.ontestautomation.com/creating-html-reports-for-your-selenium-tests-using-extentreports/ refer the above link where they have a provided a detailed way of using the same.
You can also create a separate testng.xml file. Like maintaining a separate suite file this will internally make sure with the help surefire to create a separate reports.
I have a custom report where I didn't use any testNg api, and simply created by using Java API, and HTML, CSS, and JS files. It works as expected. But I had requirement to implement parallel execution in script.
It was implemented successfully but report mechanism is not properly working now. Because I enabled 5 threads, and 5 browser will launch and each will pick one tscript. Report is showing wrong outcome, like 1 script contains 2 script check points. But If I seen a testNG report everything is good. How it does?
TestNG associates each ITestResult with its execution thread using an InheritableThreadLocal<T>. This allows TestNG to keep track of state for multiple tests running in parallel. See Reporter.java.
All,
I am using JUnit/Selenium (Java). I have over 400 test cases (separate java class files) distributed in different packages. I need to generate a basic test run report which would tell me if the test failed or passed and how much time it took.
TestNG is not an option since i am using TestWatcher along to make calls to a bug tracking tool API.
Any suggestions?
Thanks.
You may to use the RunListener class that listens to the test events. Then you also may to prepare the custom listener and create the report file. Unfortunately you probably will need add such listener to each your package.
The following link provides more details.