For instance, if regression test is failed then overall report should be failed.
I've configured jenkins with these steps, it gives the code coverage by unit test given by developer if unit tests are executed.
My question is how can I get the code coverage by executing my regression tests which are configured in Post Step.
Pre Steps: Configuring development environment, like changing some values in configs.
Pre-Build Step: clean install -fae -Pcode-coverage -DskipTests using maven.
Post Step:
Execute Shell - /var/lib/jenkins/qascripting/Vdopia_Automation/exe/hudson_tc_execute.sh to trigger my regression test.
Related
I have a big java project. Using maven to automate the build and manage dependencies and JUnit for the unit testing.
When i'm running mvn clean install from Intellij the build fails because of one unit test that fails.
The test succeeds when ran manually.
Also, executing mvn clean install from the terminal succeeds. (all tests pass)
I'm not sure where to look for the problem here.
Thanks!
Added integration test for enrichment process using Java. I've used Eclipse IDE for local testing, tried running integration test as gradle build and also did verification as integrationtest.
Also triggered build for the same in teamcity, it is getting successful and all the testcases are passed.
But the problem here is, we have set automatic trigger for build in teamcity everyday at 11 p.m.
On automatic trigger, the integration test is failed.
Showing ITTestCases: AppStartup_Tests: java.lang.AssertionError: expected [true] but found [false]
I am trying to run jmeter tests with jmeter maven plugin as shown in https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki/Configuring-the-jvm-that-the-jmeter-process-runs-in
When I run mvn verify it runs the jmeter test but also all the unit tests under src/test/. Is there a way that I can skip unit tests and only run the jmeter test?
Thanks in advance!
The easiest is just running single jmeter goal like:
mvn jmeter:jmeter
this way you will run only performance tests, all other build procedures will be skipped.
A harder but more flexible way would be using Maven Builde Profiles, see Maven – How to skip unit test
More information: How to Use the JMeter Maven Plugin
I'm using Maven's failsafe plugin to run integration test suites. The tests themselves are selenium-webdriver tests that use JUnit for the asserts and categories. When I launch an individual test with:
mvn clean verify -Dfailsafe.rerunFailingTestsCount=N -Pfunctional-test,env-stage -Denvironment=env-stage -DtestBuildNumber=${testBuildNumber} -Dit.test=TestName -Dwebdriver=${browser} -Dselenium.grid.2.hub=${hub}
It will rerun failing tests N times.
However if I launch a test suite with that same command it will not rerun failing tests. Is there some way to get rerunFailingTestsCount to work with suites?
What version of failsafe plugin do you use?
Your problem looks like this bug: https://issues.apache.org/jira/browse/SUREFIRE-1152
Option rerunFailingTestsCount silently fails with test suites
After a test failure (with rerunFailingTestsCount > 0), the
JUnit4*Provider.executeWithRerun calls execute again with a list of
the failing testMethods, but JUnit4*Provider.execute silently fails to
find the requested methods.
Should be fixed in 2.19
My development environment is Netbeans 7.4 and Maven 3.0.5 (bundled with Netbeans). I'm using JUnit 4.11 to build unit (class names ending with Test) and integration tests (class names ending with IT). I'm running unit tests and excluding integration tests with the "-DskipITs" Maven option. I have a custom action which runs just the integration tests with the failsafe plugin. Both execute successfully. However, I only see the results in the "Test Results" window when running the unit tests. How can I get the integration tests to show in the "Test Results" window? With the integration tests, I'm only seeing the output in the console.
The maven-failsafe-plugin only executes in integration-test and verify (and of course the help) goal while the maven-surefire-plugin runs during test goal.
NetBeans Test Results window only shows the tests that were executed using the 'test' goal.
My solution for this situation is to categorize my integration tests into
testintegration - just lightweight ITs, I run them with surefire (to see them in Test Results window)
testheavy - those that will require me to bootstrap something I run with the fail-safe plugin
I hope you have the option of doing something close to that.