My development environment is Netbeans 7.4 and Maven 3.0.5 (bundled with Netbeans). I'm using JUnit 4.11 to build unit (class names ending with Test) and integration tests (class names ending with IT). I'm running unit tests and excluding integration tests with the "-DskipITs" Maven option. I have a custom action which runs just the integration tests with the failsafe plugin. Both execute successfully. However, I only see the results in the "Test Results" window when running the unit tests. How can I get the integration tests to show in the "Test Results" window? With the integration tests, I'm only seeing the output in the console.
The maven-failsafe-plugin only executes in integration-test and verify (and of course the help) goal while the maven-surefire-plugin runs during test goal.
NetBeans Test Results window only shows the tests that were executed using the 'test' goal.
My solution for this situation is to categorize my integration tests into
testintegration - just lightweight ITs, I run them with surefire (to see them in Test Results window)
testheavy - those that will require me to bootstrap something I run with the fail-safe plugin
I hope you have the option of doing something close to that.
Related
I am trying to run jmeter tests with jmeter maven plugin as shown in https://github.com/jmeter-maven-plugin/jmeter-maven-plugin/wiki/Configuring-the-jvm-that-the-jmeter-process-runs-in
When I run mvn verify it runs the jmeter test but also all the unit tests under src/test/. Is there a way that I can skip unit tests and only run the jmeter test?
Thanks in advance!
The easiest is just running single jmeter goal like:
mvn jmeter:jmeter
this way you will run only performance tests, all other build procedures will be skipped.
A harder but more flexible way would be using Maven Builde Profiles, see Maven – How to skip unit test
More information: How to Use the JMeter Maven Plugin
I am working on augmenting a test framework that uses Maven. Due to the nature of the code being tested, it is necessary to run tests in VMs. We are currently doing this in a sort of hack-ish way by running shell scripts that SSH into the VMs via Vagrant and run a list of tests. However, this list of tests has to be updated every time someone adds a new integration test to our tests. Ideally, we'd like to automatically gather the relevant tests that are flagged as Component / Integration tests with #Category JUnit flags in our Java code, and then run these tests within the VMs. It seems like Failsafe has no parameters to run the integration tests outside of the local machine. Is there any way to do this using existing Maven plugins?
Ideally, the flow of things would be as follows:
Discover all component / integration tests using Failsafe.
Pass the list of these tests into a VM
Run the tests on that VM, preferably with vagrant.
The existing plugin for Vagrant in Maven shows how to run a VM during integration tests, but it doesn't make clear how to actually run the integration tests on the VMs within Maven: http://nicoulaj.github.io/vagrant-maven-plugin/examples/running-a-vm-during-integration-tests.html . The plugin hasn't been updated since 2013 either, which isn't very promising.
I'm using Maven's failsafe plugin to run integration test suites. The tests themselves are selenium-webdriver tests that use JUnit for the asserts and categories. When I launch an individual test with:
mvn clean verify -Dfailsafe.rerunFailingTestsCount=N -Pfunctional-test,env-stage -Denvironment=env-stage -DtestBuildNumber=${testBuildNumber} -Dit.test=TestName -Dwebdriver=${browser} -Dselenium.grid.2.hub=${hub}
It will rerun failing tests N times.
However if I launch a test suite with that same command it will not rerun failing tests. Is there some way to get rerunFailingTestsCount to work with suites?
What version of failsafe plugin do you use?
Your problem looks like this bug: https://issues.apache.org/jira/browse/SUREFIRE-1152
Option rerunFailingTestsCount silently fails with test suites
After a test failure (with rerunFailingTestsCount > 0), the
JUnit4*Provider.executeWithRerun calls execute again with a list of
the failing testMethods, but JUnit4*Provider.execute silently fails to
find the requested methods.
Should be fixed in 2.19
Currently I have standard structure of project:
src
|---androidTest
|-------------------|---java
|----------------------------|---robotium
|----------------------------|---unit
|---main
|---------|---AndroidManifest.xml
|---------|---java
|---------|---res
According to documentation
I've placed unit and integration tests under androidTest folder.
Since instrumentation tests need lot of time, I placed it inside robotium folder(I use Robotium).
Inside unti - jUnit tests respectively.
To run all tests I invoke ./gradlew connectedCheck - it's taking lot of time.
How to to run integration (inside robotium folder) tests and unit tests separately?
I want to run these tests using gradle - it's for CI server.
For example I want to run integration tests every night and unit tests - every hour..
Thanks!
According to the documentation JUnit tests should be places under - src/test/java and then you'll be able to switch between test sources by changing Test Artifact when selecting Build variants as shown:
(source: android.com)
How do we setup Eclipse, TestNG (and possibly TestNG-Eclipse plugin?) so that our devs can run two different test configurations conveniently? We have hundreds of unit tests but only 20 integration tests, so we would prefer to a minimum number of changes to the unit tests if possible.
We have added attributes to our unit tests:
// unit tests
#Test
// integration tests
#Test(groups = { "IntegrationTest" })
The holy grail would be that we could right-click our package and select either:
debug configuration which only runs unit tests
debug configuration which the slow running integration tests (and possibly the unit tests).
It seems like we could add these configurations to the "Favorites" list. It is an acceptable compromise.
Finally, the last requirement is that the unit tests will also run on the build server (maven surefire plugin compatible). This doesn't seem to be a challenge so I'm not too concerned about it.
Also, if there is another framework I should investigate that supports these scenarios, I would also like to hear about that as well. Right now we are using junit and are exploring TestNG.
When you create a TestNG launch configuration, you can specify various things such as which groups to run.