We run our test in Jenkins from package where we have 10 classes. Each class represents one scenario.
At the end we have one graph trend where we have trends for all tests.
But when we try to add assertion and assertion failed, the build failed too.
test run command:
mvn gatling:test -D gatling.includes=packageName.*
Is it possible to made test suite where each scenario have own assertion, not in setup?
Is it possible to made build fail after finishing all tests, even if the assertion for some test failed and after that in Jenkins getting tests trends in one graph (in one graph 10 tests trends)?
No. Assertions are currently only global, not per scenario.
Related
When I add sonar.tests I get the tests analysed and the number of tests, I only want the number of tests but not the analysis.
sonar-project.properties:
sonar.projectKey=xxx
sonar.java.binaries=target/classes
sonar.java.test.binaries=target/test-classes
sonar.sources=src/main/java
sonar.tests=src/test/java
sonar.test.exclusions=src/test/java/**
sonar.java.source=11
sonar.sourceEncoding=UTF-8
sonar.coverage.jacoco.xmlReportPaths=target/site/jacoco/jacoco.xml
sonar.dynamicAnalysis=reuseReports
sonar.junit.reportPaths=target/surefire-reports/testng-junit-results/junitreports/
You cannot do it. It is impossible.
I have a bat file that is running my JUnit Cucumber tests. This bat file is being invoked by TeamCity. Once the JUnit Cucumber tests have been run, the output more or less ends with:
[15:24:37]18 Scenarios (18 passed)
[15:24:37]187 Steps (187 passed)
[15:24:37]0m24.339s
How can I save the line "18 Scenarios (18 passed)" to a variable in TeamCity, so that the next build step in TeamCity, which is an email test results component, can use this variable to send an email that contains the content of this variable? Otherwise the test results file that it attaches just has the details of each scenario without an overall summary being visible to the recipient.
Нou have to start your tests with maven(gradle) runner. e.g. mvn test. TeamCity automatically parses your junit.xml results and can build Test tab with test counts, test names, etc. and if any tests will be failed then TeamCity failed the build. or, you can just pass junit-results.xml for TeamCity
I am trying to run individual spock unit tests using intellij idea.
Consider:
// rest of code
def "Test Something"() {
// test code below
}
In above test, when I goto the test body and right context menu, I get two kinds of tests for Test Something. One is the grails test and other is the junit test.
Referring to this question, the accepted answer recommends using the jUnit runner. But using it, the code simply does not compile(probably because certain plugins and other classes are not available).
(I am not sure though as this is the desired behavior because I am just running a single test and not all tests. So wonder why is it compiling all classes ,including plugin classes not required by the test target class.)
Using the grails runner, I check the configuration and here is the screenshot:
So nothing looks wrong with the command there.
But the test on running gives Test framework quit unexpectedly error.
I try running same command from grails console(CMD windows) and it runs without any error message.
But on checking the output html files(in target/test-reports) I see that none of the tests actually ran!
So what is going on here and why are not individual tests running?
PS:
When I run All tests using test-app command, tests run as expected. Only individual (unit)tests are not running.
Part of the price paid for Spock's nice test naming, is that you can't specify an individual test to run anymore.
Here are some articles about it. The first seems pretty on-point:
Run a specific test in a single test class with Spock and Maven
This one isn't about running a single test, but has some relevance and talks about Spock's test-name conversions, plus Peter Niederwieser chimes in with comments:
Can TestNG see my Spock (JUnit) test results?
A workaround for this could be the #IgnoreRest annotation. Simply annotate the test you want to run with #IgnoreRest, and then specify that test class to run, and only the annotated test will run. http://spockframework.github.io/spock/javadoc/1.0/spock/lang/IgnoreRest.html
Try using the grails unit test and add the following in the command line part:
-Dgrails.env=development
This will run the test as we change the running environment to development . Hope this will help to everyone facing such problems.
I am learning Selenium with the aim of exporting UI tests into JUnit tests and then running them in a CI build.
As a basic test I have a Suite.java and a BasicTest.java which is a test case.
I have used the Selenium export and saved the files to the same location.
When I try and compile the tests it fails.
1st Error - http://pastebin.com/0j37KZ08
To get past this error I add an import to the Suite.java for the BasicTest class, which then gives me the following error:
2nd Error - http://pastebin.com/PKxmdj3L
Source code:
Suite.java - http://pastebin.com/Q6HVNtqT
BasicTest.java - http://pastebin.com/fKCk6iN2
Ant build.xml - http://pastebin.com/x16zHKP0
Any help appreciated.
I don't know if you need a Suite here. The issue seems to be that you are calling addTestSuite() with your BasicTest.class which does not extend junit.framework.TestCase. You should see this if you scrutinize the second error message closely. Of course, you would not subclass junit.framework.TestCase for a JUnit4-style test, so there's a disconnect here as far as what the Suite/framework expects and what you are providing. Maybe just avoid using a suite altogether--personally, I haven't bothered much with suites for some time now. Or, if you are following an example that insists on a suite, you can switch over to the JUnit3-style for purposes of learning.
EDIT: This link should help with JUnit4-style suites if you decide you want to keep the suite.
I know that it's possible to run a specific test class with -Dtest=MyTest. But is it possible to run a specific test within that class?
I.e. if MyTest defines testFoo() and testBar(), is there a way to specify that only testfoo() should be run?
I'm aware that it's trivially easy to do this in an IDE, but I occasionally need to run tests on the command line on another server.
From Running a Single Test Using Maven Surefire Plugin
With version 2.7.3, you can run only n tests in a single Test Class.
NOTE : it's supported for junit 4.x and TestNG.
You must use the following syntax
mvn -Dtest=TestCircle#mytest test
You can use patterns too
mvn -Dtest=TestCircle#test* test
It will be available as of Surefire 2.8, see SUREFIRE-577
Don't think its available. You can work around it by passing some system properties & ignore execution of tests based on the property value. However it does not seem to add a great value add. There is also TestNG which offers additional features.
http://maven.apache.org/plugins/maven-surefire-plugin/examples/testng.html
To execute one Test at a time, run mvn test
mvn -Dtest=MyUnitlTest test
To execute one Test at a time and a specific method from it:
mvn -Dtest=MyUnitTest#method test
where MyUnitTest is the name of your test and #method is the name of your method.
Execute tests with surefire:
mvn surefire:test