we run our junit tests in Eclipse seems fine, but when we move it to hudson or jenkins, the tests seems hangs there never returns. its definitely the same code, I just wandering whats the difference between running tests in Eclipse and Hudson.
In eclipse you usually just run one test class or one test method. A "fresh" Java Virtual machine is started and stopped for this test.
I suppose that Jenkins runs all tests in one Java Virtual machine. This can make a huge difference.
Try to run all test outside eclipse and outside hudson at once (should be simple if you have a maven project). What is the result? Maybe you can configure your test run to fork the Java VM on each test (this is possible with maven by configuring the surefire plugin).
My guess would be to look at the memory limit. Try to increase the memory limit of the maven process started from Jenkins.
My other guess would be to look whether the Maven build from Jenkins is allowed to execute submodules parallel. If yes, it might cause some race conditions that are hard to find.
Are the tests executed in the same way in Eclipse and in Hudson? My guess is that the tests are executed one by one (or class by class) in Eclipse, and by a build script such as Maven or Ant in Hudson? What happens if you execute the test suite from a terminal on your local machine?
Related
I have a Gradle project that I would like to use infinitest with. However, while all test pass using the Gradle test runner, many fail in infinitest. Is this because infinitest uses the IntelliJ test runner to run the tests?
I expect this is the case since I get similar results when I try to run tests manually using the IntelliJ test runner.
How can I configure infinitest to use Gradle as the test runner?
Infinitest starts a new JVM to run tests, it gets the project classpath from IntelliJ but does not know it is a Gradle project.
I don't know what the Gradle test runner does differently but it would be great if you could open an issue on the Infinitest bug tracker with the steps to reproduce the problem (ideally with a sample project).
In case the issue is with missing JVM options you can set them using the infinitest.args file
I just received a project from my collage and when I run the whole project with Maven test/ Maven install by right click to the whole project and choose Maven test or Maven install, it runs successfully. However if I run an individual test, this test failed. I do not really understand here why it happened. Can anyone give me a hint please? Thank you!
It is possible that another unit test is establishing some state that is affecting the unit test that is failing. It is best practice to avoid writing unit tests that depend on shared state, as there is usually no guarantee as to what order the tests run in. See this article for a good explanation of unit test best practices.
I am working on augmenting a test framework that uses Maven. Due to the nature of the code being tested, it is necessary to run tests in VMs. We are currently doing this in a sort of hack-ish way by running shell scripts that SSH into the VMs via Vagrant and run a list of tests. However, this list of tests has to be updated every time someone adds a new integration test to our tests. Ideally, we'd like to automatically gather the relevant tests that are flagged as Component / Integration tests with #Category JUnit flags in our Java code, and then run these tests within the VMs. It seems like Failsafe has no parameters to run the integration tests outside of the local machine. Is there any way to do this using existing Maven plugins?
Ideally, the flow of things would be as follows:
Discover all component / integration tests using Failsafe.
Pass the list of these tests into a VM
Run the tests on that VM, preferably with vagrant.
The existing plugin for Vagrant in Maven shows how to run a VM during integration tests, but it doesn't make clear how to actually run the integration tests on the VMs within Maven: http://nicoulaj.github.io/vagrant-maven-plugin/examples/running-a-vm-during-integration-tests.html . The plugin hasn't been updated since 2013 either, which isn't very promising.
Is there a way to tell only Eclipse to skip various individual junit tests (but still run the rest of them)? I am familiar with the #Ignore annotation, but that is not what I am looking for because that will cause the tests to always be skipped.
I would like the tests to be skipped when ran by eclipse (Run As -> Junit Test) but ran in any other context, most likely during a maven build.
Any ideas?
So far the following solution has worked well. It is not a complete solution because it causes the tests to only be ran by maven when maven is invoked from the command line. For now though, it works.
When maven is invoked, the maven command line arguments are placed in the environment map (System.getenv()). When the tests are run through jUnit in eclipse, there is no entry in System.getenv() for the maven command line arguments. So, I am using jUnits Assume class to check that this property is not null for the tests that I want eclipse to skip.
Code:
Assume.assumeNotNull(System.getenv("MAVEN_CMD_LINE_ARGS"));
FYI, one downside of this solution is that eclipse marks the tests as passed instead of skipped.
I had been introduced to concept of CI lately and was trying to work on jenkins CI. I was stuck up in one thing . How to trigger executable testng files in jenkins CI. For ex locally in our machines we just run testng.xml to execute couple of test cases. In the same way how can we trigger this xml file to run in jenkins CI ?
In most cases with jenkins you wouldn't use an executable. Normally you'd run the wrapper for the tests (Junit/Nunit etc.) which Jenkins is fully capable of running on it's own.
You can use this article to run TestNG tests using Maven:
Running TestNG tests using maven
After configuration is completed just add Invoke top-level Maven targets step to the Build Steps in Jenkins (Maven plugin should be installed). The target should be test in this case.
If you will face with any errors during configuration, try to google them.
If you are not using any build tool like maven or ant, you can invoke it from command line as we'll and specify your suite file. Make sure to set the correct class paths http://testng.org/doc/documentation-main.html#running-testng
You can put this as a build step in Jenkins.
Add a compilation step prior to this step. I haven't ever tried it - have always used ant or maven, but that is where I would start exploring.