My question is about the directory location for a docker based acceptance test.
My project is a Spring Boot based command line application which extracts data from a table and builds a spreadsheet. It has unit tests and a JUnit based acceptance test. The JUnit runner for the acceptance test is a standard JUnit runner, not a Spring based runner.
Finally, I have an acceptance test structure which tests the Docker components against a dedicated DB2 instance created specifically for each test. At this point, there's a docker-compose file that:
Launches a DB2 container instance exclusively and solely for this test.
Launches a Flyway migration container to load test data.
Launches a container that does the above mentioned Spring Boot command line application.
After the close of the docker-compose, a comparison is done against the generated spreadsheet and an expected file. If they're byte for byte equivalent, the test is considered passed.
Given the acceptance test above is heavily docker laden and a few steps removed from the Java side, is it still appropriate to put this test under /src/test/acceptance?
There are many approaches to this. In general maven has two plugins: surefire and failsafe. They are very similar in terms of configurations, however surefire is mainly aimed for running unit tests, and failsafe is for integration tests.
So, first off you probably want to configure acceptance tests with a failsafe plugin. You will:
Run them during a different building phase (at least way after the unit tests run)
If your build was broken and some unit tests do not succeed you won't even want to run the acceptance/failsafe tests - it might save some build time.
You will get different reports for integration and unit tests (technically these plugins create different report folders surefire-reports and failsafe-reports)
Now to physically separate the tests you can:
Merely rely on the naming convention. These plugins look for tests named differently, say: SampleTest.class will be run with surefire while SampleTestIT.class will be run with the filesafe plugin. Of course its all customizable at the level of plugibs configurations in the pom file.
Usually unit tests are required to be put to the same package as a real class conceptually. For example: if you have a class Foo in com.myorg.Foo.java, so you place it in src/main/java/com/myorg/Foo.java and the corresponding unit test for it will be in src/main/java/com/myorg/FooTest.java. For integration tests is not usually the case so you can simply create a folder it or something and use run them with different plugins automatically, again because you'll name the tests differently.
Another approach is to separate the folders within the same module, it was already described above. So technically you maintain src/test/java and src/test/resources and next to it you will have something like src/it/java and src/it/resources. Probably you'll still want to use both surefire and failsafe plugin as I've described above. You'll still run both types of the tests in the same maven lifecycle.
The most "radical" approach is to separate the acceptance tests to different maven module. This will give you the ability to run the module with acceptance tests separately in a different build step. This might be handy in CI tool for example. Of course you can achieve a similar effect with properties or with maven profiles.
As you can refer in https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
It is optimal to have separate directories for test. I would suggest put all tests under test and have separate folder, test/acceptance/docker-test or something, but overall it is up to you.
Separate folder does help to run and de-couple different tests.
Related
I am working on understanding Maven and I'm learning about building your Java app with it.
So when I do a :
maven package
It does build my jar as expected but I see in the output console that Maven does build tests (it always say that the test a run and there are no failure).
I researched on the web about that and learned that Maven use a plugin called Maven Surefire. But I can't understand what does that plugin do to my code, what does the tests "means" ? What does the tests do with my code and how it works behind the console ?
The Maven surefire plugin runs the tests you have written. These are usually in the src/test/java folder. If you have none, the plugin does nothing.
Is this only one question? :D
So. Different things are going on.
You create an application with Java. To test the single components / packages / classes that you create most people use JUnit or TestNg. You usually have dedicated test classes that verify your production code behaves as intended without you clicking through all the things on every change.
When you now use maven to run your build the pom.xml file defines a packaging - in your case "jar" since you create a jar file. The packaging defines what set of default plugins run in the defined maven phases. You probably recognize package here. Maven executes all phases up to package and the registered / configured plugins.
To execute those tests maven provides the surefire plugin which supports running JUnit or TestNg tests. If you follow the directory conventions your tests reside in src/test/java and the surefire includes naming convention maven will execute those tests in every build (as this is the best practice). If you also want to write integration tests then there is the failsafe plugin. That plugin is not enabled by default and runs in different maven phases.
So the tests just run your production code - in fact they just do what you implement in the tests. They don't alter it in any way.
The maven introduction documentation has step by step explanations: Maven in 5 Minutes and the Getting Started Guide.
Starting from scratch this is probably a lot. So don't rush this. The build setup and test setup are very important things to have.
For junit test cases The class should be named like XXTest.java
For intergation-test the class should be named like XXIT.java
Is there any class name conventions for pre-integration-test and post-integration-test goals in maven similar to unit and integration test?
The convention is that no tests of any kind are run during pre- and post-integration-test.
As suggested by #biziclop, the way to think about this is "pre-X" and "post-X" - these are things executed before/after "X". So, pre-integration-test phase is not the phase where "pre-integration" tests are run, but rather it is the phase before "integration-test" is run.
These goals are usually used for managing system state for the running of integration tests. For example, pre-integration-test may start a container that is used by the integration tests, while post-integration-test will stop the container.
There is nothing wrong with using a variety of plugins and/or custom code to support the execution of your integration tests. If using exec-maven-plugin really bothers you, you can always create your own maven plugin that knows about whatever naming convention you want to use to find java classes in your build and execute them. Then each of your modules can use the generic plugin to invoke the specific set of classes to do the setup/tear-down.
I have a Maven project with a number of modules. When building, I have an argument that determines which directory config files and such are copied from, depending on environment it will be running in - ie UAT, DEV, TEST, etc. I do not want to use profiles. Now, I want to package all integration tests into a separate jar that can be executed from command line as well as in integration-test phase. Basically there will be only one test class with one method that does something like
Class.forName("...").getMethod("main").invoke(null, args);
Only problem, is that since I do not want to use profiles Id have to add/remove the dependency on the test solution jar depending on if I want to run integration tests or not. I would like to do something like
mvn clean install -Denv=IT
and let it be. Is there a way to do so?
The standard mechanism for running different kinds of build in Maven is to use profiles (Maven is a highly opinionated build framework, so you are forced to play by its rules)
Your also appear to be building binaries to match the system you intend to deploy. This is generally a bad idea, you are better advised to look at some mechanism that allows the run-time configuration of your application. (In J2EE there is JNDI, but could be just a simple property file). This allows you to certify a single binary that ideally is pushed into a shared repository for sharing between development, test and production.
Currently I have standard structure of project:
src
|---androidTest
|-------------------|---java
|----------------------------|---robotium
|----------------------------|---unit
|---main
|---------|---AndroidManifest.xml
|---------|---java
|---------|---res
According to documentation
I've placed unit and integration tests under androidTest folder.
Since instrumentation tests need lot of time, I placed it inside robotium folder(I use Robotium).
Inside unti - jUnit tests respectively.
To run all tests I invoke ./gradlew connectedCheck - it's taking lot of time.
How to to run integration (inside robotium folder) tests and unit tests separately?
I want to run these tests using gradle - it's for CI server.
For example I want to run integration tests every night and unit tests - every hour..
Thanks!
According to the documentation JUnit tests should be places under - src/test/java and then you'll be able to switch between test sources by changing Test Artifact when selecting Build variants as shown:
(source: android.com)
How do we setup Eclipse, TestNG (and possibly TestNG-Eclipse plugin?) so that our devs can run two different test configurations conveniently? We have hundreds of unit tests but only 20 integration tests, so we would prefer to a minimum number of changes to the unit tests if possible.
We have added attributes to our unit tests:
// unit tests
#Test
// integration tests
#Test(groups = { "IntegrationTest" })
The holy grail would be that we could right-click our package and select either:
debug configuration which only runs unit tests
debug configuration which the slow running integration tests (and possibly the unit tests).
It seems like we could add these configurations to the "Favorites" list. It is an acceptable compromise.
Finally, the last requirement is that the unit tests will also run on the build server (maven surefire plugin compatible). This doesn't seem to be a challenge so I'm not too concerned about it.
Also, if there is another framework I should investigate that supports these scenarios, I would also like to hear about that as well. Right now we are using junit and are exploring TestNG.
When you create a TestNG launch configuration, you can specify various things such as which groups to run.