I have a java webservice and want to set separate test/build/deploy stages in gitlab-ci.
A flow would probably be simple as follows:
stages:
- test
- build
- deploy
test:
stage: clean test
script:
- mvn $MAVEN_CLI_OPTS test
build:
stage: build
script:
- mvn $MAVEN_CLI_OPTS package -DskipTests=true
deploy:
stage: deploy
script:
- mvn $MAVEN_CLI_OPTS package -DskipTests=true
Problem: each maven goal will execute the preceding lifecycle goals. Eg a package or deploy goal will by default also execute the test goal. Thus having to exclude it explicit with skipTests=true.
Anyways goals like package will still be re-executed on test + deploy.
Question: can this be further optimized? I mean, I would not want to rebuild the jar on each stage. Could I tell maven to reuse the jar, and skip any preceding goals?
I know that a single deploy stage would be sufficient for maven to execute the package and test goal under the hood. But then in my gitlab I'd always have failures in the deploy stage, while eg just a junit test in the test goal failed underneath.
No, not really. Maven is not constructed to support that.
You can use skip parameters of the those plugins which have them.
Alternatively, just use two steps with mvn clean test and then mvn deploy -DskipTests=true, which should not really take longer than a single mvn clean deploy.
You can use job artifacts to access the results of previous stages (the compiled classes in this case). Also it would be reasonable to cache the local repository so that dependencies are downloaded only once. Example:
default:
image: maven:3
variables:
MAVEN_CLI_OPTS: "--batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository"
cache:
paths:
- .m2/repository
test-compile:
stage: build
script: mvn $MAVEN_CLI_OPTS test-compile
artifacts:
paths:
- target/
test:
stage: test
script: mvn $MAVEN_CLI_OPTS test
# maven-compiler-plugin outputs "Nothing to compile - all classes are up to date"
artifacts:
when: always
reports:
junit:
- target/surefire-reports/TEST-*.xml
deploy:
stage: deploy
script: mvn $MAVEN_CLI_OPTS package -DskipTests
It's also possible to have unit and integration tests run in parallel this way:
Related
I have a simple gitlab-ci pipeline. Now I want to add a maven dependency to a library that I want to share between different projects, eg my main project defines:
<dependencies>
<dependency>
<groupId>my.projects</groupId>
<artifactId>my-commons</artifactId>
<version>1.0.0</version>
</dependency>
</dependencies>
The my-commons is also a project in my gitlab.
Question: how can I trigger a mvn clean install into the ci-repository on that shared dependency during build of my main project?
This is my gitlab-ci from main project:
image: adoptopenjdk/maven-openjdk11
#TODO how to trigger a mvn clean install on my-commons?
test:
stage: test
script: mvn test -B
build:
stage: build
script: mvn package -B -Dmaven.test.skip=true
artifacts:
paths:
- target/*.jar
deploy:
stage: deploy
script: ...
Sidenote: I know I could setup a local shared repository manager like nexus, but I'd like to avoid that and simply add the common dependency into the build process.
You can build your own image and use it at gitlab-ci.
Example of Dockerfile:
FROM adoptopenjdk/maven-openjdk11
RUN mvn clean install
# ... other stuff
after you'll build your image and push it to docker hub you can use it inside your gitlab-ci.
useful links:
Best practices for writing Dockerfiles
docker image build
Pushing a Docker container image to Docker Hub
I have a Gradle Java 11 project that resides into Gitlab.
I wanted to introduce gitlab-ci.yml to force Gitlab to build a project and run tests on each push to a remote branch.
What should it look like?
I'm pretty sure that a similar question already has been asked but I couldn't find it, so I appreciate your help.
Gitlab provides an official description of the .gitlab-ci.yml file, but that could be a bit lengthy to get started out of the gate. For a basic project, you can use the following as a basis:
image: gradle:jdk11
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
cache:
paths:
- .gradle/wrapper
- .gradle/caches
package:
stage: build
script:
- ./gradlew assemble
test:
stage: test
script:
- ./gradlew check
Note that the image tag is set to maven:latest in many examples, but in order for Gitlab to compile the project using JDK 11, the image tag must be set to maven:3-jdk-11. The GRADLE_USER_HOME is set to the .gradle of the current directory that the script is read from to ensure that the Gradle environment is properly configured before starting the build.
The cache section defines the paths that Gitlab CI should look for cached artifacts and dependencies (which, for a Gradle build, is .gradle/wrapper and .gradle/cache). Builds may take a long time to execute, as each build requires downloading all dependencies each time the build is executed; to speed this up, caching can be included to remove the need to repeatedly download dependencies. The specifics for caching may vary for your project. See the official cache documentation for more information.
The assemble and check steps simply run gradle assemble and gradle check, respectively. While gradle test would be sufficient in many cases (as opposed to gradle check, the check step includes test while also including other verification steps. For more information on the difference between check and test, see Gradle difference between test and check.
For more information, see the following:
How to enable maven artifact caching for gitlab ci runner?
Cache dependencies in GitLab CI/CD
GitLab CI/CD Examples
Gitlab CI gradle dependency cache
A sample of .gitlab-ci.yml for a gradle project
gitlab 8.2.1, How to use cache in .gitlab-ci.yml
List of supported gradle images
Equivalent Maven example:
image: maven:3-jdk-11
variables:
MAVEN_CLI_OPTS: "--batch-mode"
MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"
cache:
paths:
- .m2/repository/
- target/
package:
stage: build
script:
- mvn $MAVEN_CLI_OPTS package
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS test
The --batch-mode flag runs Maven with default values and does not prompt the user to select any defaults. The -Dmaven.repo.local=.m2/repository sets the local repository to the location on the build server (Gitlab). which plays into the caching ability of Gitlab. Builds may take a long time to execute, as each build requires downloading all dependencies each time the build is executed; to speed this up, caching can be included to remove the need to repeatedly download dependencies. The specifics for caching may vary for your project. See the official cache documentation for more information.
The package and test steps simply run mvn package and mvn test, respectively (with the Maven options described above).
I am running maven like this:
mvn clean cobertura:cobertura package
I am noticing that my unit tests get run twice (thus doubling my build time). Is there a way to run cobertura AND generate the package in the same command without running tests twice?
An easy way would be to run two separate commands. In Bash it's then easy to chain them together into one line:
mvn clean cobertura:cobertura && mvn package -Dmaven.test.skip=true
The first bit:
mvn clean cobertura:cobertura
Does clean, runs the tests and generates the coverage report.
The second bit:
mvn package -Dmaven.test.skip=true
Does the packaging, but tells it not to run the tests.
The && is there so that if the first command fails, then it won't try to run the second.
I was trying to build hive-0.13.
When using -Dmaven.test.skip=true, it will not build the test jars but it will check test dependency.
When using -DskipTests, it will not build the test jars and also not check test dependency.
What's the difference between -DskipTests and -Dmaven.test.skip=true?
Maven docs:
-DskipTests compiles the tests, but skips running them
-Dmaven.test.skip=true skips compiling the tests and does not run them
Also this one might be important
maven.test.skip is honored by Surefire, Failsafe and the Compiler
Plugin
There is a third, related option described here: https://stackoverflow.com/a/21933970/3169948
"maven.test.skip.exec=true" the tests get compiled, but not executed.
So the complete set of test options for Maven would be:
-DskipTests ==> the tests get compiled, but not executed.
-Dmaven.test.skip.exec=true ==> the tests get compiled, but not executed (exactly the same as -DskipTests).
-Dmaven.test.skip=true ==> doesn't compile or execute the tests.
I have multiple questions.
Can I specify the pom.xml in mvn command?
Can I mix the goals of another project while executing mvn command on current project ?
Eg: mvn clean-otherproject comple-otherproject instal-otherproject compile-thisproject
I can do this with multiple mvn commands, but Can I do this in single maven command.
Just mvn --help would have answered the first question:
mvn -f otherPomFile.xml
No. You can simple execute the phases for the current project you are in. You can give multiple phases like
mvn clean install site site:deploy
For the first question, see khmarbaise's answer
If you want to build more than one maven project in one step, you must use modules.
In a multi-module project, if you call mvn install from the top project, all sub modules are built, unless you use the advanced reactor options (e.g. mvn install -pl util -am only builds the module 'util' and it's dependencies)