I'm working on a Java application that is build by Gradle and will be integrated into Jenkins.
Ignoring Gradle for a moment, one would setup multiple stages in Jenkins (e.g. Build, Test, Deploy) which are then executed step by step and also shown in the GUI.
In Gradle I can use the Java plugin to get a lot of predefined tasks (like build and test). Since build depends on test I basically run a pipeline (the dependency graph) every time I execute build.
What confuses me is how Jenkins and Gradle are playing together in this case.
Due to the dependencies in Gradle my Jenkins pipeline shrinks down to (more or less) a single stage and the stages executed in Gradle are hidden in the Jenkins UI.
Is this the "normal" way of doing things? Having all the build logic in Gradle (as it is a build tool after all) and having Jenkins only for the triggers that allow me to automatically execute tasks?
In case of Gradle, it is a single command to execute the compile, build goals. It basically performs the build process and finally creates the jar/war file in the end. In case of Jenkins, you need to perform the pipeline processing using the Jenkins stages. The stages may be compile classes, test for executing unit test cases, code analysis using sonar, security check using checkmarx etc. The following snippet will give you an idea of stages in Jenkins pipeline.
stages {
stage('Compile') {
steps {
gradlew('clean', 'classes')
}
}
stage('Unit Tests') {
steps {
gradlew('test')
}
post {
always {
junit '**/build/test-results/test/TEST-*.xml'
}
}
}
....
.....
To know more details about the flow, refer below the following links.
https://bmuschko.com/blog/jenkins-build-pipeline/
https://docs.gradle.org/current/userguide/jenkins.html
Related
I'm doing a POC for a Java project in an AzureDevops CI/CD pipeline. I created a Maven project that has Selenium tests (TestNG) that run against a demo website which is independent of my project. I want to run unit tests in the build pipeline and UI Selenium tests in the release pipeline.
The Visual Studio test task seems to be the building block that I need. I think you can differentiate between unit tests & UI tests using the 'Test files' field like **\unit*Test.dll, **\ui*Test. Unfortunately, this task is not available/compatible for Java projects.
I was able to run the Selenium tests with the Maven task and Surefire plugin during build but remember, I only want to run unit tests during build.
I actually was able to run the Selenium tests in the release pipeline via a workaround which was:
Copy the whole project to the artifacts directory of the release during build (copy files task).
Add a Maven task to the release pipeline
Trigger the Selenium tests in pom.xml
Normally, you would only copy artifacts to the artifact directory so I think doing that is a huge hack.
Another problem is that Maven will build the project during build and release which is wasteful. To dial back the waste, some savvy Maven configuration might help. I was thinking about skipping compilation and resolve dependencies during release, but I don't know where to find the Maven dependencies in the DevOps ecosystem.
Am I missing something or is AzureDevops maybe not supporting Java all that well?
I do follow a method for Maven selenium tests on Azure DevOps. What I do is, in the build pipeline I build my tests in such a way that it produces a jar with all the dependencies and test classes in it. I also use testng in my approach. Next I copy my build Artifact to Artifactory. This completes the build. Now during the release I download my Artifact from Artifactory and I check the environment where i want to run and I inject the right testng file by running java -jar myfile.jar testngIT.xml. This runs my tests faster and better.
You can try just adding a test task in your release pipeline just as in the build pipeline.
And add a copy task in the build pipeline to copy the test codes and files to the build artifacts and publish it to release pipeline.
Below steps is just for reference(in classic view). Hope it can be of some help.
1, Add copy file task in the build pipeline to copy the all test files and all the dependent setting files to the test folder in artifacts.
2, Publish artifacts to release pipeline
3, In the release pipeline, add the task to execute the tests just like the way you do in build pipeline
I am using IntelliJ IDEA 2019.1.1. Builds now delegate to gradle by default. For the most part, I like this idea but for one of my projects this delegation to gradle seems to be causing a build problem. I see this in the build output:
10:09:23 AM: Executing tasks ':scanrunner:testClasses :policyconsole:testClasses :scanrunner:classes :policyconsole:classes :stepsapi:querydslClasses :service-core:classes :service-core:testClasses :stepsapi:classes :stepsapi:testClasses'...
It looks like IntelliJ is choosing these tasks to run automatically from my build.gradle. The problem is that running the task ':stepsapi:querydslClasses' like this always causes compilation errors. After running that task in --debug mode, I see that the classpath is just wrong all files fail to compile. The build will work fine if that task is just omitted like this:
./gradlew :scanrunner:testClasses :policyconsole:testClasses :scanrunner:classes :policyconsole:classes :service-core:classes :service-core:testClasses :stepsapi:classes :stepsapi:testClasses
When running this way, gradle runs :stepsapi:classes which in turn runs :stepsapi:compileQuerydsl which comes from the querydsl gradle plugin. I'm assuming there is something in the querydsl plugin that sets up the classpath correctly.
So my question is, is there some IntelliJ IDEA setting or build.gradle setting where I can override which gradle tasks will be executed when a build action is delegated to gradle?
update: Here is a github project that demonstrates this issue.
After messing with this for a while, I'm going with this addition to build.gradle:
project.afterEvaluate {
project.tasks.compileQuerydslJava {
onlyIf {false}
}
}
This effectively prevents gradle from doing anything with the :querydslClasses task that IntelliJ IDEA insists on running.
FWIW, the querydsl gradle plugin extends JavaCompile from the gradle Java plugin. By doing this, it gets tasks like :querydslClasses and :compileQuerydslJava for free. The only problem is the plugin doesn't bother to make these "free" tasks work out of the box. It does, however do some tricks so that the plugin does work as long as you only specify tasks like :build.
I am trying to find a solution for the following puzzle. I have java projects, managed by maven, which needs some native dependencies to work (run unit and integration tests). Those are provided in form of deb packages, which needs to be installed prior to running a build.
I use Jenkins for CI. Native dependencies can not be installed on Jenkins nodes, because of conflicts with other builds and they can change often. What I do now is not to create a Jenkins job type 'maven', but 'freestyle' and use a pbuilder to create an clean sandbox, install all that is necessary and invoke maven build.
This is working great, but I am loosing Jenkins maven goodies like automatic upstream projects, trigger build when dependency change, etc. Jenkins simply does not know that maven is there.
Finally, my question. Is there a way how to achieve both, isolate build so installed libraries does not affect other builds and leverage Jenkins's 'magic' applied to maven builds and their dependencies?
You could split your build in three jobs, which trigger the next one.
Create needed environment
Run maven job
Clean Up
Even a Freestyle job has "Invoke top-level Maven targets". You could use that to get "maven goodies" while also having ability to run other build steps.
There is an option to "use private Maven repository" which will make sure it will use the .m2/repository folder location relative to the workspace. If you need to separate into multiple jobs, you can use "Custom/shared workspace" between those jobs.
Even in Maven-style job, there is an option to use private repository, so that one job does not affect another.
The problem can be solved by using distributed Jenkins builds. Slave agents can be configured to provision clean environment (e.g. via VMs, docker,...) for each build and tear it down after build is done. This way Jenkins job can be of Maven type and any changes done by pre-build step will not affect others.
More information can be found here.
Consider docker. There you can run processes in isolated environments just as you want. Docker works in a way that it easily communicates with Jenkins.
As a benefit you can also use that docker container to run local builds in the same environment as they run in Jenkins.
I have a Gradle custom Test task to run my Integration Tests. I would like to be able to run this without Gradle automatically going through all previous phases and just running the test. Is there a way to do this without -x for each build step?
I had been introduced to concept of CI lately and was trying to work on jenkins CI. I was stuck up in one thing . How to trigger executable testng files in jenkins CI. For ex locally in our machines we just run testng.xml to execute couple of test cases. In the same way how can we trigger this xml file to run in jenkins CI ?
In most cases with jenkins you wouldn't use an executable. Normally you'd run the wrapper for the tests (Junit/Nunit etc.) which Jenkins is fully capable of running on it's own.
You can use this article to run TestNG tests using Maven:
Running TestNG tests using maven
After configuration is completed just add Invoke top-level Maven targets step to the Build Steps in Jenkins (Maven plugin should be installed). The target should be test in this case.
If you will face with any errors during configuration, try to google them.
If you are not using any build tool like maven or ant, you can invoke it from command line as we'll and specify your suite file. Make sure to set the correct class paths http://testng.org/doc/documentation-main.html#running-testng
You can put this as a build step in Jenkins.
Add a compilation step prior to this step. I haven't ever tried it - have always used ant or maven, but that is where I would start exploring.