After some research into running Cucumber on Jenkins I have come to a dead end, I have read some post here about Running Cucumber but most are error not the process.
Running via command line the problem here is that I don't know how to call this in Jenkins after building.
I have Jenkins running on an Ubuntu server. Everything for building a maven project is setup, but how would one run the Runcukes file or setup the pom file in a way to call cucumber to start running?
Wire up a Maven task to run cucumber. As cucumber generates stubs for junit, maven's surefire plugin will run the tests nicely.
Jenkins has full support for running maven builds, so you won't have any issues there.
Related
So I have a BDD testing project for a service and what I want is that as soon as I merge a branch into the master branch of that service then that BDD testing should be triggered first and if every test case passes then only it should allow to deploy the service to production.
The BDD is written in Cucumber-Java.
You need to configure this in a CI server such as Jenkins.
You can consider these steps, when setting up Jenkins, to achieve your use case:
Use Git-hooks; This will trigger a Jenkins pipeline as soon you merge a branch into the master. Check this article for more info https://kohsuke.org/2011/12/01/polling-must-die-triggering-jenkins-builds-from-a-git-hook/
Then in Jenkins, run your BDD project. Here is a nice article, which explains how cucumber projects are setup and run in Jenkins:
https://faun.pub/running-cucumber-tests-with-jenkins-a5a3a8df07eb
When your cucumber project passes, then write a script in Jenkins to deploy into production (This would be a separate stage in Jenkins). Although I would recommend you to deploy this into a staging server or something like Artifactory, which would hold your release package and have a manual step to deploy into production.
Finally here is a detailed article that gives step by step explanation of how you could integrate NoCodeBDD with Jenkins. Though this article is for NoCodeBDD, you could copy these steps and apply them for any BDD project by changing some parts of the script.
https://blog.nocodebdd.com/integrate-nocodebdd-and-jenkins-to-run-bdd/
I'm doing a POC for a Java project in an AzureDevops CI/CD pipeline. I created a Maven project that has Selenium tests (TestNG) that run against a demo website which is independent of my project. I want to run unit tests in the build pipeline and UI Selenium tests in the release pipeline.
The Visual Studio test task seems to be the building block that I need. I think you can differentiate between unit tests & UI tests using the 'Test files' field like **\unit*Test.dll, **\ui*Test. Unfortunately, this task is not available/compatible for Java projects.
I was able to run the Selenium tests with the Maven task and Surefire plugin during build but remember, I only want to run unit tests during build.
I actually was able to run the Selenium tests in the release pipeline via a workaround which was:
Copy the whole project to the artifacts directory of the release during build (copy files task).
Add a Maven task to the release pipeline
Trigger the Selenium tests in pom.xml
Normally, you would only copy artifacts to the artifact directory so I think doing that is a huge hack.
Another problem is that Maven will build the project during build and release which is wasteful. To dial back the waste, some savvy Maven configuration might help. I was thinking about skipping compilation and resolve dependencies during release, but I don't know where to find the Maven dependencies in the DevOps ecosystem.
Am I missing something or is AzureDevops maybe not supporting Java all that well?
I do follow a method for Maven selenium tests on Azure DevOps. What I do is, in the build pipeline I build my tests in such a way that it produces a jar with all the dependencies and test classes in it. I also use testng in my approach. Next I copy my build Artifact to Artifactory. This completes the build. Now during the release I download my Artifact from Artifactory and I check the environment where i want to run and I inject the right testng file by running java -jar myfile.jar testngIT.xml. This runs my tests faster and better.
You can try just adding a test task in your release pipeline just as in the build pipeline.
And add a copy task in the build pipeline to copy the test codes and files to the build artifacts and publish it to release pipeline.
Below steps is just for reference(in classic view). Hope it can be of some help.
1, Add copy file task in the build pipeline to copy the all test files and all the dependent setting files to the test folder in artifacts.
2, Publish artifacts to release pipeline
3, In the release pipeline, add the task to execute the tests just like the way you do in build pipeline
I have configured maven surefire plugin with parameter:
<configuration>
<forkedProcessTimeoutInSeconds>60</forkedProcessTimeoutInSeconds>
</configuration>
So when test's working more then 60 seconds, the surefire plugin interrupts it.
Everything works perfectly on my local machine when I use mvn test or mvn install, but when I try to build project on Jenkins it just swallows exception, writes into log [ERROR] There was a timeout or other error in the fork and continues the build. As result I get a Finished: SUCCESS message.
Question: Have anyone got this problem? Does anyone know any solution?
One important difference between default options of a maven local build and a Jenkins maven job is that locally the maven.test.failure.ignore option of the Maven Surefire Plugin is set to false (reasonably) so that test failures will also fail the build.
From official documentation:
Set this to "true" to ignore a failure during testing. Its use is NOT RECOMMENDED, but quite convenient on occasion.
However, a Maven Jenkins job will always run setting the same option to true, as such making the Maven build successful even with test failing and turn the status of the Jenkins job to UNSTABLE (and not SUCCESSFUL or FAILED, which may be a point of debate indeed).
This behavior is also documented in an official Jenkins issue ticket
Following the Jenkins Terminology, when (surefire or failsafe) tests fail, the Jenkins build status must be UNSTABLE:
<< A build is unstable if it was built successfully and one or more publishers report it unstable. For example if the JUnit publisher is configured and a test fails then the build will be marked unstable. >>
So, in a Maven Jenkins job, if a test fails:
Maven build is SUCCESSFUL
Jenkins build is UNSTABLE
Instead, in a freestyle Jenkins job executing Maven, if a test fails:
Maven build is FAILED
Jenkins build is FAILED
Possible solutions:
Change the build to a freestyle Jenkins job running maven (which may be too much work though) or
Add the -Dmaven.test.failure.ignore=false option to your build (however, you would not have UNSTABLE builds any longer).
I've deployed a servlet based web application(War file) in Tomcat server using Jankins and Maven. Now I want to run some selenium automation scripts on same application, Can it be achieved by Jenkins.
Its like Deploy web application on Sever and once deployment is successful run automation scripts using jenkins..
Please Help..
Two possible ways :
1) Add post build step to your current task and run the scripts there.
2) Add another task ex. "app-automation-scripts" that runs every time that your current task is builded correctly.
Yep,
you can add maven plugin to jenkins, (installed by default most of times)
then declare your project as war in pom.xml
then add a post processing "maven" to your maven build config and configure it with options "war:war"
-- options you can add for a maven build is: clean install build and etc. war option is for creating war files.
Or:
simply run a bash command using your jenkins build configuration.
Former needs installation of maven plug-in and configuring it,
Later needs configuring your maven to be run on the jenkins server by user named "jenkins".
I had been introduced to concept of CI lately and was trying to work on jenkins CI. I was stuck up in one thing . How to trigger executable testng files in jenkins CI. For ex locally in our machines we just run testng.xml to execute couple of test cases. In the same way how can we trigger this xml file to run in jenkins CI ?
In most cases with jenkins you wouldn't use an executable. Normally you'd run the wrapper for the tests (Junit/Nunit etc.) which Jenkins is fully capable of running on it's own.
You can use this article to run TestNG tests using Maven:
Running TestNG tests using maven
After configuration is completed just add Invoke top-level Maven targets step to the Build Steps in Jenkins (Maven plugin should be installed). The target should be test in this case.
If you will face with any errors during configuration, try to google them.
If you are not using any build tool like maven or ant, you can invoke it from command line as we'll and specify your suite file. Make sure to set the correct class paths http://testng.org/doc/documentation-main.html#running-testng
You can put this as a build step in Jenkins.
Add a compilation step prior to this step. I haven't ever tried it - have always used ant or maven, but that is where I would start exploring.