Has anyone out in the community successfully created a Selenium build in Jenkins using Browserstack as their cloud provider, while requiring a local testing connection behind a firewall?
I can say for sure Saucelabs is surprisingly easy to execute builds with the Sauce Jenkins plugin in a continuous deployment environment as I have done it. I cannot however, say the same for Browserstack. The organization I work with currently uses Browserstack, and although their service does support automated testing using a binary application I find it troublesome with Jenkins. I need to make absolutely sure Browserstack is not a viable solution, if so. I love Saucelabs and what their organization provides, but if Browserstack works I don't want to switch if I don't need to.
The Browserstack documentation instructs you to run a command, with some available options, in order to create a local connection before execution.
nohup ./[binary file] -localIdentifier [id] [auth key] localhost,3000,0 &
I have added the above statement as a pre-build step shell command. I have to also add 'nohup' as once the binary creates a successful connection, the build never actually starts since I have not exited as displayed in the output below.
BrowserStackLocal v3.5
You can now access your local server(s) in our remote browser.
Press Ctrl-C to exit
Normally I can successfully execute the first build without a problem. Subsequent build configurations using the same command never connect. The above message displays, but during test execution Browserstack reports no local testing connection was established. This confuses me.
To give you a better idea of what's being executed, I have 15 build configurations for various projects suites and browser combinations. Two Jenkins executors exist and I have more than 5 Browserstack VM's available at any given time. Five of the builds will automatically begin execution when the associated project code is pushed to the staging server, filling up both executors. One of them will begins and end fine. None of the others will as Browserstack reports local testing is not available.
Saucelabs obviously has this figured out with their plugin, which is great. If Browserstack requires shell commands to create local testing connections, I must be doing something wrong, out of order, etc.
Environment:
Java 7
Selenium 2.45
JUnit 4.11
Maven 3.1.1
Allure 1.4.10
Jenkins 1.5
Can someone post some information who use Browserstack in a continuous testing environment while utilizing multiple parallel test executions and tell me how each build is configured?
Thanks,
I've recently looked into BrowserStack with Selenium and the BrowserStack Plugin has made this task much easier.
Features
Manage your BrowserStack credentials globally or per build job.
Set up and tear down BrowserStack Local for testing internal, dev or
staging environments.
Embed BrowserStack Automate reports in your
Jenkins job results.
Much easier integration all round.
This is Umang replying on behalf of BrowserStack.
To start with, you are using the correct command for setting up the Local Testing connection. Although you do not need to specify the ‘localhost,3000,0’ details. We would also suggest you use the “-forcelocal” parameter while initiating the connection. The command should be as follows:
nohup ./[binary file] [auth key] -localIdentifier [id] -forcelocal &
The parameter “-forcelocal” will route all traffic via your IP address. Also, the process to initiate the connection before running your tests is correct.
However, here I’d like on confirm on the “id” you’ve specified while creating the connection. As you shared, there are 15 build configurations and I understand that each build has a different “id” specified. Please make sure that “id” specified while setting up the Local Testing connection and in the tests (“browserstack.localIdentifier” = “id”) is the same. Else, you will receive the error “[browserstack.local] is set to true but local testing through BrowserStack is not connected”
Integrating BrowserStack with Jenkins is a little bit tricky, but don't worry, it's perfectly doable :-)
The BrowserStackLocal client needs to be started as a background process, as per Umang's suggestion, and that's pretty much how the SauceLabs plugin works as well.
The trouble is that when Jenkins sees that you start daemon processes all by yourself and not via a plugin, it kills them. That's why you need to convince it otherwise.
I've described how to go about it step by step in this article, but if you're using the Pipeline Plugin, you can use the below script as a starting point:
node {
with_browser_stack 'linux-x64', {
// Execute tests: here's where a step like
// sh 'mvn clean verify'
// would go
}
}
// ----------------------------------------------
def with_browser_stack(type, actions) {
// Prepare the BrowserStackLocal client
if (! fileExists("/var/tmp/BrowserStackLocal")) {
sh "curl -sS https://www.browserstack.com/browserstack-local/BrowserStackLocal-${type}.zip > /var/tmp/BrowserStackLocal.zip"
sh "unzip -o /var/tmp/BrowserStackLocal.zip -d /var/tmp"
sh "chmod +x /var/tmp/BrowserStackLocal"
}
// Start the connection
sh "BUILD_ID=dontKillMe nohup /var/tmp/BrowserStackLocal 42MyAcc3sK3yV4lu3 -onlyAutomate > /var/tmp/browserstack.log 2>&1 & echo \$! > /var/tmp/browserstack.pid"
try {
// Execute tests
actions()
}
finally {
// Stop the connection
sh "kill `cat /var/tmp/browserstack.pid`"
}
}
You'd of course need to replace the fake access key (42MyAcc3sK3yV4lu3) with yours, or provide it via an environmental variable.
The important part here is the BUILD_ID, because that's what the Jenkins ProcessTreeKiller looks for when it decides whether to kill your daemon process or not.
Hope this helps!
Jan
Paul Whelan's answer to use the Jenkin's BrowserStack Plugin is currently the simplest way to integrate Jenkins with BrowserStack. The plugin supports all Jenkins versions >1.580.1.
To ensure that you get BrowserStack test reports you will need to configure your project's pom.xml as documented on the plugin wiki.
Just in case anyone else was having problems with this
For BrowserStackLocal v4.8 I found that -localidentifier has been removed from the binary options. (This is probably old news!)
When I removed the capabilities['browserstack.localIdentifier'] property from our automated tests the connection started working.
local binary
browserstack <key> -v -forcelocal
selenium setup
Capybara.register_driver :browserstack do |app|
capabilities = Selenium::WebDriver::Remote::Capabilities.new
# If we're running the BrowserStackLocal binary, we need to
# tell the driver as well
capabilities['browserstack.local'] = true
# other useful options
capabilities['browserstack.debug'] = true
capabilities['browserstack.javascriptEnabled'] = true
capabilities['javascriptEnabled'] = true
# etc ...
Related
I’m currently seeing an issue with my Java 8/Kotlin unit tests while executing them with Maven. This started happening after upgrading to MacOS Monterey. The tests run fine through IntelliJ.
One thing the tests have in common is, that they open ports/start a service.
In one instance, I’m trying to start a Redis test server on localhost and in a different one I’m opening a random port on localhost (via a wire mock rule).
Has anybody seen this issue before, it seems to solely affect Monterey - I assume I have to somehow grant Maven more permissions?!
Any help is appreciated.
Edit: Here some more info.
Error message (using redis.embedded.RedisServer):
[ERROR] Errors:
[ERROR] RedisCacheServiceTest » RedisConnection Unable to connect to Redis server: 127...
Code piece for this:
redisServer = RedisServer(REDIS_PORT)
redisServer.start()
val config = Config().apply {
useSingleServer().address = "redis://127.0.0.1:$REDIS_PORT"
}
redissonClient = Redisson.create(config)
and a failed assertion on the other (runs fine on other MacOS versions):
response.statusCode().is2xxSuccessful -> false
Code piece:
#Rule
#JvmField
var rule = WireMockRule(Options.DYNAMIC_PORT)
lateinit var uri: URI
Maven version: 3.8.3
Testing framework: jUnit5 (Jupiter)
Looks like this self-resolved by reinstalling Maven through brew. However, it also updated from 3.8.3 to 3.8.5 - so not entirely sure whether it was the upgrade or the reinstall. Regardless, it's fixed, yay!
I'm trying to run tests of a spring boot application when it writes to mongodb using the testcontainers test library. Testcontainers should spin up a Docker image running mongodb. Then I run my test, it connects to the data store, writes something and I have assertions that make sure the stuff got stored. Then it all goes away.
The test needs to run on a Jenkins build agent (on Red Hat Linux 7.5) inside our corporate network which is pretty well locked down.
We have Artifactory set up with a proxy to docker hub. When I normally do a docker login I give it https://artifactory.example.com or just do docker run with "artifactory.example.com/docker-all/image:1.2.3"
The log on the Jenkins run has this in it:
00:02:13.052 2019-05-22 00:15:59.647 INFO 83570 --- [ main] o.t.d.DockerClientProviderStrategy : Found Docker environment with Environment variables, system properties and defaults. Resolved:
00:02:13.052 dockerHost=unix:///var/run/docker.sock
00:02:13.052 apiVersion='{UNKNOWN_VERSION}'
00:02:13.052 registryUrl='https://index.docker.io/v1/'
00:02:13.052 registryUsername='cicduser'
00:02:13.052 registryPassword='null'
00:02:13.052 registryEmail='null'
00:02:13.052 dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=cicduser,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/home/cicduser/.docker,sslConfig=<null>,apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]'
Question: I don't know how to get the registryUrl listed there to be "https://artifactory.example.com/docker-all" and the registryUsername and registryPasswords set correctly (if our Artifactory gets locked down for reads).
There is lots of info to find online about using an HTTP proxy providing access to the internet at large. I think I have found how to do that. But that's not what I need to do.
It seems you can't change the URL
You can login to your artifactory docker hub remote in the same way in Jenkins, probably by using the credentials plugin - assuming you have Jenkinsfiles
Hopefully you can also get to your artifactory from your Jenkins docker agent. If not, then that is a separate problem
I'm new to SonarQube. I would like to implement SonarQube with Jenkins on my software factory. Before I started I read a artikel on the internet how to implement SonarQube with Jenkins (book DevOps).
See pictures below:
1. First, I created on SonarQube a Token on pasted it into Jenkins --> Server authentication token
Setup SonarQube Scanner
Build configuration on Jenkins Job.
When I rebuild my project on Jenkins, I got the follow Error: SonarQube server [URL] can not be reached
When I clicked on the URL message, my browser will redirected me to SonarQube page
Can anybody help me with my problem?
Kind regards
Michael
I think you should verify your firewall, on which operating system did you install SonarQube? In most systems, port 9000 is not open and it is a common source of failure.
I am running an embedded OrientDb. We're using Spring/Spring Data and we have JUnit tests running via Maven's Surefire plugin.
The embedded database gets started before every test. I'd like to somehow change this so that it would get loaded once per Maven module's execution, if possible.
I had the following:
OServerNetworkListenerConfiguration binaryListener = new OServerNetworkListenerConfiguration();
binaryListener.ipAddress = "0.0.0.0";
binaryListener.portRange = "2424-2430";
binaryListener.protocol = "binary";
binaryListener.socket = "default";
Obviously, due to the amount of tests, 6 ports are insufficient and I get this cryptic error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project bar-api: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd /java/foo/bar/bar-api && /java/jdk1.8.0_65/jre/bin/java -Xmx1024m -Xms512m -jar /java/foo/bar/bar-api/target/surefire/surefirebooter3262843936755308263.jar /java/foo/bar/bar-api/target/surefire/surefire2085279380429297504tmp /java/foo/bar/bar-api/target/surefire/surefire_53552629494142788284tmp
There is no System.exit() anywhere in my code, but the Maven Surefire plugin exits the build.
My questions are: why the need for so many ports? Can't it just use one? What am I missing out here and how to fix it? (For the time-being, I have simply increased the number of ports, as a temporary workaround, but I'd really like to get a better understanding of the issue and sort it out properly).
When a server instance starts it tries to bind the listener to the first unused port in that range. So you can't use the same port for two different server instances.
I'm running Docker 1.12.0 on a Windows 10 machine. I'm developing a Java program, using Maven 3.3.9 as a dependency manager. I have a maven docker plugin (https://github.com/fabric8io/docker-maven-plugin), which gives the following error on clean install.
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.15.16:build (docker-build-start) on project integration-test: Execution docker-build-start of goal io.fabric8:docker-maven-plugin:0.15.16:build failed: No <dockerHost> or <machine> given, no DOCKER_HOST environment variable, and no read/writable '/var/run/docker.sock' -> [Help 1]
When I run a clean install with the following configuration option in the POM file:
<dockerHost>tcp://0.0.0.0:2376</dockerHost>
the following result is shown.
[ERROR] Failed to execute goal io.fabric8:docker-maven-plugin:0.15.16:build (docker-build-start) on project integration-test: Cannot create docker access object: Cannot extract API version from server https://0.0.0.0:2376: Connect to 0.0.0.0:2376 [/0.0.0.0] failed: Connection refused: connect -> [Help 1]
My question: is there an IP address I can use to tell this maven plugin where it can reach the daemon? Normal docker commands work perfectly fine. The plugin works without any problems on OS X.
After nearly a day of fruitless Googling I found this solution by myself. Trivial, but might still help others.
You need to enable the checkbox 'Expose daemon on...' under Settings -> General
On Windows 10 with Docker for Windows, the Docker Engine API is available in these two locations:
npipe:////./pipe/docker_engine
http://localhost:2375
I recommend trying with the localhost one.
Details here: https://docs.docker.com/docker-for-windows/faqs/#/how-do-i-connect-to-the-remote-docker-engine-api
It seems the user which is running Maven goals doesn't have access to docker.sock . The error message is telling which options are there to resolve the problem.
No <dockerHost> or <machine> given, no DOCKER_HOST environment
variable, and no read/writable '/var/run/docker.sock'
Last option is the easiest one because it requires a file permission and it doesn't need to create any docker machine or set a DOCKER_HOST, On Linux you can change read/write permission of docker.sock with the following:
sudo chmod 776 /var/run/docker.sock
On windows go through this article : Microsoft article
I had the same issue when I tried to build a project in a custom GitLab CI/CD runner, with another than root user defined in a custom build image. I fixed by setting the read/write permission for users to the docker-socket.
chmod o+rw /var/run/docker.sock
If you are using Window and Maven in eclipse to build your java project
but continue seeing that error, then you have to perform these steps:
Step1:
You need to enable the checkbox 'Expose daemon on...' under Settings -> General
As mentioned by #Adriaan Koster
If step1 does not solve the issue,then
Step2: Run your eclipse in administration mode.
Now it should work without issues.
If someone just wants to skip fabric8 docker-maven-plugin execution that prevents build to succeed with error
No given, no DOCKER_HOST environment variable, no
read/writable '/var/run/docker.sock' or '//./pipe/docker_engine' and
no external provider like Docker machine configured
then this can be achieved with -Ddocker.skip=true according to https://dmp.fabric8.io/#global-configuration.
use docker-machine if you are using toolbox.
<machine>
<name>default</name>
<autoCreate>true</autoCreate>
<createOptions>
<driver>virtualbox</driver>
<virtualbox-cpu-count>2</virtualbox-cpu-count>
</createOptions>
</machine>