I’m currently seeing an issue with my Java 8/Kotlin unit tests while executing them with Maven. This started happening after upgrading to MacOS Monterey. The tests run fine through IntelliJ.
One thing the tests have in common is, that they open ports/start a service.
In one instance, I’m trying to start a Redis test server on localhost and in a different one I’m opening a random port on localhost (via a wire mock rule).
Has anybody seen this issue before, it seems to solely affect Monterey - I assume I have to somehow grant Maven more permissions?!
Any help is appreciated.
Edit: Here some more info.
Error message (using redis.embedded.RedisServer):
[ERROR] Errors:
[ERROR] RedisCacheServiceTest » RedisConnection Unable to connect to Redis server: 127...
Code piece for this:
redisServer = RedisServer(REDIS_PORT)
redisServer.start()
val config = Config().apply {
useSingleServer().address = "redis://127.0.0.1:$REDIS_PORT"
}
redissonClient = Redisson.create(config)
and a failed assertion on the other (runs fine on other MacOS versions):
response.statusCode().is2xxSuccessful -> false
Code piece:
#Rule
#JvmField
var rule = WireMockRule(Options.DYNAMIC_PORT)
lateinit var uri: URI
Maven version: 3.8.3
Testing framework: jUnit5 (Jupiter)
Looks like this self-resolved by reinstalling Maven through brew. However, it also updated from 3.8.3 to 3.8.5 - so not entirely sure whether it was the upgrade or the reinstall. Regardless, it's fixed, yay!
I have been trying to solve this issue for 5 hours
when I try to execute the below code in the terminal it works fine. However, when I use the same code in flask application I am getting this error (sh: 1: java: not found)
import os
environ = os.environ.copy()
os.putenv("JAVA_HOME", "/usr/lib/jvm/java-14-oracle/bin/java")
os.environ["PATH"] += os.pathsep + ":/usr/lib/jvm/java-14-oracle/bin/java/bin"
os.system("java -jar /home/../myproject/application/graphseg1.jar /home/../myproject/application/entire_text /home/.../myproject/application/segmented_text 0.40 2")
the above code works fine within the terminal as shown below:
However, when I try to run the same code within flask application, I am getting the following error:
I am using Ubuntu 18.04, nginx,supervisor,gunicorn3
supervisor configration file:
[program:flask_app]
directory=/home/****/myproject
environment=PATH=/tmp/enter/envs/myenvi/bin
command=/tmp/enter/envs/myenvi/bin/gunicorn application:app --timeout 9223372036
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
stderr_logfile=/var/log/flask_app/flask_app.err.log
stdout_logfile=/var/log/flask_app/flask_app.out.log
You should probably using Subprocess instead. It provides pipes for STDIN, STOUT, STDERR and the option for a full shell with some inherent security warnings.
See: https://docs.python.org/2/library/subprocess.html#module-subprocess
Seems like your path is not setup properly, like omajd said can you print it?
See also: Problems adding path and calling external program from Python
Trying to run spark job with masterURL=yarn-client. Using SparkLauncher 2.10. The java code is wrapped in nifi processor. Nifi is currently running as root. When I do yarn application -list, I see the spark job started with USER = root. I want to run it with USER = hive.
Following is my SparkLauncher code.
Process spark = new SparkLauncher()
.setSparkHome(cp.fetchProperty(GlobalConstant.spark_submit_work_dir).toString())
.setAppResource(cp.fetchProperty(GlobalConstant.spark_app_resource))
.setMainClass(cp.fetchProperty(GlobalConstant.spark_main_class))
.addAppArgs(ps.getName())
// .setConf(SparkLauncher.DRIVER_EXTRA_JAVA_OPTIONS,"-Duser.name=hive")
.setConf(SparkLauncher.DRIVER_EXTRA_JAVA_OPTIONS, "-Dlog4j.configuration=file:///opt/eim/log4j_submitgnrfromhdfs.properties")
.setVerbose(true)
.launch();
Do I need to pass user as driver extra options? Environment is non-kerberos.
Read somewhere that I need to pass user name as driver extra java option. Cannot find that post now!!
export HADOOP_USER_NAME=hive worked. SparkLauncher has overload to accept Map of environment variables. As for spark.yarn.principle, the environment is non-kerberos. As per my reading yarn.principle works only with kerboros. Did the following
Process spark = new SparkLauncher(getEnvironmentVar(ps.getRunAs()))
.setSparkHome(cp.fetchProperty(GlobalConstant.spark_submit_work_dir).toString())
.setAppResource(cp.fetchProperty(GlobalConstant.spark_app_resource))
.setMainClass(cp.fetchProperty(GlobalConstant.spark_main_class))
.addAppArgs(ps.getName())
// .setConf(SparkLauncher.DRIVER_EXTRA_JAVA_OPTIONS,"-Duser.name=hive")
.setConf(SparkLauncher.DRIVER_EXTRA_JAVA_OPTIONS, "-Dlog4j.configuration=file:///opt/eim/log4j_submitgnrfromhdfs.properties")
.setVerbose(true)
.launch();
Instead of new SparkLancher() used SparkLauncher(java.util.Map<String,String> env).Added or replacedHADOOP_USER_NAME=hive.
Checked yarn application -listlaunches as intended withUSER=hive.
Has anyone out in the community successfully created a Selenium build in Jenkins using Browserstack as their cloud provider, while requiring a local testing connection behind a firewall?
I can say for sure Saucelabs is surprisingly easy to execute builds with the Sauce Jenkins plugin in a continuous deployment environment as I have done it. I cannot however, say the same for Browserstack. The organization I work with currently uses Browserstack, and although their service does support automated testing using a binary application I find it troublesome with Jenkins. I need to make absolutely sure Browserstack is not a viable solution, if so. I love Saucelabs and what their organization provides, but if Browserstack works I don't want to switch if I don't need to.
The Browserstack documentation instructs you to run a command, with some available options, in order to create a local connection before execution.
nohup ./[binary file] -localIdentifier [id] [auth key] localhost,3000,0 &
I have added the above statement as a pre-build step shell command. I have to also add 'nohup' as once the binary creates a successful connection, the build never actually starts since I have not exited as displayed in the output below.
BrowserStackLocal v3.5
You can now access your local server(s) in our remote browser.
Press Ctrl-C to exit
Normally I can successfully execute the first build without a problem. Subsequent build configurations using the same command never connect. The above message displays, but during test execution Browserstack reports no local testing connection was established. This confuses me.
To give you a better idea of what's being executed, I have 15 build configurations for various projects suites and browser combinations. Two Jenkins executors exist and I have more than 5 Browserstack VM's available at any given time. Five of the builds will automatically begin execution when the associated project code is pushed to the staging server, filling up both executors. One of them will begins and end fine. None of the others will as Browserstack reports local testing is not available.
Saucelabs obviously has this figured out with their plugin, which is great. If Browserstack requires shell commands to create local testing connections, I must be doing something wrong, out of order, etc.
Environment:
Java 7
Selenium 2.45
JUnit 4.11
Maven 3.1.1
Allure 1.4.10
Jenkins 1.5
Can someone post some information who use Browserstack in a continuous testing environment while utilizing multiple parallel test executions and tell me how each build is configured?
Thanks,
I've recently looked into BrowserStack with Selenium and the BrowserStack Plugin has made this task much easier.
Features
Manage your BrowserStack credentials globally or per build job.
Set up and tear down BrowserStack Local for testing internal, dev or
staging environments.
Embed BrowserStack Automate reports in your
Jenkins job results.
Much easier integration all round.
This is Umang replying on behalf of BrowserStack.
To start with, you are using the correct command for setting up the Local Testing connection. Although you do not need to specify the ‘localhost,3000,0’ details. We would also suggest you use the “-forcelocal” parameter while initiating the connection. The command should be as follows:
nohup ./[binary file] [auth key] -localIdentifier [id] -forcelocal &
The parameter “-forcelocal” will route all traffic via your IP address. Also, the process to initiate the connection before running your tests is correct.
However, here I’d like on confirm on the “id” you’ve specified while creating the connection. As you shared, there are 15 build configurations and I understand that each build has a different “id” specified. Please make sure that “id” specified while setting up the Local Testing connection and in the tests (“browserstack.localIdentifier” = “id”) is the same. Else, you will receive the error “[browserstack.local] is set to true but local testing through BrowserStack is not connected”
Integrating BrowserStack with Jenkins is a little bit tricky, but don't worry, it's perfectly doable :-)
The BrowserStackLocal client needs to be started as a background process, as per Umang's suggestion, and that's pretty much how the SauceLabs plugin works as well.
The trouble is that when Jenkins sees that you start daemon processes all by yourself and not via a plugin, it kills them. That's why you need to convince it otherwise.
I've described how to go about it step by step in this article, but if you're using the Pipeline Plugin, you can use the below script as a starting point:
node {
with_browser_stack 'linux-x64', {
// Execute tests: here's where a step like
// sh 'mvn clean verify'
// would go
}
}
// ----------------------------------------------
def with_browser_stack(type, actions) {
// Prepare the BrowserStackLocal client
if (! fileExists("/var/tmp/BrowserStackLocal")) {
sh "curl -sS https://www.browserstack.com/browserstack-local/BrowserStackLocal-${type}.zip > /var/tmp/BrowserStackLocal.zip"
sh "unzip -o /var/tmp/BrowserStackLocal.zip -d /var/tmp"
sh "chmod +x /var/tmp/BrowserStackLocal"
}
// Start the connection
sh "BUILD_ID=dontKillMe nohup /var/tmp/BrowserStackLocal 42MyAcc3sK3yV4lu3 -onlyAutomate > /var/tmp/browserstack.log 2>&1 & echo \$! > /var/tmp/browserstack.pid"
try {
// Execute tests
actions()
}
finally {
// Stop the connection
sh "kill `cat /var/tmp/browserstack.pid`"
}
}
You'd of course need to replace the fake access key (42MyAcc3sK3yV4lu3) with yours, or provide it via an environmental variable.
The important part here is the BUILD_ID, because that's what the Jenkins ProcessTreeKiller looks for when it decides whether to kill your daemon process or not.
Hope this helps!
Jan
Paul Whelan's answer to use the Jenkin's BrowserStack Plugin is currently the simplest way to integrate Jenkins with BrowserStack. The plugin supports all Jenkins versions >1.580.1.
To ensure that you get BrowserStack test reports you will need to configure your project's pom.xml as documented on the plugin wiki.
Just in case anyone else was having problems with this
For BrowserStackLocal v4.8 I found that -localidentifier has been removed from the binary options. (This is probably old news!)
When I removed the capabilities['browserstack.localIdentifier'] property from our automated tests the connection started working.
local binary
browserstack <key> -v -forcelocal
selenium setup
Capybara.register_driver :browserstack do |app|
capabilities = Selenium::WebDriver::Remote::Capabilities.new
# If we're running the BrowserStackLocal binary, we need to
# tell the driver as well
capabilities['browserstack.local'] = true
# other useful options
capabilities['browserstack.debug'] = true
capabilities['browserstack.javascriptEnabled'] = true
capabilities['javascriptEnabled'] = true
# etc ...
I've got a bunch of selenium test cases set up in a JUnit class as four methods. The first runs fine - but the remaining three close the Firefox browser before the final step of the method is complete - giving a
ERROR: Command execution failure. Please search the forum at http://clearspace.openqa.org for error details from the log window. The error message is: this.page().currentDocument is undefined
com.thoughtworks.selenium.SeleniumException: ERROR: Command execution failure. Please search the forum at http://clearspace.openqa.org for error details from the log window. The error message is: this.page().currentDocument is undefined
Setup is following:
Ant based execution
Java 1.5
Eclipse 3.5
Selenium Server 1.0.1
Selenium Client Driver 1.0.1
IDE
Java 1.5
Eclipse 3.5
Selenium Server 1.0.1
Selenium Client Driver 1.0.1
I'm hoping to track down the root cause
It is crashing on the following line:
This is the modification to the user-extensions.js file:
Selenium.prototype.getElementBody = function(elementId) {
return this.outerHTML(this.page().currentDocument.getElementById(elementId));
};
This is what is being called:
commandProcessor.getString("getElementBody", "idOfElement");
I have never seen a command called getString part of the Selenium API so think that is why your tests are failing.
The other thing is have you told Selenium RC to use the user-extensions.js file that you have modified with the following -userExtensions user-extensions.js argument? I have never found it beneficial to use user-extensions in Se:RC and instead use .getEval() commands.
what you want would be selenium.getEval("this.browserbot.getUserWindow().getElementByID(id).outerHTML");
I have cleaned up your JavaScript for you. The this.browserbot.getUserWindow() gives you access to the Selenium JavaScript object which has the page in it.
I hope that helps
Can you show us your JUnit test class ?
I know from personnal experience that once in a while Selenium tests can crash for no apparent reason, but your problem seems to be happening each time so it might have something to do with your code/script.
Afraid I can't help you without some more details.