I have a cluster of 3 workers Spark. (worker-1, worker-2, worker-3) that runs with Spark 2.0.2.
The Spark Master is started on worker-1.
I submit my application with the following script :
#!/bin/bash
sparkMaster=spark://worker-1:6066
mainClass=my.package.Main
jar=/path/to/my/jar-with-dependencies.jar
driverPort=7079
blockPort=7082
deployMode=cluster
$SPARK_HOME/bin/spark-submit \
--conf "spark.driver.port=${driverPort}"\
--conf "spark.blockManager.port=${blockPort}"\
--class $mainClass \
--master $sparkMaster \
--deploy-mode $deployMode \
$jar
When my driver is started on the worker-1 (Worker + Master), everything is ok, and my application is correctly executed using all workers
But when my driver start on another worker (worker-2 or worker-3), he fails with error :
Launch Command: "/usr/java/jdk1.8.0_181-amd64/jre/bin/java" "-cp" "/root/spark-2.0.2-bin-hadoop2.7/conf/:/root/spark-2.0.2-bin-hadoop2.7/jars/*" "-Xmx1024M" "-Dspark.submit.deployMode=cluster" "-Dspark.app.name=my.package.Main" "-Dspark.driver.port=7083" "-Dspark.blockManager.port=7082" "-Dspark.master=spark://worker-1:7077" "-Dspark.jars=file:/path/to/my/jar-with-dependencies.jar" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker#worker-2:7078" "/data/spark/work/driver-20181001132624-0001/jar-with-dependencies.jar" "my.package.Main"
========================================
org.apache.spark.internal.Logging$class.logWarning(Logging.scala:66) | Service 'Driver' could not bind on port 0. Attempting port 1.
org.apache.spark.internal.Logging$class.logWarning(Logging.scala:66) | Service 'Driver' could not bind on port 0. Attempting port 1.
...
org.apache.spark.internal.Logging$class.logWarning(Logging.scala:66) | Service 'Driver' could not bind on port 0. Attempting port 1.
org.apache.spark.internal.Logging$class.logWarning(Logging.scala:66) | Service 'Driver' could not bind on port 0. Attempting port 1.
Exception in thread "main" java.net.BindException: Cannot assign requested address: Service 'Driver' failed after 16 retries! Consider explicitly setting the appropriate port for the service 'Driver' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:125)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:485)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1089)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:430)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:415)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:903)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:198)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:348)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:748)
My 3 workers are configured as follow :
SPARK_LOCAL_IP=worker-[X]
SPARK_LOCAL_DIRS=/data/spark/tmp
SPARK_WORKER_PORT=7078
SPARK_WORKER_DIR=/data/spark/work
SPARK_WORKER_OPTS="-Dspark.worker.cleanup.enabled=true -Dspark.worker.cleanup.appDataTtl=86400 -Dspark.worker.cleanup.interval=1800"
After multiple attempts to solve this problem, I tried to force the start of the driver on the master machine by adding to my submit the following option :
--conf "spark.driver.host=worker-1"
But the driver still start on a random worker, so it does not solve my problem.
Edit :
When I submit with the spark.driver.host option, the option does not appear in the Launch Command log (but the spark.driver.port appear, so I don't understand why my option is not taken this time).
Edit 2 :
I have done some deeper tests :
I now have only one worker running on worker-2, still submitting from worker-1 where my master is running.
When I submit my application, I can see on my worker logs :
2018-10-04 11:27:39,794 | dispatcher-event-loop-6 | INFO | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54) | Asked to launch driver driver-20181004112739-0003
2018-10-04 11:27:39,833 | DriverRunner for driver-20181004112739-0003 | INFO | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54) | Copying user jar file:/path/to/myjar-with-depencies.jar to /data/spark/work/driver-20181004112739-0003/myjar-with-depencies.jar
2018-10-04 11:27:39,833 | DriverRunner for driver-20181004112739-0003 | INFO | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54) | Copying /path/to/myjar-with-depencies.jar to /data/spark/work/driver-20181004112739-0003/myjar-with-depencies.jar
2018-10-04 11:27:40,243 | DriverRunner for driver-20181004112739-0003 | INFO | org.apache.spark.internal.Logging$class.logInfo(Logging.scala:54) | Launch Command: "/usr/java/jdk1.8.0_181-amd64/jre/bin/java" "-cp" "/root/spark-2.0.0-bin-hadoop2.7/conf/:/root/spark-2.0.0-bin-hadoop2.7/jars/*" "-Xmx1024M" "-Dspark.driver.supervise=false" "-Dspark.history.fs.cleaner.interval=12h" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://worker-1:7077" "-Dspark.history.fs.cleaner.maxAge=1d" "-Dspark.app.name=my.package.Main" "-Dspark.jars=file:/path/to/myjar-with-depencies.jar" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker#worker-2:7078" "/data/spark/work/driver-20181004112739-0003/myjar-with-depencies.jar" "my.package.Main"
2018-10-04 11:27:42,692 | dispatcher-event-loop-8 | WARN | org.apache.spark.internal.Logging$class.logWarning(Logging.scala:66) | Driver driver-20181004112739-0003 exited with failure
And I still have the same error in my driver logs.
I then tried to run manually the command that is launched by the DriverRunner :
"/usr/java/jdk1.8.0_181-amd64/jre/bin/java" "-cp" "/root/spark-2.0.0-bin-hadoop2.7/conf/:/root/spark-2.0.0-bin-hadoop2.7/jars/*" "-Xmx1024M" "-Dspark.driver.supervise=false" "-Dspark.history.fs.cleaner.interval=12h" "-Dspark.submit.deployMode=cluster" "-Dspark.master=spark://worker-1:7077" "-Dspark.history.fs.cleaner.maxAge=1d" "-Dspark.app.name=my.package.Main" "-Dspark.jars=file:/path/to/myjar-with-depencies.jar" "org.apache.spark.deploy.worker.DriverWrapper" "spark://Worker#worker-2:7078" "/data/spark/work/driver-20181004112739-0003/myjar-with-depencies.jar" "my.package.Main"
And when I do that, the application start correctly (surprisingly).
What is the difference between my manual start, and the one from the Driver-Runner that can cause my binding error ?
Note :
I have made no modification on the Driver-Runner command line to work
I manually launched my command line in root, and my spark runs in root too.
Had the same behavior on Spark 2.0.0 and Spark 2.0.2
So, I answer my own question as I found the reason of this weird behavior.
It does happend when I run the spark-submit from a machine where there is a spark-env.sh file. And more precisely, when the SPARK_LOCAL_IP is set in this machine.
To avoid this problem, I created a 4th machine, running only a Spark Master and with no spark-env.sh file and from which I run my spark submit.
Related
version 1.10 , Apache geode exampples of clientSecurity
when I build the project and execute the 'start' task, the GemFireSecurityException always occurs when start the server. even I can find the file "example_security.json" in the dir build/resources/main/.
and locator can find the file but server can't, why?
> Task :clientSecurity:start
1. Executing - start locator --name=locator --bind-address=127.0.0.1 --connect=false --security-properties-file=******** --classpath=../build/resources/main/
........
Locator in C:\Users\kenneth\Desktop\geode-examples-master\clientSecurity\locator on 127.0.0.1[10334] as locator is currently online.
2. Executing - start server --name=server1 --locators=127.0.0.1[10334] --classpath=../build/resources/main/:../build/classes/java/main/ --security-properties-file=******** --server-port=0 --user=superUser --password=********
...The Cache Server process terminated unexpectedly with exit status 1. Please refer to the log file in C:\Users\kenneth\Desktop\geode-examples-master\clientSecurity\server1 for full details.
Exception in thread "main" org.apache.geode.security.GemFireSecurityException: ExampleSecurityManager: unable to find json resource "example_security.json" as specified by [security-json].
at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:842)
at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:732)
at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:251)
************************* Execution Summary ***********************
Script file: C:\Users\kenneth\Desktop\geode-examples-master\clientSecurity\scripts\start.gfsh
Command-1 : start locator --name=locator --bind-address=127.0.0.1 --connect=false --security-properties-file=example_security.properties --classpath=../build/resources/main/
Status : PASSED
Command-2 : start server --name=server1 --locators=127.0.0.1[10334] --classpath=../build/resources/main/:../build/classes/java/main/ --security-properties-file=./example_security.properties --server-port=0 --user=superUser --password=123
Status : FAILED
I've just tried this myself locally and it worked just fine, below is the execution output:
user#localhost~/git/geode-examples ((rel/v1.10.0)): cd clientSecurity/
user#localhost~/git/geode-examples/clientSecurity ((rel/v1.10.0)): ../gradlew build
> Task :clientSecurity:compileJava
Note: /Users/user/git/geode-examples/clientSecurity/src/main/java/org/apache/geode_examples/clientSecurity/ExampleAuthInit.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
BUILD SUCCESSFUL in 17s
5 actionable tasks: 4 executed, 1 up-to-date
user#localhost~/git/geode-examples/clientSecurity ((rel/v1.10.0)): ../gradlew start
> Task :clientSecurity:start
1. Executing - start locator --name=locator --bind-address=127.0.0.1 --connect=false --security-properties-file=******** --classpath=../build/resources/main/
......
Locator in /Users/user/git/geode-examples/clientSecurity/locator on 127.0.0.1[10334] as locator is currently online.
Process ID: 3103
Uptime: 8 seconds
Geode Version: 1.10.0
Java Version: 1.8.0_221
Log File: /Users/user/git/geode-examples/clientSecurity/locator/locator.log
JVM Arguments: -DgemfireSecurityPropertyFile=/Users/user/git/geode-examples/clientSecurity/example_security.properties -Dgemfire.enable-cluster-configuration=true -Dgemfire.load-cluster-configuration-from-dir=false -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/user/git/geode-examples/build/apache-geode-1.10.0/lib/geode-core-1.10.0.jar:../build/resources/main/:/Users/user/git/geode-examples/build/apache-geode-1.10.0/lib/geode-dependencies.jar
2. Executing - start server --name=server1 --locators=127.0.0.1[10334] --classpath=../build/resources/main/:../build/classes/java/main/ --security-properties-file=******** --server-port=0 --user=superUser --password=********
...==========-> 94% EXECUTING [11s]
Server in /Users/user/git/geode-examples/clientSecurity/server1 on 10.255.203.195[50649] as server1 is currently online.
Process ID: 3119
Uptime: 3 seconds
Geode Version: 1.10.0
Java Version: 1.8.0_221
Log File: /Users/user/git/geode-examples/clientSecurity/server1/server1.log
JVM Arguments: -DgemfireSecurityPropertyFile=/Users/user/git/geode-examples/clientSecurity/./example_security.properties -Dgemfire.locators=127.0.0.1[10334] -Dgemfire.security-username=superUser -Dgemfire.start-dev-rest-api=false -Dgemfire.security-password=******** -Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/user/git/geode-examples/build/apache-geode-1.10.0/lib/geode-core-1.10.0.jar:../build/resources/main/:../build/classes/java/main/:/Users/user/git/geode-examples/build/apache-geode-1.10.0/lib/geode-dependencies.jar
3. Executing - start server --name=server2 --locators=127.0.0.1[10334] --classpath=../build/resources/main/:../build/classes/java/main/ --security-properties-file=******** --server-port=0 --user=superUser --password=********
...
Server in /Users/user/git/geode-examples/clientSecurity/server2 on 10.255.203.195[50674] as server2 is currently online.
Process ID: 3120
Uptime: 3 seconds
Geode Version: 1.10.0
Java Version: 1.8.0_221
Log File: /Users/user/git/geode-examples/clientSecurity/server2/server2.log
JVM Arguments: -DgemfireSecurityPropertyFile=/Users/user/git/geode-examples/clientSecurity/./example_security.properties -Dgemfire.locators=127.0.0.1[10334] -Dgemfire.security-username=superUser -Dgemfire.start-dev-rest-api=false -Dgemfire.security-password=******** -Dgemfire.use-cluster-configuration=true -XX:OnOutOfMemoryError=kill -KILL %p -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: /Users/user/git/geode-examples/build/apache-geode-1.10.0/lib/geode-core-1.10.0.jar:../build/resources/main/:../build/classes/java/main/:/Users/user/git/geode-examples/build/apache-geode-1.10.0/lib/geode-dependencies.jar
4. Executing - connect --user=superUser --password=******** --use-ssl=true --key-store=keystore.jks --key-store-password=******** --trust-store=truststore.jks --trust-store-password=********
Connecting to Locator at [host=localhost, port=10334] ..
Connecting to Manager at [host=10.255.203.195, port=1099] ..
Successfully connected to: [host=10.255.203.195, port=1099]
5. Executing - create region --name=region1 --type=REPLICATE
Member | Status | Message
------- | ------ | --------------------------------------
server1 | OK | Region "/region1" created on "server1"
server2 | OK | Region "/region1" created on "server2"
Cluster configuration for group 'cluster' is updated.
6. Executing - create region --name=region2 --type=PARTITION
Member | Status | Message
------- | ------ | --------------------------------------
server1 | OK | Region "/region2" created on "server1"
server2 | OK | Region "/region2" created on "server2"
Cluster configuration for group 'cluster' is updated.
************************* Execution Summary ***********************
Script file: /Users/user/git/geode-examples/clientSecurity/scripts/start.gfsh
Command-1 : start locator --name=locator --bind-address=127.0.0.1 --connect=false --security-properties-file=example_security.properties --classpath=../build/resources/main/
Status : PASSED
Command-2 : start server --name=server1 --locators=127.0.0.1[10334] --classpath=../build/resources/main/:../build/classes/java/main/ --security-properties-file=./example_security.properties --server-port=0 --user=superUser --password=123
Status : PASSED
Command-3 : start server --name=server2 --locators=127.0.0.1[10334] --classpath=../build/resources/main/:../build/classes/java/main/ --security-properties-file=./example_security.properties --server-port=0 --user=superUser --password=123
Status : PASSED
Command-4 : connect --user=superUser --password=123 --use-ssl=true --key-store=keystore.jks --key-store-password=password --trust-store=truststore.jks --trust-store-password=password
Status : PASSED
Command-5 : create region --name=region1 --type=REPLICATE
Status : PASSED
Command-6 : create region --name=region2 --type=PARTITION
Status : PASSED
BUILD SUCCESSFUL in 28s
8 actionable tasks: 2 executed, 6 up-to-date
user#localhost~/git/geode-examples/clientSecurity ((rel/v1.10.0)): ../gradlew stop
> Task :clientSecurity:stop
1. Executing - connect --locator=127.0.0.1[10334] --user=superUser --password=******** --use-ssl=true --key-store=./keystore.jks --key-store-password=******** --trust-store=./truststore.jks --trust-store-password=********
Connecting to Locator at [host=127.0.0.1, port=10334] ..
Connecting to Manager at [host=10.255.203.195, port=1099] ..
Successfully connected to: [host=10.255.203.195, port=1099]
2. Executing - shutdown --include-locators=true
Shutdown is triggered
************************* Execution Summary ***********************
Script file: /Users/user/git/geode-examples/clientSecurity/scripts/stop.gfsh
Command-1 : connect --locator=127.0.0.1[10334] --user=superUser --password=123 --use-ssl=true --key-store=./keystore.jks --key-store-password=password --trust-store=./truststore.jks --trust-store-password=password
Status : PASSED
Command-2 : shutdown --include-locators=true
Status : PASSED
BUILD SUCCESSFUL in 3s
2 actionable tasks: 1 executed, 1 up-to-date
user#localhost~/git/geode-examples/clientSecurity ((rel/v1.10.0)):
I've tried on MacOS and I've noticed you're using Windows instead, maybe the problem is caused by the path separator used within the start.gfsh script?.
Can you change the scripts under geode-examples\clientSecurity\scripts to use full paths and give it a try?.
I changed to full path of locator and server, here is all output:
PS C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security> C:\Users\hw83770\Documents\pivotal-gemfire-9.8.0\bin\gfsh.bat run --file=.\scripts\start.gfsh
1. Executing - start locator --name=clocator --bind-address=127.0.0.1 --connect=false --security-properties-file=******** --classpath=C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\resources\main
......
Locator in C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\clocator on 127.0.0.1[10334] as clocator is currently online.
Process ID: 28816
Uptime: 7 seconds
Geode Version: 9.8.0
Java Version: 1.8.0_161
Log File: C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\clocator\clocator.log
JVM Arguments: -DgemfireSecurityPropertyFile=C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\example_security.properties -Dgemfire.enable-cluster-configuration=true -Dgemfire.load-cluster-configuration-from-dir=false -Dgemfire.launcher.registerSignalHandlers=true -Djava.awt.headless=true -Dsun.rmi.dgc.server.gcInterval=9223372036854775806
Class-Path: C:\Users\hw83770\Documents\pivotal-gemfire-9.8.0\lib\geode-core-9.8.0.jar;C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\resources\main;C:\Users\hw83770\Documents\pivotal-gemfire-9.8.0\lib\geode-dependencies.jar;C:\Users\hw83770\Documents\pivotal-gemfire-9.8.0\extensions\gemfire-greenplum-3.4.1.jar
2. Executing - start server --name=cserver1 --locators=127.0.0.1[10334] --classpath=C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\resources\main:C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\classes\java\main --security-properties-file=******** --server-port=0 --user=superUser --password=********
...The Cache Server process terminated unexpectedly with exit status 1. Please refer to the log file in C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\cserver1 for full details.
Exception in thread "main" org.apache.geode.security.GemFireSecurityException: ExampleSecurityManager: unable to find json resource "example_security.json" as specified by [security-json].
at org.apache.geode.distributed.ServerLauncher.start(ServerLauncher.java:824)
at org.apache.geode.distributed.ServerLauncher.run(ServerLauncher.java:716)
at org.apache.geode.distributed.ServerLauncher.main(ServerLauncher.java:236)
************************* Execution Summary ***********************
Script file: .\scripts\start.gfsh
Command-1 : start locator --name=clocator --bind-address=127.0.0.1 --connect=false --security-properties-file=example_security.properties --classpath=C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\resources\main
Status : PASSED
Command-2 : start server --name=cserver1 --locators=127.0.0.1[10334] --classpath=C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\resources\main:C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\build\classes\java\main --security-properties-file=./example_security.properties --server-port=0 --user=superUser --password=123
Status : FAILED
besides, it says
1. Please refer to the log file in C:\Users\hw83770\git\frameworkpoc\rio-geode-cli\client-security\cserver1 for full details
Actually, there is no any log file, and I'm not familiar with the geode source code so dont know how to deal with this.
I'm working the POC of geode, our team need to ensure the geode supports security of client and end points, it's very important, so I'm here for some help.
I'm still convinced the problem is caused by a problem within your environment, specially the classpath. As you can see here, the start.gfsh script sets the member's classpath to contain ../build/resources/main/, exactly the folder under which the example_security.json file should be located after building the project with Gradle.
I've just noticed that, at the very start of your code snippet, you have
C:\Users\hw83770\Documents\pivotal-gemfire-9.8.0\bin\gfsh.bat run --file=.\scripts\start.gfsh... why is that?, according to the instructions you should execute $ ../gradlew start under the clientSecurity directory instead. Using C:\Users\hw83770\Documents\pivotal-gemfire-9.8.0\bin\gfsh.bat run --file=.\scripts\start.gfsh is basically changing the folder from which the script is executed and, thus, ../build/resources/main/ doesn't point to what it should anymore, this is probably the reason why the example fails.
Last, but not least, you must not mix Pivotal GemFire with Apache Geode, things will probably not work as expected.
I am facing trouble to start the SonarQube Server as either a service or on its own.
(All commands being run as Administrator on Windows 10, with 64-bit JDK and JRE installled)
C:\sonarqube\bin\windows-x86-64>StartSonar.bat
wrapper | --> Wrapper Started as Console
wrapper | Launching a JVM...
jvm 1 | Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
jvm 1 | Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
jvm 1 |
jvm 1 | 2018.01.09 10:05:37 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory C:\sonarqube\temp
jvm 1 | 2018.01.09 10:05:37 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
jvm 1 | 2018.01.09 10:05:37 INFO app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [C:\sonarqube\elasticsearch]: C:\Program Files\jre\bin\java -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dlog4j.skipJansi=true -Xms512m -Xmx512m -XX:+HeapDumpOnOutOfMemoryError -Delasticsearch -Des.path.home=C:\sonarqube\elasticsearch -cp lib/* org.elasticsearch.bootstrap.Elasticsearch -Epath.conf=C:\sonarqube\temp\conf\es
jvm 1 | 2018.01.09 10:05:37 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
jvm 1 | 2018.01.09 10:05:38 INFO app[][o.e.p.PluginsService] no modules loaded
jvm 1 | 2018.01.09 10:05:38 INFO app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
jvm 1 | 2018.01.09 10:05:39 WARN app[][o.s.a.p.AbstractProcessMonitor] Process exited with exit value [es]: 1
jvm 1 | 2018.01.09 10:05:39 INFO app[][o.s.a.SchedulerImpl] Process [es] is stopped
jvm 1 | 2018.01.09 10:05:39 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped
wrapper | <-- Wrapper Stopped
or
C:\sonarqube\bin\windows-x86-64>InstallNTService.bat
wrapper | SonarQube installed.
C:\sonarqube\bin\windows-x86-64>StartNTService.bat
wrapper | Starting the SonarQube service...
wrapper | Waiting to start...
wrapper | The SonarQube service was launched, but failed to start.
Press any key to continue . . .
I assume the error is the same for both of the cases.
Ok, so I looked into the logs, and saw this in sonarqube\logs\es: java.lang.IllegalStateException: failed to obtain node locks
I figured that was the issue, so I went to Task Manager and ended the Java task that was running. Then I was able to run StartNTService.bat. Not sure what the actual issue was, maybe the process didn't end correctly the first time I started it and it failed.
C:\sonarqube-8.9.3.48735\sonarqube-8.9.3.48735\bin\windows-x86-64\StartSonar.bat
I tried with the StartNTservice.bat file so many times then I ran the StartSonar.bat file in the same location.It started the sonarqube service in my windows 10 system
I'm setting teamcity agent using my account and there are no admin rights, so I got error
Current Java runtime version is 1.7
INFO | jvm 1 | 2015/08/21 16:03:57 | Starting new process with command line ('|' arguments separator): C:\QA\volAgent\jre\bin\java|-classpath|C:\QA\volAgent\update\launcher\lib\launcher.jar|jetbrains.buildServer.agent.upgrade.Upgrade|NT_SERVICE|C:\QA\volAgent|-ea|-Xmx512m|-XX:+HeapDumpOnOutOfMemoryError|-Xrs|-Dlog4j.configuration=file:../conf/teamcity-agent-log4j.xml|-Dteamcity_logs=../logs/|jetbrains.buildServer.agent.AgentMain|-file|../conf/buildAgent.properties| in directory: C:\QA\volAgent
INFO | jvm 1 | 2015/08/21 16:03:57 | Launcher is waiting to be killed by upgrade process
FATAL | wrapper | 2015/08/21 16:04:01 | OpenSCManager failed - Access is denied. (0x5)
FATAL | wrapper | 2015/08/21 16:04:01 | OpenSCManager failed - Access is denied. (0x5)
Error occurs because my local user does not have required permission, but I have admin console opened, and all I need is to run script above manually in admin console. I tried many cases but it doesn't work, can anybody help?
I am trying to deploy our designer using the Tooltwist Controller and I keep receiving the following error:
+------------------------------------------------------------------------------------------+
| |
| GENERATION PHASE |
| |
+------------------------------------------------------------------------------------------+
...
**
** Check the server is running
**
Setting JAVA_OPTS=-Xms512m -Xmx5g -XX:MaxPermSize=512m
Starting the launchpad...
$ ./startup.sh
Wait a bit...
-
Error with http request: Connection reset by peer
==>> Status is error - Connection reset by peer
==>> Status is down
**
** Fatal error: Could not start the launchpad.
**
Finished: SUCCESS
I have tried changing the tomcat version in the payloads using both Tomcat 7.0.54 and Tomcat 7.0.40 but the issue persists
It appears that the launchpad server is not starting correctly. There are many reasons why Tomcat might not start, so the best first step is to look at the Tomcat log file, which will be located somewhere like /ControllerV8/launchpads/<launchpad-name>/image/tomcat/logs/catalina.out.
One possibility could be that another launchpad's server is already running using the same launchpad ports.
I have the commons-daemon-x.jar in the classpath. The Jsvc is successfully launching the Java VM. But jsvc is reporting that it cannot find the daemon loader class even though it is in the classpath.
Does anyone know how to solve this problem? I’m running Mac OS X 10.8.3. Thanks.
Here's an excerpt of the jsvc debug output:
...
+-- DUMPING JAVA VM CREATION ARGUMENTS -----------------
| Version: 0x010004
| Ignore Unrecognized Arguments: False
| Extra options: 1
| "-Djava.class.path=commons-daemon-1.0.15.jar:./api-monitor.jar" (0x00000000)
+———————————————————————————
...
Java VM created successfully
Cannot find daemon loader org/apache/commons/daemon/support/DaemonLoader
java_init failed
Service exit with a return value of 1
...
Here’s the full debug output if it helps:
sudo jsvc -jvm server -debug -cp commons-daemon-1.0.15.jar:./api-monitor.jar ApiMonitorDaemon
+-- DUMPING PARSED COMMAND LINE ARGUMENTS --------------
| Detach: True
| Show Version: No
| Show Help: No
| Check Only: Disabled
| Stop: False
| Wait: 0
| Run as service: No
| Install service: No
| Remove service: No
| JVM Name: "server"
| Java Home: "null"
| PID File: "/var/run/jsvc.pid"
| User Name: "null"
| Extra Options: 1
| "-Djava.class.path=commons-daemon-1.0.15.jar:./api-monitor.jar"
| Class Invoked: "ApiMonitorDaemon"
| Class Arguments: 0
+-------------------------------------------------------
Home not specified on command line, using environment
Home not on command line or in environment, searching
Attempting to locate Java Home in /System/Library/Frameworks/JavaVM.framework/Home
Attempting to locate VM configuration file /System/Library/Frameworks/JavaVM.framework/Home/jre/lib/jvm.cfg
Attempting to locate VM configuration file /System/Library/Frameworks/JavaVM.framework/Home/lib/jvm.cfg
Found VM configuration file at /System/Library/Frameworks/JavaVM.framework/Home/lib/jvm.cfg
Found VM client definition in configuration
Checking library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libclient.dylib
Found VM jvm definition in configuration
Checking library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libjvm.dylib
Found VM hotspot definition in configuration
Checking library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libhotspot.dylib
Found VM server definition in configuration
Checking library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libserver.dylib
Found VM classic definition in configuration
Checking library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libclassic.dylib
Cannot locate library for VM classic (skipping)
Java Home located in /System/Library/Frameworks/JavaVM.framework/Home
+-- DUMPING JAVA HOME STRUCTURE ------------------------
| Java Home: "/System/Library/Frameworks/JavaVM.framework/Home"
| Java VM Config.: "/System/Library/Frameworks/JavaVM.framework/Home/lib/jvm.cfg"
| Found JVMs: 4
| JVM Name: "client"
| "/System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libclient.dylib"
| JVM Name: "jvm"
| "/System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libjvm.dylib"
| JVM Name: "hotspot"
| "/System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libhotspot.dylib"
| JVM Name: "server"
| "/System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libserver.dylib"
+-------------------------------------------------------
my-server:Applications developer$ redirecting stdout to /dev/null and stderr to /dev/null
Switching umask back to 022 from 077
Using specific JVM in /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libserver.dylib
Attemtping to load library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libserver.dylib
JVM library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libserver.dylib loaded
Attemtping to load library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libverify.dylib
Shell library /System/Library/Frameworks/JavaVM.framework/Home/../Libraries/libverify.dylib loaded
JVM library entry point found (0x02A49762)
+-- DUMPING JAVA VM CREATION ARGUMENTS -----------------
| Version: 0x010004
| Ignore Unrecognized Arguments: False
| Extra options: 1
| "-Djava.class.path=commons-daemon-1.0.15.jar:./api-monitor.jar" (0x00000000)
+-------------------------------------------------------
| Internal options: 4
| "-Dcommons.daemon.process.id=8919" (0x00000000)
| "-Dcommons.daemon.process.parent=8918" (0x00000000)
| "-Dcommons.daemon.version=1.0.15-dev" (0x00000000)
| "abort" (0x022bef30)
+-------------------------------------------------------
Java VM created successfully
Cannot find daemon loader org/apache/commons/daemon/support/DaemonLoader
java_init failed
Service exit with a return value of 1
You probably found the solution by now, but just in case someone stumble over this in the future, what I had missed in mine is that current directory defaults to '/', so you have to specify it if you want to use relative paths.
sudo jsvc -jvm server -cwd /path/to/my/daemon/ -cp commons-daemon-1.0.15.jar:./api-monitor.jar ApiMonitorDaemon
Note: my OS is ubuntu 12.10
Check the permissions of the commons-daemon-1.0.15.jar file, make sure your current user can read it.