I recently set up a RHEL7 slave node in a dev Jenkins environment which simply ran a few Unix specific scripts via SSH.
The time has come to essentially replicate this yet in my companies Production environment (using a different slave node (Ubuntu 16.04), I have replicated the slave node configuration from the Dev jenkins and been working through the errors produced by the log, however I've been stuck at this stage for awhile and can't find any documentation which is relevant to my situation.
I am getting the following error:
[02/15/17 10:47:11] [SSH] Starting slave process: cd "/home/jenkins" && java -jar slave.jar
Exception in thread "main" java.lang.RuntimeException: Root directory not writable
at hudson.remoting.FileSystemJarCache.<init>(FileSystemJarCache.java:44)
at hudson.remoting.Launcher.runWithStdinStdout(Launcher.java:477)
at hudson.remoting.Launcher.run(Launcher.java:251)
at hudson.remoting.Launcher.main(Launcher.java:201)
hudson.util.IOException2: Slave JVM has terminated. Exit code=1
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:984)
at hudson.plugins.sshslaves.SSHLauncher.access$400(SSHLauncher.java:137)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:725)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:706)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.EOFException: unexpected stream termination
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:365)
at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:310)
at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:389)
at hudson.plugins.sshslaves.SSHLauncher.startSlave(SSHLauncher.java:976)
... 7 more
[02/15/17 10:47:11] Launch failed - cleaning up connection
[02/15/17 10:47:11] [SSH] Connection closed.
I have double, triple checked the permissions of the root directory (home/jenkins/jenkins_slave) and they're fine. If I attempt to manually run java -jar slave.jar on the slave node I get the following error:
WARNING: Are you running slave agent from an interactive console?
If so, you are probably using it incorrectly.
See http://wiki.jenkins ci.org/display/JENKINS/Launching+slave.jar+from+from+console
<===[JENKINS REMOTING CAPACITY]===>
I just don't understand, because the permissions and config are identical so it must be something environmental? Any help/guidance would be appreciated.
Your home is not /home/jenkins/jenkins_slave but home/jenkins - at least according to the log output you provided (Starting slave process: cd "/home/jenkins" [...]). So you should change your startup configuration of the slave and cd into /home/jenkins/jenkins_slave
Related
Using GOCD pipeline. We have two nodes, one of which clones code fine, this older node does not. We are seeing Cannot run program "git": error=11, Resource temporarily unavailable in the logs. Is there some configuration I am able to check?
Logs:
020-08-25 17:39:18,629 [loopThread] ERROR thoughtworks.go.work.DefaultGoPublisher:135 - Error while executing [git clone --branch=20.08_OMS_15.09_Endpoint git#github.comcast.com:TPP-Provisioning/PortOutGui.git /app/go-agent1/pipelines/PATGUI-build]
Make sure this command can execute manually.
java.lang.RuntimeException: Error while executing [git clone --branch=20.08_OMS_15.09_Endpoint git#example.com:TPP-Provisioning/PortOutGui.git /app/go-agent1/pipelines/PATGUI-build]
Make sure this command can execute manually.
at com.thoughtworks.go.util.ExceptionUtils.bomb(ExceptionUtils.java:36)
at com.thoughtworks.go.config.materials.git.GitMaterial.updateTo(GitMaterial.java:159)
at com.thoughtworks.go.domain.MaterialRevision.updateTo(MaterialRevision.java:151)
at com.thoughtworks.go.domain.materials.AbstractMaterialAgent.prepare(AbstractMaterialAgent.java:41)
at com.thoughtworks.go.remote.work.BuildWork.prepareJob(BuildWork.java:193)
at com.thoughtworks.go.remote.work.BuildWork.build(BuildWork.java:141)
at com.thoughtworks.go.remote.work.BuildWork.doWork(BuildWork.java:96)
at com.thoughtworks.go.agent.JobRunner.run(JobRunner.java:60)
at com.thoughtworks.go.agent.AgentHTTPClientController.retrieveWork(AgentHTTPClientController.java:123)
at com.thoughtworks.go.agent.AgentHTTPClientController.work(AgentHTTPClientController.java:99)
at com.thoughtworks.go.agent.AgentController.loop(AgentController.java:78)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:273)
at org.springframework.scheduling.support.MethodInvokingRunnable.run(MethodInvokingRunnable.java:65)
at org.springframework.scheduling.timer.DelegatingTimerTask.run(DelegatingTimerTask.java:70)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Caused by: com.thoughtworks.go.util.command.CommandLineException: Error while executing [git clone --branch=20.08_OMS_15.09_Endpoint git#example.com:TPP-Provisioning/PortOutGui.git /app/go-agent1/pipelines/PATGUI-build]
Make sure this command can execute manually.
at com.thoughtworks.go.util.ProcessManager.startProcess(ProcessManager.java:112)
at com.thoughtworks.go.util.ProcessManager.createProcess(ProcessManager.java:64)
at com.thoughtworks.go.util.command.CommandLine.createProcess(CommandLine.java:361)
at com.thoughtworks.go.util.command.CommandLine.execute(CommandLine.java:354)
at com.thoughtworks.go.util.command.CommandLine.run(CommandLine.java:513)
at com.thoughtworks.go.domain.materials.SCMCommand.run(SCMCommand.java:34)
at com.thoughtworks.go.domain.materials.git.GitCommand.clone(GitCommand.java:82)
at com.thoughtworks.go.config.materials.git.GitMaterial.git(GitMaterial.java:247)
at com.thoughtworks.go.config.materials.git.GitMaterial.updateTo(GitMaterial.java:153)
... 17 more
Caused by: java.io.IOException: Cannot run program "git": error=11, Resource temporarily unavailable
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at com.thoughtworks.go.util.ProcessManager.startProcess(ProcessManager.java:105)
... 25 more
Caused by: java.io.IOException: error=11, Resource temporarily unavailable
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:248)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
GO CD Build:
20:21:26.207 [go] Job Started: 2020-08-25 20:21:26 UTC
20:21:26.207 [go] Start to prepare PATGUI-build/131/Execute_Script/1/build_job on exmaple-po-b4p.sys.comcast.net [/app/go-agent3]
20:21:26.212 [go] Start to update materials.
20:21:26.212 [go] Start updating files at revision 1c9e6a31946f62af1285305f338178d2a8fd13f2 from git#github.comcast.com:TPP-Provisioning/PortOutGui.git
20:21:26.214 Error while executing [git clone --branch=20.08_OMS_15.09_Endpoint git#example.com:TPP-Provisioning/PortOutGui.git /app/go-agent3/pipelines/PATGUI-build]
Make sure this command can execute manually.
20:21:26.231 [go] Job completed PATGUI-build/131/Execute_Script/1/build_job on exmaple-po-b4p.sys.comcast.net [/app/go-agent3]
This issue was resolved by restarting the go agent. A new issue arouse but this issue was resolved by the following.
Login into GoCD UI.
Click on "Agents"
Select the agent which is required to be restarted
click on Disable button
refresh the page.
You will see that agent disabled and at the last in the list
Copy the "Agent Name". This is the host of agent. Login into this host with Cada credentials.
Run command : sudo su -
Now execute : sudo /etc/init.d/go-agent1 restart (Note : The last digit after agent will change as per Agents/"sandbox" value in the GoCD UI)
Above command will restart the agent
Now go back to the GoCD UI, select the disabled agent and click on Enable.
Refresh the page, agent should be back in idle state.Restarting GoCD agents:
Sorry this question might be similar to any other but those questions did not solve my issue.
I had JDK 11 [primary] and JDK 8 [secondary] installed for Jenkins slave agent due to requirement.
Then JDK 11 started giving JNLP connection error after builds are successful but build status is FAILED.
Now I want to use JDK8 [as primary JDK] on my macOS slave to avoid JNLP connection error as my Master Jenkins is having JDK8.
But as soon I remove JDK11 , all my projects on that particular MacOS node fails before starting with error :-
check out build error logs when I removed JDK11 :-
FATAL: Command execution failed.
java.io.IOException: error=2, No such file or directory
at java.lang.ProcessImpl.forkAndExec(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:340)
at java.lang.ProcessImpl.start(ProcessImpl.java:271)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from 192.xx.xx.50/192.xx.xx.50:49xxx
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1737)
at hudson.remoting.UserResponse.retrieve(UserRequest.java:313)
at hudson.remoting.Channel.call(Channel.java:952)
at hudson.Launcher$RemoteLauncher.launch(Launcher.java:1052)
at hudson.Launcher$ProcStarter.start(Launcher.java:449)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:109)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:66)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1727)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: java.io.IOException: Cannot run program "/bin/sh" (in directory "Jenkins_home/workspace/java11-failure-on-mac-test"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at hudson.Proc$LocalProc.<init>(Proc.java:249)
at hudson.Proc$LocalProc.<init>(Proc.java:218)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:929)
at hudson.Launcher$ProcStarter.start(Launcher.java:449)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1299)
at hudson.Launcher$RemoteLaunchCallable.call(Launcher.java:1259)
at hudson.remoting.UserRequest.perform(UserRequest.java:153)
at hudson.remoting.UserRequest.perform(UserRequest.java:50)
at hudson.remoting.Request$2.run(Request.java:336)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at hudson.remoting.Engine$1$1.run(Engine.java:94)
at java.lang.Thread.run(Thread.java:834)
Build step 'Execute shell' marked build as failure
Finished: FAILURE
Help would be appreciated..!
I'm running a Java app in Eclipse EE Oxygen 32 bit in a 64 machine since the program will be transferred to a 32 bit machine later. The app is connected to mysql installed through Wampserver, and the server I'm using in the app is Apache Tomcat 8.0.35. When I run the app in "Run as> Run on Server" inside Eclispe. I get the following error message in "Console":
Illegal access: this web application instance has been stopped already. Could not load []. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load []. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1372)
at org.apache.catalina.loader.WebappClassLoaderBase.getResource(WebappClassLoaderBase.java:1042)
at com.mysql.jdbc.AbandonedConnectionCleanupThread.checkContextClassLoaders(AbandonedConnectionCleanupThread.java:90)
at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:63)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
How can I debug that?
The java code is totally portable across 32-bit and 64-bit platforms, doesn't make sense to use a 32 bit Eclipse version.
To solve your issue, just restart the Tomcat server to get rid of the older cache then try again.
If still doesn't work:
Restart Eclipse
Clean up Eclipse working directory (see #nitind comment).
Run Eclipse in clean mode (see this)
I have some tests in selenium. When I run them using maven on my local machine then everything works fine, but when I try to start them using jenkins I get an error. Can anyone help me out and tell why it works on my machine, and doesnt on Jenkins? Thanks
[SNOWDEN_develop_Selenium_AutoTests] $ java -cp /var/lib/jenkins/maven33-agent.jar:/var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/maven /boot/plexus-classworlds-2.5.2.jar:/var/lib/jenkins/tools /hudson.tasks.Maven_MavenInstallation/maven/conf/logging jenkins.maven3.agent.Maven33Main /var/lib/jenkins/tools/hudson.tasks.Maven_MavenInstallation/maven /var/lib/jenkins/slave.jar /var/lib/jenkins/maven33-interceptor.jar /var/lib/jenkins/maven3-interceptor-commons.jar 43010
Exception in thread "main" java.lang.ClassNotFoundException: hudson.remoting.Launcher
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
at jenkins.maven3.agent.Maven33Main.main(Maven33Main.java:133)
at jenkins.maven3.agent.Maven33Main.main(Maven33Main.java:64)
[SNOWDEN_develop_Selenium_AutoTests] $ /bin/bash -xe /tmp/hudson5734623695552292225.sh
+ cd /var/lib/jenkins/workspace/SNOWDEN_develop_Selenium_AutoTests/snowden-docker
+ ./docker.snowden.sh stop devsel
WORKSPACE: /var/lib/jenkins/workspace/SNOWDEN_develop_Selenium_AutoTests
Docker host : 10.0.0.27:4243 (SNOWDEN-DEV)
Stop docker container snowden_devsel ...
Stoping container snowden_devsel ...
snowden_devsel_tomcat
snowden_devsel_mongodb
ERROR: Failed to parse POMs
java.io.EOFException: unexpected stream termination
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:365)
at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:310)
at hudson.slaves.Channels.forProcess(Channels.java:115)
at hudson.maven.AbstractMavenProcessFactory.newProcess(AbstractMavenProcessFactory.java:294)
at hudson.maven.ProcessCache.get(ProcessCache.java:236)
at hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:798)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1728)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:544)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Finished: FAILURE
I'm using CentOS 5 and Jenkins 1.430. When I try to build, I get an error:
hudson.util.IOException2: remote file operation failed: /home/build/jenkins/workspace/tests at hudson.remoting.Channel#6c89db9a:build-test
at hudson.FilePath.act(FilePath.java:754)
at hudson.FilePath.act(FilePath.java:740)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:705)
at hudson.scm.SubversionSCM.checkout(SubversionSCM.java:654)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1193)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:566)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:454)
at hudson.model.Run.run(Run.java:1376)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:230)
Caused by: java.io.IOException: Remote call on build-test failed
at hudson.remoting.Channel.call(Channel.java:690)
at hudson.FilePath.act(FilePath.java:747)
... 10 more
Caused by: java.lang.NoClassDefFoundError: Could not initialize class hudson.model.Hudson
at hudson.scm.SubversionWorkspaceSelector.syncWorkspaceFormatFromMaster(SubversionWorkspaceSelector.java:85)
at hudson.scm.SubversionSCM.createSvnClientManager(SubversionSCM.java:782)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:725)
at hudson.scm.SubversionSCM$CheckOutTask.invoke(SubversionSCM.java:712)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1995)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:287)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Email was triggered for: Failure
Sending email for trigger: Failure
An attempt to send an e-mail to empty list of recipients, ignored.
Finished: FAILURE
This thread seems to say it's something wrong with Master/Slave:
https://groups.google.com/forum/?fromgroups=#!topic/jenkinsci-issues/MaMqjK3iW8Y
But it doesn't really offer a solution. The Jenkin slave says it's connected and it's reflected on the Jenkins project page.
Well apparently disabling the slave and rebooting it worked. bleh
If you don't want to reboot then simply kill these processes on the slave
$ ps -ef | grep java
1006 11948 11930 0 Jul17 ? 00:00:00 bash -c cd "/mnt/jenkins" && java -jar slave.jar
1006 11949 11948 0 Jul17 ? 00:02:55 java -jar slave.jar
Then launch the slave agent again
Got a quite similar error, and fixed it by reverting the "publish over ssh" plug-in from 1.14 to 1.13
Our Jenkins server is run on Debian 8.2 (Jessie), and our slaves under Ubuntu 14.04 (trusty)
Terminated the slave and launched it again. Working perfectly.