Develop Reference

Using Freenas 11.2 my java process closes when leaving jail shell

For the past 2 weeks I've been busy trying to figure out how to setup my minecraft server onto my freenas server.
I was able to get it up and running stably when going into the jail manually typing in my startup command:
cd /root/Minecraft_Server
java -Xmx4096M -Xms4096M -jar forge-1.12.2-14.23.4.2757-universal.jar
And then just close the shell.
I tried looking to automate this command and put it into and sh file in crontab and everything, that didn't work so i decided to upgrade to 11.2 to see if that has any solutions.
Now the main problem already is that if I try to run my command manually in the shell, and i leave the webui, it will just close the server down to unlike in the 11.1 freenas.
Does anyone have any more ideas here?
In the same location as the server I have a minecraft.sh script with this command.
If I manually run the script it works, but if I use crontab it won't start it either.
The corntab command that i've used is:
#reboot /root/Minecraft_Server/minecraft.sh
I also tried putting in the command directly but this also was useless.
I even tried the exec.poststart but when i direct it to /root/minecraft_Server/minecraft.sh it won't start either, it won't even run the jail anymore

use “screen java ...”
on relog to shell do screen -x to get on the server shell

You can configure your Java command as a service that starts whenever the jail starts. That way, the Java server doesn’t depend on the shell or webui.
Basically, create a usr/local/etc/rc.d/minecraftd file that includes the following script:
#!/bin/sh
#
# PROVIDE: minecraftd
# REQUIRE: LOGIN DAEMON NETWORKING mountcritlocal
# KEYWORD: shutdown
#
# Use the following variables to configure the minecraft server. For example, to
# configure the ON/OFF knob variable:
# sysrc minecraftd_enable="YES"
#
# minecraftd_enable="YES"
# minecraftd_user_dir="/root/minecraft"
# minecraftd_jar_path="/root/minecraft/server.jar"
# minecraftd_java_opts="-Xms512M -Xmx1024M"
. /etc/rc.subr
name=minecraftd
rcvar=`set_rcvar`
pidfile=/var/run/minecraftd.pid
load_rc_config $name
start_cmd="${name}_start"
: ${minecraftd_enable="NO"}
: ${minecraftd_user_dir="/root/minecraft"}
: ${minecraftd_jar_path="/root/minecraft/server.jar"}
: ${minecraftd_java_opts="-Xms512M -Xmx1024M"}
minecraftd_start() {
if [ -e $pidfile ]; then
echo "$name already running."
else
echo "Starting $name..."
/usr/sbin/daemon -f -p $pidfile \
/usr/local/bin/java -Duser.dir=$minecraftd_user_dir \
$minecraftd_java_opts \
-jar $minecraftd_jar_path nogui
echo "$name started."
fi
}
run_rc_command $1
Then configure the service to start on boot:
sysrc minecraftd_enable="YES"
And restart your jail.
For more information, check Installing a Minecraft server on FreeNAS.
Disclaimer: My team published that article.
Hope this helps somebody.

How do I launch a service on linux? [duplicate]

I have written a Java server application that runs on a standard virtual hosted Linux solution. The application runs all the time listening for socket connections and creating new handlers for them. It is a server side implementation to a client-server application.
The way I start it is by including it in the start up rc.local script of the server. However once started I do not know how to access it to stop it and if I want to install an update, so I have to restart the server in order to restart the application.
On a windows PC, for this type of application I might create a windows service and then I can stop and start it as I want. Is there anything like that on a Linux box so that if I start this application I can stop it and restart it without doing a complete restart of the server.
My application is called WebServer.exe. It is started on server startup by including it in my rc.local as such:
java -jar /var/www/vhosts/myweb.com/phpserv/WebServer.jar &
I am a bit of a noob at Linux so any example would be appreciated with any posts. However I do have SSH, and full FTP access to the box to install any updates as well as access to a Plesk panel.

I wrote another simple wrapper here:
#!/bin/sh
SERVICE_NAME=MyService
PATH_TO_JAR=/usr/local/MyProject/MyJar.jar
PID_PATH_NAME=/tmp/MyService-pid
case $1 in
start)
echo "Starting $SERVICE_NAME ..."
if [ ! -f $PID_PATH_NAME ]; then
nohup java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
echo $! > $PID_PATH_NAME
echo "$SERVICE_NAME started ..."
else
echo "$SERVICE_NAME is already running ..."
fi
;;
stop)
if [ -f $PID_PATH_NAME ]; then
PID=$(cat $PID_PATH_NAME);
echo "$SERVICE_NAME stoping ..."
kill $PID;
echo "$SERVICE_NAME stopped ..."
rm $PID_PATH_NAME
else
echo "$SERVICE_NAME is not running ..."
fi
;;
restart)
if [ -f $PID_PATH_NAME ]; then
PID=$(cat $PID_PATH_NAME);
echo "$SERVICE_NAME stopping ...";
kill $PID;
echo "$SERVICE_NAME stopped ...";
rm $PID_PATH_NAME
echo "$SERVICE_NAME starting ..."
nohup java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
echo $! > $PID_PATH_NAME
echo "$SERVICE_NAME started ..."
else
echo "$SERVICE_NAME is not running ..."
fi
;;
esac
You can follow a full tutorial for init.d here and for systemd (ubuntu 16+) here
If you need the output log replace the 2
nohup java -jar $PATH_TO_JAR /tmp 2>> /dev/null >> /dev/null &
lines for
nohup java -jar $PATH_TO_JAR >> myService.out 2>&1&

A simple solution is to create a script start.sh that runs Java through nohup and then stores the PID to a file:
nohup java -jar myapplication.jar > log.txt 2> errors.txt < /dev/null &
PID=$!
echo $PID > pid.txt
Then your stop script stop.sh would read the PID from the file and kill the application:
PID=$(cat pid.txt)
kill $PID
Of course I've left out some details, like checking whether the process exists and removing pid.txt if you're done.

Linux service init script are stored into /etc/init.d. You can copy and customize /etc/init.d/skeleton file, and then call
service [yourservice] start|stop|restart
see http://www.ralfebert.de/blog/java/debian_daemon/. Its for Debian (so, Ubuntu as well) but fit more distribution.

Maybe not the best dev-ops solution, but good for the general use of a server for a lan party or similar.
Use screen to run your server in and then detach before logging out, this will keep the process running, you can then re-attach at any point.
Workflow:
Start a screen: screen
Start your server: java -jar minecraft-server.jar
Detach by pressing: Ctl-a, d
Re-attach: screen -r
More info here: https://www.gnu.org/software/screen/manual/screen.html

Another alternative, which is also quite popular is the Java Service Wrapper. This is also quite popular around the OSS community.

Referring to Spring Boot application as a Service as well, I would go for the systemd version, since it's the easiest, least verbose, and best integrated into modern distros (and even the not-so-modern ones like CentOS 7.x).

The easiest way is to use supervisord. Please see full details here: http://supervisord.org/
More info:
https://askubuntu.com/questions/779830/running-an-executable-jar-file-when-the-system-starts/852485#852485
https://www.digitalocean.com/community/tutorials/how-to-install-and-manage-supervisor-on-ubuntu-and-debian-vps

Here is a sample shell script (make sure you replace the MATH name with the name of the your application):
#!/bin/bash
### BEGIN INIT INFO
# Provides: MATH
# Required-Start: $java
# Required-Stop: $java
# Short-Description: Start and stop MATH service.
# Description: -
# Date-Creation: -
# Date-Last-Modification: -
# Author: -
### END INIT INFO
# Variables
PGREP=/usr/bin/pgrep
JAVA=/usr/bin/java
ZERO=0
# Start the MATH
start() {
echo "Starting MATH..."
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "The service is already running"
else
#Run the jar file MATH service
$JAVA -jar /opt/MATH/MATH.jar > /dev/null 2>&1 &
#sleep time before the service verification
sleep 10
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "Service was successfully started"
else
echo "Failed to start service"
fi
fi
echo
}
# Stop the MATH
stop() {
echo "Stopping MATH..."
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
#Kill the pid of java with the service name
kill -9 $($PGREP -f MATH)
#Sleep time before the service verification
sleep 10
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "Failed to stop service"
else
echo "Service was successfully stopped"
fi
else
echo "The service is already stopped"
fi
echo
}
# Verify the status of MATH
status() {
echo "Checking status of MATH..."
#Verify if the service is running
$PGREP -f MATH > /dev/null
VERIFIER=$?
if [ $ZERO = $VERIFIER ]
then
echo "Service is running"
else
echo "Service is stopped"
fi
echo
}
# Main logic
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart|reload)
stop
start
;;
*)
echo $"Usage: $0 {start|stop|status|restart|reload}"
exit 1
esac
exit 0

From Spring Boot application as a Service, I can recommend the Python-based supervisord application. See that stack overflow question for more information. It's really straightforward to set up.

Other answers do a good job giving custom scripts and setups depending on your platform. In addition to those, here are the mature, special purpose programs that I know of:
JSW from TanukiSoftware
YAJSW is an open source clone from the above. It is written in Java, and it is a nanny process that manages the child process (your code) according to configurations. Works on windows / linux.
JSVC is a native application. Its also a nanny process, but it invokes your child application through the JNI, rather than as a subprocess.

You can use Thrift server or JMX to communicate with your Java service.

From Spring Boot Reference Guide
Installation as an init.d service (System V)
Simply symlink the jar to init.d to support the standard start, stop, restart and status commands.
Assuming that you have a Spring Boot application installed in /var/myapp, to install a Spring Boot application as an init.d service simply create a symlink:
$ sudo ln -s /var/myapp/myapp.jar /etc/init.d/myapp
Once installed, you can start and stop the service in the usual way. For example, on a Debian based system:
$ service myapp start
If your application fails to start, check the log file written to /var/log/<appname>.log for errors.
Continue reading to know how to secure a deployed service.
After doing as written I've discovered that my service fails to start with this error message in logs: start-stop-daemon: unrecognized option --no-close. And I've managed to fix it by creating a config file /var/myapp/myapp.conf with the following content
USE_START_STOP_DAEMON=false

It is possible to run the war as a Linux service, and you may want to force in your pom.xml file before packaging, as some distros may not recognize in auto mode. To do it, add the following property inside of spring-boot-maven-plugin plugin.
<embeddedLaunchScriptProperties>
<mode>service</mode>
</embeddedLaunchScriptProperties>
Next, setup your init.d with:
ln -s myapp.war /etc/init.d/myapp
and you will be able to run
service myapp start|stop|restart
There are many other options that you can find in Spring Boot documentation, including Windows service.

Im having Netty java application and I want to run it as a service with systemd. Unfortunately application stops no matter of what Type I'm using. At the end I've wrapped java start in screen. Here are the config files:
service
[Unit]
Description=Netty service
After=network.target
[Service]
User=user
Type=forking
WorkingDirectory=/home/user/app
ExecStart=/home/user/app/start.sh
TimeoutStopSec=10
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
start
#!/bin/sh
/usr/bin/screen -L -dmS netty_app java -cp app.jar classPath
from that point you can use systemctl [start|stop|status] service.

To run Java code as daemon (service) you can write JNI based stub.
http://jnicookbook.owsiak.org/recipe-no-022/
for a sample code that is based on JNI. In this case you daemonize the code that was started as Java and main loop is executed in C. But it is also possible to put main, daemon's, service loop inside Java.
https://github.com/mkowsiak/jnicookbook/tree/master/recipes/recipeNo029
Have fun with JNI!

However once started I don't know how to access it to stop it
You can write a simple stop script that greps for your java process, extracts the PID and calls kill on it. It's not fancy, but it's straight forward.
Something like that may be of help as a start:
#!/bin/bash
PID = ps ax | grep "name of your app" | cut -d ' ' -f 1
kill $PID

Getting PID of a process that has just started

I am working on a Gradle Java project. Which starts Tomcat for testing and stops it later.
I am supposed to kill this Tomcat instance when the test fails.
I tried using "ps aux| grep tomcat | grep -v grap | awk {print $2}" command to get the process id and kill the process.
But on Production, there will be so many Tomcat processes running simultaneously by many users, I just want the tomcat process started by my build.gradle for test execution.
So how can I accomplish the task? Please provide me some guidelines.

You need to find a unique string in the output of 'ps aux' which differentiates your test tomcat and others'.
I currently use the below script to run 'shutdown.sh' first and then kill the PID as most of the times, the application stops but the process does not stop.
PID=`ps -ef | grep $JAVA_HOME/bin/java | grep "$TOMCAT_LOC"/conf | grep -v grep | awk '{ print $2 }'`
if [ $PID ]; then
echo tomcat is running with PID:$PID.
# Stop or Kill running Tomcat
if [[ -f $TOMCAT_LOC/bin/shutdown.sh ]]; then
[[ ! -x $TOMCAT_LOC/bin/shutdown.sh ]] && chmod a+x $TOMCAT_LOC/bin/shutdown.sh
$TOMCAT_LOC/bin/shutdown.sh >>/dev/null
sleep 20
fi
kill -9 $PID
sleep 3
else
echo tomcat is not running
fi
You may also look at configuring a PID file by editing the 'catalina.sh' which you can read later to find out your PID.
# CATALINA_PID (Optional) Path of the file which should contains the pid
# of the catalina startup java process, when start (fork) is
# used

Java JRE has tool called jps in $JAVA_HOME/bin folder.
It's similar to unix ps command but for java only.
You can use it to determined exac java process you need.
Using this tool is more recommended and actually it is more useful, when you have more than one java applications is running on your host...
for example I have running h2 database and many other apps, but wanna kill only h2, so I can use jps to get it PID
$ jps
17810 GradleDaemon
17798 GradleWrapperMain
17816 h2-1.4.197.jar
17817 GradleDaemon
17818 GradleDaemon
18011 Jps
16479
and then just kill needed process:
kill -9 17816
and all other java apps will continue work normally. I not sure about tomcat, but I think it can be done in similar way, something like that:
kill -9 $(jps | grep tomcat | awk '{print $1}')
Lastly, little bit offtopic, but a specially to your case: correct way would be using start/stop/restart scripts provided by tomcat

The correct way to terminate a Tomcat instance is via its own shutdown command. You should not be thinking of processes, or PIDs, or kills, at all.

so if you want to kill tomcat from that user from which you have logged in then try following and let me know if this helps you.
ps -ef | grep -v grep | grep `whoami` | grep tomcat
So by ps -ef I am listing all the processes then grep -v grep will remove the grep command's process then grep whoami will look for your currently logged in user then grep tomcat will look only for tomcat process, test it once and if All is Well then you could kill it.
By the way how about tomcat stop script? In case it is there you could use that also.

You can use shell variable $!. It represents the PID of the most recent background command.
yourCommand &
CMD_PID=$!
echo $CMD_PID

How to keep a GlassFish server's domain active while disconnected from SSH to an EC2 instance? [duplicate]

I connect to the linux server via putty SSH. I tried to run it as a background process like this:
$ node server.js &
However, after 2.5 hrs the terminal becomes inactive and the process dies. Is there anyway I can keep the process alive even with the terminal disconnected?
Edit 1
Actually, I tried nohup, but as soon as I close the Putty SSH terminal or unplug my internet, the server process stops right away.
Is there anything I have to do in Putty?
Edit 2 (on Feb, 2012)
There is a node.js module, forever. It will run node.js server as daemon service.

nohup node server.js > /dev/null 2>&1 &
nohup means: Do not terminate this process even when the stty is cut
off.
> /dev/null means: stdout goes to /dev/null (which is a dummy
device that does not record any output).
2>&1 means: stderr also goes to the stdout (which is already redirected to /dev/null). You may replace &1 with a file path to keep a log of errors, e.g.: 2>/tmp/myLog
& at the end means: run this command as a background task.

Simple solution (if you are not interested in coming back to the process, just want it to keep running):
nohup node server.js &
There's also the jobs command to see an indexed list of those backgrounded processes. And you can kill a backgrounded process by running kill %1 or kill %2 with the number being the index of the process.
Powerful solution (allows you to reconnect to the process if it is interactive):
screen
You can then detach by pressing Ctrl+a+d and then attach back by running screen -r
Also consider the newer alternative to screen, tmux.

You really should try to use screen. It is a bit more complicated than just doing nohup long_running &, but understanding screen once you never come back again.
Start your screen session at first:
user#host:~$ screen
Run anything you want:
wget http://mirror.yandex.ru/centos/4.6/isos/i386/CentOS-4.6-i386-binDVD.iso
Press ctrl+A and then d. Done. Your session keeps going on in background.
You can list all sessions by screen -ls, and attach to some by screen -r 20673.pts-0.srv command, where 0673.pts-0.srv is an entry list.

This is an old question, but is high ranked on Google. I almost can't believe on the highest voted answers, because running a node.js process inside a screen session, with the & or even with the nohup flag -- all of them -- are just workarounds.
Specially the screen/tmux solution, which should really be considered an amateur solution. Screen and Tmux are not meant to keep processes running, but for multiplexing terminal sessions. It's fine, when you are running a script on your server and want to disconnect. But for a node.js server your don't want your process to be attached to a terminal session. This is too fragile. To keep things running you need to daemonize the process!
There are plenty of good tools to do that.
PM2: http://pm2.keymetrics.io/
# basic usage
$ npm install pm2 -g
$ pm2 start server.js
# you can even define how many processes you want in cluster mode:
$ pm2 start server.js -i 4
# you can start various processes, with complex startup settings
# using an ecosystem.json file (with env variables, custom args, etc):
$ pm2 start ecosystem.json
One big advantage I see in favor of PM2 is that it can generate the system startup script to make the process persist between restarts:
$ pm2 startup [platform]
Where platform can be ubuntu|centos|redhat|gentoo|systemd|darwin|amazon.
forever.js: https://github.com/foreverjs/forever
# basic usage
$ npm install forever -g
$ forever start app.js
# you can run from a json configuration as well, for
# more complex environments or multi-apps
$ forever start development.json
Init scripts:
I'm not go into detail about how to write a init script, because I'm not an expert in this subject and it'd be too long for this answer, but basically they are simple shell scripts, triggered by OS events. You can read more about this here
Docker:
Just run your server in a Docker container with -d option and, voilá, you have a daemonized node.js server!
Here is a sample Dockerfile (from node.js official guide):
FROM node:argon
# Create app directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Install app dependencies
COPY package.json /usr/src/app/
RUN npm install
# Bundle app source
COPY . /usr/src/app
EXPOSE 8080
CMD [ "npm", "start" ]
Then build your image and run your container:
$ docker build -t <your username>/node-web-app .
$ docker run -p 49160:8080 -d <your username>/node-web-app
Always use the proper tool for the job. It'll save you a lot of headaches and over hours!

another solution disown the job
$ nohup node server.js &
[1] 1711
$ disown -h %1

nohup will allow the program to continue even after the terminal dies. I have actually had situations where nohup prevents the SSH session from terminating correctly, so you should redirect input as well:
$ nohup node server.js </dev/null &
Depending on how nohup is configured, you may also need to redirect standard output and standard error to files.

Nohup and screen offer great light solutions to running Node.js in the background. Node.js process manager (PM2) is a handy tool for deployment. Install it with npm globally on your system:
npm install pm2 -g
to run a Node.js app as a daemon:
pm2 start app.js
You can optionally link it to Keymetrics.io a monitoring SAAS made by Unitech.

$ disown node server.js &
It will remove command from active task list and send the command to background

I have this function in my shell rc file, based on #Yoichi's answer:
nohup-template () {
[[ "$1" = "" ]] && echo "Example usage:\nnohup-template urxvtd" && return 0
nohup "$1" > /dev/null 2>&1 &
}
You can use it this way:
nohup-template "command you would execute here"

Have you read about the nohup command?

To run command as a system service on debian with sysv init:
Copy skeleton script and adapt it for your needs, probably all you have to do is to set some variables. Your script will inherit fine defaults from /lib/init/init-d-script, if something does not fits your needs - override it in your script. If something goes wrong you can see details in source /lib/init/init-d-script. Mandatory vars are DAEMON and NAME. Script will use start-stop-daemon to run your command, in START_ARGS you can define additional parameters of start-stop-daemon to use.
cp /etc/init.d/skeleton /etc/init.d/myservice
chmod +x /etc/init.d/myservice
nano /etc/init.d/myservice
/etc/init.d/myservice start
/etc/init.d/myservice stop
That is how I run some python stuff for my wikimedia wiki:
...
DESC="mediawiki articles converter"
DAEMON='/home/mss/pp/bin/nslave'
DAEMON_ARGS='--cachedir /home/mss/cache/'
NAME='nslave'
PIDFILE='/var/run/nslave.pid'
START_ARGS='--background --make-pidfile --remove-pidfile --chuid mss --chdir /home/mss/pp/bin'
export PATH="/home/mss/pp/bin:$PATH"
do_stop_cmd() {
start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 \
$STOP_ARGS \
${PIDFILE:+--pidfile ${PIDFILE}} --name $NAME
RETVAL="$?"
[ "$RETVAL" = 2 ] && return 2
rm -f $PIDFILE
return $RETVAL
}
Besides setting vars I had to override do_stop_cmd because of python substitutes the executable, so service did not stop properly.

Apart from cool solutions above I'd mention also about supervisord and monit tools which allow to start process, monitor its presence and start it if it died. With 'monit' you can also run some active checks like check if process responds for http request

For Ubuntu i use this:
(exec PROG_SH &> /dev/null &)
regards

Try this for a simple solution
cmd & exit

Get pid from a java subprocess started with xvfb-run

I'm trying to get the PID of a java process started with xvfb-run. When started without xvfb-run, I use $! to get the PID of the last backgrounded process but as soon as I use xvfb-run I obviously get the PID of xvfb-run.
Here is the code:
#! /bin/bash
logfile=/var/log/SleepTest.log
pidfile=/var/run/SleepTest.pid
command="java -jar /data/test/SleepTest.jar"
( eval exec -c xvfb-run $command < /dev/null >> $logfile 2>&1 ) &
$! > $pidfile
If I remove the xvfb-run part in the second last line, everything works ok (except the part that I don't have a display and the program crash). I probably have to play with the "()" and "&" but I'm not an expert.
The program SleepTest.jar is a small program I wrote so I don't have to deal with the real thing. It only sleep for 2 minutes.
For those wondering why I use xvfb-run, it's because the java application I need to start use SWT and I don't have display on my server.
For those wondering why I need the pid of the process, it's because I want to create a init.d file to be able to start|stop|status my application
So is there a simple way to get it ?

Run your entire script with xvfb-run (e.g. xvfb-run name_of_script.sh), and remove the xvfb-run from your eval line.