Develop Reference

Why does spark-submit fail with “Error executing Jupyter command”?

When trying to run Spark locally on my Mac (which used to work) ...
/Library/Java/JavaVirtualMachines/jdk1.8.0_192.jdk/Contents/Home/bin/java \
-cp /usr/local/Cellar/apache-spark/2.4.0/libexec/conf/:/usr/local/Cellar/apache-spark/2.4.0/libexec/jars/* \
-Xmx1g org.apache.spark.deploy.SparkSubmit \
--packages org.mongodb.spark:mongo-spark-connector_2.11:2.4.0 \
/Users/crump/main.py
I'm now getting the following error:
Error executing Jupyter command '/Users/crump/main.py': [Errno 2] No such file or directory
The file is there. Since I know this used to work, I must have installed something recently that changed a library, sdk, etc.

Ok, I found the answer finally: PYSPARK_DRIVER_PYTHON=jupyter in my environment. I set this up to launch Jupyter/Spark notebooks with just the pyspark command, but it causes spark-submit to fail.
The solution is set the variable to use python, not jupyter: PYSPARK_DRIVER_PYTHON=python.

Using Freenas 11.2 my java process closes when leaving jail shell

For the past 2 weeks I've been busy trying to figure out how to setup my minecraft server onto my freenas server.
I was able to get it up and running stably when going into the jail manually typing in my startup command:
cd /root/Minecraft_Server
java -Xmx4096M -Xms4096M -jar forge-1.12.2-14.23.4.2757-universal.jar
And then just close the shell.
I tried looking to automate this command and put it into and sh file in crontab and everything, that didn't work so i decided to upgrade to 11.2 to see if that has any solutions.
Now the main problem already is that if I try to run my command manually in the shell, and i leave the webui, it will just close the server down to unlike in the 11.1 freenas.
Does anyone have any more ideas here?
In the same location as the server I have a minecraft.sh script with this command.
If I manually run the script it works, but if I use crontab it won't start it either.
The corntab command that i've used is:
#reboot /root/Minecraft_Server/minecraft.sh
I also tried putting in the command directly but this also was useless.
I even tried the exec.poststart but when i direct it to /root/minecraft_Server/minecraft.sh it won't start either, it won't even run the jail anymore

use “screen java ...”
on relog to shell do screen -x to get on the server shell

You can configure your Java command as a service that starts whenever the jail starts. That way, the Java server doesn’t depend on the shell or webui.
Basically, create a usr/local/etc/rc.d/minecraftd file that includes the following script:
#!/bin/sh
#
# PROVIDE: minecraftd
# REQUIRE: LOGIN DAEMON NETWORKING mountcritlocal
# KEYWORD: shutdown
#
# Use the following variables to configure the minecraft server. For example, to
# configure the ON/OFF knob variable:
# sysrc minecraftd_enable="YES"
#
# minecraftd_enable="YES"
# minecraftd_user_dir="/root/minecraft"
# minecraftd_jar_path="/root/minecraft/server.jar"
# minecraftd_java_opts="-Xms512M -Xmx1024M"
. /etc/rc.subr
name=minecraftd
rcvar=`set_rcvar`
pidfile=/var/run/minecraftd.pid
load_rc_config $name
start_cmd="${name}_start"
: ${minecraftd_enable="NO"}
: ${minecraftd_user_dir="/root/minecraft"}
: ${minecraftd_jar_path="/root/minecraft/server.jar"}
: ${minecraftd_java_opts="-Xms512M -Xmx1024M"}
minecraftd_start() {
if [ -e $pidfile ]; then
echo "$name already running."
else
echo "Starting $name..."
/usr/sbin/daemon -f -p $pidfile \
/usr/local/bin/java -Duser.dir=$minecraftd_user_dir \
$minecraftd_java_opts \
-jar $minecraftd_jar_path nogui
echo "$name started."
fi
}
run_rc_command $1
Then configure the service to start on boot:
sysrc minecraftd_enable="YES"
And restart your jail.
For more information, check Installing a Minecraft server on FreeNAS.
Disclaimer: My team published that article.
Hope this helps somebody.

Error trying to run headless BehaviorSpace

I feel I must apologize for such a basic question, but I am getting an error simply trying to run BehaviorSpace experiments in headless mode. I tried running my own model experiments from the command line, but got an error. So I then tried following the exact instructions on the BehaviorSpace documentation. To do this, I created a BehaviorSpace experiment in the Fire.nlogo model called "experiment1" (see screen shot) and then tried to execute commands to run experiment1 from the command line. The screen shot of the terminal shows that I first set the directory where I have NetLogo 5.3 installed, and then tried to run the commands from the BehaviorSpace documentation. The screen shot of the terminal also shows the Java error I am getting. I have never used the terminal before and am not sure what I am doing wrong, but I am sure I am missing something simple.
I am using Mac OS X and NetLogo 5.3. Thank you for your time.

Seems you're not working in the correct directory.
You need to cd into the netlogo directory:
For me:
netlogo_directory = "/Applications/NetLogo 5.2"
so
cd /Applications/NetLogo\ 5.2
Then you can execute your command:
java -Xmx2048m -Dfile.encoding=UTF-8 -cp ./Netlogo.jar org.nlogo.headless.Main --model /path/to/your/file/name/filename.nlogo --experiment experimentname --table /path/to/log/with/filename.csv --spreadsheet /path/tp/spreadsheet/with/filename.csv

The problem is that the Java file that comes with NetLogo is where the .jar file and lib file are located. Hence, a simple addition of Java/ in the below code allows all files to be found.
java -Xmx1024m -Dfile.encoding=UTF-8 -cp Java/NetLogo.jar \
org.nlogo.headless.Main \
--model Fire.nlogo \
--experiment experiment1 \
--table mytable.csv

jmeter plugins cmd runner

I am running the the following:
java -jar /opt/jmeter/apache-jmeter-2.13/lib/ext/CMDRunner.jar \
--tool Reporter --generate-png /home/nune/jmeter_test/target/jmeter/results/RTOT.png \
--input-jtl A.jtl --plugin-type ResponseTimesOverTime
command from the terminal.PNG file is successfully creating but the process is not ending.
How can I resolve this issue. I want to end process after picture is successfully created.

I suspect the image is still being generated possibly due to lack of memory, in this case add after java
-Xmx256m
An other option is head issues, then add after java
-Djava.awt.headless=true
If none of those work, run on the pid of process:
jstack -l pid
and show output here

Running Equinox stand-alone with --launcher.ini option

I'm trying to run Equinox and antRunner in "headless" mode with a custom eclipse.ini file. The "--launcher.ini" option should work according to:
http://wiki.eclipse.org/Equinox_Launcher .
However, when I run the following command line:
java.exe
-cp "C:\Program Files\eclipse\plugins\org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar" \
org.eclipse.core.launcher.Main \
--launcher.ini "C:\ini\my_eclipse.ini" \
-application org.eclipse.ant.core.antRunner \
-console \
-data "c:\my_workspace" \
-file "c:\my_buildfiles\build.xml" \
I get the following error message:
osgi> Unknown argument: --launcher.ini
Unknown target: C:\ini\my_eclipse.ini
Buildfile: .\build.xml
How can I load a custom eclipse.ini when starting Eclipse with Equinox from the command line?

The problem is that you are trying to launch using only the Java part of the launcher, while the wiki page describes the arguments for the native part of the launcher (eclipse.exe or any name you want).
The launcher.ini describes how to setup the Java process (memory size, vm location, arguments to vm, etc.). So, it makes sense that you pass refernece to the launcher.ini to the native launcher.