Can not run java in spark worker - java

When I was running a spark program in the cluster, I got this error in log:
java.io.IOException: Cannot run program "java" (in directory "/cloud/packages/spark-0.9.0-incubating-bin-hadoop1/work/app-20140424114752-0000/0"): java.io.IOException: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at org.apache.spark.deploy.worker.ExecutorRunner.fetchAndRunExecutor(ExecutorRunner.scala:129)
at org.apache.spark.deploy.worker.ExecutorRunner$$anon$1.run(ExecutorRunner.scala:59)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 2 more
I have set JAVA_HOME(/cloud/packages/jdk1.6.0_38) and SPARK_HOME(/cloud/packages/spark-0.9.0-incubating-bin-hadoop1).
What's the causes of this exception? How to fixed it?

I encountered the same issue on Ubuntu 12.04, and fixed it by adding JAVA_HOME in /etc/environment.

Check your java version.
java -version
If java is installed properly it displays Java version.
If not Install java(Ubundu)
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
update
Please check echo $JAVA_HOME.
Else set java home in .bashrc
export JAVA_HOME=/cloud/packages/jdk1.6.0_38
export PATH=$PATH:$JAVA_HOME/bin

In a shell, you are accustomed to being able to run java and letting the shell consult the PATH to go find where it is. Here you are invoking a command directly in the OS. It can't find java as it says. You can use a shell like bash to invoke the command for you, or give a full path to java.
Why though? why not run Java code inside the worker?

Related

Unable to locate an executable at "/Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/Home/bin/apt" (-1)

I am trying to install bowtie2, but I fail.
sudo apt install bowtie2
But I always got error messages:
Unable to locate an executable at "/Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/Home/bin/apt" (-1)
I have been tried:
sudo ln -s /usr/bin/Java /Library/Java/JavaVirtualMachines/jdk9.0.4.jdk/Contents/Home/bin/apt
and
export JAVA_HOME='/usr/bin/Java'
But I am still unable to install my bowtie2 function.
Other backgrounds of this question:
Bowtie2 can be downloaded from https://www.java.com/zh-TW/download/
I store the unzipped file on the Desktop.
My Java files are at the below path:
/Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/Home/
My operating system version is macOS Mojave Version 10.14.6
Thanks in advance.

Strange advise when loading external library with JNA [duplicate]

I am trying to run the first example in rootbeer library, which you can find from here.
The issue is, even though I followed all of their instructions, I am getting the following message when I am trying to run the Jar
root#ubuserver3:/home/ubuadmin/JavaJars/HelloWordGPU# java -jar HelloWorldGPU.jar
OpenJDK Client VM warning: You have loaded library /usr/lib/libcuda.so.319.37 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Exception in thread "main" java.lang.NullPointerException
at org.trifort.rootbeer.runtime.Rootbeer.run(Rootbeer.java:104)
at ScalarMult.test(ScalarMult.java:13)
at Main.main(Main.java:17)
I am not a Linux person, so I do not understand how to execute this command properly. I also don't understand what they mean by that message, because this is the first time I am on GPU programming. But I know messing with GPU with wrong commands would be a problem.
Type: execstack --version to check if you already have execstack installed.
If you get command not found type: sudo apt-get install execstack -y --force-yes
Type sudo execstack -c <file>
e.g sudo execstack -c /usr/lib/libcuda.so
1: Check execstack version using following command to find execstack installed or not
execstack --version
2: If execstack not installed then install using following command
sudo apt-get install execstack -y --force-yes
3: After successful installation execute following command
sudo execstack -c <filepath>
eg: sudo execstack -c /usr/lib/libjpcap.so
Make sure you are not running 32-bit Java. If you run java -d64, the 32-bit version will print an error.
make sure you are not running 32-bit jar on 64-bit jvm first
What does the file /usr/lib/libcuda.so.319.37 contain?
If the file is a 0-byte-file, there is something really wrong, so try to repair your system by reinstalling that library.
I had the same problem in smartgit, where a library in the local config folder was used, which was somehow corrupted and was a 0 byte file. So deleting the config folder and restarting the app worked for me. This could be a solution here too.

Stack guard might have disabled?

I am trying to run the first example in rootbeer library, which you can find from here.
The issue is, even though I followed all of their instructions, I am getting the following message when I am trying to run the Jar
root#ubuserver3:/home/ubuadmin/JavaJars/HelloWordGPU# java -jar HelloWorldGPU.jar
OpenJDK Client VM warning: You have loaded library /usr/lib/libcuda.so.319.37 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
Exception in thread "main" java.lang.NullPointerException
at org.trifort.rootbeer.runtime.Rootbeer.run(Rootbeer.java:104)
at ScalarMult.test(ScalarMult.java:13)
at Main.main(Main.java:17)
I am not a Linux person, so I do not understand how to execute this command properly. I also don't understand what they mean by that message, because this is the first time I am on GPU programming. But I know messing with GPU with wrong commands would be a problem.
Type: execstack --version to check if you already have execstack installed.
If you get command not found type: sudo apt-get install execstack -y --force-yes
Type sudo execstack -c <file>
e.g sudo execstack -c /usr/lib/libcuda.so
1: Check execstack version using following command to find execstack installed or not
execstack --version
2: If execstack not installed then install using following command
sudo apt-get install execstack -y --force-yes
3: After successful installation execute following command
sudo execstack -c <filepath>
eg: sudo execstack -c /usr/lib/libjpcap.so
Make sure you are not running 32-bit Java. If you run java -d64, the 32-bit version will print an error.
make sure you are not running 32-bit jar on 64-bit jvm first
What does the file /usr/lib/libcuda.so.319.37 contain?
If the file is a 0-byte-file, there is something really wrong, so try to repair your system by reinstalling that library.
I had the same problem in smartgit, where a library in the local config folder was used, which was somehow corrupted and was a 0 byte file. So deleting the config folder and restarting the app worked for me. This could be a solution here too.

Adb permission denied when run from Java program

I'm trying to run chimpchat from Java code and I get this error:
Unexpected exception 'Cannot run program
"/home/asco/adt-bundle-linux-x86_64/sdk/platform-tools": error=13,
Permission denied' while attempting to get adb version from
'/home/asco/adt-bundle-linux-x86_64/sdk/platform-tools
I can run adb from the shell as a normal user. I've chmod 777'ed the adb program.
What can I do? I run Linux Mint 14 (64bit) (have ia32-libs installed) and use java7-openjdk-amd64.
When I tried to open the Android SDK from Eclipse I also received the error
"...Permission denied' while attempting to get adb version from '..."
initially, I tried
sudo apt-get install ia32-libs
but it did not fix the problem. i had them already.
What fixed the problem was
sudo chmod -R 777 /name-of-root-directory-containing-SDK
Recommendation! Place SDK, JDK and all other manually installed non-system resources in their own root directory.
[ example: /resource ]
Permissions set on this directory make no change to any critical files.
Warning!
Erroneous use of chmod -R on system directories can lead to "must reinstall OS from scratch"
it then complains about not finding the adb program
you need to install ia32-libs package:
sudo apt-get install ia32-libs
The error is specifying the directory that contains adb, rather than the adb executable itself. Check the permissions on the directory, and that your program is attempting to run the proper command (it may be trying to run the directory, which clearly won't work...).

JAVA_HOME environment variable not set Validator.nu HTML Parser

I am trying to install a local version of the Validator.nu server and it keeps failing on trying to build the HTML Parser.
It says it can't find the JAVA_HOME variable which I have set in my .bashrc file and shows correctly when I type "echo $JAVA_HOME" at the prompt
Ideas appreciated thanks
Error output
"hg" pull --update -R build https://bitbucket.org/validator/build/
Not trusting file build/.hg/hgrc from untrusted user dave, group dave
Not trusting file /home/dave/src/checker/build/.hg/hgrc from untrusted user dave, group dave
warning: bitbucket.org certificate with fingerprint 81:2b:08:90:dc:d3:71:ee:e0:7c:b4:75:ce:9b:6c:48:94:56:a1:fe not verified (check hostfingerprints or web.cacerts config setting)
pulling from https://bitbucket.org/validator/build/
warning: bitbucket.org certificate with fingerprint 81:2b:08:90:dc:d3:71:ee:e0:7c:b4:75:ce:9b:6c:48:94:56:a1:fe not verified (check hostfingerprints or web.cacerts config setting)
searching for changes
no changes found
Error: The JAVA_HOME environment variable is not set.
Set the JAVA_HOME environment variable to the pathname of the directory where your JDK is installed.
Instead of:
$ sudo python build/build.py all
try:
$ sudo -E python build/build.py all
The sudo command for security reasons resets the environment (so your JAVA_HOME for the python process is wiped out even when it's exported). The "sudo -E" will preserve the environment.
I assume (from tag) you use ubuntu.
list versions of installed javas in your system:
dave#ubuntu:~$ update-java-alternatives --list
java-6-openjdk 1061 /usr/lib/jvm/java-6-openjdk
Note, that if you set JAVA_HOME in ~/.bashrc it will be set only in your terminal sessions.
Unless you export it, it will be set only for your current shell process (not subprocesses like mercurial).
add line to your .bashrc:
export JAVA_HOME="/usr/lib/jvm/java-6-openjdk"
open a new terminal and test it:
$JAVA_HOME/bin/java -version && echo java seen by bash
bash -c '$JAVA_HOME/bin/java -version && echo java seen by bash subprocesses'
If you want to set environment for all processes (not only started by hand from terminal), you can:
dave#ubuntu:~$ sudo $EDITOR /etc/environment
After tackling this for the last 4 days I have managed to get the validator.nu server running on my local Ubuntu VM and so I thought I would update this thread in case anyone else runs in to the same issues.
I am still not 100% sure where the original issue with the JAVA_HOME variable was coming from but I suspect (although I am not an expert at this) that it had something to do with the way I was using sudo to run the python build.
I was initially following the instructions on http://about.validator.nu/#src but using
$ sudo python build/build.py all
This was because part of the build needed the correct permissions to work.
This is my step-by-step process which starts from a clean install of Ubuntu 11.
installed ubuntu 11
opened the terminal
sudo /bin/bash <----------- I THINK THIS IS THE CRUCIAL LINE
apt-get install mercurial
apt-get install subversion
apt-get install openjdk-6-jre
apt-get install openjdk-6-jdk
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
follow rest of http://about.validator.nu/#src instructions
I'm going to need to do it again when I set this up for the internal network for our build scripts so i'll edit this if I've missed out on anything.
Hope this saves another person's headache and lost days!

Categories

Resources