pyspark not able to start - java

I am having Windows 8.1 operating system. I installed apache spark and then, installed Java JDK. I am using git bash in windows. I am having below setting in my .bash_profile of git bash.
export SPARK_HOME="/c/tools/spark-2.3.1-bin-hadoop2.7"
export PATH="$SPARK_HOME/bin:$PATH"
export JAVA_HOME="/c/Program Files (x86)/Java/jdk1.8.0_181/"
When I try to call pyspark, I am getting error as given below.
/c/tools/spark-2.3.1-bin-hadoop2.7/bin/pyspark: line 45: python:
command not found Error: Could not find or load main class
org.apache.spark.launcher.Main
I tried running spark-shell in the path "/c/tools/spark-2.3.1-bin-hadoop2.7/bin/". Still getting the same error.
Please help me in making pyspark running in my machine.

I was finally able to make pyspark work in Windows, by following the steps in the blog
http://nishutayaltech.blogspot.com/2015/04/how-to-run-apache-spark-on-windows7-in.html
I had to install few more tools and set environment variables and finally make it working.

Related

Java Unable to load libary 'tesseract' not found in resource path [duplicate]

I've had tesseract and Tess4J running on my MBP for a while now. Today I started to migrate my app to the server and started installing everything on the server. Prior to running Tess4J in tomcat I tried to run a simple java program to make sure everything is fine and dandy. It's not...
I'm on a centOS 64bit server
I've installed tesseract and its working fine - tesseract myimage.jpg mytext produces data
However, running my simple class that useses Tess4j produces this error:
Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to
load library 'tesseract': libtesseract.so: cannot open shared object
file: No such file or directory
What I've done so far
I've ran sudo ldconfig after installing tesseract
I've search for libtesseract.so and its found in /usr/local/lib/libtesseract.so
Folder on server contains these files:
myimge.png
ghost4j-0.3.1.jar
jai_imageio.jar
jna.jar
maslab.jar
pngj.jar
tess4j.jar
TesseractExample.class
TesseractExample.java
tesseract -v produces:
tesseract -v
tesseract 3.02.02
leptonica-1.69
libjpeg 6b : libpng 1.2.49 : libtiff 3.9.4 : zlib 1.2.3
Question
How can I make Tess4J aware that libtesseract.so does exist?enter code here
I had this problem too.
Did you run the following after installing tesseract:
sudo ldconfig
That fixed it for me.
You must set LD_LIBRARY_PATH environment variable to the path where libtesseract.so is.
export LD_LIBRARY_PATH=/usr/local/lib
It is necessary to define the variable jna.platform.library.path. For instance:
-Djna.platform.library.path=/usr/local/lib/
Maybe it is a 32 bit library .so installed.

Can't compile GWT application on AWS instance (Linux)

I'm getting an error when trying to compile a GWT application on an AWS EC2 Instance under AMI Linux.
I already installed Java and have it in directory /usr/lib/jvm/java-1.7.0-openjdk.x86_64.
I also set up the enviroment variables:
$JAVA_HOME = /usr/lib/jvm/java-1.7.0-openjdk.x86_64
$PATH = /sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin;/usr/lib/jvm/java-1.7.0-openjdk.x86_64/bin
So, when I try to run the command:
java -cp
"/var/lib/gwt/gwt-2.6.1/gwt-dev.jar;/var/my-project-source-folder"
com.google.gwt.dev.Compiler com.mypackage.MyClass
I'm getting this error:
Error: Could not find or load main class com.google.gwt.dev.Compiler
I downloaded and unziped GWT 2.6.1 again but the files are fine. The same command is working perfectly in my Windows machine. Do you know what's happening?
Ok, this is embarrassing.
Here is the answer:
I was separating the paths with ; (Windows) instead of : (Linux). BTW, who would think in separate things with :, you Linux guys know how to make hard the easy...

Error running 'webdriver-manager start' on Windows 8.1

I am attempting to run end-to-end test for an angular project using protractor and am getting the following error when I run the command 'webdriver-manager start'
I have run webdriver-manager update, and selenium standalone and chromedriver are up to date.
I have java installed. java -version returns
I also have the path to java in my system environment variables (C:\Program Files\Java\jre1.8.0_31\bin)
Does anyone have any idea how to fix this error?
You can also start the web driver as below,
java -Dwebdriver.ie.driver=D:\Selenium_download_New\IEDriverServer.exe -jar D:\Selenium_download_New\selenium-server-standalone-2.45.0.jar
java -Dwebdriver.chrome.driver=D:\Selenium_download_New\chromedriver.exe -jar D:\Selenium_download_New\selenium-server-standalone-2.45.0.jar
Note the Standalone.jar should be placed in the exact location as above, you can also download the standalone.jar from selenium website
I had the same issue. Probably the webdriver update you are running is installing Selenium required files in another folder that it isn't inside your project folder. At least that was my case.
I was using a grunt task for running Protractor tests. If you are using the same plugin check this folder:
C:\{yourprojectfolder}\node_modules\grunt-protractor-runner\node_modules\protractor\selenium
If you don't find that folder then you are under the same issue as me. Try with another bash. You're using cmd, try with Git Bash. Locate yourself in the following directory:
C:\{yourprojectfolder}\node_modules\grunt-protractor-runner\node_modules\protractor\bin
Then run:
webdriver-manager update
Finally check the selenium folder inside your Protractor plugin. It should be created now.
I didn't have time to find why cmd is behaving different than Git Bash when running the same command inside the same folder. Each one installs the Selenium files in different places.
Maybe you are missing a system variable(C:\Windows\System32\ variable.). Adding this variable would solve your problem.
Hope, it helps!
Add "C:\Windows\System32\" to your PATHs.
On windows, right-click My Computer -> Properties -> Advanced system settings -> Environment Variables -> Edit... PATH and add "C:\Windows\System32\" to the end (with a semi-colon separating each variable).
Otherwise
SET PATH=%PATH%;C:\Windows\System32
Then start web driver using below command
webdriver-manager start
SPAWN command is not available in windows.
Install git. git bash will be installed too.
Now add git bash path to the PATH environment variable.
Restart your command prompt and it will run.

jnetpcap library unable to build on ubuntu 12.04

I am trying to install JNetPcap and followed the instructions given at here. At step 12, I am unable to run the ant command and i see the error
Error: JAVA_HOME is not defined correctly.
We cannot execute /usr/lib/jvm/java-6-sun/bin/java
As I am able to run Java classes from eclipse or from command line I don't think if it's a problem with JAVA_HOME.
echo $PATH shows
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/lib/jvm/java-6-sun/bin
Kindly let me know if am missing something here.
thanks in advance
Neither running java from the command line or running eclipse will require JAVA_HOME to be set. However, the build procedure you are trying to use ant, and ant often does require JAVA_HOME to be set appropriately. (It actually depends on the version of ant that you are using. The use of JAVA_HOME is typically in the wrapper script for ant.)
Just set it.
JAVA_HOME should probably be set to /usr/lib/jvm/java-6-sun ... based on what you gave said PATH to.
However, it is also possible that the problem is that your PATH is incorrect. Or that you have (somehow) managed to get the owner/group/permissions on your Java install incorrect, such that the java command isn't executable.
Check that running java -version displays the installed Java version.
Repeat with /usr/lib/jvm/java-6-sun/bin/java -version.

Neo4J Server Installation Linux AWS Centos OS -- service user cannot find java

I have downloaded Neo4J enterprise from the Neo site and have untar'd it under /opt
I have successfully downloaded and installed the Java 7 Server edition from the Oracle web site. Note after indicating I want a server JRE, Oracle downloads this : jdk1.7.0_51
I have untar'd both under /opt giving me /opt/neo4j-2.0.0 and /opt/jdk1.7.0_51
I have successfully installed Neo4J 2.O on AWS with Centos Linux, such that it should run as a Service under a service account. To do this I used the Neo install command. The command ran successfully
I can successfully start Neo4J as myself. After editing my ~/.bash_profile to define JAVA_HOME and exporting.
export JAVA_HOME="/opt/jdk1.7.0_51/"
export NEO4J_HOME="/opt/neo4j-2.0.0"
export PATH=$JAVA_HOME/bin:$NEO4J_HOME:$PATH
Apply
source ~/.bash_profile
cd /$NEO$J_HOME
bin/neo4j start
Works just fine.
However, we are not able to successfully get Neo to start as a service
sudo service neo4j-service start
which: no java in (/sbin:/usr/sbin:/bin:/usr/bin)
ERROR: Unable to find java. (Cannot execute )
* Please use Oracle(R) Java(TM) 7 to run Neo4j Server. Download "Java Platform (JDK) 7" from:
http://www.oracle.com/technetwork/java/javase/downloads/index.html
* Please see http://docs.neo4j.org/ for Neo4j Server installation instructions.
If I echo $JAVA_HOME it returns as expected
If I echo $PATH it returns as expected
by as expected we can see java on the path.
If I simply type java -version before I run sudo service neo4j-service start
the response shows the java version.
A bit of background: the Linux service command is not available unless I first run a special script per our normal it policies. However, the script clobbers the exports that I would otherwise use.
Accordingly the command
service neo4j-service start
returns
-bash: neo4j-service: command not found
indicating that the 'service' command cannot be found
So sudo is required in our environment to execute the service command
We have tried simply placing jdk1.7.0_51 under /sbin/jdk1.7.0_51 that does not work. We have also tried extracting the jre folder from jdk1.7.0_51 and placing it under /sbin/jre
Also tried creating a java.sh file under /etc/profile.d/
Still no success.
So the simple question....
Where is Neo4J 2.0.0 looking for Java when running as a service? Is it using the environment variable JAVA_HOME, or is it looking in one of the following locations as the Neo4J server error message would seem to indicate? "no java in (/sbin:/usr/sbin:/bin:/usr/bin)"
Under my installation configuration the neo4j service is intended to run under the account neo4jservice. Is this as simple as making sure JAVA_HOME is available to the neo4jservice accound? Why the error message "no java in (/sbin:/usr/sbin:/bin:/usr/bin)"?
Thanks in advance
I ran a test on my solaris and Linux Mint install and this seems to work for me
Perhaps it's not elegant, but it will get you going. Just drop the sudo part for your startup script.
sudo NEO4J_HOME="/opt/neo4j" JAVA_HOME="/usr/java" /opt/neo4j/bin/neo4j start
Obviously, modify the environment variables to your situation:
JAVA_HOME="/opt/jdk1.7.0_51/" NEO4J_HOME="/opt/neo4j-2.0.0" service neo4j-service start
Not quite the answer I was hoping for... as the answer seems to disregard this part of the original POST: "So sudo is required in our environment to execute the service command"
What does seem to work is modifying the neo4j-service script to include the exports.
neo4j-service is found at /etc/init.d

Categories

Resources