Does anybody know a tutorial how HBase can be run on Windows under Cygwin?
I managed to setup everything such as the keys for auto login with SSH but Im stuck with following error message:
localhost: +======================================================================+
localhost: | Error: JAVA_HOME is not set and Java could not be found |
localhost: +----------------------------------------------------------------------+
localhost: | Please download the latest Sun JDK from the Sun Java web site |
localhost: | > http://java.sun.com/javase/downloads/ < |
localhost: | |
localhost: | HBase requires Java 1.6 or later. |
localhost: | NOTE: This script will find Sun Java whether you install using the |
localhost: | binary or the RPM based installer. |
localhost: +======================================================================+
...
java.lang.NoClassDefFoundError: org/apache/zookeeper/KeeperException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Unknown Source)
at java.lang.Class.getMethod0(Unknown Source)
at java.lang.Class.getMethod(Unknown Source)
at sun.launcher.LauncherHelper.getMainMethod(Unknown Source)
at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)
Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.KeeperException
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
I already set .bash_profile to point to:
export JAVA_HOME='/usr/lib/jre1.6.1'
which is symbolic link on the real jre
/usr/lib/jre1.6.1 -> /cygdrive/c/Java/jre7/
I had the same issue and I found following:
Cygwin use ":" as a path separator.
But script that creates CLASS_PATH works incorrectly. Final CLASS_PATH looks like
"...:zookeeper.jar:" with ":" at the end. Before run java script converting CLASS_PATH to Windows style. But with ":" at the end !!
So my solution is: just create an empty file "zzz.jar" under hbase/lib.
zzz.jar will be very last in CLASS_PATH and zookeeper.jar will be added to CLASS_PATH correctly.
I know it is better to fix script, but I who cares :)
I need HBase for 2 days only, so it works and I am happy :)
For JAVA_HOME error, you can fix it by setting JAVA_HOME in the hbase\conf\hbase-env.sh.
For the class not found exception, try adding the hbase\lib to the HBASE_CLASSPATH under the same file.
Personally, I use the following two lines in my hbase-env.sh on a Windows 7 Environment + Cygwin:
export JAVA_HOME=/cygdrive/c/Program\ Files/Java/jdk1.7.0/
export HBASE_CLASSPATH=/cygdrive/c/programs/hbase-0.92.1/lib/
Use a VM to run hadoop in a Linux environment. With Java you may be facing issues typical under windows, adding Cygwin to the mix adds a whole other set, for example:
spaces in the include path
multiple installations of java from different applications (e.g. browser plugins) can mess up the consistency of the the CLASSPATH, JAVA_HOME and PATH variables
shell script path/syntax issues
For tutorial purposes I highly recommend the cloudera demo VM and tutorial:
https://ccp.cloudera.com/display/SUPPORT/Cloudera%27s+Hadoop+Demo+VM
https://ccp.cloudera.com/display/SUPPORT/Hadoop+Tutorial#HadoopTutorial-Purpose
EDIT:
The first error looks like Java is not being found, but the second error is clearly coming from Java, so the combination of these two errors is somewhat confusing. Are you sure that they are both being produced by the same script at the same time?
The second error is caused by not having the zookeeper jar file on the classpath. Try modifying the script or copying the jar into the hadoop lib dir.
hbase_classpath is not need, just change zookeeper.jar to azookeeper.jar
Related
Since upgrading my os to Mac Catalina,
I'm getting the following error when trying to run the following:
MysqldResource mysqldResource = new MysqldResource(uri); // Uri - temp db dir
mysqldResource.start("embedded-mysqld-thread-" + System.currentTimeMillis(), databaseOptions);
Looks like the source of the problem is with the dependency "mysql-connector-mxj-db-files" having binary files in 32bit format, while catalina stopped supporting it.
This is the error I'm getting:
Exception in thread "embedded-mysqld-thread-1574931580040" com.mysql.management.util.WrappedException: java.io.IOException: Cannot run program "/private/var/folders/l5/4ywgvbbj1kvgvrwxzp6dz82c0000gn/T/test_db_24166997810717/bin/mysqld": error=86, Bad CPU type in executable
at com.mysql.management.util.Exceptions.toRuntime(Exceptions.java:55)
at com.mysql.management.util.Exceptions$Block.exec(Exceptions.java:89)
at com.mysql.management.util.RuntimeI$Default.exec(RuntimeI.java:180)
at com.mysql.management.util.Shell$Default.run(Shell.java:140)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Cannot run program "/private/var/folders/l5/4ywgvbbj1kvgvrwxzp6dz82c0000gn/T/test_db_24166997810717/bin/mysqld": error=86, Bad CPU type in executable
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at java.lang.Runtime.exec(Runtime.java:620)
at com.mysql.management.util.RuntimeI$Default$1.inner(RuntimeI.java:177)
at com.mysql.management.util.Exceptions$Block.exec(Exceptions.java:86)
... 3 more
Caused by: java.io.IOException: error=86, Bad CPU type in executable
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 6 more
Does anyone know of an alternative to mysql-connector-java or knows of a way to run the existing dependency on Mac OS Catalina?
For anyone who has the same issue on MacOS Catalina, you can follow the README.md in my repo (https://github.com/pengyue/mysql-connector-mxj-mac-os-catalina), which has the 64 bits executables for MacOS Catalina, and this improved connector works on Catalina.
To explain what I have done:
Download the 64 bits executables from MySQL to the folder 5-5-9
Add the platform mapping for 64 bits MacOS in platform-map.properties
Create a jar by using jar cvf mysql-connector-mxj-db-files-5.0.12.jar .
Replace the jar in your mvn settings folder ~/.m2/repository/mysql/mysql-connector-mxj-db-files/5.0.12/ with the jar you just created.
This solution works fine for my projects, Unfortunately I could not find a github for the mysql-connector-mxj, as it is deprecated and not maintainable anymore.
The other option is to use wix-embedded-mysql(https://github.com/wix/wix-embedded-mysql) instead of mysql-connector-mxj, but this probably requires some code changes in your projects.
When i run the install.bat file on a windows 7 machine. i receive the following error:
Detecting Java version installed. Detecting if it is 64 bit machine
CurrentVersion Detecting if it is 32 bit machine CurrentVersion 1.8
Java 1.8 found! Installing thingsboard ... Error: Could not find or
load main class Files\thingsboard\lib\thingsboard.jar ThingsBoard
installation failed!
I assumed it was reading the BASE file directory incorrectly. So I it to the absolute directory location as shown below:
SET BASE="C:\Program Files\thingsboard"
SET LOADER_PATH=%BASE%\conf,%BASE%\extensions
SET SQL_DATA_FOLDER=%BASE%\data\sql
SET jarfile=%BASE%\lib\thingsboard.jar
SET installDir=%BASE%\data
I get the following error:
Detecting Java version installed. Detecting if it is 64 bit machine
CurrentVersion Detecting if it is 32 bit machine CurrentVersion 1.8
Java 1.8 found! Installing thingsboard ... Logging system failed to
initialize using configuration from 'C:\Program
Files\thingsboard\install\logback.xml'
java.lang.IllegalStateException: Logback configuration error detected:
ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[fileLogAppender] -
Failed to create parent directories for [C:\ Program
Files\thingsboard\conf\"C:\Program
Files\thingsboard"\logs\install.log] ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[fileLogAppender] -
openFile("C:\Program Files\thingsboard"\logs /install.log,true) call
failed. java.io.FileNotFoundException: "C:\Program
Files\thingsboard"\logs\install.log (The file name, directory name, or
volume label syntax is incorrect)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.loadConfiguration(LogbackLoggingSystem.java:161
)
at org.springframework.boot.logging.AbstractLoggingSystem.initializeWithSpecificConfig(AbstractLoggingSystem.jav
a:57)
at org.springframework.boot.logging.AbstractLoggingSystem.initialize(AbstractLoggingSystem.java:47)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.initialize(LogbackLoggingSystem.java:114)
at org.springframework.boot.logging.LoggingApplicationListener.initializeSystem(LoggingApplicationListener.java:
304)
at org.springframework.boot.logging.LoggingApplicationListener.initialize(LoggingApplicationListener.java:272)
at org.springframework.boot.logging.LoggingApplicationListener.onApplicationEnvironmentPreparedEvent(LoggingAppl
icationListener.java:235)
at org.springframework.boot.logging.LoggingApplicationListener.onApplicationEvent(LoggingApplicationListener.jav
a:208)
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMult
icaster.java:166)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMult
icaster.java:138)
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMult
icaster.java:121)
at org.springframework.boot.context.event.EventPublishingRunListener.environmentPrepared(EventPublishingRunListe
ner.java:68)
at org.springframework.boot.SpringApplicationRunListeners.environmentPrepared(SpringApplicationRunListeners.java
:54)
at org.springframework.boot.SpringApplication.prepareEnvironment(SpringApplication.java:337)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
at org.thingsboard.server.ThingsboardInstallApplication.main(ThingsboardInstallApplication.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:521)
java.lang.IllegalStateException: Logback configuration error detected:
ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[fileLogAppender] -
Failed to create parent directories for [C:\ Program
Files\thingsboard\conf\"C:\Program
Files\thingsboard"\logs\install.log] ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[fileLogAppender] -
openFile("C:\Program Files\thingsboard"\logs /install.log,true) call
failed. java.io.FileNotFoundException: "C:\Program
Files\thingsboard"\logs\install.log (The file name, directory name, or
volume label syntax is incorrect) ThingsBoard installation failed!
What am I doing wrong?
The quotes seem to be a problem.
C:\ Program Files\thingsboard\conf\"C:\Program
Files\thingsboard"\logs\install.log java.io.FileNotFoundException:
"C:\Program Files\thingsboard"\logs\install.log
Have you tried to use SET BASE=%ProgramFiles%\thingsboard or escape the space in the path with a ^ SET BASE=C:\Program^ Files\thingsboard?
edit:
The problem is because of the space in the install dir.
I have tried to install it in "c:\test folder" and got the same problem.
There was no problem when installing in a folder without space like "c:\test"
I changed the following part in the install.bat file and was able to install it in the folder with a space.
SET "BASE=%~dp0"
SET "LOADER_PATH=%BASE%conf,%BASE%extensions"
SET "SQL_DATA_FOLDER=%BASE%data\sql"
SET "jarfile=%BASE%lib\thingsboard.jar"
SET "installDir=%BASE%data"
PUSHD %BASE%conf
java -cp "%jarfile%" -Dloader.main=org.thingsboard.server.ThingsboardInstallApplication "-Dinstall.data_dir=%installDir%" -Dinstall.load_demo=%loadDemo% -Dspring.jpa.hibernate.ddl-auto=none -Dinstall.upgrade=false "-Dlogging.config=%BASE%install\logback.xml" org.springframework.boot.loader.PropertiesLauncher
if errorlevel 1 (
#echo ThingsBoard installation failed!
POPD
exit /b %errorlevel%
)
POPD
"%BASE%thingsboard.exe" install
When I received this error, I ran the command with Run as Administrator and my problem was solved.
if you receive the same result:
Install JRE
Delete the space in the installation directory
Run command with Run as Administrator
I usually use Weka from command line on Linux systems to perform feature selection on attributes as:
java -cp PATH_TO_WEKA_JAR weka.attributeSelection.CfsSubsetEval ... (other parameters)
I'm trying to run the same code on Mac OS but I have this error:
Exception in thread "main" java.lang.NoClassDefFoundError: weka.attributeSelection.CfsSubsetEval
Caused by: java.lang.ClassNotFoundException: weka.attributeSelection.CfsSubsetEval
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
How can I run the same command on Mac OS? Shouldn't be the same as it's a UNIX based OS?
I also tried to include the path within " " but there is no difference. What's is wrong?
Try this command:
jar tf PATH_TO_WEKA_JAR | grep weka.attributeSelection.CfsSubsetEval
In the output you should see a line with weka.attributeSelection.CfsSubsetEval.class.
If you don't see such line, then the jar file doesn't contain that class,
and the command cannot work.
In that case, try to run this:
jar tf PATH_TO_WEKA_JAR | less
to just see what is in the jar file.
One way or another, this is a simple class path issue:
the class weka.attributeSelection.CfsSubsetEval is simply not on your classpath.
You need to find the correct path to the jar,
possible to other dependencies as well,
and construct the correct parameter to use in:
java -cp CORRECT_CLASSPATH weka.attributeSelection.CfsSubsetEval # ... your other params
I guess there is something wrong with the Weka jar file. It tells you it can't find a particular class in the file.
Mac OS has evolved away from Unix quite a bit, which may make it necessary to use a differend JAR file.
This may help you: Weka Site download
I am in the folder :
/cygdrive/d/ragsxq where I have kept some simple X-Query stmts to be executed via Saxon X-Query engine...(Saxon 9.4 HE )
The following statement :
$ java -classpath /cygdrive/d/saxon/saxon9he.jar net.sf.saxon.Query -q:ragsXQuery.xq
gives :
java.lang.NoClassDefFoundError: net/sf/saxon/Query Caused by:
java.lang.ClassNotFoundException: net.sf.saxon.Query at
java.net.URLClassLoader$1.run(Unknown Source) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(Unknown Source) at
java.lang.ClassLoader.loadClass(Unknown Source) at
sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at
java.lang.ClassLoader.loadClass(Unknown Source) Could not find the main class:
net.sf.saxon.Query. Program will exit. Exception in thread "main"
But the below statement works fine:
$ java -classpath ../saxon/saxon9he.jar net.sf.saxon.Query -q:ragsXQuery.xq
?!?!
Something to do with the Class-loader..?? Permmissions..?? Cygwin ? Hmmmmm...
Thanks a lot for pointing out this Cygwin behavior!
But the strange thing is, i run into this java.lang.NoClassDefFoundError (exactly the same as above) when I try to Run an Xquery via my Eclipse (Indigo) on Mac OS Lion too...via the XQDT plugin, where I have to configure the XQuery engine by providing the full-path of the saxon9he.jar & the main-class viz., net.sf.saxon.Query
Note that a similar set-up on Windows-XP works fine.
This time, I guess it could be some Classpath issue or the way Mac OS's Java treats things differently ?
I think I have faced this before. It is probably a Cygwin nuance. Just try
$ java -classpath `cygpath -m /cygdrive/d/saxon/saxon9he.jar` net.sf.saxon.Query -q:ragsXQuery.xq
Here is a possible explanation as to why:
http://cygwin.com/ml/cygwin/2008-01/msg00083.html
Below is the output from my Cygwin for an executable JAR file. I use Windows Vista 32 bit (I know, it sucks).
adarshr$ java -jar /cygdrive/d/adarshr/data/cf.jar
Error: Unable to access jarfile /cygdrive/d/adarshr/data/cf.jar
adarshr$
adarshr$ java -jar `cygpath -m /cygdrive/d/adarshr/data/cf.jar`
Usage: java -jar cf.jar SEARCH [DIRECTORY] [OPTIONS]...
Try -h or --help for more information
As you can see, the second command worked and printed a help (as expected).
I've downloaded the latest com4j jars and I'm trying to run through the most simple of their examples.
I'm on Windows 7 64 bit and using a 64 bit JVM.
When I run the command (from the com4j tutorial):
java -jar tlbimp.jar -o wsh -p test.wsh %WINDIR%\system32\wshom.ocx
I get:
Exception in thread "main" java.lang.UnsatisfiedLinkError: com4j-amd64.dll: %1 is
not a valid Win32 application
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at com4j.COM4J.loadNativeLibrary(COM4J.java:569)
at com4j.COM4J.<clinit>(COM4J.java:522)
at com4j.tlbimp.driver.Lib.getLibid(Lib.java:90)
at com4j.tlbimp.driver.Driver.addLib(Driver.java:38)
at com4j.tlbimp.driver.Main.doMain(Main.java:123)
at com4j.tlbimp.driver.Main.main(Main.java:56)
Exception in thread "Com4J shutdown hook" java.lang.NoClassDefFoundError: Could
not initialize class com4j.COM4J
at com4j.COM4J$3.run(COM4J.java:476)
The java code extracts "com4j-amd64.dll" next to the jar.
The latest version has just been released:
http://kohsuke.org/2012/04/27/com4j-updates/
https://github.com/kohsuke/com4j/downloads
Using this one solved all my problems.
You have two options:
Download and use the full latest version (located here:
https://github.com/kohsuke/com4j/downloads)
Extract the latest com4j-amd64.dll from the most recent version
Download the latest version (located here: https://github.com/kohsuke/com4j/downloads).
Open up the com4j.jar with your favorite zip utility (read 7zip)
Extract com4j.jar\com4j\com4j-x64.dll to the base directory of your application
(optional) Register the dll using REGSVR32 <path to dll>\com4j-x64.dll
I had a similar result at first. I found the com4j-amd64.dll that is produced by com4j is empty. The remedy is to extract the file from the com4j jar file, and put it in the relevant directory by manual action. Com4j will then not auto-produce the file since it is already there. So check if the file is empty as a first step -Carl