java.lang.NoClassDefFoundError: running Mahout example on a hadoop Cluster - java

I'm following the Mahout In Action tutorial for kmeans clustring, i use the same code found here:
with the same pom.xml also.
On my local machine using eclipse every thing works fine, so i build the jar file (clustering-0.0.1-SNAPSHOT.jar) and bring it to the cluster (Hortonworks 2.3) when trying to run it using: hadoop jar clustering-0.0.1-SNAPSHOT.jar com.digimarket.clustering.App (I named my project differently) I get this error:
java.lang.NoClassDefFoundError:
org/apache/mahout/common/distance/DistanceMeasure
I know it's a dependency issue, I found questions asked by users who had this issue before but couldn't understand how they solved it.
here and here
This is the content of mahout directory in my cluster:
ls /usr/hdp/2.3.4.0-3485/mahout/
bin
conf
doc
lib
mahout-examples-0.9.0.2.3.4.0-3485.jar
mahout-examples-0.9.0.2.3.4.0-3485-job.jar
mahout-integration-0.9.0.2.3.4.0-3485.jar
mahout-math-0.9.0.2.3.4.0-3485.jar
mahout-mrlegacy-0.9.0.2.3.4.0-3485.jar
mahout-mrlegacy-0.9.0.2.3.4.0-3485-job.jar
Thanks.

It looks like you have a dependency that is not available to your code on your cluster.
Based on the pom.xml from that project you should be using:
<properties>
<mahout.version>0.5</mahout.version>
<mahout.groupid>org.apache.mahout</mahout.groupid>
</properties>
...
<dependencies>
<dependency>
<groupId>${mahout.groupid}</groupId>
<artifactId>mahout-core</artifactId>
<version>${mahout.version}</version>
</dependency>
...
</dependencies>
The class org.apache.mahout.common.distance.DistanceMeasure is included in the mahout-core-0.*.jar I have mahout-core-0.7.jar and the class is present in there.
You can download that jar and include it with the -libjars flag or you can put it on the hadoop classpath.

Related

Get NoClassDefFound Error in Apache Maven 3.8.6 (JDK-19+Gson2.10)

I am new to java and maven. I am learning and trying to make a system to help me build up JSON data by GOOGLE's gson and maven. This is my using command order:
mvn -f my-app\pom xml clean compile
mvn -f my-app\pom xml install
cd my-app\target
java -jar my-app-1.0.jar
After I run it it show me this error.
in GSONExample.java Line 13:
Gson gson = new Gson();
The following is my using software/IDE version.
Source Editor: VS Code
JDK: JDK-19
apache-maven: 3.8.6
Gson: 2.10 (https://github.com/google/gson/releases)
I have already gone through other websites and StackOverflow to find solutions.
GSON is not being imported into the maven project Changed scope role and still crash
error even though it is defined in my classpath /WEB-INF/lib can't find in my situation
Now below a part of my pom.xml
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.10</version>
</dependency>
I also had try to import it(Gson) to local and try to fix it out.
mvn install:install-file -Dfile=C:\gson-2.10.jar -DgroupId=com.myself.gson -DartifactId=example-gson -Dversion=2.10 -f my-app\pom.xml
After I use the command, I changed pom.xml as follow. It can compile and install. But still show me "NoClassDefFound"
<dependency>
<groupId>com.myself.gson</groupId>
<artifactId>example-gson</artifactId>
<version>2.10</version>
</dependency>
May anyone provide any solution to this? Thanks in advance.
I had tried
changed scope to provided in pom.xml
put Gson in src\main\resources\lib and change pom.xml
Local gson-2.10.jar and change pom.xml
put gson-2.10.jar in C:\Program Files\apache-maven-3.8.6\lib
The problem is this command:
java -jar my-app-1.0.jar
Most likely your current Maven pom.xml creates a JAR file which contains only the classes of your project, but not the classes of dependencies, such as Gson. This is also not specific to Gson but applies to any dependencies you are trying to use. You can either:
use java -cp and specify the path to your JAR as well as the paths to all the JARs of dependencies you are using, for example java -cp gson-2.10.jar;my-app-1.0.jar com.mycompany.app.GSONExample
configure Maven to build a "JAR with dependencies", see this question whose answers describe multiple ways of how this can be achieved

Trying to run libGDX with Maven and Eclipse

Maybe a newbie question...
I've been working on a LWJGL project, where I use Maven to manage dependencies. In it, I want to use some parts of the libgdx library. So I figured I will first run at least a helloworld working with it before I add it to my main project.
So in my pom.xml I have this:
<!-- https://mvnrepository.com/artifact/com.badlogicgames.gdx/gdx-backend-lwjgl -->
<dependency>
<groupId>com.badlogicgames.gdx</groupId>
<artifactId>gdx-backend-lwjgl</artifactId>
<version>1.9.11</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/com.badlogicgames.gdx/gdx-platform -->
<dependency>
<groupId>com.badlogicgames.gdx</groupId>
<artifactId>gdx-platform</artifactId>
<version>1.9.11</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/com.badlogicgames.gdx/gdx -->
<dependency>
<groupId>com.badlogicgames.gdx</groupId>
<artifactId>gdx</artifactId>
<version>1.9.11</version>
</dependency>
The other contents of the file are the same as in a working project and are 100% working.
I tried creating a separate libgdx project before that and... it didn't work. But, I saw that the code that was supposed to run the program was:
public static void main (String[] arg) {
LwjglApplicationConfiguration config = new LwjglApplicationConfiguration();
new LwjglApplication(new SomeApplicationListenerFile(), config);
}
So I used that in my maven project.
When I do "run as a Java Application", the error is the following:
Exception in thread "main" java.lang.NoClassDefFoundError: com/badlogic/gdx/backends/lwjgl/LwjglApplicationConfiguration
at org.boby.RayTracing.main.Main.main(Main.java:179)
And if I do a Maven Build, it tells me that "package com.badlogic.gdx.backends.lwjgl does not exist"
I looked for that package in the jars Maven downloaded in the "Maven dependencies" folder and I found it in gdx-backend-lwjgl-1.9.11.jar - right where it should be.
The package is apparently there, but Java cannot find it. How can I fix that?
Some additional information:
Windows 10, eclipse oxygen, Maven 3.6.0, JRE 1.8.0_191, JDK 8
Thank you in advance! I've been banging my head on this for hours.
Edit: I made some progress. Looks like the "test" was messing things up so I removed those statements. Now I get the following Error:
Exception in thread "main" com.badlogic.gdx.utils.GdxRuntimeException: Couldn't load shared library 'gdx64.dll' for target: Windows 10, 64-bit
at com.badlogic.gdx.utils.SharedLibraryLoader.load(SharedLibraryLoader.java:125)
at com.badlogic.gdx.utils.GdxNativesLoader.load(GdxNativesLoader.java:33)
at com.badlogic.gdx.backends.lwjgl.LwjglNativesLoader.load(LwjglNativesLoader.java:47)
at com.badlogic.gdx.backends.lwjgl.LwjglApplication.<init>(LwjglApplication.java:83)
at com.badlogic.gdx.backends.lwjgl.LwjglApplication.<init>(LwjglApplication.java:71)
at org.boby.RayTracing.main.Main.main(Main.java:178)
It looks like I need to include gdx-natives.jar in my dependencies, but I can't find a maven repository for it.
I downloaded gdx-natives.jar (saw it in a forum thread). In there, was a file named "gdx-64.dll". As I need "gdx64.dll", I just renamed the dll and now it runs.
You can let Maven do the work if you define the gdx-platform dep like this:
<dependency>
<groupId>com.badlogicgames.gdx</groupId>
<artifactId>gdx-platform</artifactId>
<version>1.9.11</version>
<classifier>natives-desktop</classifier>
</dependency>
This will load the natives jar including the gdx64.dll so you don't have to add any external jar to your project in the build path.
A side note is: if you use the standard Maven repo directory structure and you load assets with the Gdx.files.internal("fileName") statement you need to define a folder in the main/repository with the same name as the package you have your code in. (i.e. main/java/myPackage relates to /main/repository/myPackage). I struggled a bit with this because I don't normaly have to define a package folder int the repository dir.

Hadoop: How to write to HDFS using a Java application

I am new to Hadoop and trying to learn. I am trying to run the below Hadoop sample code in Eclipse on Ubuntu Linux. I have Hadoop v 2.7.0 and I have the required jars.
Configuration conf = new Configuration();
conf.addResource(new Path("/usr/local/hadoop/conf/core-site.xml"));
Path pt=new Path("hdfs://localhost:9000/myhome/a.txt");
FileSystem fs = FileSystem.get(conf);
When I run the application in Eclipse I get Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/PlatformName. The hadoop-common-2.7.0.jar file I am referencing does not contain the class application is looking for. I am referencing that jar file Hadoop/common folder.
Any help is resolving this issue will be much appreciated.
If I create a jar file of the class for above above code and run it using hadoop -jar <jar file> <class name>, it works. So I am wondering whether it's possible at all to run a Hadoop Java application from Eclipse or command line without using hadoop command.
It seems that the JVM doesn't load all required Hadoop artifacts.
If you are a maven user, please ensure that you have these dependencies.
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-client</artifactId>
<version>${hadoop.client.version}</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${hadoop.client.version}</version>
</dependency>

java.lang.NoSuchMethodError: javax.media.opengl.GLDrawableFactory.initSingleton()V

I am trying to use the JZY3D libraries in my Java project.
Starting with the basic, I set up the following main():
public static void main(final String[] args) {
new GLCapabilities(GLProfile.getDefault());
}
The jogl-all.jar, gluegen-rt.jar and gluegen.jar files are in the build path. I get the following exception:
Exception in thread "main" java.lang.NoSuchMethodError: javax.media.opengl.GLDrawableFactory.initSingleton()V
at javax.media.opengl.GLProfile.initProfilesForDefaultDevices(GLProfile.java:1444)
at javax.media.opengl.GLProfile.access$100(GLProfile.java:76)
at javax.media.opengl.GLProfile$1.run(GLProfile.java:174)
at java.security.AccessController.doPrivileged(Native Method)
at javax.media.opengl.GLProfile.initSingleton(GLProfile.java:140)
at javax.media.opengl.GLProfile.initSingleton(GLProfile.java:199)
at javax.media.opengl.GLProfile.getProfileMap(GLProfile.java:1914)
at javax.media.opengl.GLProfile.get(GLProfile.java:815)
at javax.media.opengl.GLProfile.getDefault(GLProfile.java:602)
at javax.media.opengl.GLProfile.getDefault(GLProfile.java:613)
at eu.crisis_economics.abm.model.SpecsCompetitionStrategyCFO.main(SpecsCompetitionStrategyCFO.java:312)
It seems like there is a problem with the 3d libraries, but I cannot figure out what.
The jogl-all.jar, gluegen-rt.jar and gluegen.jar files are in the build path. I get the following exception:
Remove gluegen.jar from the build path.
Make sure these two jars are added to the classpath at runtime as well!
You only need jogl-all.jar and gluegen-rt.jar (rt == runtime) at compile time and runtime.
Also you need to have a copy of the matching native jars gluegen-rt-natives-os.and.arch.jar and jogl-all-natives-os.and.arch.jar for the platform you use next to the gluegen-rt.jar and jogl-all.jar
http://jogamp.org/jogl/doc/deployment/JOGL-DEPLOYMENT.html#NativeJARFiles
Have you tried the Maven dependency instead of add those jars by hand?
I made a clean maven project, add a new user library with the following files:
org.jzy3d-0.9.jar
dependencies/misc/opencsv-2.1.jar
dependencies/misc/org.convexhull.jar
dependencies/misc/swt.jar
dependencies/misc/swt-debug.jar
After that, I add the following maven dependencies:
<dependency>
<groupId>org.jogamp.gluegen</groupId>
<artifactId>gluegen-rt-main</artifactId>
<version>2.0.2</version>
</dependency>
<dependency>
<groupId>org.jogamp.jogl</groupId>
<artifactId>jogl-all-main</artifactId>
<version>2.0.2</version>
</dependency>
Build & Run with no exceptions.
A much cleaner solution is:
Clone Jzy3D repository (https://github.com/jzy3d/jzy3d-api)
install jzy3d-master
add the jzy3d-api dependency for your project

Getting java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory exception

i am executing simple Dependency Injection program of spring & getting this exception.
I have already included common-logging1.1.1.jar and spring.jar file. Could you please help to out?
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.springframework.context.support.AbstractApplicationContext.<init>(AbstractApplicationContext.java:119)
at org.springframework.context.support.AbstractXmlApplicationContext.<init>(AbstractXmlApplicationContext.java:55)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:77)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:65)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:56)
at com.client.StoryReader.main(StoryReader.java:15)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)
... 6 more
If you're using maven for managing dependencies, add the following lines in your pom.xml:
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</dependency>
I have also faced the same issues, to fix, download the jar files from the below url
http://commons.apache.org/logging/download_logging.cgi
and copy to your lib folder, will resolve your issue.
You just download commons-logging-1.1.2.jar and then copy this file in to libs
finally, it works.
commons-logging-1.1.1.jar or jcl-over-slf4j-1.7.6.jar al
If you are using maven, use the below code.
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>${slf4j.version}</version>
</dependency>
I had the same problem, and solved it by just adding the commons-logging.jar to the class path.
Setting the scope to compile did it for me
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.2</version>
<scope>compile</scope>
</dependency>
Adding commons-logging.jar or commons-logging-1.1.jar will solve this...
I have already included common-logging1.1.1.jar and ...
Are you sure you spelled the name of the JAR file exactly right? I think it should probably be commons-logging-1.1.1.jar (note the extra - in the name). Also check if the directory name is correct.
NoClassDefFoundError always means that a class cannot be found, so most likely your class path is not correct.
Try doing a complete clean of the target/deployment directory for the app to get rid of any stale library jars. Make a fresh build and check that commons-logging.jar is actually being placed in the correct lib folder. It might not be included when you are building the library for the application.
Issue solved by adding commons-logging.jar
Imp files are ,
antlr-runtime-3.0.1
org.springframework.aop-3.1.0.M2
org.springframework.asm-3.1.0.M2
org.springframework.aspects-3.1.0.M2
org.springframework.beans-3.1.0.M2
org.springframework.context.support-3.1.0.M2
org.springframework.context-3.1.0.M2
org.springframework.core-3.1.0.M2
org.springframework.expression-3.1.0.M2
commons-logging-1.1.1
Two options (at least):
Add the commons-logging jar to your file by copying it into a local folder.
Note: linking the jar can lead to problems with the server and maybe the reason why it's added to the build path but not solving the server startup problem.
So don't point the jar to an external folder.
OR...
If you really don't want to add it locally because you're sharing the jar between projects, then...
If you're using a tc server instance, then you need to add the jar as an external jar to the server instance run configurations.
go to run as, run configurations..., {your tc server instance}, and then the Class Path tab.
Then add the commons-logging jar.
I got the same trouble than you.
Finally I checked the version of apache possessing the class.
I found that the version 1.0.4 has the class.
Try to use the version 1.0.4 instead of 1.1.X or 1.2.X
My dependencies :
<dependencies>
<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-core</artifactId>
<version>1.3.5</version>
</dependency>
<dependency>
<groupId>org.jolokia</groupId>
<artifactId>jolokia-client-java</artifactId>
<version>1.3.5</version>
</dependency>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.0.4</version>
</dependency>
</dependencies>
My Java Code
J4pClient j4pClient = new J4pClient("http://localhost:8080/jolokia");
J4pReadRequest req = new J4pReadRequest("java.lang:type=Memory","HeapMemoryUsage");
req.setPath("used");
J4pReadResponse resp = j4pClient.execute(req);
System.out.println(resp.getValue());
My Result :
130489168
Double check also that your maven dependencies are well imported.
http://commons.apache.org/logging/download_logging.cgi
use this url to download jar files and include them in your class path, issue will be solved
The topic is very outdated. But it still can be met ourdays.
commons-logging, or also known as jcl is a deprecated library. The last version was exposed in 2014
You should avoid adding dependency on it directly in your projects. I assume the most of answers and the accepted one are not actual anylonger.
A preferrable way to use in your projects new alternatives, like slf4j or log4j2, which play the same role, as jcl. The reasons and motivation is another big topic, not for the scope of this issue.
If your application uses log4j2, and you meet the error, add dependency:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-jcl</artifactId>
<version>2.y.z</version>
</dependency>
If you prefer slf4j, (already offered in previous comments/replies ) use:
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>${slf4j.version}</version>
</dependency>
If you use Spring, most probably you have in the dependency tree:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jcl</artifactId>
</dependency>
and it solves the issue as well.
In the examples I skipped certain versions by purpose, they get deprecated very quickly, see Offitial Maven repository.
In some cases you should not use version attribute at all, preferring using dependencies from BOM files. Spring is an example.
Just check whether the commons-logging.jar has been added to your libs and the classpath.. I had the same issue and that was because of this.
dhammikas-
I generally assign the classpath to a variable and then verify it. I've written a small ruby script which I include in a my startup scripts which validates the classpath before launching java. Validating the classpath before the JVM starts has saved me lots of time troubleshooting these types of problems.
Hey I was following the tutorial on tutorialpoint.com. Add after you complete Step 2 - Install Apache Common Logging API: You must import external jar libraries to the project from the files downloaded at this step. For me the file name was "commons-logging-1.1.1".
If you're running this on Android then note that apparently java.beans package is not complete on Android. To attempt to fix it on Android try the following:
Download android-java-air-bridge.jar (currently the download button is on the bottom of the page or direct link here)
Copy the downloaded jar to your [APPROOT]/app/libs directory (or link the jar in any other way)
Change the import *** statements to that of air-bridge. Eg import javadz.beanutils.BeanUtils instead of import org.apache.commons.beanutils.BeanUtils;
Clean and rebuild the project
source 1, source 2
I apologise as I realise this is not exactly answering the question, though this SO page comes up a lot when searching for android-generated NoClassDefFoundError: Failed resolution of: beanUtils errors.
I was getting the same error while the jar was present. No solution worked. What worked was deleting the jar from the file system (from .m2 directory) and then cleaning the maven project.
I have the same problem in eclipse IDE, my solution was:
Right click in My project > Properties
Click in Maven and write: jar in the Active Maven Project
Finally, Apply and Close
In my case I was testing a Tomcat app in eclipse and got this error. I solved it by checking the .classpath file and corrected this entry:
<classpathentry kind="con" path="org.eclipse.m2e.MAVEN2_CLASSPATH_CONTAINER">
<attributes>
<attribute name="maven.pomderived" value="true"/>
<attribute name="org.eclipse.jst.component.dependency" value="/WEB-INF/lib"/>
</attributes>
</classpathentry>
The attribute org.eclipse.jst.component.dependency had been missing.
Check whether the jars are imported properly. I imported them using build path. But it didn't recognise the jar in WAR/lib folder. Later, I copied the same jar to war/lib folder. It works fine now. You can refresh / clean your project.
Hello friends if your getting any not class found exception in hibernate code it is the problem of jar files.here mainly two problems
1.I mean to say your working old version of hibernate may be 3.2 bellow.So if u try above 3.6 it will works fine
2.first checkes database connection.if it database working properly their was a mistake in ur program or jar file.
please check these two prioblems if it also not working you tried to IDE . I am using netbeanside 6.9 version.here hibernate working fine.you dont get any error from class not founnd exception..
I hope this one helps more
try adding this dependency
org.apache.commons
commons-exec
1.3
If all else fails, as it had for me, try putting the commons-logging-x.y.z.jar in your Tomcat lib directory. It solved the problem! BTW, I am using Tomcat 6.
Solution is to Add common-logging.x.x jar file

Categories

Resources