Hadoop: strange ClassNotFoundException - java

I am getting a classnotfound exception. The class which is claimed to be not found does not exist, but the class name is set as the path to the list of input files for my map reduce jobs.
INFO server Running: /usr/lib/hadoop/bin/hadoop --config /var/run/cloudera-scm-agent/process/155-hue/JOBSUBD/hadoop-conf jar tmp.jar /user/hduser/datasets/ /user/hduser/tmp/job_20/ mongodb://slave15/db_8.job_20
Exception in thread "main" java.lang.ClassNotFoundException: /user/hduser/datasets/
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(Runjar.java:190)
As we can see, the /user/hduser/datasets/ is the path to the input files. Why am I getting this error as ClassNotFoundException? Why does it see it as a class?
I found my own error. I have a package structure. I need to specify my package information
/usr/lib/hadoop/bin/hadoop
--config /var/run/cloudera-scm-agent/process/155-hue/JOBSUBD/hadoop-conf
jar tmp.jar org.myorg.tmp /user/hduser/datasets/
/user/hduser/tmp/job_20/ mongodb://slave15/db_8.job_20
In my tool, there is no option for giving the package as argument to Java. So I need to have no packaging. But then I am having the following error since the argument before this input file path is missing.
My classes are directly in the tmp.jar in its root. I mean no org.myorg etc...
SOLUTION:
jar cmf [manifest_file] [jar_name.jar] -C [folder_of_classes] [path_for_jar_file]
it will merge the content of the manifest_file with the generated manifest file in the jar archive. Include the following line in the manifest_file
Main-Class: [Name_Of_Class]

Whether or not there's a package hierarchy (and if there isn't, you've done Something Wrong) you still need to give it the name of the class containing main.
For example, the docs have an example where the main class is org.myorg.WordCount. Even if WordCount was in the default package, it should be specified if the jar file doesn't include a main class in the manifest:
bin/hadoop jar /usr/joe/wordcount.jar WordCount /usr/joe/wordcount/input /usr/joe/wordcount/output
I'd assume you could also specify the main class in the manifest as with any jar; the class argument is shown as optional in those same docs.

Related

How do you prepend paths to a java class?

I'm building an older kafka (0.8.2.2) and am getting runtime errors about a missing java class:
java.lang.NoClassDefFoundError: org/apache/kafka/common/KafkaException
When I look in libs/kafka_2.10-0.8.2.2.jar, I see the existence of kafka/common/KafkaException:
$ jar tf $CLASSPATH | grep KafkaException
kafka/common/KafkaException.class
So it appears that the build (gradlew jar) constructed the class with the org/apache missing at the start of the class path. How do I fix the path?
You can't prepend paths to a Java class. The path must correspond to the package name. In your case the class kafka.common.KafkaException is found in libs/kafka_2.10-0.8.2.2.jar and org.apache.kafka.common.KafkaException is found in libs/kafka-clients-0.8.2.2.jar.
You must include libs/kafka-clients-0.8.2.2.jar in your classpath as well.

NoClassDefFoundError when attempting to run from command line

Apologies in advance - I know there are many, many, many very similar questions but I wanted to ask something specific to my situation.
I have a bunch of java and jar files in the same directory. I am able to compile fine and thus end up with a number of class files in the same dir. But when I go to execute the program it gives a NoClassDefFoundError saying it can't find the specified class:
C:\Users\DB\Desktop\nextreports-integration-demo\src\ro\nextreports\integration>
java -cp ".;*.jar" SimpleDemo
Exception in thread "main" java.lang.NoClassDefFoundError: SimpleDemo (wrong nam
e: ro/nextreports/integration/SimpleDemo)
I tried the same thing from a higher-level dir but it made no difference:
C:\Users\DB\Desktop\nextreports-integration-demo\src>java -cp ".\ro\nextreports\
integration\*.jar;.\ro\nextreports\integration" SimpleDemo
Exception in thread "main" java.lang.NoClassDefFoundError: SimpleDemo (wrong nam
e: ro/nextreports/integration/SimpleDemo)
The package statement in the source file is:
package ro.nextreports.integration;
I have a feeling I'm overlooking something very elementary. Thanks in advance.
Edit: Thanks very much. It works with the following:
java -cp ".\ro\nextreports\integration\nextreports-engine-6.3.jar;.\ro\nextreports\integration\commons-jexl-2.1.1.jar;.\ro\nextreports\integration\commons-logging-1.1.1.jar;.\ro\nextreports\integration\derby-10.10.1.1.jar;.\ro\nextreports\integration\itext-2.1.7.jar;.\ro\nextreports\integration\itext-rtf-2.1.7.jar;.\ro\nextreports\integration\itextpdf-5.0.6.jar;.\ro\nextreports\integration\jcalendar-1.3.2.jar;.\ro\nextreports\integration\jcommon-1.0.15.jar;.\ro\nextreports\integration\jfreechart-1.0.12.jar;.\ro\nextreports\integration\jofc2-1.0.1.jar;.\ro\nextreports\integration\mysql-connector-java-5.1.23-bin.jar;.\ro\nextreports\integration\mysql-connector-java-5.1.23-bin.jar;.\ro\nextreports\integration\poi-3.7.jar;.\ro\nextreports\integration\winstone-lite-0.9.10.jar;.\ro\nextreports\integration\xstream-1.3.1.jar;" ro.nextreports.integration.SimpleDemo
But why can I not use wildcards for the *.jar files? For instance, the following leads to a NoClassDefFoundError for a class in any jar file I don't make explicit:
java -cp ".;.\ro\nextreports\integration\*.jar" ro.nextreports.integration.
SimpleDemo
Suppose this is your directory tree:
src/
ro/
nextreports/
integration/
SimpleDemo.class
From your package declaration, your compiled class should be in the integration subdiretory. Check that there really is a SimpleDemo.class in that directory. If that is correct, its classpath includes the contents of the src directory.
That means that, if you didn't have any JAR dependencies, you could run your application from the src directory like this:
java -cp . ro.nextreports.integration.SimpleDemo
You need to use the fully qualified class name.
Since you do have jars, you have to include them in the classpath as well. Suppose you have one JAR in the directory above src, and another in the current directory, you could use:
java -cp ../one.jar;another.jar;. ro.nextreports.integration.SimpleDemo
If you run it from another directory, it will still work if you review the classpath's relative directories or use absolute directories to describe your classpath. I am not sure but usually the current directory is always included in the classpath by the java executable.

default classpath current directory anomaly

I am trying to compile and run simple Java program. This program basically prints out hello world phrase. I am not specifying -cp option and I don't have CLASSPATH environment variable. Hence the user classpath is limited only to current directory.
Now, compilation works beautifully.
rustam#rustam-laptop:~/temp/bird_test$ javac Sparrow.java
This command produces needed .class file. The weird stuff happens when I try to run .class file. The following command works good.
rustam#rustam-laptop:~/temp/bird_test$ java Sparrow
But when I try the following command
rustam#rustam-laptop:~/temp/bird_test$ java ./Sparrow
I receive the following error:
Error: Could not find or load main class ..Sparrow
WTF! i thought that symbol ./ refers to current directory.
java takes a class name as argument. It doesn't take a file path. The class name (Sparrow) is then resolved by the java class loader to a .class file based on the classpath, i.e. it looks for a Sparrow.class file in every directory and jar listed in the classpath.
Let's take an example that respects good practices, and thus doesn't use the default package:
package foo.bar;
public class Baz {
...
}
The class name of the above class is foo.bar.Baz. To execute it, you must use
java foo.bar.Baz
and java will look for a foo/bar/Baz.class in all the directories listed in the classpath. So if the classpath is set to /hello/world, it will look for the file /hello/world/foo/bar/Baz.class.

Java Classpath Woes

Luckily or unluckily, I haven't had to work too extensively with invoking java from the commandline up until this point, I've usually been using something like Maven, Ant, or running things within a servlet container. I've just compiled my application in Maven into one JAR using the assembly:single goal, and am having serious problems running it from the commandline.
Here's what I'm attempting to do:
export JAVA_CLASSPATH="`pwd`:/path/to/remote/libs/"
java -cp "${JAVA_CLASSPATH}" -jar groupId-artifactId-version-jar-with-dependencies.jar com.my.main.Class
This is failing with the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: com/remote/lib/IServer
Caused by: java.lang.ClassNotFoundException: com.remote.lib.IServer
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: com.my.main.Class. Program will exit.
The IServer class is in the /path/to/remote/libs directory and isn't being found. It also seems to not be able to find the main class, which is really odd. What am I doing wrong?
The -jar option ignores the classpath ( and all other ) options.
You need to edit the manifest file in your jar and set the Class-path variable in there.
http://download.oracle.com/javase/1.4.2/docs/tooldocs/linux/java.html
#Kal's answer is correct, but an easier way to fix it would be this:
export JAVA_CLASSPATH="groupId-artifactId-version-jar-with-depencies.jar:`pwd`:/path/to/remote/libs/iserver.jar"
java -cp "${JAVA_CLASSPATH}" com.my.main.Class
No need to tinker with the manifest file; just add the jar you want to the classpath.
Also, if the IServer stuff is in a jar inside /path/to/remove/libs/ then you need to explicitly include the name of the jar file. I've included it above and assumed it is called "iserver.jar"
If you have many dependencies then it's often easier to write a shell script for launching your code. The script can build a (potentially very large) classpath string using shell glob goodness. The java executable doesn't do any globbing.
EDIT: Note that I've assumed that you don't have access to Ant or Maven or any of that good stuff, since you stated that you usually use those tools but are not doing it right now.
The classpath parameter must be specified with the exact path to the jar file. If your IServer class is found in server.jar, you need to specify "$pwd\:/path/to/remote/libs/server.jar", for example. (in other words, you need to include a :-separated list of paths to jar files)
As others have noted, if you choose this method you should remove the -jar parameter and instead include the class name you intend to be your main class on the command line, such as java -cp $PWD:/path/to/server.jar com.remote.lib.IServer.
Side note: usually names that start with I are reserved for interfaces; it seems a bit odd to use an interface as your main class.

Can you run a .class file from terminal that is outputted by an IDE

I am trying to run a file from command line. The file is a .class file and is apart of a larger project that I compiled in Netbeans. I navigated to the .class file and ran
java MyFile
And I got:
Exception in thread "main" java.lang.NoClassDefFoundError: PersonTest/class
Caused by: java.lang.ClassNotFoundException: PersonTest.class
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: PersonTest.class. Program will exit
Whats up with that? (I should mention that i'm running ubuntu)
You need to check this useful link java - the Java application launcher:
By default, the first non-option
argument is the name of the class to
be invoked. A fully-qualified class
name should be used
So, you have to write the full qualified name of the class (this includes the package name).
So, the right way to execute your command is this (from the root dir where your class files are stored):
> java my.package.MyFile
Also, make sure to include all the needed dependencies at the classpath (-cp) argument (check the referenced link).
UPDATE: to include a classpath setting example:
java -classpath C:\MyProject\classes;C:\MyProject\lib\utility.jar my.package.MyFile
With this, the java runtime will search for the classes at the C:\MyProject\classes directory, and at the C:\MyProject\lib\utility.jar JAR file. You'll need not only your class direct dependencies, but the dependencies needed by the referenced files (the whole tree).
The answer appears to be in this line:
Exception in thread "main" java.lang.NoClassDefFoundError: PersonTest/class
It means you didn't type:
java MyFile
as you said in your original post, you typed
java PersonTest.class
you should have typed
java PersonTest
Yes you can, they are compiled by a java compiler. If you have the right version of the jvm (often other versions work aswell) than it can be run. The information about your error is not enough to tell what went wrong.
Your probably in the wrong folder, mistyped the classname, used a class in your code that couldn't be found, etc.
Unless your class is entirely standalone (i.e. only references java.lang classes like String), you'll need to add other classes/JARs to the classpath when you invoke Java.
The NoClassDefFoundError (which usually states the name of the class by the way, and always includes a stacktrace) indicates that an external class that was available when your class was compiled, is not available on the classpath at runtime.
EDIT based on update:
You're invoking your process incorrectly. You don't need to append the .class suffix of the file - doing so makes Java look for a file class class in a subpackage.
(P.S. you said you ran java MyFile. That's a lie, you actually ran java PersonTest.class. If you'd noted that to start with, it would have made it much easier for people to answer the question!)
Just consider this example
say I already have a folder src and I wrote in my notepad
package test.oye;
class testclass {
static public void main (String [] args)
{
int a=3;
System.out.println(a);
}
}
then what go to src folder and you ,yourself create a folder named test and inside it oye . Then put your .java file in it . Then cd src/test/oye only(in Command prompt or terminal).From there itself
javac testclass.java
cd src
java test.oye.testclass
This will work for sure.
If you don’t want to put .java file there … then just compile your .java file and get the .class file .
Now create the test folder and then oye inside it ….and put .class file inside it ….
Now go back to src …and then type
java test.oye.testclass;
according to terminal ide, android requires classes in DEX format when running them.
Try:
dx --dex --output=practice.jar practice.class
Then run using this:
java -jar practice.jar practice

Categories

Resources