I am working on map reduce join problem using joiner and splitter functions. I have searched a lot on Google and finds about adding guava-18.0.jar file in the referenced libraries of the project. I also attached the javadoc location to the jar file but still getting this error mentioned below:
Error: java.lang.ClassNotFoundException: com.google.common.base.Splitter
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at edu.cs.okstate.cs.Partitioning.Partition_Mapper.setup(Partition_Mapper.java:29)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
I'm guessing you're using your IDE to manage your project and not a dependency management tool (Maven, Gradle, Ant+Ivy...), and the Guava jar doesn't actually end up in the classpath of your Hadoop job.
Make sure it gets bundled with what's deployed.
Related
There's a need for my team to use old version of StreamSet, version 2.5. But there is some important processors in version 3.8 we want to include in the old environment, namely the JDBC Meta Data Processor.
What have been done is below:
Create a sample custom processor project with StreamSets version 2.5.1.1
mvn archetype:generate -DarchetypeGroupId=com.streamsets -DarchetypeArtifactId=streamsets-datacollector-stage-lib-tutorial -DarchetypeVersion=2.5.1.1 -DinteractiveMode=true
Modify the code based on JDBC Meta Data classes in StreamSets 3.8, with reference library in POM to StreamSets 3.8:
2.1 streamsets-datacollector-api
2.2 streamsets-datacollector-jdbc-protolib
Package and upload the customized jar to external library of StreamSets UI.
Upload the dependency jars to StreamSets UI:
4.1 streamsets-datacollector-api-3.8.jar
4.2 streamsets-datacollector-jdbc-protolib-3.8.jar
Everything is find except that once I uploaded
streamsets-datacollector-jdbc-protolib-3.8.jar
and restart StreamSets, there's error in the log:
java.lang.NoClassDefFoundError: com/streamsets/pipeline/api/base/configurablestage/DPushSource
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at com.streamsets.pipeline.BlackListURLClassLoader.findClass(BlackListURLClassLoader.java:77)
at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:341)
at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:323)
at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.loadStages(ClassLoaderStageLibraryTask.java:325)
at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.initTask(ClassLoaderStageLibraryTask.java:192)
at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:66)
at com.streamsets.datacollector.task.CompositeTask.initTask(CompositeTask.java:48)
at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:66)
at com.streamsets.datacollector.task.TaskWrapper.init(TaskWrapper.java:44)
at com.streamsets.datacollector.main.Main$1.run(Main.java:101)
at com.streamsets.datacollector.main.Main$1.run(Main.java:98)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at com.streamsets.datacollector.main.Main.doMain(Main.java:98)
at com.streamsets.datacollector.main.DataCollectorMain.main(DataCollectorMain.java:46)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:308)
I noticed this class exists in API-3.8 jar and the class in API-2.5 exists but with a different package path,
API-3.8
com/streamsets/pipeline/api/base/configurablestage/DPushSource
API-2.5
com/streamsets/pipeline/configurablestage/DPushSource
Any hints I am missing to make it work? Thanks.
Needless to say, there are many fixes and enhancements since StreamSets Data Collector 2.5.0, and I would strongly recommend you upgrade.
Having said that, since DPushSource is specific to the origin, and you're focused on a processor, the easiest way forward is to simply remove the JDBC origin. Delete everything under jdbc-protolib/src/main/java/com/streamsets/pipeline/stage/origin/jdbc and try again.
I'm totally a newbie in springs so I wanted to try a helloworld program in springs. I came across this tutorial.
I downloaded the source code with library provided on this site, I have the latest Spring STS installed on my Eclipse Kepler. I got this error :
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.springframework.context.support.AbstractApplicationContext.(AbstractApplicationContext.java:153)
at org.springframework.context.support.AbstractApplicationContext.(AbstractApplicationContext.java:217)
at org.springframework.context.support.AbstractRefreshableApplicationContext.(AbstractRefreshableApplicationContext.java:88)
at org.springframework.context.support.AbstractRefreshableConfigApplicationContext.(AbstractRefreshableConfigApplicationContext.java:58)
at org.springframework.context.support.AbstractXmlApplicationContext.(AbstractXmlApplicationContext.java:61)
at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:136)
at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:83)
at com.vaannila.HelloWorldApp.main(HelloWorldApp.java:9)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 8 more
I also tried manually doing the entire program with a different class name and It throws the same error.
Things I have tried:
built path and included the springframework jars
also included commons-logging-1.1.1.jar
I got past this exception by adding commons-logging-1.1.1.jar to the classpath of my project. I had it in my maven repository from other projects.
Plus, to make the example work, the file name for the instantiation of ClassPathXmlApplicationContext needs to be qualified with the package path. If you are following the example exactly, you need to instantiate your context like this:
ApplicationContext context = new ClassPathXmlApplicationContext("com/vaannila/beans.xml");
Hi I am working on Java & am using HtmlUnitDriver.I have created the object
`WebDriver webDriver = new HtmlUnitDriver(true);`
I am using selenium-htmlunit-driver-2.24.1.jar & selenium-java-2.24.1.jar.
When I run my code I am getting following exception.
Exception in thread "Thread-5" java.lang.NoClassDefFoundError: org/openqa/selenium/internal/FindsByCssSelector
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Please guide me how to solve this issue.If there is jar for this Exception then which Jar should I use?
Make sure that all the relevant jars (such as package and library jars) are in the class path. Always make sure to clean and recompile the project once you copied jars.
[You can use a jar search engine to find jars of missing classes.][1]
[http://www.jarvana.com/jarvana/search?search_type=class&java_class=org.openqa.selenium.internal.FindsByCssSelector][1]
Note : It is always advisable to use jars from official release, because of possible compatibility issues due to version difference.
Imagine I have a following project structure of folders
A
B
C
and many external jars.
Now I am trying to build two different executable-jars like this A+B, A+C.
At first, I do Eclipse -> Export -> Executable Jar and save script for Ant. So it generates Ant-script and jar-in-jar-loader. Now I modify ant script to exclude folder B
<fileset dir="C:/Users/Nikolay/bin">
<exclude name="org/B"/>
</fileset>
and keep jar-in-jar-loader.zip file unmodified.
After building the jar running gives this error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/B
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoa
der.java:56)
Caused by: java.lang.ClassNotFoundException: org.B
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
... 3 more
But I am sure B class is not used in the A.
It seems like JarRscrLoader tries to load all listed classes at application startup. How can I prevent that?
Please, don't suggest me to split the code at Eclipse into 2-3 projects.
Create two different Run configurations (Run --> Run configurations...) and modify the included libraries in the classpath tab according to your requirements.
When you export the project, just select the appropriate configuration.
I got "java.lang.NoClassDefFoundError" Exception when I want to use a certain jar file.
Exception in thread "main" java.lang.NoClassDefFoundError: .
org/apache/http/client/ClientProtocolException
at ?uk.org.taverna.server.client.connection.ConnectionFactory.getConnection(ConnectionFactory.java:63)
at uk.org.taverna.server.client.Server.<init>(Server.java:99)
at uk.org.taverna.server.client.Server.<init>(Server.java:126)
at uk.org.taverna.server.client.Server.connect(Server.java:293)
at uk.org.taverna.server.usage.ServerUsage.Usage(ServerUsage.java:24)
at Test.main(Test.java:23)
Caused by: java.lang.ClassNotFoundException: org.apache.http.client.ClientProtocolException
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
I import this jar file to jar library, when I call the method in that jar file I got this Exception. By the way, This jar file be wrote by myself, and I can run it correctly.
I have no idea how to solve this problem, I am quite new in java.
Thanks in advance.
Sandy
You need also at least HttpClient jar-library. You can get it at Apache Commons web-site.
Jar libraries may and often refer to another jar libraries. In your case library you are using refers to HttpClient library from Apache Commons framework. You need to add this library to your project.
You should but all the jar files you use in you application in the libs folder in your application directory .
so if you use the apachecommons jar file copy and paste the jar file to the libs folder in your application .