Develop Reference

Dataflow and Bigtable dependency conflict

The error I'm getting:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/beam/sdk/transforms/DoFn
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.beam.sdk.transforms.DoFn
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
My pom dependencies:
<dependencies>
<dependency>
<groupId>com.google.cloud.bigtable</groupId>
<artifactId>bigtable-hbase-beam</artifactId>
<version>1.11.0</version>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>2.13.0</version>
<scope>runtime</scope>
</dependency>
</dependencies>
I assumed that the issue was coming from the two dependencies using different versions of the beam sdk, so I changed the beam-runners-google-cloud-dataflow-java version to 2.11.0, which uses the same beam sdk version. But I got the same error.
Any help would be greatly appreciated!

How are you running the pipeline? I'm guessing that the issue is that the beam jars are not on your classpath. You either need to use the maven exec plugin to invoke your main() with all of the maven dependencies on the classpath or you need to create a fat jar using something like the maven shade plugin or the maven assembly plugin. Please see here for example of how to use the maven exec plugin to run a pipeline:
https://github.com/GoogleCloudPlatform/cloud-bigtable-examples/blob/a017a237fa8dae764451a658150190743c1b4127/java/dataflow-connector-examples/pom.xml#L146-L161

Spring boot : Unable to load cache item: javax/servlet/Filter

I have a spring boot project and getting some error related to javax servlet filter.
Is it some kind of version compatibility or I am missing something in pom file.
Following is the log from the console and the pom file.
I have been searching all over, couldn't find anything.
Exception in thread "main" java.lang.IllegalStateException: Cannot load configuration class: org.springframework.security.oauth2.config.annotation.web.configuration.AuthorizationServerSecurityConfiguration
at org.springframework.context.annotation.ConfigurationClassPostProcessor.enhanceConfigurationClasses(ConfigurationClassPostProcessor.java:403)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanFactory(ConfigurationClassPostProcessor.java:249)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:281)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:125)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:687)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:525)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
at com.abc.application.Application.main(Application.java:15)
Caused by: java.lang.IllegalStateException: Unable to load cache item
at org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:79)
at org.springframework.cglib.core.internal.LoadingCache.get(LoadingCache.java:34)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData.get(AbstractClassGenerator.java:116)
at org.springframework.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:291)
at org.springframework.cglib.proxy.Enhancer.createHelper(Enhancer.java:480)
at org.springframework.cglib.proxy.Enhancer.createClass(Enhancer.java:337)
at org.springframework.context.annotation.ConfigurationClassEnhancer.createClass(ConfigurationClassEnhancer.java:138)
at org.springframework.context.annotation.ConfigurationClassEnhancer.enhance(ConfigurationClassEnhancer.java:110)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.enhanceConfigurationClasses(ConfigurationClassPostProcessor.java:393)
... 11 more
Caused by: java.lang.NoClassDefFoundError: javax/servlet/Filter
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getDeclaredConstructors(Class.java:2020)
at org.springframework.cglib.proxy.Enhancer.generateClass(Enhancer.java:566)
at org.springframework.cglib.transform.TransformingClassGenerator.generateClass(TransformingClassGenerator.java:33)
at org.springframework.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanFactoryAwareGeneratorStrategy.generate(ConfigurationClassEnhancer.java:252)
at org.springframework.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:329)
at org.springframework.cglib.proxy.Enhancer.generate(Enhancer.java:492)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:93)
at org.springframework.cglib.core.AbstractClassGenerator$ClassLoaderData$3.apply(AbstractClassGenerator.java:91)
at org.springframework.cglib.core.internal.LoadingCache$2.call(LoadingCache.java:54)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.springframework.cglib.core.internal.LoadingCache.createEntry(LoadingCache.java:61)
... 19 more
Caused by: java.lang.ClassNotFoundException: javax.servlet.Filter
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 33 more

For me problem were in pom file (in spring boot 2 project),
when I used a parent spring boot in version 2.1.9 which under the hood uses spring-core in version 5.1.10.RELEASE
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.9.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
I had dependency:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>5.2.0.RELEASE</version>
</dependency>
removing it resolved problem

I was getting the similar error in my Gradle project(might be a problem with Maven too) when I was trying to upgrade my Spring Boot version to latest one.
A quick gradle clean solved that problem for me.

Usually, this Filter class can be found in embedded tomcat jar.
So please make sure that your application (jar) contains this file:
Open up the Jar file of the compiled spring boot application and check whether the jar named like tomcat-embed-core resides in the BOOT-INF\lib folder.
This answer is valid as long as you're running tomcat of course + use JAR artifact for spring boot application (for WAR the folder is different, probably WEB-INF/lib although I've never worked with WARs of Spring Boot application so I might be wrong here)

java.lang.NoClassDefFoundError while using SourceAFIS in my Java Application using Maven [duplicate]

I am trying simple Maven app with Log4J ver 2-beta 9. In my pom.xml file I have these two dependencies (as is mentioned in Log4J Maven webpage):
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.0-beta9</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.0-beta9</version>
</dependency>
Eclipse sees Log4J library:
But when I package the app and run it this Exception is thrown:
java -cp target/notification-1.0.0.jar com.example.Sandbox
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/LogManager
at com.example.Sandbox.<clinit>(Sandbox.java:13)
Caused by: java.lang.ClassNotFoundException: org.apache.log4j.LogManager
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 1 more
I searched for this Exception and it seems to be related to CLASSPATH variable.
How should be CLASSPATH set for Maven project?

Maven is a build tool. It doesn't help you much running the final application.
You can use mvn exec:java -Dexec.mainClass="com.example.Sandbox" to run your app (see the question Maven Run Project) but this gets tedious when you have to pass arguments to it.
You can get the classpath that Maven used to compile your application with mvn dependency:build-classpath
That will print the classpath to the console. Note that it will be missing target/notification-1.0.0.jar
Another useful tool in this area is the assembly plugin; it will create one very big JAR with all the dependencies rolled into a single file when you specify the descriptor jar-with-dependencies.

Nutch: NoClassDefFoundError: org/springframework/retry/RetryCallback

I am running the normal Nutch indexing, it used to work 2 days ago perfectly but today out of nowhere it just stopped working with the previous error. Following the stack error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/springframework/retry/RetryCallback
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructor0(Class.java:3075)
at java.lang.Class.newInstance(Class.java:412)
at org.apache.nutch.plugin.Extension.getExtensionInstance(Extension.java:161)
at org.apache.nutch.indexer.IndexWriters.<init>(IndexWriters.java:55)
at org.apache.nutch.indexer.IndexingJob.index(IndexingJob.java:121)
at org.apache.nutch.indexer.IndexingJob.run(IndexingJob.java:228)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.indexer.IndexingJob.main(IndexingJob.java:238)
Caused by: java.lang.ClassNotFoundException: org.springframework.retry.RetryCallback
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 10 more
I ended up seeing that the error occurred at this exact piece of code :
Object object = extensionClazz.newInstance();
Another question on SO but with different project was able to fix the issue by adding the spring-framework-messaging dependency
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-messaging</artifactId>
<version>4.2.4.RELEASE</version>
</dependency>
well i am using nutch version 1.12 with ivy for dependency management so in my config is the following:
<dependency org="org.springframework.retry" name="spring-retry" rev="1.2.0.RELEASE">
<exclude module="spring-beans"/>
<exclude module="spring-context"/>
<exclude module="spring-core"/>
<exclude module="spring-aop"/>
</dependency>
and even with adding the spring-messaging dependency it's still not working.
Any ideas?

These kind of errors might come when you have more than one version of same jar (same class) in your classpath, You can see the dependency tree by executing
<report conf="compile" /> for ivy projects
And
mvn dependency:tree for maven projects
Narrow down to the version that you want and you can exclude the other jar by explicitly excluding that jar from the module which is importing it .

No idea what went wrong but this was able to fix the problem
ant clean
ant runtime

Unable to resolve a dependency on URLUtil, conflicting requirements

I'm getting the following error when trying to compile my project:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/util/URIUtil
at org.apache.commons.vfs2.provider.URLFileName.getPathQueryEncoded(URLFileName.java:91)
at org.apache.commons.vfs2.provider.URLFileName.getURIEncoded(URLFileName.java:161)
at org.apache.commons.vfs2.provider.url.UrlFileObject.createURL(UrlFileObject.java:74)
at org.apache.commons.vfs2.provider.url.UrlFileObject.doAttach(UrlFileObject.java:63)
at org.apache.commons.vfs2.provider.AbstractFileObject.attach(AbstractFileObject.java:1505)
at org.apache.commons.vfs2.provider.AbstractFileObject.getType(AbstractFileObject.java:489)
.... (More project-scope traces)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.httpclient.util.URIUtil
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 13 more
I have the following maven dependencies in my project, among others:
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.4</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-vfs2</artifactId>
<version>2.0</version>
</dependency>
I absolutely must depend on a HttpClient version 4.3.4 or greater because of other dependencies and various bugs we ran into with previous versions. After a bit of research, I discovered the class in question (URLUtil) was removed from HttpClient after HttpClient 3.1.
Is there any way to resolve this? Or another library I could use instead of vfs2? I'm attempting to write files over sftp using both user/pass and key authentication schemes.

As it turns out, the artifacts are different between the 3.x and 4.x releases of HttpClient. All I had to do was include:
<dependency>
<groupId>commons-httpclient</groupId>
<artifactId>commons-httpclient</artifactId>
<version>3.1</version>
</dependency>
Silly me, thinking that they'd override each-other.