Kafka Connect cant' find class of developed plugin - java

I created a plugin for kafka connect that implements SinkConnector, I packaged it into a jar using gradle jar task:
jar {
archiveName='name.jar'
}
I copy it into the kafka cluster in a folder, and I set CLASSPATH=where my jar is.
Then I execute the kafka script to start the standalone connect and it gives me an error saying that my class can't be found:
[2017-07-25 05:15:52,084] WARN could not get type for name mypackage.SplunkSinkConnector from any class loader (org.reflections.Reflections:384)
org.reflections.ReflectionsException: could not get type for name mypackage.SplunkSinkConnector
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:378)
at org.reflections.ReflectionUtils.forNames(ReflectionUtils.java:397)
at org.reflections.Reflections.getSubTypesOf(Reflections.java:367)
at org.apache.kafka.connect.runtime.PluginDiscovery.connectorPlugins(PluginDiscovery.java:76)
at org.apache.kafka.connect.runtime.PluginDiscovery.scanClasspathForPlugins(PluginDiscovery.java:70)
at org.apache.kafka.connect.runtime.AbstractHerder$1.run(AbstractHerder.java:354)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: mypackage.SplunkSinkConnector
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:376)
... 6 more
[2017-07-25 05:15:52,419] INFO Reflections took 1586 ms to scan 62 urls, producing 3002 keys and 15379 values (org.reflections.Reflections:229)
[2017-07-25 05:15:52,420] WARN could not get type for name mypackage.SplunkSinkConnector from any class loader (org.reflections.Reflections:384)
org.reflections.ReflectionsException: could not get type for name mypackage.SplunkSinkConnector
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:378)
at org.reflections.ReflectionUtils.forNames(ReflectionUtils.java:397)
at org.reflections.Reflections.getSubTypesOf(Reflections.java:367)
at org.apache.kafka.connect.runtime.ConnectorFactory.getConnectorClass(ConnectorFactory.java:69)
at org.apache.kafka.connect.runtime.ConnectorFactory.newConnector(ConnectorFactory.java:38)
at org.apache.kafka.connect.runtime.AbstractHerder.getConnector(AbstractHerder.java:334)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:233)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:159)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:93)
Caused by: java.lang.ClassNotFoundException: mypackage.SplunkSinkConnector
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.reflections.ReflectionUtils.forName(ReflectionUtils.java:376)
... 8 more
Any idea why is not picking up my jar?
Thank you
================================
EDIT: Kakfa Connect version 10.2.1, as per script, classpath is calculated with: CLASSPATH="$CLASSPATH":"$KAFKA_HOME/libs/*"

Can you check your .jar file to make sure that class is there. Using Scala as a JVM shell:
// Or Maybe try $CLASSPATH with the full classpath that you are using.
scala -classpath path-to-jar.jar
// If class is not loaded, this will trigger an error.
classOf[mypackage.SplunkSinkConnector]
FYI, I am doing exactly this, using a custom .jar plugin with Kafka Connect loaded via the CLASSPATH environment variable.
UPDATE: Here is my build.gradle file for my plugin. IMO, this is the simplest way to build a Java .jar with some simple dependencies. I build the jar with gradle jar and it will be created at ./build/libs/(project-name).jar:
apply plugin: 'java'
apply plugin: 'idea'
sourceCompatibility = 1.8
repositories {
mavenLocal()
mavenCentral()
maven { url "http://packages.confluent.io/maven/" }
}
configurations {
all*.exclude group: 'org.slf4j', module: 'slf4j-log4j12'
all*.exclude group: 'log4j'
}
dependencies {
compile 'io.confluent:kafka-connect-storage-partitioner:3.2.1'
compile 'org.apache.kafka:connect-api:0.10.2.1'
testCompile 'junit:junit:4.+'
}
idea {
project {
languageLevel = '1.8'
}
}

Related

Java doesn't load classes from manifest-defined classpath

I'm using Gradle to build a project. This is my build.gradle:
plugins { id 'application' }
group 'my.group'
version '1.0'
sourceCompatibility = 1.11
repositories { mavenCentral() }
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.12'
implementation group: 'org.xerial', name: 'sqlite-jdbc', version: '3.7.2'
}
mainClassName = 'my.group.App'
jar {
manifest {
attributes(
'Main-Class': mainClassName,
'Class-Path': configurations.runtimeClasspath.files.collect { it.name }.join(' ')
)
}
from configurations.runtimeClasspath
into ''
}
This correctly generates a jar file with the following META-INF/MANIFEST.MF:
Manifest-Version: 1.0
Main-Class: my.group.App
Class-Path: sqlite-jdbc-3.7.2.jar
[newline]
And in the root of this jar file, there is indeed the sqlite-jdbc-3.7.2.jar file referenced in the manifest; I manually verified that inside this jar file there is a class called org.sqlite.JDBC.
However, running the generated jar with java -jar jarfile.jar results in:
Exception in thread "main" java.lang.ExceptionInInitializerError
at my.group.LEManagerSqlite.<init>(LEManagerSqlite.java:64)
at my.group.LEManager.createLEManager(LEManager.java:80)
at my.group.GuiFrame.<init>(GuiFrame.java:60)
at my.group.App.openFromFile(App.java:23)
at my.group.App.main(App.java:19)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.sqlite.JDBC
at my.group.SqliteConnectionManager.<clinit>(SqliteConnectionManager.java:17)
... 5 more
Caused by: java.lang.ClassNotFoundException: org.sqlite.JDBC
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at my.group.SqliteConnectionManager.<clinit>(SqliteConnectionManager.java:14)
... 5 more
For reference, SqliteConnectionManager has a static initializer which loads org.sqlite.JDBC, which is what is referenced as SqliteConnectionManager.java:14 in the stack trace:
Class.forName("org.sqlite.JDBC");
I tested this with OpenJDK 11 and OpenJ9 11, with identical results. I conclude that I'm doing something wrong, but I cannot understand what.
I found out what I was doing wrong: the documentation clearly says (doh!) that the Class-Path directive looks for additional classpath resources in the local filesystem or the network.
Loading additional classes from jar files included in the main jar file is not supported by the JVM, and thus requires custom loading code. A simpler alternative is to unpack the additional classes from the jar files and include them directly. A jar file built this way is referred to as a "fat jar" (or "uber jar").
There is a Gradle plugin called Shadow which can build such jars without any further configuration.

NoClassDefFoundError caused by ClassNotFoundException in Spring Boot App w/ Gradle in Eclipse

I'm new to creating a web application using Spring Boot with Gradle and I've been getting along alright, then all of a sudden without knowledge of what's changed, I can't run it as a Spring Boot app and I get a popup error that says "Error: A JNI error has occurred. Please check your installation and try again." I hit okay and then another popup occurs that says "A Java exception has occurred". I hit okay again and then the console reads:
Exception in thread "main" java.lang.NoClassDefFoundError:
javax/servlet/http/HttpServletResponse
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at
sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at
sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException:
javax.servlet.http.HttpServletResponse
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
I have import javax.servlet.http.HttpServletResponse; and I'm not getting any errors in the code until I compile so I know something is getting lost when I compile it. I've read about possible errors in the class path file? I'm not sure where that is or how to edit that. It's all new to me and I've just hit a brick wall. I've even downloaded other JARs to import and then I get the same error but for a different class and it's just an infinite loop of errors.
My build.gradle file looks something like this:
plugins {
id 'org.springframework.boot' version '2.1.5.RELEASE'
id 'java'
}
apply plugin: 'io.spring.dependency-management'
group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'
configurations {
developmentOnly
runtimeClasspath {
extendsFrom developmentOnly
}
}
repositories {
mavenCentral()
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.boot:spring-boot-starter-thymeleaf'
developmentOnly 'org.springframework.boot:spring-boot-devtools'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
What other information will you need to assist me with this? Any pointers in the right direction would be greatly appreciated.

Trying to use Kotlin-gradle plugin from local url

I'm trying to setup the gradle-kotlin-plugin locally, but it doesnt work at all. Im using Gradle 4.9 and Kotlin version 1.2.71. The Gradle sync fails and this exceptions is thrown:
org/jetbrains/kotlin/cli/common/PropertiesKt
NoClassDefFoundError: org/jetbrains/kotlin/cli/common/PropertiesKt
Im put all needed resources into the directory /lib/kotlin, meaning kotlin-gradle-plugin-1.2.71.jar, kotlin-stdlib-1.2.70.jar,kotlin-stdlib-1.2.70.pom,kotlin-stdlib-common-1.2.70.jar,kotlin-stdlib-common-1.2.70.pom.
If i set the repo to url "https://plugins.gradle.org/m2/" everything is working fine(i always want to load the plugin from the local file and NOT from the online repo).
The other kotlin dependencies are also included.
build.gradle
buildscript {
ext.kotlin_version = '1.2.71'
repositories {
flatDir dirs: '/lib/kotlin'
mavenLocal()
maven{
url uri('lib/kotlin')
}
dependencies {
classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
}
...
allprojects {
apply plugin: 'java'
apply plugin: 'application'
apply plugin: "kotlin"
..
dependencies {
compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
Edit:
Now the plugin is working but Kotlin is not compiling and is telling me:
Could not perform incremental compilation: Could not connect to Kotlin compile daemon
Could not connect to kotlin daemon. Using fallback strategy.
Exception in thread "main" java.lang.NoClassDefFoundError: kotlin/jvm/internal/Intrinsics
at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler$Companion.main(K2JVMCompiler.kt)
at org.jetbrains.kotlin.cli.jvm.K2JVMCompiler.main(K2JVMCompiler.kt)
Caused by: java.lang.ClassNotFoundException: kotlin.jvm.internal.Intrinsics
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
Try adding below dependency also
compile "org.jetbrains.kotlin:kotlin-reflect:$kotlin_version"
Do clean your ~/.gradle directory

Java jar cannot find class on run

I have created a simple Java program in IntelliJ ide. I used few libraries and when I try to export the jar as an artifact and run it from command line I get the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/util/concurrent/FutureCallback
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2615)
at java.lang.Class.getMethod0(Class.java:2856)
at java.lang.Class.getMethod(Class.java:1668)
at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.FutureCallback
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 6 more
Compiling & Running the program inside IntelliJ works great.
My project is configured as a gradle project with the following build.gradle:
group 'marianpavel.com'
version '1.0'
apply plugin: 'java'
sourceCompatibility = 1.5
repositories {
mavenCentral()
maven {
url "http://repo.bastian-oppermann.de"
}
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
compile 'de.btobastian.javacord:javacord:2.0.11'
compile 'ch.qos.logback:logback-classic:1.0.13'
compile 'de.btobastian.sdcf4j:sdcf4j-core:1.0.2'
compile 'de.btobastian.sdcf4j:sdcf4j-javacord:1.0.2'
compile 'org.jsoup:jsoup:1.9.2'
}
I have exported the jar as followed: Project Structure -> Artifacts -> Jar -> From Module with dependencies -> Added the main source folder and all the libraries from library files, added a manifest and a main class and exported the jar.
I am trying to figure this out for days and I don't understand why it cant find the class.
I think there is a misconception on your end.
Your project consists of two important parts:
Your own code
External libraries that your code is using
When you run your project within an IDE, and your project setup is correct, then both things are in your classpath.
But when you create a JAR for your project, it very much depends on your setup. Meaning: the "default" would be that a JAR created for your project only contains the "1." parts. Because you do not want that all external classes are added to "your" JAR in the first place.
So, to resolve that: when you want to run your program from your JAR on the command line, then you will need all those external JARs to be in your classpath as well! Because, as said: to run, you need both "1" and "2" parts to be present in the classpath.
Conclusion: you should check what gradle puts in the JAR it creates for you. I am pretty sure: that JAR only contains "1".
(and for the record, it is fine like that. it is possible to bundle everything into a single jar, but that will require work on your end)

Testning with Android Studio: NoClassDefFoundError: org/testng/TestNG

So I am trying to use the built in testTool to generate some test for my mainActivity, so far I have managed to generate a test class and I got empty methods in it to test the methods of mainActivity. If I run the testClass with coverage I get this error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/testng/TestNG
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:117)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.ClassNotFoundException: org.testng.TestNG
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 18 more
I'm not sure what this really means, there is a class that is not defined?
I might also add that I added:
task testClasses {
doLast {
println 'This is a dummy testClasses task'
}
}
in the build.gradle file for my sqlite module, otherwise I couldn't run any test at all.
I might also add that under Dependencies tab under Module settings, I do have "org.testng:testnb:6.9.6" set to "Test compile".
This is how my dependencies look in my gradle file:
dependencies {
compile fileTree(include: ['*.jar'], dir: 'libs')
testCompile 'junit:junit:4.12'
androidTestCompile 'org.testng:testng:6.9.6'
compile project(':sqlite-jdbc-3.8.11.2')
compile 'com.android.support:appcompat-v7:23.3.0'
compile 'com.getbase:floatingactionbutton:1.10.1'
compile 'com.android.support:design:23.3.0'
compile 'com.github.PhilJay:MPAndroidChart:v2.1.6'
compile 'com.android.support:support-v4:23.3.0'
compile 'com.numetriclabz.numandroidcharts:numandroidcharts:1.0.13'
}
EDIT: I saw that Android studio also created an ExampleUnitTest file and If I add my testcases to this file and run them, it works just fine. The problem only seems to happen when I try to run the whole MainActivityTest class, but individual methods seems to be ok to test.
Check your module structure. Probably one of your high level module folders doesn`t have dependencies. (You can link modules together and everything should work).
Go to -> File|Project structure|Modules|*your module*|Dependencies.
It's not a TestNG issue, it's a module structure issue.
More detailed description Click This -> Link
Try changing
testCompile 'junit:junit:4.12'
to
androidTestCompile 'junit:junit:4.12'
Also, make sure you are running it as an Android Test.

Categories

Resources