Missing dependency on EmbeddedKafka on integration test with groovy and spock - java

I've been trying to create an integration test using the embeddedKafka, but I'm getting problem of missing dependency when trying to run it, this is the error:
Unable to load class org.springframework.kafka.test.EmbeddedKafkaBroker due to missing dependency org/I0Itec/zkclient/serialize/ZkSerializer
I saw some stuff saying that this is related to my dependencies, so here is my dependencies:
springBootVersion = '2.3.5.RELEASE'
compile("org.springframework.boot:spring-boot-starter-web:${springBootVersion}")
compile("org.springframework.kafka:spring-kafka:${springBootVersion}")
testCompile("org.springframework.boot:spring-boot-starter-test:${springBootVersion}",
'org.spockframework:spock-core:1.2-groovy-2.4',
'org.spockframework:spock-spring:1.2-groovy-2.4',
'com.microsoft.azure:spring-data-cosmosdb:2.3.0',
'com.nimbusds:oauth2-oidc-sdk:5.64.4',
)
testCompile("org.springframework.kafka:spring-kafka-test:${springBootVersion}")
So, my question is, am I missing something?
EDIT
After changed the versions as indicated, I got a different error:
Error creating bean with name 'embeddedKafka': Invocation of init
method failed; nested exception is java.lang.NoClassDefFoundError:
scala/math/Ordering$$anon$7
I've added the scala dependencies, but still having the same issue:
testImplementation("org.scala-lang:scala-library:2.12.11")
testImplementation("org.scala-lang:scala-reflect:2.12.11")

You somehow have a mismatched kafka Vs. kafka-clients jars on the classpath; they all must be the same version.
You generally should not specify a version on boot's dependencies and use its dependency management instead.
You are pulling in spring-kafka 2.3.5 whereas spring-boot 2.3.5 requires spring-kafka 2.5.7.
Spring-kafka 2.5.x uses the kafka-clients 2.5.1.
See here for how to override versions of kafka jars when using a different version to the version that Boot prescribes.

the Kafka Client libraries for a time were inlining a particular version of the Scala library. This caused problems for those of us wanting to use the kafka client library with a slightly different version of Scala than that inline version.
In this cases the version of Scala they were using inline is Scala 2.12.10
They removed this dependency in later versions and this was backported as fixes (the earliest being 2.8.0) https://archive.apache.org/dist/kafka/2.8.0/RELEASE_NOTES.html)

Related

Setting up JNA in Gradle Project

I try to import a custom .dll inside my Gradle project. I add the dependencie inside build.gradle
dependencies {
implementation group: 'net.java.dev.jna', name: 'jna', version: '5.6.0'
}
but when I try to run the gradle build I receive this error
..java:170: error: cannot find symbol
CustomLibrary INSTANCE = (CustomLibrary) Native.load("xxx", CustomLibrary.class);
^
symbol: method load()
location: class Native
1 error
Any suggestions?
You have a transitive dependency on an older version of JNA in another dependency.
As you have tagged this with spring boot that is the likely cause. Older Spring Boot versions used a 4.x (I think 4.3) JNA dependency and the syntax has changed.
The POM for Spring Boot uses a property jna.version that you could override if you were using Maven, but I don't think that is (easily) possible using Gradle. However, updating to the latest version of Spring Boot should solve your problem.

java.lang.NoSuchMethodError: org.springframework.util.Assert.noNullElements

I upgraded the spring-web module version from 5.1.2.RELEASE to 5.2.4.RELEASE for my Micronaut application deployed on AWS. I had to upgrade to a more stable version (suggested fix by WhiteSource) as the older version jar was considered to be highly vulnerable from a security standpoint by the WhiteSource tool.
The issue is that most of my application's endpoints worked just fine as they were mainly database queries but I noticed that if I make a REST call, the endpoint fails with the following error:
Invocation with requestId [ca31a9a5-35b3-4b52-a955-e304d9021880] failed: org.springframework.util.Assert.noNullElements(java.util.Collection, java.lang.String)java.lang.NoSuchMethodError: org.springframework.util.Assert.noNullElements(java.util.Collection, java.lang.String)
at org.springframework.web.client.HttpMessageConverterExtractor.<init>(HttpMessageConverterExtractor.java:77)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.<init>(RestTemplate.java:988)
at org.springframework.web.client.RestTemplate.responseEntityExtractor(RestTemplate.java:819)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:582)
at com.connector.getToken(Connector.java:58)
Downgrading the version is not much of an option as those jars will be rejected/deemed vulnerable by WhiteSource. Is there another workaround/solution for this? Please let me know! Thanks.
Make sure to upgrade the spring-core dependency to 5.2.4.RELEASE as well.
Or rather: Make sure that all spring- dependencies have the same version number.

Using Spring Boot, QueryDSL, and Springfox Swagger together - Guava version mismatch

I'm trying to use QueryDSL for Spring Data predicate resolution as well as Swagger API documentation for my Spring Boot service. However I've run into a problem. When my application starts, I get this error message:
java.lang.NoSuchMethodError: 'com.google.common.collect.FluentIterable com.google.common.collect.FluentIterable.concat(java.lang.Iterable, java.lang.Iterable)
An attempt was made to call a method that does not exist. The attempt was made from the following location:
springfox.documentation.schema.DefaultModelDependencyProvider.dependentModels(DefaultModelDependencyProvider.java:79)
The following method did not exist:
'com.google.common.collect.FluentIterable com.google.common.collect.FluentIterable.concat(java.lang.Iterable, java.lang.Iterable)'
The method's class, com.google.common.collect.FluentIterable, is available from the following locations:
jar:file:/my_m2/com/google/guava/guava/18.0/guava-18.0.jar!/com/google/common/collect/FluentIterable.class
Action:
Correct the classpath of your application so that it contains a single, compatible version of com.google.common.collect.FluentIterable
I've discovered that this is happening because QueryDSL is dependent on Guava 18.0 library, but Springfox / Swagger is dependent on Guava 20.0 library, so I end up with both versions of the library on my classpath and maven seems to prioritize the 18.0 one. How can I fix this dependency mismatch? Is there any way to force QueryDSL to try to use Guava 20.0 (in the hopes that it will still function)? Or could there possibly be any other way around this?
Versions:
Spring Boot version: 2.1.9.RELEASE
This version of Spring Boot uses QueryDSL version: 4.2.1
Springfox Swagger version: 2.9.2
If using Gradle, you can force the use of a specific library version. In this case, you may use the following syntax -
configurations.all {
resolutionStrategy.force "com.google.guava:guava:$guavaVersion"
}
I'm sure there's a similar solution if you use a different build tool.

spark and httpclient dependency

I have a spark application that runs in YARN. One of the dependencies in application needs httpclient 4.5.1. I explicitly include that as a dependency on my pom.
However, when I run the application in YARN. I get the following stacktrace
Factory method 'createClient' threw exception; nested exception is java.lang.NoSuchMethodError: org.apache.http.impl.conn.PoolingHttpClientConnectionManager.setValidateAfterInactivity(I)V
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1123)
Looks like a dependency issue with Spark. Is there any way that I could override httpclient?
The workaround I used was to run the spark application in YARN with the flag
spark.executor.userClassPathFirst=true
That fixed my issue.
It seems that this method was added in httpclient 4.4. The fact that the run reports that is a java.lang.NoSuchMethodError implies that your class is finding another version of this class (PoolingHttpClientConnectionManager) on the Spark platform. Maybe another version of Apache HTTP Client is bundled with Spark?
I suggest you use the Maven shade plugin to create a self-contained application such that it can (hopefully) use the classes and JAR's that it needs.

Grails NoSuchField exception because of wrong denpendecy version of apache http core

I have Grails 2.0.4 application that depends on a jar that depends on org.apache.httpcomponents:httpcore:4.3.1
I'm getting a NoSuchField exception which I managed to track down to org.apache.http.impl.conn.BasicLineParser.INSTANCE.
This field is only present from httpcore 4.3, before this version the field was BasicLineParser.DEFAULT instead
My BuildConfig.groovy is like:
dependencies {
runtime 'mysql:mysql-connector-java:5.1.16'
runtime 'org.apache.httpcomponents:httpclient:4.3.1'
runtime 'org.apache.httpcomponents:httpcore:4.3.1'
}
Running the dependecy report I found out the there is a signpost-commonshttp4 by oauth.signpost that depends on httpcore:4.0.1 which is probably the root cause of my issue, because it doesn't have the INSTANCE property.
I've tried to exclude this dependency adding
runtime('oauth.signpost:signpost-commonshttp4:1.2.1.1'){
excludes 'httpcore'
}
with no success.
Is there any other way to prevent the code to use the older library version?
Regards
RESOLVED
I managed to resolve this issue adding an extra dependecies configuration
build 'org.apache.httpcomponents:httpclient:4.3.1'
build 'org.apache.httpcomponents:httpcore:4.3.1'
along with the runtime
Thank you all

Categories

Resources