Develop Reference

spring-boot-starter-webflux dependency issue with gradle build

I tried to use the Springframework WebClient (I use IntelliJ with gradle), and in the "dependencies" section of the build.gradle, I added
“compile group: ‘org.springframework.boot’, name: ‘spring-boot-starter-webflux’, version: versions.spring_boot”
However, when I do gradle build, it failed in the lintGradle task with the following errors:
This project contains lint violations. A complete listing of the violations follows.
Because some were serious, the overall build status has been changed to FAILED
error unused-dependency one or more classes in jakarta.annotation:jakarta.annotation-api:1.3.5 are required by your code directly (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:compile’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:runtime’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:testCompile’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:implementation’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:integrationTestCompile’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error undeclared-dependency one or more classes in jakarta.annotation:jakarta.annotation-api:1.3.5 are required by your code directly
error unused-dependency this dependency should be removed since its artifact is empty (no auto-fix available)
build.gradle:122
compile group: ‘org.springframework.boot’, name: ‘spring-boot-starter-webflux’, version: versions.spring_boot
Not sure if I need to add something else to the gradle build setting environment? Or if the webflux dependency conflicts with other dependencies?
There is a parent folder Par and child folder Child, they both have their build.gradle respectively. The build task in the Child folder passed but failed in the Par folder with the errors above. The use of WebClient is in the child folder and the dependency is added to the build.gradle file of the Child foler.
The build.gradle file for Par folder:
plugins {
id 'org.springframework.boot' version '2.2.6.RELEASE'
id 'nerv-java-war-module-plugin' version '10.9.0'
id 'nerv-tab-checkstyle-plugin' version '4.5.0'
id 'hackathon-teamcity-auto-increment' version '3.1.0'
id 'microservice-incremental-deploy-plugin' version '1.2.0'
id 'protobuf-java-consumer-plugin' version '4.2.0'
id 'nebula.lint' version '16.0.2'
id 'application'
id 'eclipse-wtp'
}
subprojects {
apply plugin: 'nerv-java-war-module-plugin'
apply plugin: 'nerv-tab-checkstyle-plugin'
apply plugin: 'hackathon-teamcity-auto-increment'
}
incrementalDeploy.packageNames += 'Par'
// Notice about Spring Dependency Management: It doesn't work in this project due to a
// nebula.lint bug: https://github.com/nebula-plugins/gradle-lint-plugin/issues/227
gradleLint {
criticalRules += 'all-dependency'
excludedRules += ['overridden-dependency-version']// Ignore for dependency lock
/*gradle daemon locks the Child.jar file which prevents from
cleaning the build
cause: linting rules 'undeclared-dependency','unused-dependency'
Workaround: run gw --stop and then clean */
alwaysRun = false
build.finalizedBy lintGradle
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
// Integration tests related setup for sources
sourceSets {
integrationTest {
java {
compileClasspath += main.output + test.output
runtimeClasspath += main.output + test.output
srcDir file('src/integrationTest/java')
}
resources.srcDir file('src/integrationTest/resources')
}
}
// When running locally, configure Log4J via system property
application {
mainClassName = 'Par.ParApplication'
applicationDefaultJvmArgs = ["-Dlog4j.defaultInitOverride=false",
"-Dlogging.config=${buildDir}/config/log4j2.xml"]
}
// When running locally, we need some config files that would normally live in
// resources. However, resources get included in the published WAR file, which
// we need to avoid. We copy these to the build dir so that the same config files
// can be used by both this service, and the accompanying contractTest.
task copyConfig(type: Copy) {
from("${buildscript.sourceFile.parent}/config/")
into("${buildDir}/config")
}
bootRun.dependsOn(copyConfig)
ext.versions = [
// These definitions are shared with the contractTest application,
// so version numbers can be updated in one place.
config_decrypt: '^3.0',
grpc_helpers_java: '^19.0',
// Fixed versions for third-party packages
grpc: '1.32.1',
guava: '29.0-jre',
javax_annotation: '1.3.2',
junit: '4.12',
log4j: '2.13.1',
protobuf_java: '3.12.0',
servlet_api: '4.0.1',
slf4j: '1.7.30',
spring: '5.2.9.RELEASE',
spring_boot: '2.2.10.RELEASE'
]
configurations {
gradleLint.ignore {
// These exclusions are useful for application projects to make sure they aren't getting any log4j v1 dependencies transitively
all*.exclude group: 'log4j', module: 'log4j'
all*.exclude group: 'org.slf4j', module: 'slf4j-log4j12'
// ignore logging libraries
all*.exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
}
// Favor org.slf4j:jcl-over-slf4j
all*.exclude group: 'org.springframework', module: 'spring-jcl'
integrationTestCompile.extendsFrom testCompile
integrationTestRuntime.extendsFrom testRuntime
integrationTestRuntimeOnly.extendsFrom testRuntimeOnly
}
dependencies {
compile project(':Child')
compile nerv('grpc-helpers-java', versions.grpc_helpers_java)
compile group: 'com.google.guava', name: 'guava', version: versions.guava
compile group: 'com.google.protobuf', name: 'protobuf-java', version: versions.protobuf_java
compile group: 'io.grpc', name: 'grpc-api', version: versions.grpc
compile group: 'io.grpc', name: 'grpc-protobuf', version: versions.grpc
compile group: 'io.grpc', name: 'grpc-stub', version: versions.grpc
compile group: 'javax.annotation', name: 'javax.annotation-api', version: versions.javax_annotation
compile group: 'org.springframework', name: 'spring-beans', version: versions.spring
compile group: 'org.springframework', name: 'spring-context', version: versions.spring
gradleLint.ignore {
// Avoid unused-dependency error on spring-web. It is detected as a service provider which gradleLint thinks
// should be a runtime dependency. There are compile time dependencies on spring-web so use an ignore block.
compile group: 'org.springframework', name: 'spring-web', version: versions.spring
compile group: 'org.springframework.boot', name: 'spring-boot', version: versions.spring_boot
compile group: 'org.springframework.boot', name: 'spring-boot-autoconfigure', version: versions.spring_boot
compile group: 'org.springframework', name: 'spring-core', version: versions.spring
}
gradleLint.ignore { // See https://github.com/nebula-plugins/gradle-lint-plugin/issues/126
// Add javax.servlet-api for compiling but the implementations will be provided at runtime by the container
compileOnly group: 'javax.servlet', name: 'javax.servlet-api', version: versions.servlet_api
}
runtime group: 'org.apache.logging.log4j', name: 'log4j-api', version: versions.log4j
runtime group: 'org.apache.logging.log4j', name: 'log4j-core', version: versions.log4j
runtime group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: versions.log4j // For SLF4J 1.7.x releases or older.
// to consume encrypted configuration generated by tsm/tab-core-crypto, include this runtime dependency
// and set features.SecureSecretStorage = true anywhere in your application properties
runtime nerv('tab-microservice-config-decrypt-spring', versions.config_decrypt)
testCompile ( group: 'junit', name: 'junit', version: versions.junit )
gradleLint.ignore { // See https://github.com/nebula-plugins/gradle-lint-plugin/issues/126
// Add javax.servlet-api for compiling but the implementations will be provided at runtime by the container
testCompile group: 'org.springframework.boot', name: 'spring-boot-test', version: versions.spring_boot
testCompile group: 'org.springframework', name: 'spring-test', version: versions.spring
}
constraints {
implementation("com.netflix.nebula:gradle-scm-plugin:4.1.0"){
because("Breaking changes for Gradle 7 compatibility")
}
}
// Bring in logging modules. Code should be written to the slf4j API.
// At runtime, this is resolved to log4j for this service.
// PLEASE Remember to include these dependencies in all WAR files. Failure to include this will lead
// to very difficult to diagnose problems with logging not working correctly.
compile group: 'org.slf4j', name: 'slf4j-api', version: versions.slf4j
runtime group: 'org.slf4j', name: 'jcl-over-slf4j', version: versions.slf4j // slf4j implementation for commons-logging
runtime group: 'org.slf4j', name: 'slf4j-log4j12', version: versions.slf4j // Send slf4j calls to log4j
// Applications projects only: Spring Boot add the compatible Log4J implementation (omit for libraries)
// All 3 are necessary! Logging still may work without all 3 because you're getting some of these transitively, but that shouldn't be relied on.
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-api', version: versions.log4j
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-core', version: versions.log4j
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: versions.log4j // For SLF4J 1.7.x releases or older.
integrationTestCompile nerv('grpc-helpers-java',versions.grpc_helpers_java)
// to consume encrypted configuration generated by tsm/tab-core-crypto, include this runtime dependency
// and set features.SecureSecretStorage = true anywhere in your application properties
integrationTestCompile group: 'junit', name: 'junit', version: versions.junit
// Bring in logging modules. Code should be written to the slf4j API.
// Runtime configuration resolves this to a specific output implementation.
// The jcl-over-slf4j module directs java common logging to slf4j
// slf4j is then directed to log4j for the service and consumer, like we expect in Production.
integrationTestCompile group: 'org.springframework', name: 'spring-test', version: versions.spring
integrationTestCompile group: 'org.springframework.boot', name: 'spring-boot-test', version: versions.spring_boot
}
def testSystemProperties = [
"spring.config.location": "${project.projectDir}/build/resources/integrationTest/application-integration-test.properties",
"logging.config": "${project.projectDir}/build/resources/integrationTest/log4j2.xml"
]
task integrationTest(type: Test) {
testClassesDirs = sourceSets.integrationTest.output.classesDirs
classpath = sourceSets.integrationTest.runtimeClasspath
outputs.upToDateWhen { false }
}
eclipse {
classpath {
defaultOutputDir = file('build-eclipse/classes')
}
}
distZip.enabled = false
distTar.enabled = false
bootRun.systemProperties = application.properties
The build.gradle file for Child folder:
version '0.1.0'
sourceCompatibility = 1.8
targetCompatibility = 1.8
ext.versions = [
spring: '5.2.9.RELEASE',
spring_boot: '2.2.10.RELEASE',
slf4j: '1.7.30',
grpc_helpers_java: '^19.0'
]
dependencies {
compile nerv('grpc-helpers-java', versions.grpc_helpers_java)
compile group: 'org.springframework', name: 'spring-core', version: versions.spring
compile group: 'org.springframework', name: 'spring-beans', version: versions.spring
compile group: 'org.springframework', name: 'spring-context', version: versions.spring
compile group: 'org.springframework', name: 'spring-web', version: versions.spring
compile group: 'com.fasterxml.jackson.core', name: 'jackson-annotations', version:'2.10.5'
compile group: 'com.fasterxml.jackson.core',name:'jackson-core', version:'2.10.5'
compile group: 'com.fasterxml.jackson.core', name:'jackson-databind', version:'2.10.5'
compile group: 'org.slf4j', name: 'slf4j-api', version: versions.slf4j
compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: versions.spring_boot
runtime group: 'org.slf4j', name: 'slf4j-log4j12', version: versions.slf4j
gradleLint.ignore {
// Avoid unused-dependency error on spring-web. It is detected as a service provider which gradleLint thinks
// should be a runtime dependency. There are compile time dependencies on spring-web so use an ignore block.
compile group: 'org.springframework', name: 'spring-web', version: versions.spring
//compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: versions.spring_boot
}
}
Thank you

If you are using Spring MVC and the new WebClient from Spring WebFlux in the same application, Spring MVC will be used by default. You can override that easily by calling setWebApplicationType(WebApplicationType).
It will be helpful if you can share build.gradle.

Failed to parse TextMessage payload=[{"id" :1}], byteCount=9, last=true] in session Sending STOMP ERROR to client

I am using STOMP with spring java, I am able to connect to the application using simple in memory broker. But whenever I try to send any data to any of the destinations defined in my application I get the following error
Failed to parse TextMessage payload=[{"id" :1}], byteCount=9, last=true] in session b1e46a4d-c608-af0b-1d1e-bd6bb896bae7. Sending STOMP ERROR to client.
java.lang.NoSuchMethodError: org.springframework.util.StreamUtils.copyToString(Ljava/io/ByteArrayOutputStream;Ljava/nio/charset/Charset;)Ljava/lang/String;
Any idea if I missing any parser or necessary decoder required for STOMP to handle the incoming messages.

It appears that this method StreamUtils::copyToString(ByteArrayOutputStream baos, Charset charset) has been added to Spring quite recently (since v.5.2.6 released on Apr 28, 2020) according to documentation.
So you need to check the dependencies in your project and make sure that spring-integration-stomp uses appropriate release of Spring.

I was eventually able to solve this issue. I was missing 2 major packages that STOMP required for proper decoding of the message:
compile group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.10.2'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.10.2'
So my final gradle build( which included all dependencies for message broker i.e. RabbitMQ
and also Web Socket Security ) with all the necessary dependencies required fore Websocket with STOMP on Spring 2.2.6.RELEASE looked like as below
compile group: 'org.springframework', name: 'spring-messaging', version: '5.2.2.RELEASE'
compile group: 'org.springframework', name: 'spring-websocket', version: '5.2.2.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-amqp', version: '2.3.1.RELEASE'
compile group: 'org.springframework.security', name: 'spring-security-messaging', version: '5.3.3.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-reactor-netty', version: '2.3.1.RELEASE'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.10.2'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.10.2'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-reactor-netty', version: '2.3.1.RELEASE'

ClassNotFoundException: /org/jdom2/JDOMException thrown after the project is compiled

Edit: This question is a duplicate to this question, which got incorrectly marked as duplicate, forcing me to restate my question.
To preface this, yes, I have seen this thread, but it did not help me. Specificly, while multiple people mentioned that different class structures between the building branch and the environment branch can exists, noone said anything about how to test for that, let alone fix it. And even then, I am not certain that this is really root cause.
I am working on a gradle project and I wanted to compile the project I had, to test it outside of my IDE, Netbeans 8.2, in which it works fine.
When I try to run the program with the usual java -jar command, it initally works fine, until the programs comes across a class that can throw a JDOMException. Then I get the error described in the title, followed by a NoClassDefFoundError.
In Netbeans, the JDOMException.class is included under Dependencies>Compile for main>jdom2-2.0.6.jar>org.jdom2
And the build.gradle script reads:
apply plugin: 'java'
sourceCompatibility = 1.8
[compileJava, compileTestJava]*.options*.encoding = 'UTF-8'
if (!hasProperty('mainClass')) {
ext.mainClass = 'bikesys.hda.BikeSys'
}
repositories {
mavenCentral()
jcenter()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.10'
compile files('libs/jcalendar-1.4.jar')
compile group: 'org.jdom', name: 'jdom2', version: '2.0.6'
compile ("com.byteowls:jopencage:1.2.0")
compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.0.1'
compile group: 'org.slf4j', name:'slf4j-api', version: '1.7.2'
compile group: 'ch.qos.logback', name:'logback-classic', version: '1.0.9'
compile group: 'ch.qos.logback', name:'logback-core', version: '1.0.9'
compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.6'
compile group: 'org.json', name: 'json', version: '20180813'
compile group: 'com.google.code.gson', name: 'gson', version: '2.8.5'
compile group: 'log4j', name: 'log4j', version: '1.2.17'
compile group: 'me.atlis', name: 'atlis-location-base', version: '1.0.0'
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.8.1'
compile group: 'me.atlis', name: 'nominatim-api', version: '1.0.0'
}
jar {
manifest {
attributes(
'Main-Class': 'bikesys.hda.BikeSys'
)
}
}
I am using java jdk 1.8.0_202 in Netbeans and jre 1.8.0_211-b12 as runtime environment. I have found answers along the lines of "You have to add jdom2 to your runtime environment", however I neither know how to do that, nor if that is a general solution to the problem.
I'd appreciate the help!

The problem was that, by default, gradle does not include external dependencies when building a normal .jar file. I had to instruct gradle to build a "fat jar" file instead.
I did this by altering the
jar {
...
}
part of my gradle script into the following instruction:
jar {
manifest {
attributes(
'Main-Class': 'bikesys.hda.BikeSys'
)
}
from {
configurations.compileClasspath.filter{ it.exists() }.collect { it.isDirectory() ? it : zipTree(it) }
}
}
Be sure to edit the class path, which in my case was bikesys.hda.BikeSys, to your main class.

Building a simple MapReduce project with gradle: Hadoop dependencies don't have Mapper and Reducer

I'm trying to build a simple Hadoop mapreduce program and I chose Java for that job. I checked out the example codes around and tried to build myself. I created the following gradle script and when I looked at the installed dependencies, none had Mapper or Reducer. Not even org.apache.hadoop.mapreduce package.
group 'org.ardilgulez.demoprojects'
version '1.0-SNAPSHOT'
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
compile group: 'org.apache.hadoop', name: 'hadoop-common', version:'2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-hdfs', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-yarn-common', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-minicluster', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-core', version:'2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-jobclient', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-app', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-shuffle', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-common', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-client', version: '2.7.3'
}
I know I'm not going to be needing at least 7 out of those 10 hadoop dependencies, but I don't know which of these dependencies have org.apache.hadoop.mapreduce package (I know these 11 don't though).
Which dependency/repository should I add so that I can actually build a mapreduce job?
Can I do that with raw org.apache.hadoop packages and not vendor packages (such as Cloudera)?
Thanks in advance for all the help.

This should be the correct dependency:
compile 'org.apache.hadoop:hadoop-mapreduce-client-core:2.7.3'
Be sure to refresh your gradle project.

Add
compile group: 'org.apache.hadoop', name:'hadoop-core', version: '2.7.3'

For newer versions of Gradle (I'm using 7.5.1) compile isn't an option anymore. In that case, you should use implementation.
dependencies {
implementation 'org.apache.hadoop:hadoop-common:2.7.7'
implementation 'org.apache.hadoop:hadoop-mapreduce-client-core:2.7.7'
}

Gradle distZip - adds excluded dependency

My project uses
compile group: 'javax.servlet', name: 'javax.servlet-api', version: '3.1.0'
I added Google Sheets API to dependencies. Now
gradle dependencies says Google Sheets API uses org.mortbay.jetty:servlet-api:2.5-20081211 which I assume to exclude, because classes are loaded from the wrong jar:
compile (group: 'com.google.apis', name: 'google-api-services-sheets', version: 'v4-rev25-1.22.0') {
exclude(module: 'servlet-api')
}
compile (group: 'com.google.gdata', name: 'core', version: '1.47.1') {
exclude(module: 'servlet-api')
}
Now gradle dependencies does not show servlet-api:2.5 to be used anywhere. But after gradle clean distZip I see that archive contains servlet-api-2.5-20081211.jar! How can I exclude it from the build?