Related
I tried to use the Springframework WebClient (I use IntelliJ with gradle), and in the "dependencies" section of the build.gradle, I added
“compile group: ‘org.springframework.boot’, name: ‘spring-boot-starter-webflux’, version: versions.spring_boot”
However, when I do gradle build, it failed in the lintGradle task with the following errors:
This project contains lint violations. A complete listing of the violations follows.
Because some were serious, the overall build status has been changed to FAILED
error unused-dependency one or more classes in jakarta.annotation:jakarta.annotation-api:1.3.5 are required by your code directly (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:compile’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:runtime’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:testCompile’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:implementation’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error transitive-duplicate-dependency-classjakarta.annotation:jakarta.annotation-api:1.3.5 in configuration ‘:integrationTestCompile’ has 15 classes duplicated by javax.annotation:javax.annotation-api:1.3.2 (use --info for detailed class list) (no auto-fix available)
error undeclared-dependency one or more classes in jakarta.annotation:jakarta.annotation-api:1.3.5 are required by your code directly
error unused-dependency this dependency should be removed since its artifact is empty (no auto-fix available)
build.gradle:122
compile group: ‘org.springframework.boot’, name: ‘spring-boot-starter-webflux’, version: versions.spring_boot
Not sure if I need to add something else to the gradle build setting environment? Or if the webflux dependency conflicts with other dependencies?
There is a parent folder Par and child folder Child, they both have their build.gradle respectively. The build task in the Child folder passed but failed in the Par folder with the errors above. The use of WebClient is in the child folder and the dependency is added to the build.gradle file of the Child foler.
The build.gradle file for Par folder:
plugins {
id 'org.springframework.boot' version '2.2.6.RELEASE'
id 'nerv-java-war-module-plugin' version '10.9.0'
id 'nerv-tab-checkstyle-plugin' version '4.5.0'
id 'hackathon-teamcity-auto-increment' version '3.1.0'
id 'microservice-incremental-deploy-plugin' version '1.2.0'
id 'protobuf-java-consumer-plugin' version '4.2.0'
id 'nebula.lint' version '16.0.2'
id 'application'
id 'eclipse-wtp'
}
subprojects {
apply plugin: 'nerv-java-war-module-plugin'
apply plugin: 'nerv-tab-checkstyle-plugin'
apply plugin: 'hackathon-teamcity-auto-increment'
}
incrementalDeploy.packageNames += 'Par'
// Notice about Spring Dependency Management: It doesn't work in this project due to a
// nebula.lint bug: https://github.com/nebula-plugins/gradle-lint-plugin/issues/227
gradleLint {
criticalRules += 'all-dependency'
excludedRules += ['overridden-dependency-version']// Ignore for dependency lock
/*gradle daemon locks the Child.jar file which prevents from
cleaning the build
cause: linting rules 'undeclared-dependency','unused-dependency'
Workaround: run gw --stop and then clean */
alwaysRun = false
build.finalizedBy lintGradle
}
sourceCompatibility = 1.8
targetCompatibility = 1.8
// Integration tests related setup for sources
sourceSets {
integrationTest {
java {
compileClasspath += main.output + test.output
runtimeClasspath += main.output + test.output
srcDir file('src/integrationTest/java')
}
resources.srcDir file('src/integrationTest/resources')
}
}
// When running locally, configure Log4J via system property
application {
mainClassName = 'Par.ParApplication'
applicationDefaultJvmArgs = ["-Dlog4j.defaultInitOverride=false",
"-Dlogging.config=${buildDir}/config/log4j2.xml"]
}
// When running locally, we need some config files that would normally live in
// resources. However, resources get included in the published WAR file, which
// we need to avoid. We copy these to the build dir so that the same config files
// can be used by both this service, and the accompanying contractTest.
task copyConfig(type: Copy) {
from("${buildscript.sourceFile.parent}/config/")
into("${buildDir}/config")
}
bootRun.dependsOn(copyConfig)
ext.versions = [
// These definitions are shared with the contractTest application,
// so version numbers can be updated in one place.
config_decrypt: '^3.0',
grpc_helpers_java: '^19.0',
// Fixed versions for third-party packages
grpc: '1.32.1',
guava: '29.0-jre',
javax_annotation: '1.3.2',
junit: '4.12',
log4j: '2.13.1',
protobuf_java: '3.12.0',
servlet_api: '4.0.1',
slf4j: '1.7.30',
spring: '5.2.9.RELEASE',
spring_boot: '2.2.10.RELEASE'
]
configurations {
gradleLint.ignore {
// These exclusions are useful for application projects to make sure they aren't getting any log4j v1 dependencies transitively
all*.exclude group: 'log4j', module: 'log4j'
all*.exclude group: 'org.slf4j', module: 'slf4j-log4j12'
// ignore logging libraries
all*.exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
}
// Favor org.slf4j:jcl-over-slf4j
all*.exclude group: 'org.springframework', module: 'spring-jcl'
integrationTestCompile.extendsFrom testCompile
integrationTestRuntime.extendsFrom testRuntime
integrationTestRuntimeOnly.extendsFrom testRuntimeOnly
}
dependencies {
compile project(':Child')
compile nerv('grpc-helpers-java', versions.grpc_helpers_java)
compile group: 'com.google.guava', name: 'guava', version: versions.guava
compile group: 'com.google.protobuf', name: 'protobuf-java', version: versions.protobuf_java
compile group: 'io.grpc', name: 'grpc-api', version: versions.grpc
compile group: 'io.grpc', name: 'grpc-protobuf', version: versions.grpc
compile group: 'io.grpc', name: 'grpc-stub', version: versions.grpc
compile group: 'javax.annotation', name: 'javax.annotation-api', version: versions.javax_annotation
compile group: 'org.springframework', name: 'spring-beans', version: versions.spring
compile group: 'org.springframework', name: 'spring-context', version: versions.spring
gradleLint.ignore {
// Avoid unused-dependency error on spring-web. It is detected as a service provider which gradleLint thinks
// should be a runtime dependency. There are compile time dependencies on spring-web so use an ignore block.
compile group: 'org.springframework', name: 'spring-web', version: versions.spring
compile group: 'org.springframework.boot', name: 'spring-boot', version: versions.spring_boot
compile group: 'org.springframework.boot', name: 'spring-boot-autoconfigure', version: versions.spring_boot
compile group: 'org.springframework', name: 'spring-core', version: versions.spring
}
gradleLint.ignore { // See https://github.com/nebula-plugins/gradle-lint-plugin/issues/126
// Add javax.servlet-api for compiling but the implementations will be provided at runtime by the container
compileOnly group: 'javax.servlet', name: 'javax.servlet-api', version: versions.servlet_api
}
runtime group: 'org.apache.logging.log4j', name: 'log4j-api', version: versions.log4j
runtime group: 'org.apache.logging.log4j', name: 'log4j-core', version: versions.log4j
runtime group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: versions.log4j // For SLF4J 1.7.x releases or older.
// to consume encrypted configuration generated by tsm/tab-core-crypto, include this runtime dependency
// and set features.SecureSecretStorage = true anywhere in your application properties
runtime nerv('tab-microservice-config-decrypt-spring', versions.config_decrypt)
testCompile ( group: 'junit', name: 'junit', version: versions.junit )
gradleLint.ignore { // See https://github.com/nebula-plugins/gradle-lint-plugin/issues/126
// Add javax.servlet-api for compiling but the implementations will be provided at runtime by the container
testCompile group: 'org.springframework.boot', name: 'spring-boot-test', version: versions.spring_boot
testCompile group: 'org.springframework', name: 'spring-test', version: versions.spring
}
constraints {
implementation("com.netflix.nebula:gradle-scm-plugin:4.1.0"){
because("Breaking changes for Gradle 7 compatibility")
}
}
// Bring in logging modules. Code should be written to the slf4j API.
// At runtime, this is resolved to log4j for this service.
// PLEASE Remember to include these dependencies in all WAR files. Failure to include this will lead
// to very difficult to diagnose problems with logging not working correctly.
compile group: 'org.slf4j', name: 'slf4j-api', version: versions.slf4j
runtime group: 'org.slf4j', name: 'jcl-over-slf4j', version: versions.slf4j // slf4j implementation for commons-logging
runtime group: 'org.slf4j', name: 'slf4j-log4j12', version: versions.slf4j // Send slf4j calls to log4j
// Applications projects only: Spring Boot add the compatible Log4J implementation (omit for libraries)
// All 3 are necessary! Logging still may work without all 3 because you're getting some of these transitively, but that shouldn't be relied on.
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-api', version: versions.log4j
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-core', version: versions.log4j
runtimeOnly group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: versions.log4j // For SLF4J 1.7.x releases or older.
integrationTestCompile nerv('grpc-helpers-java',versions.grpc_helpers_java)
// to consume encrypted configuration generated by tsm/tab-core-crypto, include this runtime dependency
// and set features.SecureSecretStorage = true anywhere in your application properties
integrationTestCompile group: 'junit', name: 'junit', version: versions.junit
// Bring in logging modules. Code should be written to the slf4j API.
// Runtime configuration resolves this to a specific output implementation.
// The jcl-over-slf4j module directs java common logging to slf4j
// slf4j is then directed to log4j for the service and consumer, like we expect in Production.
integrationTestCompile group: 'org.springframework', name: 'spring-test', version: versions.spring
integrationTestCompile group: 'org.springframework.boot', name: 'spring-boot-test', version: versions.spring_boot
}
def testSystemProperties = [
"spring.config.location": "${project.projectDir}/build/resources/integrationTest/application-integration-test.properties",
"logging.config": "${project.projectDir}/build/resources/integrationTest/log4j2.xml"
]
task integrationTest(type: Test) {
testClassesDirs = sourceSets.integrationTest.output.classesDirs
classpath = sourceSets.integrationTest.runtimeClasspath
outputs.upToDateWhen { false }
}
eclipse {
classpath {
defaultOutputDir = file('build-eclipse/classes')
}
}
distZip.enabled = false
distTar.enabled = false
bootRun.systemProperties = application.properties
The build.gradle file for Child folder:
version '0.1.0'
sourceCompatibility = 1.8
targetCompatibility = 1.8
ext.versions = [
spring: '5.2.9.RELEASE',
spring_boot: '2.2.10.RELEASE',
slf4j: '1.7.30',
grpc_helpers_java: '^19.0'
]
dependencies {
compile nerv('grpc-helpers-java', versions.grpc_helpers_java)
compile group: 'org.springframework', name: 'spring-core', version: versions.spring
compile group: 'org.springframework', name: 'spring-beans', version: versions.spring
compile group: 'org.springframework', name: 'spring-context', version: versions.spring
compile group: 'org.springframework', name: 'spring-web', version: versions.spring
compile group: 'com.fasterxml.jackson.core', name: 'jackson-annotations', version:'2.10.5'
compile group: 'com.fasterxml.jackson.core',name:'jackson-core', version:'2.10.5'
compile group: 'com.fasterxml.jackson.core', name:'jackson-databind', version:'2.10.5'
compile group: 'org.slf4j', name: 'slf4j-api', version: versions.slf4j
compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: versions.spring_boot
runtime group: 'org.slf4j', name: 'slf4j-log4j12', version: versions.slf4j
gradleLint.ignore {
// Avoid unused-dependency error on spring-web. It is detected as a service provider which gradleLint thinks
// should be a runtime dependency. There are compile time dependencies on spring-web so use an ignore block.
compile group: 'org.springframework', name: 'spring-web', version: versions.spring
//compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: versions.spring_boot
}
}
Thank you
If you are using Spring MVC and the new WebClient from Spring WebFlux in the same application, Spring MVC will be used by default. You can override that easily by calling setWebApplicationType(WebApplicationType).
It will be helpful if you can share build.gradle.
I am using STOMP with spring java, I am able to connect to the application using simple in memory broker. But whenever I try to send any data to any of the destinations defined in my application I get the following error
Failed to parse TextMessage payload=[{"id" :1}], byteCount=9, last=true] in session b1e46a4d-c608-af0b-1d1e-bd6bb896bae7. Sending STOMP ERROR to client.
java.lang.NoSuchMethodError: org.springframework.util.StreamUtils.copyToString(Ljava/io/ByteArrayOutputStream;Ljava/nio/charset/Charset;)Ljava/lang/String;
Any idea if I missing any parser or necessary decoder required for STOMP to handle the incoming messages.
It appears that this method StreamUtils::copyToString(ByteArrayOutputStream baos, Charset charset) has been added to Spring quite recently (since v.5.2.6 released on Apr 28, 2020) according to documentation.
So you need to check the dependencies in your project and make sure that spring-integration-stomp uses appropriate release of Spring.
I was eventually able to solve this issue. I was missing 2 major packages that STOMP required for proper decoding of the message:
compile group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.10.2'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.10.2'
So my final gradle build( which included all dependencies for message broker i.e. RabbitMQ
and also Web Socket Security ) with all the necessary dependencies required fore Websocket with STOMP on Spring 2.2.6.RELEASE looked like as below
compile group: 'org.springframework', name: 'spring-messaging', version: '5.2.2.RELEASE'
compile group: 'org.springframework', name: 'spring-websocket', version: '5.2.2.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-amqp', version: '2.3.1.RELEASE'
compile group: 'org.springframework.security', name: 'spring-security-messaging', version: '5.3.3.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-reactor-netty', version: '2.3.1.RELEASE'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-core', version: '2.10.2'
compile group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.10.2'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-reactor-netty', version: '2.3.1.RELEASE'
I'm using gradle with selenium (Java) Testng to automate my test cases and choose Jenkins to build the scripts.
I have many testng.xml files in my workspace under src/test/resources --> ACE_Regression (package)
My build.gradle looks like this
buildscript
{
ext
{
testNgVersion = "6.14.3"
}
}
apply plugin: 'java-library'
repositories {
jcenter()
mavenCentral()
}
dependencies {
api 'org.apache.commons:commons-math3:3.6.1'
implementation 'com.google.guava:guava:27.0.1-jre'
implementation 'com.google.guava:guava:27.0.1-jre'
testImplementation 'junit:junit:4.12'
implementation "org.testng:testng:$testNgVersion"
compile group: 'org.freemarker', name: 'freemarker', version: '2.3.23'
compile group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '3.14.0'
compile group: 'com.aventstack', name: 'extentreports', version: '3.0.6'
compile group: 'com.relevantcodes', name: 'extentreports', version: '2.41.2'
compile group: 'org.apache.poi', name: 'poi', version: '4.0.0'
compile group: 'org.apache.poi', name: 'poi-ooxml', version: '4.0.0'
compile group: 'commons-io', name: 'commons-io', version: '2.4'
compile group: 'ru.yandex.qatools.ashot', name: 'ashot', version: '1.5.2'
testCompile group: 'io.rest-assured', name: 'rest-assured', version: '3.0.0'
compile group: 'com.googlecode.json-simple', name: 'json-simple', version: '1.1.1'
compile group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.12.1'
compile group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.12.1'
}
test {
useTestNG()
{
suites "src/test/resources/ACE_Regression/Acelogin.xml"
suites "src/test/resources/ACE_Regression/CLAs.xml"
}
}
Right now i use "gradlew test" to run my tests from jenkins.
I want to give input from jenkins to run only CLAs.xml file. Like this i have many XML files avaliable . I want to parameterize the job to pick only required testng files and run.
Can someone help me how to setup this test task in build.gradle file and jenkins
Thanks in advance
Modify the useTestNG() in your build.grade . It should accept one dynamic parameter e.g. xmlName.
test {
useTestNG()
{
suites "src/test/resources/ACE_Regression/${xmlName}.xml"
}
}
Pass the xmlName using -P option in your gradle invocation block. i.e. gradlew test -PxmlName=CLAs , it should work.
To make it more dynamic, Make your jenkins build parameterized. Create a choice parameter by the name of testSuites and add all the possible xml file names in it.
Then pass this choice parameter to the gradle invocation command block i.e. gradle test -PxmlName=${testSuites}
I am having a portlet deployed on Liferay which has webservice call and is using internally axis libraries. When a webservice is called I am getting following error
Caused by: java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Provider com.ctc.wstx.stax.WstxInputFactory not a subtype
I checked various other posts
java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory: Provider com.ctc.wstx.stax.WstxInputFactory not a subtype
List item
I analyzed the error line in breaking it up into two parts
Caused by: java.util.ServiceConfigurationError: javax.xml.stream.XMLInputFactory
Provider com.ctc.wstx.stax.WstxInputFactory not a subtype
And came to conclusion that the error is due to Stax Implementation mismatch.
So I check the lib directory and found the following jars
I see that WstxInputFactory is present in woodstox-core-asl-4.1.4
To see from where this jar gets added I checked the build.gradle
buildscript {
repositories {
maven {
url "https://cdn.lfrs.sl/repository.liferay.com/nexus/content/groups/public"
}
}
dependencies {
classpath group: "com.liferay", name: "com.liferay.gradle.plugins.css.builder", version: "2.0.0"
}
}
apply plugin: "com.liferay.css.builder"
dependencies {
def withoutServletAPI = {
exclude group: 'javax.servlet', module: 'servlet-api'
}
compile group: "org.springframework", name: "spring-webmvc-portlet", version: "4.1.5.RELEASE"
compileOnly group: "com.liferay.portal", name: "com.liferay.portal.kernel", version: "2.6.0"
compileOnly group: "javax.portlet", name: "portlet-api", version: "2.0"
compileOnly group: "javax.servlet", name: "javax.servlet-api", version: "3.0.1"
compileOnly group: "org.osgi", name: "org.osgi.service.component.annotations", version: "1.3.0"
compile group: "org.apache.axis", name:"axis", version: "1.4"
compile group: "org.apache.axis2", name:"axis2-adb", version: "1.6.3", withoutServletAPI
compile group: "org.apache.axis2", name:"axis2-transport-local", version: "1.6.3", withoutServletAPI
compile group: "org.apache.axis2", name:"axis2-transport-http", version: "1.6.3", withoutServletAPI
compile group: "com.google.code.gson", name:"gson", version: "2.3.1"
compile group: "jstl", name:"jstl", version: "1.2"
compile group: "taglibs", name:"standard", version: "1.1.2"
//compile group: "memberservice", name:"memberservice", version: "1.0.0"
compileOnly group: "com.liferay.portal", name: "com.liferay.util.java", version: "2.4.0"
//compile group: "com.sun.xml.ws", name: "jaxws-rt", version: "2.2.10"
testCompile group: 'junit', name: 'junit', version: '4.8.1'
testCompile group: 'org.mockito', name: 'mockito-all', version: '1.9.5'
compile files('../../modules/memberservice/build/libs/memberservice-1.0.0.jar')
}
war {
dependsOn buildCSS
exclude "**/*.scss"
filesMatching("**/.sass-cache/") {
it.path = it.path.replace(".sass-cache/", "")
}
includeEmptyDirs = false
}
I checked the dependency tree in gradle and found that woodstox-core-asl-4.1.4 is a transitive dependency for axis2-adb-1.6.3 which is in build.gradle.
Obviously there is some conflict in jar which is causing the above error but I am not able to find which jar is causing it. My suspicion is that JDK 8 already has XMLInputFactory and in my war I see that there is geronimo-stax-api which again is a transitive dependency and that might be causing conflict.
I am looking for what is actually causing the issue.
Note: This code is not created by me. My client has given this code which was running on older version of Liferay(6.2) on JBoss and created by different vendor and we are moving to the newer version of Liferay(DXP) on Tomcat.
PS: I also checked Tomcat's lib directory and there is no stax implementation there too
I have a similar issue. When upgraded portlet is deployed, spring application boots up successfully. When attempted to view a portlet page, this error occurs.
2018-08-29 14:31:50.955 ERROR [http-nio-8080-exec-8][render_portlet_jsp:131] null
java.util.ServiceConfigurationError: javax.xml.parsers.SAXParserFactory: Provider org.apache.xerces.jaxp.SAXParserFactoryImpl not a subtype
at java.util.ServiceLoader.fail(ServiceLoader.java:239)
at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.xml.parsers.FactoryFinder$1.run(FactoryFinder.java:294)
at javax.xml.parsers.FactoryFinder.findServiceProvider(FactoryFinder.java:289)
at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:267)
at javax.xml.parsers.SAXParserFactory.newInstance(SAXParserFactory.java:127)
at org.apache.taglibs.standard.tlv.JstlBaseTLV.validate(JstlBaseTLV.java:194)
at org.apache.taglibs.standard.tlv.JstlCoreTLV.validate(JstlCoreTLV.java:138)
at org.apache.jasper.compiler.TagLibraryInfoImpl.validate(TagLibraryInfoImpl.java:957)
at org.apache.jasper.compiler.Validator.validateXmlView(Validator.java:1929)
at org.apache.jasper.compiler.Validator.validate(Validator.java:1896)
at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:223)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:451)
at org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:625)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:375)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:473)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:377)
I'm trying to build a simple Hadoop mapreduce program and I chose Java for that job. I checked out the example codes around and tried to build myself. I created the following gradle script and when I looked at the installed dependencies, none had Mapper or Reducer. Not even org.apache.hadoop.mapreduce package.
group 'org.ardilgulez.demoprojects'
version '1.0-SNAPSHOT'
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
testCompile group: 'junit', name: 'junit', version: '4.11'
compile group: 'org.apache.hadoop', name: 'hadoop-common', version:'2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-hdfs', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-yarn-common', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-minicluster', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-core', version:'2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-jobclient', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-app', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-shuffle', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-mapreduce-client-common', version: '2.7.3'
compile group: 'org.apache.hadoop', name: 'hadoop-client', version: '2.7.3'
}
I know I'm not going to be needing at least 7 out of those 10 hadoop dependencies, but I don't know which of these dependencies have org.apache.hadoop.mapreduce package (I know these 11 don't though).
Which dependency/repository should I add so that I can actually build a mapreduce job?
Can I do that with raw org.apache.hadoop packages and not vendor packages (such as Cloudera)?
Thanks in advance for all the help.
This should be the correct dependency:
compile 'org.apache.hadoop:hadoop-mapreduce-client-core:2.7.3'
Be sure to refresh your gradle project.
Add
compile group: 'org.apache.hadoop', name:'hadoop-core', version: '2.7.3'
For newer versions of Gradle (I'm using 7.5.1) compile isn't an option anymore. In that case, you should use implementation.
dependencies {
implementation 'org.apache.hadoop:hadoop-common:2.7.7'
implementation 'org.apache.hadoop:hadoop-mapreduce-client-core:2.7.7'
}