Develop Reference

updating gradle version for a project that depends on another shared library project doesn't work in a toolchain (kubernetes)

I'm getting this error while trying to commit my changes into gitlab repo and deploy it in kubernetes after upgrading gradle version to 7.4
Also I think the problem in how I include my shared library inside the project it self, that's what I sense from the error output below
Could not determine the dependencies of task ':distTar'.
Could not resolve all task dependencies for configuration ':runtimeClasspath'.
Could not resolve project :vlc-shc.
Required by:
project :
> No matching configuration of project :vlc-shc was found. The consumer was configured to find a runtime of a library compatible with Java 11, packaged as a jar, preferably optimized for standard JVMs, and its dependencies declared externally but:
- None of the consumable configurations have attributes.
build.gradle:
dependencies {
implementation project(':vlc-shc')
}
setting.gradle:
rootProject.name = 'vlc-myProject'
include ('vlc-shc')

groovy-all:3.0.8 in my build.gradle is downloading different jars versions

After adding groovie-all:3.0.8 into my build.gradle, the downloaded jars are in different version as shown bellow. Each of this jars has its 3.0.8 version and I don't know why they aren't download instead.
Appreciate any help
Thanx

Since Groovy 2.5, groovy-all is just a pom which brings in the equivalent component jars, see the according release notes. This explains that you see no groovy-all.
Regarding the version numbers: spring-boot defines the grooy version, you can print it with:
ext {
// versions taken from Spring BOM
GROOVY_VERSION = dependencyManagement.importedProperties['groovy.version']
println GROOVY_VERSION
}
and I expect it is 2.5.13 which for example is included by Spring Boot Version 2.3.4. Unfortunately you didn't specify your used Spring version.
With Spring Boot you include Groovy normally just with
compile "org.codehaus.groovy:groovy-all"
without a version number since Spring Boot defines the version.
So the question is how to override the provided Spring version - alternatively you can increase the used Spring version to get a more recent Groovy version.
To override the Spring version just exclude Groovy, eg see this SO posts:
https://stackoverflow.com/a/50972674/3181392
https://stackoverflow.com/a/47355002/734687
E.g. use:
ext['groovy.version'] = '3.0.8'
To see which component included the wrong version you can execute the Gradle task "dependencies" (find it in IDEA under the help categegory). In this answer I assumed it was Spring (and I am sure it was).

How do you invoke schemagen in Java 11?

According to Oracle documentation the schemagen tool is removed from the JDK as part of JEP 320 (http://openjdk.java.net/jeps/320).
That JEP points to Maven artifacts that now supply the missing tools. The coordinates of the artifacts are wrong in the JEP, updated coordinates are found in an answer to this question:
Which artifacts should I use for JAXB RI in my Maven project?
What is missing however is how to invoke the tools. There are shell scripts pointed to in the JEP that are in the JAXB-RI Git repository. However those scripts remain undocumented and difficult to invoke. The build instructions in that git repo indicated it is built with a standard "mvn clean install", however that does not produce an output structure that matches the 'bin' folder used in the documentation here: https://javaee.github.io/jaxb-v2/doc/user-guide/ch04.html#tools-schemagen
Ideally I would like to run schemagen from Gradle, avoiding the shell scripts as they are not obtained from the maven dependency.
My current attempt, adapted from a working version that called the old schemagen.exe, looks like this:
(There is more in the 'real' build.gradle file to specify my application's dependencies, etc.)
configurations {
schemagenTool
}
dependencies {
schemagenTool "org.glassfish.jaxb:jaxb-jxc:${jaxbVersion}"
}
task schemaGen(type: Exec, dependsOn: [compileJava,configurations.schemaGenTool]) {
workingDir projectDir
executable 'java'
doFirst {
file('build/Schemas').mkdirs()
args '--module-path', "${configurations.schemaGenTool.asPath}",
'-m', 'jaxb.jxc/com.sun.tools.jxc.SchemaGeneratorFacade',
// Note: automatic module name is NOT com.sun.tool.jxc
// as documented
// Args to schemagen (these worked for the schemagen.exe)
'-d', 'build/Schemas',
'-classpath', "${compileJava.destinationDir}${File.pathSeparator}${configurations.compile.asPath}",
"src/main/java/path/to/my/ModelClass.java"
//println "\nschemagen: ${args.join(' ')}\n"
}
doLast {
// Above creates "build/Schemas/schema1.xsd" (despite printing a different path!)
// Rename it
def destFile = file('build/Schemas/model.xsd')
destFile.delete()
if (!file('build/Schemas/schema1.xsd').renameTo(destFile)) {
throw new GradleException("Failed to write new build/Schemas/model.xsd")
}
}
}
However, that results in an error:
Error occurred during initialization of boot layer
java.lang.module.ResolutionException: Modules jaxb.runtime and jaxb.core export package com.sun.xml.bind.marshaller to module relaxngDatatype

The issue seems to be known with the jaxb-*:2.3.0 version - #jaxb-v2/issues/1168. Additionally, this would be resolved with a future release as marked in the known issues of jaxb running over java-9.
You can resolve this following the comment -
Please try 2.4.0-b180725.0644 - this is JPMS modularized RI working on
JDKs with java.xml.bind module (9,10) and those without it (11-ea)
or try downloading the binary distribution from the same link.

Schemagen and xjc shell scripts are only put into binary distribution in ./bin directory.
For build tools there are plugins out there (Maven / Gradle) which does invoke schemagen and xjc APIs, providing user with simple configuration.
Your attempt to invoke com.sun.tools.jxc.SchemaGeneratorFacade manually is also correct, here is similar example for Maven. However you are probably putting 2.3.0 on module path, which has a split package problem. Putting on classpath will resolve the issue for 2.3.0. Next release of JAXB will be JPMS ready and have module descriptors declared. You can try the beta build (2.4.0-b180725.0644), here is a correct set of dependencies:
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-runtime</artifactId> <!--jaxb runtime-->
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-xjc</artifactId> <!--java generation-->
</dependency>
<dependency>
<groupId>org.glassfish.jaxb</groupId>
<artifactId>jaxb-jxc</artifactId> <!--schema generation-->
</dependency>

If trying to use the schemagen in the jaxb-ri download directly, note the JAXB_PATH in schemagen.bat is inconsistent with the mod directory in the jaxb-ri.zip file.
E.g. to use schemagen.bat:
As at now, download jaxb-ri-2.3.1.zip from https://maven.java.net/content/repositories/releases/com/sun/xml/bind/jaxb-ri/2.3.1/ (the 2.4.0-betas also had the mod/javax.activation-api.jar missing)
Use these steps to run schemagen.bat, setting JAXB_HOME as appropriate:
set JAXB_HOME=C:\Java\jaxb-ri-2.3.1
set CLASSPATH=%JAXB_HOME%/mod/relaxng-datatype.jar;%JAXB_HOME%/mod/javax.activation-api.jar
%JAXB_HOME%\bin\schemagen.bat -cp myjar1.jar;myjar2.jar -d target com.company.ClassName1 com.company.ClassName2
(This was against JDK 11)

Ivy can not resolve the scope of a dependency which is a dependency of a transitive dependency

I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.
I defined A in ivy.xml like this:
<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>
In pom file of B, C is defined both in compile and test scopes like below:
<dependency>
<groupId>X</groupId>
<artifactId>C</artifactId>
</dependency>
<dependency>
<groupId>X</groupId>
<artifactId>C</artifactId>
<type>test-jar</type>
<scope>test</scope>
</dependency>
When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
<artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>
For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?

I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:
"compile" and "test" targets are issuing separate calls to the resolve task
Each plugin is also calling an ivy resolve task
Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
Build plugins are not managed by ivy (Sonar, eclipse, rat)
I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).
I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.
In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?
Refactoring ivy
Here follows what I achieved so far:
Private "development" fork of the nutch project
Diff with trunk
Benefits:
Single ivy report showing all configurations (New ivy-resolve target)
New mechanism for installing ivy (New ivy-install target)
Classpaths are managed using ivy configurations (See use of ivy cachepath task and configurations in ivy file)
Eclipse, sonar and rat ANT tasks automatically installed using ivy (The Eclipse plugin is noteworthy as it uses a packager resolver to download and extract jar from a tar archive).
Impacts the following Nutch issues
NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)

How to configure a longer version Number in artifactory

The version-numbers for our jars have to be longer them x.x.x.
We would rather need x.x.x.x to integrate some old-fashioned self-made mechanism.
This is, because we tag our software with x.x.x and as soon as we have a delivery to a customer one specific jar has to be build exactly at this point of time to fit to another backend,
which communicates with our program. For that reason this one jar has the version 2.3.4.1,
when generated and in next delivery of the same Version it is build and named 2.3.4.2.
Now artifactory cannot handle this an doesn't save more than x.x.x.2 in some cases.
So we thought of maybe edit the regular expression in the maven repository layout (see attached Screenshot) Because testing the path in the field below shows,
that it cannot handle the version number. Of course for the rest of our jars still x.x.x has to work..
For Example here is the maven-metadata.xml
<?xml version="1.0" encoding="UTF-8"?>
<metadata>
<groupId>com.firm</groupId>
<artifactId>someid</artifactId>
<version>1.5.1</version>
<versioning>
<latest>1.5.1</latest>
<release>1.5.1</release>
<versions>
<version>1.4.62</version>
</versions>
<lastUpdated>20120926073942</lastUpdated>
</versioning>
</metadata>
The folder structure looks like:
someid
- 1.4.62
- 1.4.62.1
- 1.4.62.2
- 1.4.62.3
If we deploy an new artifact version (1.4.62.1), the maven-metadata.xml contains the 1.4.62.1 version.
But the artifactory overrides the version number (1.4.62.x) to (1.4.62) after an unspecified time. It seems that the artifactory only support major, minor and revision numbers, and deletes the buildnumber.
Now we looking for a solution do disable this behavior.
We use the JFrog Artifactory version 2.5.0 (rev. 13086).

I apologize in advance that this will be an unsatisfactory answer for you.
Your version format is incompatible with the version format used by Maven. Maven uses a .. format for version numbers. Anything after this needs to be prefaced with a dash for it to be properly used by Maven. For more information about how version numbers are used and formatted in Maven, please refer to this section of the Maven book: http://www.sonatype.com/books/mvnref-book/reference/pom-relationships-sect-pom-syntax.html