Develop Reference

Changed jdeps behavior in OpenJDK 11.0.11 (JDK-8214213)

Summary
Our build pipeline has been broken after some machines have updated from JDK 11.0.10- to JDK 11.0.11+. This happens due to changed jdeps behavior. After some research it became evident, this is likely due to changes introduced with JDK-8214213:
https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2021-April/005860.html
Assuming we were retrieving dependencies for sentry-1.7.25.jar, then our usage of jdeps via CLI is as follows:
jdeps --list-deps -filter:module --multi-release=11 "..\somePath\sentry-1.7.25.jar
The resulting dependency lists look like this:
11.0.10 and below
java.base
java.logging
java.naming
11.0.11 and above
Error: Missing dependencies: classes not found from the module path and classpath.
To suppress this error, use --ignore-missing-deps to continue.
sentry-1.7.25.jar
io.sentry.event.helper.BasicRemoteAddressResolver -> javax.servlet.http.HttpServletRequest not found
io.sentry.event.helper.ForwardedAddressResolver -> javax.servlet.http.HttpServletRequest not found
io.sentry.event.helper.HttpEventBuilderHelper -> javax.servlet.http.HttpServletRequest not found
io.sentry.event.helper.RemoteAddressResolver -> javax.servlet.http.HttpServletRequest not found
io.sentry.event.interfaces.HttpInterface -> javax.servlet.http.Cookie not found
io.sentry.event.interfaces.HttpInterface -> javax.servlet.http.HttpServletRequest not found
io.sentry.servlet.SentryServletContainerInitializer -> javax.servlet.ServletContainerInitializer not found
io.sentry.servlet.SentryServletContainerInitializer -> javax.servlet.ServletContext not found
io.sentry.servlet.SentryServletContainerInitializer -> javax.servlet.ServletException not found
io.sentry.servlet.SentryServletRequestListener -> javax.servlet.ServletRequest not found
io.sentry.servlet.SentryServletRequestListener -> javax.servlet.ServletRequestEvent not found
io.sentry.servlet.SentryServletRequestListener -> javax.servlet.ServletRequestListener not found
io.sentry.servlet.SentryServletRequestListener -> javax.servlet.http.HttpServletRequest not found
In order to fix this on OpenJDK 11.0.11+ it's necessary to set --ignore-missing-deps when calling jdeps. If done, then the output again looks correct again:
java.base
java.logging
java.naming
Question
So I am able to produce the same output with jdeps using JDK 11.0.11+ as I was able to do with JDK 11.0.10-. That being said, this output is used to create a custom runtime and in the description of JDK-8214213 is explicitely stated:
Note that a
custom image is created with the list of modules output by jdeps when
using the --ignore-missing-deps option for a non-modular
application. Such an application, running on the custom image, might
fail at runtime when missing dependence errors are suppressed.
From my understanding this means that if there is a transitive dependency involved, where the dependency of a dependency requires a runtime module that is not required by any of the top level dependencies, then this can lead to a custom runtime uncapable of running the application, since the transitive dependency cannot be resolved. In other words, if my application requires dependency A, which requires dependency B and module C, but dependency B also requires module D, then my application is at risk of encountering runtime errors, since my custom runtime is not being provided with module D.
My question now is this, since I am unable to derive it from documentation:
With JDK 11.0.11+ I can only get the same dependency list output, if --ignore-missing-deps is used. Does that mean that...
...jdeps was able to resolve transitive dependencies prior to 11.0.11, but cannot do so anylonger above said version, e.g. because dependency analysis is done differently internally in jdeps?
...jdeps acted as if it was using --ignore-missing-deps prior to 11.0.11 by default, hence if the default changed, jdeps is now throwing an error on 11.0.11+?
...something else is going on?
The resulting dependency list might be the same, simply because there are a lot of libraries, so most modules are used either way. However I am trying to determine, whether
jdeps --list-deps -filter:module --multi-release=11 "..\somePath\sentry-1.7.25.jar (11.0.10)
and
jdeps --list-deps --ignore-missing-deps -filter:module --multi-release=11 "..\somePath\sentry-1.7.25.jar (11.0.11)
behave exactly the same, or whether using --ignore-missing-deps introduces a new risk when adding new libraries to our project, as they may at some point require a module that is not part of the current jdeps-list.
Bear in mind, to me this is rather a deep-dive into OpenJDK specifics, so if there is faulty terminoogy or problems with my understanding of these scenarios, then feel free to point out and correct them.

Why does Gradle resolve a package with wrong version number?

I am building a web application and I am using Dropwizard 1.3.0, which has a dependency on jetty-io 9.4.8. This dependency has conflicts with another package (dropwizard-websocket-jee7-bundle 2.0.0), because it seem to fetch the wrong version number.
I looked into tha package, and found the method that has been renamed in 9.4.x - AbstractWebSocketConnection.java from 9.3.x - AbstractWebSocketConnection.java. The issue is that even though in Gradle the dependency tree shows I fetched 9.4.8 (the new one which I need), I still get the older, 9.3.x java file which causes the conflicts. I tried to Invalidate Caches / Restart and rebuild the whole project, but I seem to get the outdated file all the time.
What are the possible solutions for this?

If your bad class are imported by a transitive dependency, try to exclude explicit the transitive dependency.
For example if your required library is 'my.group:requiredLibrary:2.0.0' and there are another version in 'my.group:someDependency:0.1.5' you can do like this:
dependencies{
compile 'my.group:requiredLibrary:2.0.0'
compile ('my.group:someDependency:0.1.5'){
exclude group: 'my.group' module:'requiredLibrary'
}
}

Try forcing a particular version in your build.gradle
Example here: https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.ResolutionStrategy.html

Gradle: What is the difference between classpath and compile dependencies?

When adding dependencies to my project I am never sure what prefix I should give them, e.g. "classpath" or "compile".
For example, should my dependencies below be compile time or classpath?
Also, should this be in my applications build.gradle or in the module specific build.gradle?
Current build.gradle (at application level):
apply plugin: 'java'
repositories {
mavenCentral()
}
dependencies {
compile 'org.hibernate:hibernate-core:5.0.5.Final'
compile 'mysql:mysql-connector-java:5.1.38'
}

If buildscript itself needs something to run, use classpath.
If your project needs something to run, use compile.
The buildscript{} block is for the build.gradle itself.
For multi-project building, the top-level build file is for the root project, the specific build file is for sub-project (module).
Top-level build file where you can add configuration options common to all sub-projects/modules.
Do not place your application dependencies in top-level build file, they belong in the individual module build.gradle files

I'm going to guess that you're referencing compile and classpath within the dependencies {} block. If that is so, those are dependency Configurations.
A configuration is simply a named set of dependencies.
The compile configuration is created by the Java plugin. The classpath configuration is commonly seen in the buildScript {} block where one needs to declare dependencies for the build.gradle, itself (for plugins, perhaps).

If I understand correctly, you're confusing Project.dependencies script block with the Project.buildscript.dependencies script block (just like I did when I reached this question).
I'll try to answer this with what I found.
I think you should be already familiar with the Project.dependencies script block. In this block, we declare dependencies that are required by our source code. There are several ways to declare a dependency that we need for the project. See Gradle Tutorial: Dependency Types. I'll only mention the part that is the most relevant to this problem:
compile 'org.hibernate:hibernate-core:5.0.5.Final' is a module dependency declaration. The compile configuration (which is now deprecated by the implementation configuration.) is merely a keyword for Implementation only dependencies. It is not a keyword describing which type of dependency it is (by type here I'm following the three types defined in the tutorial, i.e. module, file, and project.)
In Gradle Tutorial: Organizing Build Logic it says:
If your build script needs to use external libraries, you can add them
to the script’s classpath in the build script itself. You do this
using the buildscript() method, passing in a closure which declares
the build script classpath.
This is the same way you declare, for example, the Java compilation
classpath. You can use any of the dependency types described in
Dependency Types, except project dependencies.
Having declared the build script classpath, you can use the classes in
your build script as you would any other classes on the classpath.
I hope things are getting clear to you now.
With classpath "com.android.tools.build:gradle:${Versions.android_gradle_plugin}" we're setting classpath method with com.android.tools.build:gradle:${Versions.android_gradle_plugin} which is a module dependency that is used by the build script itself rather than the source in your project.
On the other hand, with compile 'org.hibernate:hibernate-core:5.0.5.Final' we're declaring a module dependency required for your project with the compile configuration.
tl;dr: The classpath, compile, and implementation are all keywords that can be used against dependencies under different circumstances. The former is used when you want to pass in a dependency to the build script, and the latter is one of the configuration you may want to declare.

Android:
classpath in project build.gradle —— the implementation after classpath is only used by gradle it self, used in build script. So if i add the implementation (such as retrofit) in the project build.gradle classpath 'retrofit...', i can't get retrofit in my code!! Because —— my code can't see it, only the buildscript can see it.
implementation in app build.gradle —— add the implementation your code can use!!

Can I force the order of dependencies in my classpath with Gradle?

A project runs on Google App Engine. The project has dependency that uses a class that can't be invoked on App Engine due to security constraints (it's not on the whitelist). My (very hacky) solution was to just copy a modified version of that class into my project (matching the original Class's name and package) that doesn't need the restricted class. This works on both dev and live, I assume because my source appears in the classpath before my external dependencies.
To make it a bit cleaner, I decided to put my modified version of that class into it's own project that can be packaged up in a jar and published for anyone else to use should they face this problem.
Here's my build.gradle:
// my jar that has 'fixed' version of Class.
compile files('path/to/my-hack-0.0.1.jar')
// dependency that includes class that won't run on appengine
compile 'org.elasticsearch:elasticsearch:1.4.4'
On my local dev server, this works fine, the code finds my hacked version of the class first at runtime. On live, for some unknown reason, the version in the elasticsearch dependency is loaded first.
I know having two versions of the same class in the classpath isn't ideal but I was hoping I could reliably force my version to be at the start of the classpath. Any ideas? Alternatively, is there a better way to solve this problem?

Not really sure if this is what people visiting this question were looking for, but this was what my problem and a solution that I reached at.
Jar A: contains class XYZ
Jar B: also contains class XYZ
My Project needs Jar B on the classpath before Jar A to be able to get compiled.
Problem is Gradle sorts the dependencies based on alphabetical order post resolving them which meant Jar B will be coming after Jar A in the generated classpath leading to error while compiling.
Solution:
Declare a custom configuration and patch the compileClasspath. This is how the relevant portion of build.gradle might look like.
configurations {
priority
sourceSets.main.compileClasspath = configurations.priority + sourceSets.main.compileClasspath
}
dependencies {
priority 'org.blah:JarB:2.3'
compile 'org.blah:JarA:2.4'
...
}

It's the app engine classloader I should have been investigating, not gradle...
App Engine allows you to customise the class loader JAR ordering with a little bit of xml in your appengine-web.xml. In my case:
<class-loader-config>
<priority-specifier filename="my-hack-0.0.1.jar"/>
</class-loader-config>
This places my-hack-0.0.1.jar as the first JAR file to be searched for classes, barring those in the directory war/WEB-INF/classes/.
...Thanks to a nudge in the right direction from #Danilo Tommasina :)
UPDATE 2020:
I just hit the same problem again and came across my own question... This time, live appengine was loading a different version of org.json than was being loaded in dev. Very frustrating and no amount of fiddling the build script would fix it. For future searchers, if you're getting this:
java.lang.NoSuchMethodError: org.json.JSONObject.keySet()Ljava/util/Set;
It's because it's loading an old org.json dependency from god-knows-where. I fixed it by adding this to my appengine-web.xml:
<class-loader-config>
<priority-specifier filename="json-20180130.jar"/>
</class-loader-config>
You'll also need a matching dependency in build.gradle if you don't already have one:
compile 'org.json:json:20180130'

According to gradle dependencies documentation, the order of dependencies defines the order in the classpath. So, we can simply put the libraries in the correct order in "dependencies".
But beware! here are two rules with higher priorities:
For a dynamic version, a 'higher' static version is preferred over a 'lower' version.
Modules declared by a module descriptor file (Ivy or POM file) are preferred over modules that have an artifact file only.

Missing dependency 'class javax.annotation.Nullable'

I was trying to use the jira-rest-java-client provided by Atlassian in a Scala program I am developing. I am using Eclipse as my IDE.
When I have an object of type Issue and I try to look at the properties I see there are far fewer properties than are declared in the Java code.
I thought perhaps this was just Eclipse not finding all properties/methods of an object so I tried putting Issue.getSummary() and doing an sbt compile. The compile showed me this error:
Missing dependency 'class javax.annotation.Nullable'
Any ideas?

I found the answer in this issue on googlecode: http://code.google.com/p/guava-libraries/issues/detail?id=1095. To correct the problem in sbt you need to add this dependency:
"com.google.code.findbugs" % "jsr305" % "1.3.+"

The Scala compiler requires all annotation classes in the classpath. As this
class is not available in the classpath, the compilation fails.
In my particular case, the class is not used by the application. Hence, it is sufficient to disable the
fatal-warnings option in the build.
In my built.sbt I had the following line:
scalacOptions ++= Seq("-Yno-adapted-args", "-Ywarn-dead-code", "-Ywarn-numeric-widen", "-Ywarn-value-discard", "-Xfatal-warnings")
I removed the "-Xfatal-warnings" and compilation was successful.