Java 8 here.
Say there is an old version of the widget libray, with Maven coordinates widgetmakers:widget:1.0.4, that has a class defined in it like so:
public class Widget {
private String meow;
// constructor, getters, setters, etc.
}
Years pass. The maintainers of this widget library decide that a Widget should never meow, rather, that it should in fact bark. And so a new release is made, with Maven coordinates widgetmakers:widget:2.0.0 and with Widget looking like:
public class Widget {
private Bark bark;
// constructor, getters, setters, etc.
}
So now I go to build my app, myapp. And, wanting to use the latest stable versions of all my dependencies, I declare my dependencies like so (inside of build.gradle):
dependencies {
compile (
,'org.slf4j:slf4j-api:1.7.20'
,'org.slf4j:slf4j-simple:1.7.20'
,'bupo:fizzbuzz:3.7.14'
,'commons-cli:commons-cli:1.2'
,'widgetmakers:widget:2.0.0'
)
}
Now let's say that this (fictional) fizzbuzz library has always depended on a 1.x version of the widget library, where Widget would meow.
So now, I'm specifying 2 versions of widget on my compile classpath:
widgetmakers:widget:1.0.4 which is pulled in by the fizzbuzz library, as a dependency of it; and
widgetmakers:widget:2.0.0 which I am referencing directly
So obviously, depending on which version of Widget gets classloaded first, we will either have a Widget#meow or a Widget#bark.
Does Gradle provide any facilities for helping me out here? Is there any way to pull in multiple versions of the same class, and configure fizzbuzz classes to use the old version of Widget, and my classes to use the new version? If not, the only solutions I can think of are:
I might be able to accomplish some kind of shading- and/or fatjar-based soltuion, where perhaps I pull in all my dependencies as packages under myapp/bin and then give them different version-prefixes. Admittedly I don't see a clear solution here, but am sure something is feasible (yet totally hacky/nasty). Or...
Carefully inspect my entire dependency graph and just make sure that all of my transitive dependencies don't conflict with each other. In this case for me, this means either submitting a pull-request to the fizzbuzz maintainers to upgrade it to the latest widget version, or, sadly, downgrading myapp to use the older widget version.
But Gradle (so far) has been magic for me. So I ask: is there any Gradle magic that can avail me here?
Don't know the specifics of Gradle, as I'm a Maven person, but this is more generic anyway. You basically have two options (and both are hacky):
ClassLoader magic. Somehow, you need to convince your build system to load two versions of the library (good luck with that), then at runtime, load the classes that use the old version with a ClassLoader that has the old version. I have done this, but it's a pain. (Tools like OSGI may take away some of this pain)
Package shading. Repackage the library A that uses the old version of library B, so that B is actually inside A, but with a B-specific package prefix. This is common practice, e.g. Spring ships its own version of asm. On the Maven side, the maven-shade-plugin does this, there probably is a Gradle equivalent. Or you can use ProGuard, the 800 pound gorilla of Jar manipulation.
Gradle will only set up the classpath with your dependencies, it doesn't provide its own runtime to encapsulate dependencies and its transitive dependencies. The version active at runtime will be the one according to the classloading rules, which I believe is the first jar in the classpath order to contain the class. OSGI provides runtime that can deal with situations like this and so will the upcoming module system.
EDIT: Bjorn is right in that it will try to resolve conflicts in different versions; it'll compile the classpath based on its strategies, so the order you put your dependencies in the file doesn't matter. However you still only get one class per classname, it won't resolve OP's issue
If you have different versions of a library with otherwise equal coordinates, Gradles conflict resolution mechanism comes into play.
The default resolution strategy is to use the newest requested version of the library. You will not get multiple versions of the same library in your dependendcy graph.
If you really need different versions of the same library at runtime you would have to either do some ClassLoader magic which definitely is possible or do some shading for one of the libraries or both.
Regarding conflict resolution, Gradle has built-in the newest strategy that is default and a fail strategy that fails if different versions are in the dependency graph and you have to explicitly resolve version conflicts in your build files.
Worse case is when the same class appears in multiple jars. This is more insidious - look at the metrics jars from Codahale and Dropwizard with incompatible versions of the same class in the two jars.
The gradle classpath-hell plugin can detect this horror.
Related
My program relies on the following code to get available system memory:
import oshi.SystemInfo;
import oshi.hardware.HardwareAbstractionLayer;
SystemInfo si = new SystemInfo();
HardwareAbstractionLayer hal = si.getHardware();
// Next line throws exception: NoClassDefFoundError -> com/sun/jna/platform/win32/Psapi
long availableBytes = hal.getMemory().getAvailable();
double availableMegabytes = ((double) availableBytes) / 1048576;
double availableGigabytes = ((double) availableMegabytes)/1024;
Update: After deleting every occurrence of oshi-core from every project in Workspace (to remove possibility of transient conflict dependency - only 4.2.1 is left). Now the error I get is -> java.lang.NoClassDefFoundError: com/sun/jna/platform/win32/VersionHelpers
In pom.xml I've added oshi-core dependency - I've tried almost every version starting from version 3.4.0 to latest version 4.2.1 and they all result in the same error.
I realize oshi-core relies on jna and jna-platform. In Dependency Hierarchy I see both have resolved (compiled) to version 5.5.0.
What is causing this error and how can it be solved?
Thanks!
P.S
I've seen some other threads with similar error but could not find any thread with this exact problem (missing com/sun/jna/platform/win32/Psapi)
While you've pointed out in your comments that you think the latest version of JNA is being resolved, the errors indicate that your project does not have the most recent version of jna-platform (or possibly it has multiple versions linked on the classpath). This is nearly always the case for NoClassDefFoundError and while you're troubleshooting in the right direction, evidence indicates there's an old jna-platform version in your project somewhere.
The com.sun.jna.platform.win32.VersionHelpers class is in jna-platform version 5.3.0 and newer. The GetPerformanceInfo() method required for the method call giving you the error is in the com.sun.jna.platform.win32.Psapi class is in jna-platform version 4.3.0 and newer. If your classloader can't find these classes, then you don't have the correct jars linked to your project -- or you have incorrect jars linked alongside the correct ones.
Maven resolves dependencies by level... first it does all the dependencies you list in your POM (in order), then the transitive dependencies of those projects (in order) and so on. Ensuring the most recent version of JNA is used can be enforced by either (or both) of:
Specify oshi-core dependency earlier in your list of dependencies
in your POM, specifically, before any project that depends on an
earlier version of JNA.
Explicitly specify the jna and
jna-platform versions (5.5.0) in your top-level POM.
Also, in Eclipse, be sure to go through the menus to Update Maven Project to ensure your dependencies are in sync after changes in the POM.
It's possible that your local repository is not downloading the updated jar, in which case you can purge it (or just delete any JNA artifacts, or everything, from C:\Users\<username>\.m2\repository and let it rebuild.)
Also check the classpath in Eclipse. If you have manually added dependencies (e.g., to JNA) before setting up your POM to get them from Maven, you could be using those.
If the above hints do not resolve your problem, please post the contents of the dependencies section your pom.xml file so we can provide additional advice.
Seems oshi-core relies on internal undocumented features of the Sun / Oracle JVM, and you're running on a different and/or newer JVM that doesn't have that undocumented feature anymore. That's the risk of using undocumented features.
Get a newer/other version of oshi-core that supports the version of the JVM you're using, or switch to use a JVM that oshi-core supports.
Is it a good idea to publish a Maven project using this kind of scheme:
<groupId>com.oresoftware</groupId>
<artifactId>async.1</artifactId>
<groupId>com.oresoftware</groupId>
<artifactId>async.2</artifactId>
<groupId>com.oresoftware</groupId>
<artifactId>async.3</artifactId>
where these represent major releases of the projects? Would this not be an effective way to create a different namespace so that different dependencies in the tree could depend on different versions of this library? Does anyone do this or is this a bad practice?
I was even thinking about namespacing them by minor version too:
<groupId>com.oresoftware</groupId>
<artifactId>async.1.1</artifactId>
<groupId>com.oresoftware</groupId>
<artifactId>async.1.2</artifactId>
<groupId>com.oresoftware</groupId>
<artifactId>async.1.3</artifactId>
Update, supposedly this is what Apache Commons did between version 3 and 4, here are two different imports:
version 3:
<dependency>
<groupId>commons-collections</groupId>
<artifactId>commons-collections</artifactId>
<version>3.2.2</version>
</dependency>
version 4:
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>4.0</version>
</dependency>
So my question is - to have a different package name or namespace, is it sufficient to use a different artifactId or do we also need a different groupId?
No, it is only a different namespace if the package name of the classes are also different.
Since they are likely not, you're just setting yourself up for more trouble, since you can now add more than one version of the classes. Given that the classpath order is usually unordered (e.g. in a webapp), it'll be arbitrary which one you'll actually see at compile-time and/or runtime, causing unpredictable behavior.
Now, if you also rename the packages, like what Apache Commons Collections did between version 3.x and 4.x, then you should rename the maven group/artifact too, so you can have both on the classpath.
Otherwise you should instead try to make the code backwards compatible, so a newer version of the library can be used without causing issues.
You can do this, as Andreas talked about in the other answer. But it is rarely done.
I thought about concepts like these myself, and I guess the main reason is that upgrading from version n to n+1 becomes much harder. You need to go through all the code and replace the imports. Furthermore, if you use objects from your library in interfaces, you bind your method parameters to a specific version of the library (through the imports) and make interoperability harder.
Often, upgrading from n to n+1 does not mean that everything breaks, but maybe just a little bit, so going through all the code may be harder than accepting the problems of the usual version scheme.
If you do that, then you must also change the package of your classes; otherwise you will cause even more conflicts to your clients. Looking at the commons-collections jars; the package for version3 is org\apache\commons\collections\.. and in version 4, they changed the package to org\apache\commons\collections4\.. .
So that a client can import both libraries and use both of them at runtime, and still have no conflicts.
So to answer your question, it is not enough to change the groupId/artifactId. Either you bump the version (and leave groupId/artifactId unchanged); so that maven/gradle will select eg. version 1, if version 1 and 2 are imported. But if you change the artifactId, you circumvent the conflict resolution mechanism based on version-s; and a client could import both. Then in order to not have conflicts, you must change the package name (for all classes included in you jar).
Alright so here is the plan
Keep the groupId the same
Change the artifactId from async1 to async2 to async3 for each major version change and the version field would be 1.x.x and then 2.x.x and then 3.x.x
in the codebase the directory structure needs to change from src/main/java/com/oresoftware/async1 to src/main/java/com/oresoftware/async2 to src/main/java/com/oresoftware/async3
and that's it.. AFAICT
I have a Java web application that exposes several REST endpoints. They all work. They can be hit and return the expected data (plain text). I need them to return JSON instead. To do this, the application needs to:
Include the "jersey-media-moxy" jar
Include #XmlRootElement atop any POJO we want converted to JSON
I added the #XmlRootElement to the class. Our project builds with gradle. I added this line to the bottom of the dependencies section of our build.gradle file:
compile group: 'org.glassfish.jersey.media', name: 'jersey-media-moxy', version: '2.26'
It looks almost identical to the line above it:
compile group: 'org.glassfish.jersey.media', name: 'jersey-media-multipart', version: '2.22.2'
Except for the name and version, of course. It builds fine. I had to add the jar (jersey-media-moxy-2.26.jar) to our dependency hierarchy, but no big deal.
It runs fine, until the user logs in. A number of components on the main screen are missing, and it spews errors (the application always spews errors, but they are expected). The new errors are varied, but a repeated one is:
SEVERE: StandardWrapper.Throwable
java.lang.NoClassDefFoundError: jersey/repackaged/com/google/common/base/Predicate
at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177)
at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
...
The stack trace never includes any of our source code as the culprit. It's all org.glassfish.jersey and org.apache.catalina.core errors (we are using Tomcat as our server). Clearly, something is wrong.
I've tried more things to resolve it than I like to recall, but some include:
Adding the jersey-media-moxy in a different location of the 40 or so dependencies we have (no difference)
Explicitly adding all the jersey-media-moxy dependencies (even though it builds fine without them; no difference)
Removing the jersey-media-multipart dependency (build fails)
Scrapping the workspace and doing a fresh "get" from source control, and re-adding the jersey-media-moxy dependency (no difference)
Bringing the jersey-media-multipart up to the latest version (doesn't build)
Removed the jersey-media-moxy dependency to see if I really need it (who knows? Maybe jersey-media-multipart already contains it) and changed the object I'm returning to a very simple class (it just contains a string). (that fails, it really does need jersey-media-moxy)
I am the only Java developer here, and this is the first time I've used gradle or Jersey. There are other ways to produce JSON, I know (such as using gson), but we really want to have Jersey do it, since we're using the framework already.
We use Eclipse (Oxygen) as our IDE and Tomcat as our server.
One big question I have is why would a new jar break an existing one?
The second question is the big one, of course: what can I do to resolve this issue?
Your 2.26 and 2.22.2 discrepancy is probably the problem. Don't mix your Jersey module versions.
why would a new jar break an existing one?
Because it isn't just a single jar that you are adding. When using a dependency management system like with Maven or Gradle, you are dealing with dependencies not just jars, and the ones that you explicitly list can have their own dependencies that implicitly get pulled in. For instance if you were to just add jersey-media-moxy, it would pull in like 10+ extra jars.
The problem with this is that if you use incompatible version, both jersey-media-moxy and jersey-media-multipart pull in some of the same jars, put you can't have more than one version of a class. So only one of the classes from those two jars will be used. And as you know, software changes with version changes. So one class might be trying to use a class that exists in one version and not in another version. This is just one of the problems you can face if you don't manage your dependencies versions correctly.
I have a Codename One project on Netbeans using their plugin.
Is there a way to make it work? I enabled it in project's settings and still doesn't show in final jar.
The annotations are in the libraries of the project. and I can see it being done in the output:
warning: Supported source version 'RELEASE_6' from annotation processor 'org.netbeans.modules.openide.util.ServiceProviderProcessor' less than -source '1.8'
I used instructions here: https://netbeans.org/kb/docs/java/annotations-lombok.html
Update:
I thought it was clear but seems it's not. All this is using Netbean's Lookup. Let's say I have one jar as project dependency with one interface in it, let's say ITest. Also a class implementing the interface, for example:
#ServiceProvider(service=ITest.class)
public class Test implements ITest{
..
}
So in the Codenamone Project I call it like this:
Lookup.getDefault().lookupAll(ITest.class);
But it come up empty. I know the system works as it does in other projects, just porting it to Codename one. Seems like it is not seeing the annotations in the dependencies.
I don't know if that will work and I'm pretty curious about it myself. Make sure you created a Java 8 version of the project and you are running on top of Java 8 to get started.
In the past things like this were done using bytecode manipulation e.g. see this code from the work done by Steve.
A project runs on Google App Engine. The project has dependency that uses a class that can't be invoked on App Engine due to security constraints (it's not on the whitelist). My (very hacky) solution was to just copy a modified version of that class into my project (matching the original Class's name and package) that doesn't need the restricted class. This works on both dev and live, I assume because my source appears in the classpath before my external dependencies.
To make it a bit cleaner, I decided to put my modified version of that class into it's own project that can be packaged up in a jar and published for anyone else to use should they face this problem.
Here's my build.gradle:
// my jar that has 'fixed' version of Class.
compile files('path/to/my-hack-0.0.1.jar')
// dependency that includes class that won't run on appengine
compile 'org.elasticsearch:elasticsearch:1.4.4'
On my local dev server, this works fine, the code finds my hacked version of the class first at runtime. On live, for some unknown reason, the version in the elasticsearch dependency is loaded first.
I know having two versions of the same class in the classpath isn't ideal but I was hoping I could reliably force my version to be at the start of the classpath. Any ideas? Alternatively, is there a better way to solve this problem?
Not really sure if this is what people visiting this question were looking for, but this was what my problem and a solution that I reached at.
Jar A: contains class XYZ
Jar B: also contains class XYZ
My Project needs Jar B on the classpath before Jar A to be able to get compiled.
Problem is Gradle sorts the dependencies based on alphabetical order post resolving them which meant Jar B will be coming after Jar A in the generated classpath leading to error while compiling.
Solution:
Declare a custom configuration and patch the compileClasspath. This is how the relevant portion of build.gradle might look like.
configurations {
priority
sourceSets.main.compileClasspath = configurations.priority + sourceSets.main.compileClasspath
}
dependencies {
priority 'org.blah:JarB:2.3'
compile 'org.blah:JarA:2.4'
...
}
It's the app engine classloader I should have been investigating, not gradle...
App Engine allows you to customise the class loader JAR ordering with a little bit of xml in your appengine-web.xml. In my case:
<class-loader-config>
<priority-specifier filename="my-hack-0.0.1.jar"/>
</class-loader-config>
This places my-hack-0.0.1.jar as the first JAR file to be searched for classes, barring those in the directory war/WEB-INF/classes/.
...Thanks to a nudge in the right direction from #Danilo Tommasina :)
UPDATE 2020:
I just hit the same problem again and came across my own question... This time, live appengine was loading a different version of org.json than was being loaded in dev. Very frustrating and no amount of fiddling the build script would fix it. For future searchers, if you're getting this:
java.lang.NoSuchMethodError: org.json.JSONObject.keySet()Ljava/util/Set;
It's because it's loading an old org.json dependency from god-knows-where. I fixed it by adding this to my appengine-web.xml:
<class-loader-config>
<priority-specifier filename="json-20180130.jar"/>
</class-loader-config>
You'll also need a matching dependency in build.gradle if you don't already have one:
compile 'org.json:json:20180130'
According to gradle dependencies documentation, the order of dependencies defines the order in the classpath. So, we can simply put the libraries in the correct order in "dependencies".
But beware! here are two rules with higher priorities:
For a dynamic version, a 'higher' static version is preferred over a 'lower' version.
Modules declared by a module descriptor file (Ivy or POM file) are preferred over modules that have an artifact file only.