Lets say there exists a library x.y.z-1.0, x.y.z-2.0, x.y.z-3.0 that uses a.b.c-1.0, a.b.c-2.0, a.b.c-3.0 respectively.
I am currently writing another library (com.mysdk) that uses x.y.z-2.0. But I have not check the compatability of my library with x.y.z-1.0 and x.y.z-3.0. Thus I am going to shade x.y.z-2.0 to com.mylibrary.shaded.x.y.z-2.0. However, after investigation, a.b.c-2.0 will not be shaded.
So the situation looks like the picture underneath.
My question comes in when the user of com.mysdk is also using a.b.c-1.0 at the same time. I could not make the project to use the specified a.b.c-1.0, instead it is using the a.b.c-2.0 in the sdk.
Related
We have a Java program that relies on a specific library. We have created a second library that has a very similar API to the first library, however, this one is made in-house and we are ready to begin testing it.
To test, we would like to replace the jar in the Java program with the jar of our new library. The issue is that the new library does not have the exact same namespace, so the import statements will not align. For example,
Java program
import someLibrary.x.y.Foo;
public class Main {
public static void main(String[] args){
new Foo().bar();
}
}
New Library has the same API but different namespace
anotherLibrary.x.y.Foo;
Question: How can I use the classloader or another tool to run a Java program but replace a dependency and redirect import statements to another namespace?
[EDIT] - We do not have access to the Java program's source code. We can have this program changed to use our new library but we do not want to do that until after it has been thoroughly tested.
The only solution I can think of would involve writing a custom ClassLoader that would alter the bytecode to change the method references and field references to change the class name.
How about the straightforward solution:
Create a branch of your main program (in git or whatever source control tool you use):
Apply all the changes required to work with the new library (change all the imports)
Deploy on test environment and test extensively
Merge back to master when you feel confident enough
Another solution could be:
Create a branch out of new library
Change the imports so that it will look exactly as the old one (with all the packages)
Substitute the old library with a new one in your application
Deploy on test environment and test extensively
When you're ready with the new library deploy to production and keep working in production for a grace period of month or something (until you really feel confident)
In a month change back all the imports (basically move from branch with the "old" imports to the branch with your real imports in both library and application.
Update
Its also possible to relocate packages of your version of the library automatically if you use maven.
Maven shade plugin has relocate goal that can be used to "relocate" the packages of your library to be just like packages of existing library. See shade plugin's documentation
I have a Java web application that exposes several REST endpoints. They all work. They can be hit and return the expected data (plain text). I need them to return JSON instead. To do this, the application needs to:
Include the "jersey-media-moxy" jar
Include #XmlRootElement atop any POJO we want converted to JSON
I added the #XmlRootElement to the class. Our project builds with gradle. I added this line to the bottom of the dependencies section of our build.gradle file:
compile group: 'org.glassfish.jersey.media', name: 'jersey-media-moxy', version: '2.26'
It looks almost identical to the line above it:
compile group: 'org.glassfish.jersey.media', name: 'jersey-media-multipart', version: '2.22.2'
Except for the name and version, of course. It builds fine. I had to add the jar (jersey-media-moxy-2.26.jar) to our dependency hierarchy, but no big deal.
It runs fine, until the user logs in. A number of components on the main screen are missing, and it spews errors (the application always spews errors, but they are expected). The new errors are varied, but a repeated one is:
SEVERE: StandardWrapper.Throwable
java.lang.NoClassDefFoundError: jersey/repackaged/com/google/common/base/Predicate
at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:177)
at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:369)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
...
The stack trace never includes any of our source code as the culprit. It's all org.glassfish.jersey and org.apache.catalina.core errors (we are using Tomcat as our server). Clearly, something is wrong.
I've tried more things to resolve it than I like to recall, but some include:
Adding the jersey-media-moxy in a different location of the 40 or so dependencies we have (no difference)
Explicitly adding all the jersey-media-moxy dependencies (even though it builds fine without them; no difference)
Removing the jersey-media-multipart dependency (build fails)
Scrapping the workspace and doing a fresh "get" from source control, and re-adding the jersey-media-moxy dependency (no difference)
Bringing the jersey-media-multipart up to the latest version (doesn't build)
Removed the jersey-media-moxy dependency to see if I really need it (who knows? Maybe jersey-media-multipart already contains it) and changed the object I'm returning to a very simple class (it just contains a string). (that fails, it really does need jersey-media-moxy)
I am the only Java developer here, and this is the first time I've used gradle or Jersey. There are other ways to produce JSON, I know (such as using gson), but we really want to have Jersey do it, since we're using the framework already.
We use Eclipse (Oxygen) as our IDE and Tomcat as our server.
One big question I have is why would a new jar break an existing one?
The second question is the big one, of course: what can I do to resolve this issue?
Your 2.26 and 2.22.2 discrepancy is probably the problem. Don't mix your Jersey module versions.
why would a new jar break an existing one?
Because it isn't just a single jar that you are adding. When using a dependency management system like with Maven or Gradle, you are dealing with dependencies not just jars, and the ones that you explicitly list can have their own dependencies that implicitly get pulled in. For instance if you were to just add jersey-media-moxy, it would pull in like 10+ extra jars.
The problem with this is that if you use incompatible version, both jersey-media-moxy and jersey-media-multipart pull in some of the same jars, put you can't have more than one version of a class. So only one of the classes from those two jars will be used. And as you know, software changes with version changes. So one class might be trying to use a class that exists in one version and not in another version. This is just one of the problems you can face if you don't manage your dependencies versions correctly.
I have a Codename One project on Netbeans using their plugin.
Is there a way to make it work? I enabled it in project's settings and still doesn't show in final jar.
The annotations are in the libraries of the project. and I can see it being done in the output:
warning: Supported source version 'RELEASE_6' from annotation processor 'org.netbeans.modules.openide.util.ServiceProviderProcessor' less than -source '1.8'
I used instructions here: https://netbeans.org/kb/docs/java/annotations-lombok.html
Update:
I thought it was clear but seems it's not. All this is using Netbean's Lookup. Let's say I have one jar as project dependency with one interface in it, let's say ITest. Also a class implementing the interface, for example:
#ServiceProvider(service=ITest.class)
public class Test implements ITest{
..
}
So in the Codenamone Project I call it like this:
Lookup.getDefault().lookupAll(ITest.class);
But it come up empty. I know the system works as it does in other projects, just porting it to Codename one. Seems like it is not seeing the annotations in the dependencies.
I don't know if that will work and I'm pretty curious about it myself. Make sure you created a Java 8 version of the project and you are running on top of Java 8 to get started.
In the past things like this were done using bytecode manipulation e.g. see this code from the work done by Steve.
Java 8 here.
Say there is an old version of the widget libray, with Maven coordinates widgetmakers:widget:1.0.4, that has a class defined in it like so:
public class Widget {
private String meow;
// constructor, getters, setters, etc.
}
Years pass. The maintainers of this widget library decide that a Widget should never meow, rather, that it should in fact bark. And so a new release is made, with Maven coordinates widgetmakers:widget:2.0.0 and with Widget looking like:
public class Widget {
private Bark bark;
// constructor, getters, setters, etc.
}
So now I go to build my app, myapp. And, wanting to use the latest stable versions of all my dependencies, I declare my dependencies like so (inside of build.gradle):
dependencies {
compile (
,'org.slf4j:slf4j-api:1.7.20'
,'org.slf4j:slf4j-simple:1.7.20'
,'bupo:fizzbuzz:3.7.14'
,'commons-cli:commons-cli:1.2'
,'widgetmakers:widget:2.0.0'
)
}
Now let's say that this (fictional) fizzbuzz library has always depended on a 1.x version of the widget library, where Widget would meow.
So now, I'm specifying 2 versions of widget on my compile classpath:
widgetmakers:widget:1.0.4 which is pulled in by the fizzbuzz library, as a dependency of it; and
widgetmakers:widget:2.0.0 which I am referencing directly
So obviously, depending on which version of Widget gets classloaded first, we will either have a Widget#meow or a Widget#bark.
Does Gradle provide any facilities for helping me out here? Is there any way to pull in multiple versions of the same class, and configure fizzbuzz classes to use the old version of Widget, and my classes to use the new version? If not, the only solutions I can think of are:
I might be able to accomplish some kind of shading- and/or fatjar-based soltuion, where perhaps I pull in all my dependencies as packages under myapp/bin and then give them different version-prefixes. Admittedly I don't see a clear solution here, but am sure something is feasible (yet totally hacky/nasty). Or...
Carefully inspect my entire dependency graph and just make sure that all of my transitive dependencies don't conflict with each other. In this case for me, this means either submitting a pull-request to the fizzbuzz maintainers to upgrade it to the latest widget version, or, sadly, downgrading myapp to use the older widget version.
But Gradle (so far) has been magic for me. So I ask: is there any Gradle magic that can avail me here?
Don't know the specifics of Gradle, as I'm a Maven person, but this is more generic anyway. You basically have two options (and both are hacky):
ClassLoader magic. Somehow, you need to convince your build system to load two versions of the library (good luck with that), then at runtime, load the classes that use the old version with a ClassLoader that has the old version. I have done this, but it's a pain. (Tools like OSGI may take away some of this pain)
Package shading. Repackage the library A that uses the old version of library B, so that B is actually inside A, but with a B-specific package prefix. This is common practice, e.g. Spring ships its own version of asm. On the Maven side, the maven-shade-plugin does this, there probably is a Gradle equivalent. Or you can use ProGuard, the 800 pound gorilla of Jar manipulation.
Gradle will only set up the classpath with your dependencies, it doesn't provide its own runtime to encapsulate dependencies and its transitive dependencies. The version active at runtime will be the one according to the classloading rules, which I believe is the first jar in the classpath order to contain the class. OSGI provides runtime that can deal with situations like this and so will the upcoming module system.
EDIT: Bjorn is right in that it will try to resolve conflicts in different versions; it'll compile the classpath based on its strategies, so the order you put your dependencies in the file doesn't matter. However you still only get one class per classname, it won't resolve OP's issue
If you have different versions of a library with otherwise equal coordinates, Gradles conflict resolution mechanism comes into play.
The default resolution strategy is to use the newest requested version of the library. You will not get multiple versions of the same library in your dependendcy graph.
If you really need different versions of the same library at runtime you would have to either do some ClassLoader magic which definitely is possible or do some shading for one of the libraries or both.
Regarding conflict resolution, Gradle has built-in the newest strategy that is default and a fail strategy that fails if different versions are in the dependency graph and you have to explicitly resolve version conflicts in your build files.
Worse case is when the same class appears in multiple jars. This is more insidious - look at the metrics jars from Codahale and Dropwizard with incompatible versions of the same class in the two jars.
The gradle classpath-hell plugin can detect this horror.
I would like to be able to determine what versions I am running of a dependency at runtime as well as the version of the web application itself.
Each web application I deploy is packaged with a pom.xml which I can read from, that part is trivial. The next part is parsing the pom without much effort.
As the web application is running, I want to be able to understand what version I am, and what versions my dependencies are.
Ideally, I would like to do something like:
MavenPom pom = new MavenPom(webApplicationPomInputStream);
pom.getVersion();
pom.getArtifactId();
pom.getGroupId();
for(Dependency dependency:pom.getDependencies())
{
dependency.getVersion();
dependency.getArtifactId();
dependency.getGroupId();
}
Should I just use XPath notation here, or is there a library I can call to do this type of thing?
After these posts, I am thinking the quickest/most reliable way is to generate a text file with the dependency tree in it: mvn dependency:tree. Then I will parse the text file, separate the groupId, artifactId, and version, and then determine the structure by the indentation level.
If I do that, can I export to XML instead of text? I can then use JAXB and easily parse that file without doing any/much work.
It is a hack, but looks promising.
Walter
I will just use the mvn dependency:tree plugin to generate a text file with the dependency tree. Then I will parse that in and create the dependency tree/graph from that. I will get the scope of the artifact, groupId, artifactId, version, and its parent.
I successfully implemented this type of lookup, it simply takes the dependency output, parses it and organizes dependencies simply using the indentation, nothing fancy. The artifact, group, version, and scope are easily parsed since the separator is a :.
Walter
Maven has of course such an API. Have a look at org.apache.maven.project.MavenProject. But, to be honest, I don't think it will be that easy to create a MavenProject instance. The source code will be helpful here, check for example MavenProjectTest or maybe the Maven Plugin API (actually, this task would be much, really much, simpler to achieve from a Mojo) for some guidance.
I'd suggest to search for or ask this question on the Maven Mailing Lists, org.apache.maven.dev would be appropriate here IMHO.