Develop Reference

How to create a custom ClassLoader for nested JAR files

I am working with a Java library that has some nested JAR files in lib package.
I have 2 issues:
I cannot see referenced types in my IDE (I am using JetBrains IntelliJ)
Of course I get class not defined at runtime
I understand that I have to create and use a custom ClassLoader, will it solve both problems?
Is this the recommended way of achieving this result?
The JAR file is an Italian government provided library and I cannot modify it as it will be periodically updated as the regulation changes.

Yes, as far as I know, the standard ClassLoaders do not support nested JARs. Which is sad, since it would be a really nice idea, but Oracle just doesn't give a damn about it. Here is a 18-year old ticket:
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4735639
If you are getting those JARs from somebody else, the best thing would be to contact the vendor and ask them for a delivery in standards-compatible format. From your answer I realize that this might be difficult to achieve, but I would still try to talk to them, because it's the right thing to do. I'm pretty sure that everybody else in your position has the same issue. According to industry standards, such situation would usually hint your vendor into using Maven repository for their deliverables.
If talking to your vendor fails, you can re-pack the JARs as you get them. I would recommend writing an automated script for that and making sure it gets run on each delivery. You can either put all .class files into one uber-JAR, or just move the nested JARs outside the enclosing JAR. Caveat 1: there can be more than one class with the same name, so you need to make sure to take the correct one. Caveat 2: if the JARs were signed, you will lose the signature (unless you sign them with your own).
Option 3: you can always implement your own ClassLoader to load the classes from anywhere, even from the kitchen sink.
This guy did exactly this: https://www.ibm.com/developerworks/library/j-onejar/index.html
The short summary is that such a ClassLoader has to perform recursive unzipping, which is a bit of a pain-in-the-ass because archives are essentially made for stream access and not for random access, but apart from that it's perfectly doable.
You can use his solution as a "wrapper loader" which will replace your main class.
As far as IntelliJ IDEA goes, I don't believe it supports this functionality out-of-the box. The best thing would be either to re-package JARs as described above and add them as separate classpath entries, or to search if anybody has written a plugin for nested JAR support.

I don't know what you want to do after load jars.
In my case, use jar dynamic loading for Servlet samples.
try{
final URLClassLoader loader = (URLClassLoader)ClassLoader.getSystemClassLoader();
final Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
new File(dir).listFiles(new FileFilter() {
#Override
public boolean accept(File jar) {
// load file if it is 'jar' type
if( jar.toString().toLowerCase().contains(".jar") ){
try {
method.invoke(loader, new Object[]{jar.toURI().toURL()});
XMLog.info_arr(logger, jar, " is loaded.");
JarInputStream jarFile = new JarInputStream(new FileInputStream(jar));
JarEntry jarEntry;
while (true) {
// load jar file
jarEntry = jarFile.getNextJarEntry();
if (jarEntry == null) {
break;
}
// load .class file in loaded jar file
if (jarEntry.getName().endsWith(".class")) {
Class loadedClass = Class.forName(jarEntry.getName().replaceAll("/", "\\.").replace(".class",""));
/*
* In my case, I load jar file for Servlet.
* If you want to use it for other case, then change below codes
*/
WebServlet annotaions = (WebServlet) loadedClass.getAnnotation(WebServlet.class);
// load annotation and mapping if it is Servlet
if (annotaions.urlPatterns().length > 0) {
ServletRegistration.Dynamic registration = servletContextEvent.getServletContext().addServlet(annotaions.urlPatterns().toString(), loadedClass);
registration.addMapping(annotaions.urlPatterns());
}
}
}
} catch (Exception e) {
System.err.println("Can't load classes in jar");
}
}
return false;
}
});
} catch(Exception e) {
throw new RuntimeException(e);
}

Interestingly I just solved a version of this problem for JesterJ, though I had the additional requirement of loading dependencies for the code in the jar file as well. JesterJ (as of this evening's commits!) runs from a fat jar and accepts an argument denoting a second fat jar containing the classes, dependencies and configuration for a document ingestion plan (the user's code that I need to run).
The way my solution works is I borrow the knowledge of how to load jars inside of jars from Uno-Jar (the library that produces the fat jar), and stuff my own classloader in above it to control the evaluation order of the class loaders.
The key bit from https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/Main.java looks like this:
JesterJLoader jesterJLoader;
File jarfile = new File(javaConfig);
URL planConfigJarURL;
try {
planConfigJarURL = jarfile.toURI().toURL();
} catch (MalformedURLException e) {
throw new RuntimeException(e); // boom
}
jesterJLoader = (JesterJLoader) ClassLoader.getSystemClassLoader();
ClassLoader loader;
if (isUnoJar) {
JarClassLoader jarClassLoader = new JarClassLoader(jesterJLoader, planConfigJarURL.toString());
jarClassLoader.load(null);
loader = jarClassLoader;
} else {
loader = new URLClassLoader(new URL[]{planConfigJarURL}, jesterJLoader);
}
jesterJLoader.addExtLoader(loader);
My JesterJLoader is here:
https://github.com/nsoft/jesterj/blob/jdk11/code/ingest/src/main/java/org/jesterj/ingest/utils/JesterJLoader.java
Though if you are happy to simply delegate up and rely on existing classes on the main class path (rather than loading additional dependencies from the sub-fat-jar like I'm doing) yours could be much simpler. I go to a lot of effort to allow it to check the sub-jar first rather than delegating up to the parent immediately, and then have to keep track of what's been sent to the sub-jar to avoid loops and subsequent StackOverflowError...
Also note that the line where I get the system class loader is going to NOT be what you want, I'm also monkeying with the system loader to work around impolite things that some of my dependencies are doing with class loading.
If you decide to try to check out Uno-Jar pls note that resource loading for this nested scenario may yet be wonky and things definitely won't work before https://github.com/nsoft/uno-jar/commit/cf5af42c447c22edb9bbc6bd08293f0c23db86c2
Also: recently committed thinly tested code warning :)
Disclosure: I maintain both JesterJ and Uno-Jar (a fork of One-JAR the library featured in the link supplied by jurez) and welcome any bug reports or comments or even contributions!

Loading java code at runtime

I got a little project where I have to compute a list. The computation depends on serveal factors.
The point is that these factors change from time to time and the user should be allowed to change this by it's self.
Up to now, the factors are hard-coded and no changes can be done without recompiling the code.
At the moment the code looks like this:
if (someStatement.equals("someString")) {
computedList.remove("something");
}
My idea is to make an editable and human readable textfile, configfile, etc. which is loaded at runtime/ at startup? This file should hold the java code from above.
Any ideas how to do that? Please note: The targeted PCs do not have the JDK installed, only an JRE.

An effective way of going about this is using a static initializer. Static Block in Java A good and concise explanation can be found under this link.

One option here that would allow this would be to use User Input Dialogs from the swing API - then you could store the users answer's in variables and export them to a text file/config file, or just use them right in the program without saving them. You would just have the input dialogs pop up at the very beginning of the program before anything else happens, and then the program would run based off those responses.

You could use Javascript for the configuration file language, instead of java. Java 7 SE and later includes a javascript interpreter that you can call from Java. it's not difficult to use, and you can inject java objects into the javascript environment.
Basically, you'd create a Javascript environment, insert the java objects into it which the config file is expected to configure, and then run the config file as javascript.

Okay, here we go... I found an quite simple solution for my problem.
I am using Janino by Codehaus (Link). This library has an integrated Java compiler and seems to work like the JavaCompiler class in Java 7.
BUT without having the JDK to be installed.
Through Janino you can load and compile *.java files(which are human readable) at runtime, which was exactly what I needed.
I think the examples and code-snippets on their homepage are just painful, so here's my own implementation:
Step one is to implement an interface with the same methods your Java file has which is loaded at runtime:
public interface ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList);}
Then you call the Janino classloader when you need the class:
File janinoSourceDir = new File(PATH_TO_JAVAFILE);
File[] srcDir = new File[] { janinoSourceDir };
String encoding = null;
ClassLoader parentClassLoader = this.getClass().getClassLoader();
ClassLoader cl = new JavaSourceClassLoader(parentClassLoader, srcDir,
encoding);
And create an new instance
ZuordnungsInterface myZuordnung = (ZuordnungInterface) cl.loadClass("zuordnung")
.newInstance();
Note: The class which is loaded is named zuordnung.java, so there is no extension needed in the call cl.loadClass("zuordnung").
And finaly the class I want to load and compile at runtime of my program, which can be located wherever you want it to be (PATH_TO_JAVAFILE):
public class zuordnung implements ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList){
ArrayList<String> computedList = (ArrayList<String>) rawList.clone();
if (Model.getSomeString().equals("Some other string")) {
computedList.add("Yeah, I loaded an external Java class");
}
return computedList;
}}
That's it. Hope it helps others with similar problems!

Make JAR as a standalone executable

Is there a way to convert JAR lib into JAR standalone?
I need to find a standalone java executable that convert PDF into TIFF and I've found these JARs: http://www.icefaces.org/JForum/posts/list/17504.page
Any ideas?

Easiest might be to create another Jar with a Main() entry point, and then just use the java.exe executable to run it:
e.g.
> java.exe -cp MyJarMain.jar;MyPDFJar.jar com.mydomain.MyMain myPDF.pdf
Where MyMain is a class with a Main static method.
You'll need something with a main entry point to pass in and interpret some command line arguments (myPDF.pdf in my made-up example)

You could do an assembly (are you using maven?) and make sure the Main-Class entry in the manifest.mf points to the main class.

Since there is no main-Method, you have to write one, or write a whole new class to call the class/method TiffConver.convertPDF .
The question is, how you're going to use it. From the command line, you need no executable jar. From the Gui, maybe you want to pass a file to be converted by drag and drop? Then you should take the parameter(s) passed to main as Input-PDF-Names (if they end in .pdf) and pass the names iteratively to TiffConverter, for "a.pdf b.pdf" =>
TiffConver.convertPDF ("a.pdf", "a.tiff");
TiffConver.convertPDF ("b.pdf", "b.tiff");
TiffCoverter will silently overwrite existing tiffs, so check that before or change the code there - this is clearly bad habit, and look out for more such things - I didn't.
/*
* Remove target file if exists
*/
File f = new File(tif);
if (f.exists()) {
f.delete();
}
Maybe you wan't to write a swing-wrapper, which let's you choose Files interactively to be converted. This would be a nice idee, if no filename is given.
If the user passes "a.pdf xy.tiff" you could rename the converted file to xy, as additional feature.
Without a main-class, however, a standalone jar would be magic.
However, building a native executale is almost always a bad idea. You loose portability, you don't profit from security- and performance improvements to the JVM or fixed bugs. For multiple programs you need always an independend bugfix, which you might have to manage yourself, if you don't have a package-management as most linux distros have.
after clearing some questions:
public static void main (String [] args) {
if (args.length == 1 && args[0].endsWith (".pdf")) {
String target = args[0].replaceAll (".pdf$", ".tif");
convertPDF (args[0], target);
}
}
This method you put into TiffConvert. It will allow you to convert a simple pdf-File, and generate a tif-File with the same basename but ending in .tif, silently overwriting an existing one of the same name.
I guess you now need to know how to start it?

Dumping java.lang.Class's originated from jar files

I am trying to find a way to collected all java.lang.Class's loaded from jar files but ignore the ones from the source code itself.
I have found java.lang.instrument.Instrumentation interface and thought it might serve the purpose, but it turned out not quite.... One of the available functions "getAllLoadedClasses" dump all java.lang.Class's out (which is good), but it not only dump onces got loaded from jar file and also loaded from the source file.
Is there a configuration that allows us to customize the setting so only the java.lang.Class's originated from jar files are dumped or there is better solution in the wild?
What I want to achieve in code representation will be something like below.
java.lang.Class[]
classesLoadedFromJars = getClassesLoadedFromJars();
for (java.lang.Class class : classesLoadedFromJars) {
// ..............
}
A word or two on the suggestion will be helpful!
Thanks in advance.

The class's classloader should be able to give you a clue as to where a certain class was loaded from.
ClassLoader loader = myClass.getClassLoader()
if (loader instanceof URLClassLoader) {
URLClassLoader uLoader = (URLClassLoader)loader;
URL cURL = uLoader.getResource(myClass.getName().replace('.', '/')+".class");
}
if cURL starts with jar:// , the class originated from a jar file

Launching Java Subprocess using parent process Classpath

I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process. The following is not enough, because it doesn't include any dynamically loaded classes:
String classpath = System.getProperty("java.class.path");
Currently I'm searching for each needed class with the code below. However, on some machines this fails for some classes/libs, the source variable is null. Is there a more reliable and simpler way to get the location of libs that are used by the current jvm process?
String stax = ClassFinder.classPath("javax.xml.stream.Location");
public static String classPath(String qualifiedClassName) throws NotFoundException {
try {
Class qc = Class.forName( qualifiedClassName );
CodeSource source = qc.getProtectionDomain().getCodeSource();
if ( source != null ) {
URL location = source.getLocation();
String f = location.getPath();
f = URLDecoder.decode(f, "UTF-8"); // decode URL to avoid spaces being replaced by %20
return f.substring(1);
} else {
throw new ClassFinder().new NotFoundException(qualifiedClassName+" (unknown source, likely rt.jar)");
}
} catch ( Exception e ) {
throw new ClassFinder().new NotFoundException(qualifiedClassName);
}
}

See my previous question which covers getting the classpath as well as how to launch a sub-process.

I want to launch a java subprocess, with the same java classpath and dynamically loaded classes as the current java process.
You mean invoke a new JVM?
Given that...
it is possible to plug in all sorts of agents and instrumentation into a JVM that can transform classes at load time
it is possible to take a byte array and turn it into a class
it is possible to have complex class loader hierarchies with varying visibility between classes and have the same classes loaded multiple times
...there is no general, magic, catch-all and foolproof way to do this. You should design your application and its class loading mechanisms to achieve this goal. If you allow 3rd party plug-ins, you'll have to document how this works and how they have to register their libraries.

If you look at the javadoc for Class.getClassLoader, you'll see that the "bootstrap" classloader is typically represented as the null. "String.class.getClassLoader()" will return null on the normal sun jvm implementations. i think this implementation detail carries over into the CodeSource stuff. As such, I wouldn't imagine you would need to worry about any class which comes from the bootstrap classloader as long as your sub-process uses the same jvm impl as the current process.