We have a java based (jersey+grizzly) REST Server which exposes calls like
foo.com/{game}/rules
foo.com/{game}/players
foo.com/{game}/matches
There can be arbitrary number of games
Each game has different implementations for rules, players, matches
For historical reasons, we would want separate jars for each game implementation
So there is REST Server
as and when there is a call like foo.com/tennis/rules
we want the REST Server to dynamically load 'tennis' jar. The jar does its operation. Then jar should be unloaded
If the next call was for foo.com/football/players
we want the REST Server to dynamically load 'football' jar. The jar does its operation. Then jar should be unloaded
Is there a technique to do this ?
Apparently there is a very old question around [this]: java: is there a framework that allows dynamically loading and unloading of jars (but not osgi)?
I don't know how it works on Java 8, but unloading a class in Java 7 requires unloading not only the class, but its loader, along with all references from other objects that this class might have.
Once all of them were unloaded the System.gc will be called. If other classes are still holding references then the gc won't do its job.
OSGI (as suggested by #Joop Eggen) is a viable option. JRebel, is not.
proxy-object proxy-object library
Load java jar files dynamically in isolated class loader to avoid dependency conflicts and enable modular updates. All jar files in the [main jar folder]/lib/myLib/2.0/*.jar will be loaded.
Code Examples
Create Object from a JAR file located in the myLib/2.0 folder:
File libDir = new File("myLib/2.0");
ProxyCallerInterface caller = ObjectBuilder.builder()
.setClassName("net.proxy.lib.test.LibClass")
.setArtifact(DirArtifact.builder()
.withClazz(ObjectBuilderTest.class)
.withVersionInfo(newVersionInfo(libDir))
.build())
.build();
String version = caller.call("getLibVersion").asString();
Related
I am trying to access a java package loaded into memory and dump it to a file. Here is how the security works: there is an exe packed with Themida that contains the java main class code to be loaded. At runtime the Themida exe loads the clean main class java code into memory. The software is structured with the loader being contained within the exe, but several external libraries can access the packages contained within the exe. So, exe contains com.mysoft.mainloader. But the clean jar library Mylib.jar can call functions within com.mysoft.mainloader. How to I dump com.mysoft.mainloader to a jar file? Can I modify Mylib.jar to dump it as it has access to the package once it is loaded as well?
There is no supported Java SE mechanism to read / retrieve a ".class" that has been loaded by a classloader. So your options would be:
Modify the custom classloader you are using to capture the ".class" before (or after) the classloader calls defineClass.
Burrow into the JVM data structures to try and figure out whether the entire ".class" stream is captured somewhere and then retrieve it.
Modify the JVM ...
Any of these could be feasible. All will be relatively difficult.
It is possible to get loaded classes in runtime using Dynamic Attach and Instrumentation API.
The idea is to inject a Java Agent into the running application.
The agent gets an array of all loaded classes with Instrumentation.getAllLoadedClasses method, then gets their bytecode using Instrumentation.retransformClasses.
The working implementation can be found in the class-file-extractor project.
Usage:
java -jar extractor.jar <pid> mainloader.jar com.mysoft.mainloader
where
<pid> is the process ID of the target JVM application;
mainloader.jar is the output file name;
com.mysoft.mainloader is the name prefix of the classes to extract.
I read about CDS in Oracle doc https://docs.oracle.com/javase/8/docs/technotes/guides/vm/class-data-sharing.html
What I understood is the system class files needed for loading the jvm are parsed, verified and then stored in a archive at jre/lib/[arch]/client/classes.jsa. Moreover they also provide their memory mapping for jvm,so jvm directly maps the memory according to the mapping information given in the archive. So this reduces the overhead of class loading everytime a jvm instance starts. Please correct me if was wrong.
Now coming to java 10, how can I achieve this for my application code ?
Secondly, would the complete application code be eligible for CDS or are there some restrictions?
There are three essential steps to creating and using an archive with application class-data (for more details, read my post about application class-data sharing):
Creating a list of classes to include in the archive:
java -XX:+UseAppCDS
-XX:DumpLoadedClassList=classes.lst
-jar app.jar
Creating an archive:
java -XX:+UseAppCDS -Xshare:dump
-XX:SharedClassListFile=classes.lst
-XX:SharedArchiveFile=app-cds.jsa
--class-path app.jar
Using the archive:
java -XX:+UseAppCDS -Xshare:on
-XX:SharedArchiveFile=app-cds.jsa
-jar app.jar
Keep the following in mind:
you can’t use wildcards or exploded JARs for the class path when creating the archive
the class path used to launch the application must have the one used to create the archive as a prefix
if you have any problems, use -Xlog:class+load (more on -Xlog) to get more information
The JEP for AppCDS has the example show casing how to add your application classes to shared archive.
As for the restrictions, there are few:
Straight classes (.class) present in directory on class path
cannot be added to the shared archive. See this thread.
Classes loaded by custom
class loaders cannot be added to the shared archive. See this thread.
There are other practical considerations to be aware of when using CDS/AppCDS, such as:
If you update the jar files on the file system, then you will have to recreate the shared archive.
If you are using Java or JVMTI agent(s) that modify/re-transform/redefine the class file at run-time, then the shared archive won't be useful as the classes will be loaded from the disk since the agents need actual classfile data which I believe is not stored in the shared archive.
Another nice and detailed article on CDS and AppCDS is https://simonis.github.io/cl4cds/.
Author of the article has also written a tool that allows sharing of application classes even if they get loaded by a custom class loaders.
If you are interested in using CDS, you can also try OpenJ9 JVM which has this feature for a long time and is much more mature and complete. Read more about it here.
This question goes a little bit further than my previous question:
Obtaining a list of all Java classes used from all JVM's?
Now I need to know the physical location from where those classes are loaded from. I have checked out the jcmd help for other commands but it wasn't useful for me. I also can't find it in jvisualvm, but the information is also not there. Anyone can help me with this?
EDIT:
This is my situation: My company has got different individual java projects (jars) for which we can control whether it starts or stops. We can control this in our own custom build webinterface. Each of such a process gets a process ID (PID) when started, and then runs on the background.
My need: I need a list of all loaded classes by each running PID java process. I already have jcmd <pid> GC.class_histogram, but this only contains a list of which classes are loaded. I also want the information where the classes are actually loaded from (which jar, location on file system).
The classes are loaded from java.lang.ClassLoader's loadClass(String name) method which in turn calls the findClass(String name) method.Usually,the custom ClassLoader overrides the findClass method to retrieve the definition of classes using a specific protocol and location.It may be that classes are loaded from database or from network location,the location of which may be dynamically generated.So you can never know location of all Java classes. The best example is AppletClassLoader which loads classes from network stream or from a remote location.
Bootstrap ClassLoader - core library package such as rt.jar present in JRE lib folder
Extension ClassLoader - jar files present in ext folder or as specified in the environment variable for ext
System ClassLoader - Application's classpath or as specified in environment variable for classpath or through JVM's startup argument parameter for -cp or -classpath
CustomClassLader -accordingly to the classLoader's class loading policy (mostly defined in findClass() method)
I have a a library that is bundled as an executable jar file and added to weblogic / tomcat classpath, how can I execute a main method from the jar file when the server is starting and loading the classes from the jar file.
what I want to is to have some initialization code to be executed first thing when the jar file is loaded and server is starting without any user intervention.
Note: I know I can bundle my jar in a war file, but I have some aspectj code in my library that I want to weave all running applications in the jvm, when I bundle my jar in war file, the aspectj code will only weave into the classes in the war file so I added my library jar file in the classpath.
Thanks in advance.
Add a class inside your JAR with the following code:
public class TomcatStartupListener implements org.apache.catalina.LifecycleListener {
public void lifecycleEvent(org.apache.catalina.LifecycleEvent event) {
if (event.getType().equals("after_start")) {
// call your main method here
}
}
}
Note: In order to compile this, you need to add <tomcat-dir>/lib/catalina.jar to your classpath. Otherwise when compiling it won't be able to find the necessary interfaces (org.apache.catalina.LifecycleListener and org.apache.catalina.LifecycleEvent). Once you're done with the compiling, put the JAR as usual under <tomcat-dir>/lib.
Now open <tomcat-dir>/conf/server.xml and add the following under the <Server> section:
<Listener className="com.yourpackage.TomcatStartupListener" />
Now whenever your Tomcat server starts, this TomcatStartupListener class inside your JAR will be called, and you can invoke your main method. There are a whole lot of other event types too! You can use any of these event types:
before_init
after_init
before_start
configure_start
start
after_start
before_stop
stop
configure_stop
after_stop
before_destroy
after_destroy
This approach is necessary because of the way the classloaders work in Tomcat (or even most JVMs). Here are the important points from that link:
There are three aspects of a class loader behavior
Lazy Loading
Class Caching
Separate Namespaces
The JVM will get very heavy if all classes inside all JARs get loaded indiscriminately. So the classes inside shared JARs are loaded only on-demand. The only way for you to invoke the main method is to add the above lifecycle listener.
Perhaps the simplest thing to do is to deploy a trivial servlet in a .war file that references your .jar file. The servlet can be configured to start up upon deployment/container start, and then it can invoke the class containing your main() method.
As application servers / servlet containers typically have a lot of different classloaders, you'll most likely need a different strategy for weaving aspects into your code than in standalone applications.
I would recommend to add the aspects to every war file deployed at build time. This might be following a common technique - as opposed to a server specific one.
Further, I'm not sure it can actually be done (properly & supported) on a server. Typically a server is built to separate all webapps from each other. You might get it to work, but it might break on the next update of the server.
It might be easier to suggest an alternative technique if you'd state the problem that you want to solve with your proposed approach.
Edit after your comment: Consider the standard web application lifecycle: You can execute some code, e.g. in a servlet, upon it being deployed. If you insist on your code being contained in main, you can call this method from your webapp's initialization code.
You need to register a Java Agent. See this link: java.lang.instrument.
java.lang.instrument provides services that allow Java programming language agents to instrument programs running on the JVM.
This is the right way to do this.
I imagine my application like this:
I have an application for encrypting files. And I want to extend ciphers in my application(jar) with another cipher later.
So, I will have a folder with ciphers (jar or java files) and my application will read the files from this folder. Then, in GUI, there will be a list with files(jar or java), which have a method encrypt and decrypt (I would test it with reflection). And user will choose one.
Could someone give me an advice? How make it, that it could be extended? Or how my application(.jar) could work with another .jar, .java file(read them and run them)?
Another way to do this would be the java.util.ServiceLoader mechanism. (Read the documentation.)
Your additional jar would contain a file META-INF/services/my.package.Cipher listing all implementations of your Cipher interface (or abstract class), and then you can say
ClassLoader cipherLoader =
new URLClassLoader(new URL[]{new URL("jar:file:myAdditional.jar")},
Cipher.class.getClassLoader());
ServiceLoader<Cipher> serviceLoader = ServiceLoader.load(Cipher.class, cipherLoader);
for(Cipher c : serviceLoader) {
c.encrypt(...);
}
Using ServiceLoader mandates that your implementations have public no-argument constructors - if they don't have, use instead a ServiceLoader for some factory interface.
The URL should be either a jar: URL to the jar file in which the classes are, or a non-jar URL which points to the root directory where the .class files are in (in their package structure).
I would check out the Java Extension Mechanism.
This document describes the mechanism
provided by the Java™ platform for
handling optional packages. An
optional package is a group of
packages housed in one or more JAR
files that implement an API that
extends the Java platform. Optional
package classes extend the platform in
the sense that the virtual machine can
find and load them without their being
on the class path, much as if they
were classes in the platform's core
API.