Currently I'm trying to modify method bodies residing in classes already loaded by the JVM. I'm aware of the JVM actually not allowing to change the definition of classes that have already been loaded. But my researches brought me to implementations like JRebel or Java Instrumentation API, both using an agent-based approach. I know how to do it right before a class is loaded o behalf of Javassist. But considering e.g. JRebel in an EJB environment where class definitions are loaded on application startup, shouldn't bytecode modification be possible on JVM-loaded classes?
Well, you learned that the Instrumentation API exists and it offers redefinition of classes as an operation. So then it is time to rethink you initial premise of “the JVM actually not allowing to change the definition of classes that have already been loaded”.
You should note that
as the links show, the Instrumentation API is part of the standard API
the support of redefinition of classes is, however, optional. You may ask whether the current JVM supports this feature
it might be limited to not support every class; you may ask whether it’s possible for a particular class
Even if it is supported, the changes may be limited, to cite the documentation:
The redefinition may change method bodies, the constant pool and attributes. The redefinition must not add, remove or rename fields or methods, change the signatures of methods, or change inheritance. These restrictions maybe be lifted in future versions.
at the time you perform the redefinition, there might be threads executing code of methods of these classes; these executions will complete using the old code then
So the Instrumentation is merely useful for debugging and profiling, etc.
But other frameworks, like EJB containers, offering class reloading in production code, usually resort to creating new ClassLoaders which create different versions of the classes which then are entirely independent to the older versions.
In a Java runtime environment, the identity of a class consists of a pair of <ClassLoader, Qualified Name> rather than just a qualified name…
I wasn't aware that you can use the instrumentation API to redefine classes (see #Holger's answer). However, as he notes, there are some significant limitations on that approach. Furthermore, the javadoc says:
"This method is intended for use in instrumentation, as described in the class specification."
Using it to materially change the semantics of a class is ... all sorts of bad from the perspective of the Java type system.
Related
Is there a standard way in Java to mark classes, methods etc. that are used by other parts of the program in indirect ways (think: reflection) which are not discoverable by the usual search-functions of IDEs?
In one particular example I have a bunch of classes with a couple of hundred small validation methods. Validation occurs basically by listing all methods of those classes via reflection and executing one by one them on the given object. (It's more complicated than that, but that's the underlying idea)
Now my IDE understandably marks each and everyone of those methods as "unused" because there are never directly called, only via reflection.
A similar problem occurs in another part of the program where several dozen helper classes reside, some of which are almost certainly unused and could be deleted. But: In some rare cases the fields of these classes are accessed via reflection and the usual search functions of the IDE cannot find these usages (again: very understandably so).
I know that it is impossible for the IDE to solve this problem without outside help. Hence my question whether there are already established ways like annotations for example to clearly mark these cases. Of course I could define such an annotation myself, but I'd rather go with an accepted standard if one exists.
Is there even an IDE that can recognise them and warn me automatically if I'm doing stuff like that?
No, there is not a standard way to mark indirect control flow (reflection, Android intents, callbacks, etc.).
There are some tools that provide their own ways to analyze indirect control flow.
For example, the Checker Framework's reflection resolution uses the #MethodVal annotation to indicate the possible targets of a reflective invocation. It also has ways to indicate Android intents.
You typically annotate those classes with #SuppressWarnings("unused") to get rid of IDE warnings
My initial question was an exact duplicate of this one; that is, why is it that this interface has a runtime retention policy.
But the accepted answer does not satisfy me at all, for two reasons:
the fact that this interface is #Documented has (I believe) nothing to do with it (although why #Documented has a runtime retention policy is a mystery to me as well);
even though many "would be" functional interfaces existed in Java prior to Java 8 (Comparable as the answer mentions, but also Runnable etc), this does not prevent them from being used as "substitutes" (for instance, you can perfecty well use a DirectoryStream.Filter as a substitute to a Predicate if all you do is filter on Path, for instance).
But still, it has this retention. Which means that it has to influence the JVM behavior somehow. How?
I've found the thread in core-libs-dev mailing list which discusses the retention of #FunctionalInterface annotation. The main point mentioned here is to allow third-party tools to use this information for code analysis/validation and to allow non-Java JVM languages to map correctly their lambdas to functional interfaces. Some excerpts:
Joe Darcy (original committer of #FunctionalInterface):
We intentionally made this annotation have runtime retention to
allow it to also be queried to various tools at runtime, etc.
Brian Goetz
There is a benefit for languages other than Java, that can use this as a means to determine whether the interface is suitable for passing to the SAM conversion machinery. The JDK support for lambda conversion is available to other languages as well.
So seems that it's not used by JVM itself, it's just an additional possibility for third-party tools. Making the annotation runtime-visible is not a big cost, so seems there were no strong reasons not to do this.
The only requirement for annotations with retention policy runtime is
Annotations are to be recorded in the class file by the compiler and retained by the VM at run time, so they may be read reflectively. (https://docs.oracle.com/javase/7/docs/api/java/lang/annotation/RetentionPolicy.html#RUNTIME)
Now this has some consequences on runtime behaviour, since the class loader must load these annoations and the VM must keep these annotations in memory for reflective access (for example by third party libraries).
There is however no requirement for the VM to act on such annotations.
I don't want to use the URL Classloader to load classes.
I want to implement this myself.
I don't want to use a solution like JRebel (although it's great).
I've got prior experience of JavaAssist, bytecode generation, implementing javaagent class transformers etc.
I would like to write a javaagent which hooks into the classloader or defines it's own system classloader.
I'll store the class files in an in memory cache, and for particular files, periodically reload them from disk.
I'd prefer to do this in a way which doesn't involve continuously polling the file system and manually invalidating specific classes. I'd much rather intercept class loading events.
I last messed around with this stuff 4 years ago, and I'm sure, although my memory may deceive me that it was possible to do, but 8 hours of searching google doesn't present an obvious solution beyond building a patched JVM.
Is this actually possible?
I've created a stub implementation at https://github.com/packetops/poc_agent if anyone's interested in a simple example of javaagent use.
update
Just found this post - I may have been using the wrong approach, I'll investigate further.
It depends on what you want to do. If you want to reload your classes and define new ones, then you are fine with implementing your own classloader, as you already found.
If you want to replace existing classes, things become more "envolved". You can do this by implementing your own tiny Java agent. See the Java documentation, how to do this: http://docs.oracle.com/javase/7/docs/api/java/lang/instrument/package-summary.html
With the instrumentation mechanism you can not freely redefine classes, quote from Instrumentation.redefineClass:
The redefinition may change method bodies, the constant pool and attributes. The redefinition must not add, remove or rename fields or methods, change the signatures of methods, or change inheritance. These restrictions maybe be lifted in future versions. The class file bytes are not checked, verified and installed until after the transformations have been applied, if the resultant bytes are in error this method will throw an exception.
If you want to do more, you need to load it again. This can be done under the same name, by using a different classloader. The previous class definition will be unloaded, if no one else is using it any more. So, you need to reload any class that uses your previous class also. Utlimatly, you end up reinventing something like OSGi. Take a look at: Unloading classes in java?
I want to redefine the bytecode of the StackOverflowError constructor so I have a "hook" for when a stack overflow occurs. All I want to do is insert a single method call to a static method of my choosing at the start of the constructor. Is it possible to do this?
You should be able to do it using one of two ways (unless something changed in the last 1-2 years, in which case I'd love some links to changelogs/docs):
Mentioned in a comment, not very feasible I guess, modify the classes you are interested in, put them in a jar and then use the -bootclasspath option to load them instead of the default ones. As was mentioned before this can have some legal issues (and is a pain to do in general).
You should be able to (or at least you used to be able to) instrument almost all core classes (iirc Class was the only exception I've seen). One of many problems you might have is the fact that many of core classes are being initialized before the agents you provide (or well their premain methods to be exact) are consulted. To overcome this you will have to add Can-Retransform-Classes property to your agent jar and then re-transform the classes you are interested in. Be aware that re-transformation is a bit less powerful and doesn't give you all the options you'd have normally with instrumentation, you can read more about it in the doc.
I am assuming you know how to do instrumentation?
There are several things to consider.
It is possible to redefine java.lang.StackOverflowError. I tried it successfully on 1.7.0_40. isModifiableClass(java.lang.StackOverflowError.class) return true and I successfully redefined it inserting a method invocation into all of its constructors
You should be aware that when you insert a method call into a class via Instrumentation you still have to obey the visibility imposed by the ClassLoader relationships. Since StackOverflowError is loaded by the bootstrap loader it can only invoke methods of classes loaded by the bootstrap loader. You would have to add the target method’s class(es) to the bootstrap loader
This works if the application’s code throws a StackOverflowError manually. However, when a real stackoverflow occurs, the last thing the JVM will do is to invoke additional methods (keep in mind what the error says, the stack is full). Consequently it creates an instance of StackOverflowError without calling its constructor (a JVM can do that). So your instrumentation is pointless in this situation.
As already pointed out by others, a “Pure Java Application” must not rely on modified JRE classes. It is only valid to use Instrumentation as add-on, i.e. development or JVM management tool. You should keep in mind that the fact that Oracle’s JVM 1.7.0_40 supports the redefinition of StackOverflowError does not imply that other versions or other JVMs do as well.
Why at runtime is anyone interested in knowing that a method is deprecated? Can some provide me with some examples?
There are some frameworks and tools that instantiate objects to work with them.
For example, many JavaBean UI editors create instances of the beans and interact with them as the user manipulates the UI they're designing.
Having the #Deprecated annotation available at runtime allows tools such as this to flag deprecated methods, events, properties for the user.
You're assuming that #deprecated is only of interest in the compile phase (IDE, compiler), but its not a stretch to imaging instrumentation scenarios where you require that information.
For example, an IDE can inform you of the number of call sites for a deprecated method, but how would you go about determining the percentage of time your application spends in deprecated methods?
One's runtime is another one's design time, e.g. when writing code that uses an API.
Good question, and I'm stretching to come up with a convincing scenario. All I've got is that I could imagine a application which used a classloader which didn't allow the use of deprecated code. This would require RetentionPolicy.RUNTIME.
That's all I've got...
Couple practical uses that come to mind:
With Velocity you can have a custom Uberspector which logs the actual calls from Velocity templates to any deprecated method and then just by reading the log you can see where the method is used and you can go and edit it out.
With Wicket you can have a security policy which disallows instantiating any class based on the .class contents so it could be possible to make a system which prevents the instantiation of #Deprecated classes if you're not an admin.
Imagine you compile MyClass.class with deprecated methods. If your #Deprecated annotations got lost, your IDE or compiler couldn't warn you when you call those methods from another class.