We have huge codebase and some classes are often used via reflection all over the code. We can safely remove classes and compiler is happy, but some of them are used dynamically using reflection so I can't locate them otherwise than searching strings ...
Is there some reflection explorer for Java code?
No simple tool to do this. However you can use code coverage instead. What this does is give you a report of all the line of code executed. This can be even more useful in either improving test code or removing dead code.
Reflections is by definition very dynamic and you have to run the right code to see what it would do. i.e. you have to have reasonable tests. You can add logging to everything Reflection does if you can access this code, or perhaps you can use instrumentation of these libraries (or change them directly)
I suggest, using appropriately licensed source for your JRE, modifying the reflection classes to log when classes are used by reflection (use a map/WeakHashMap to ignore duplicates). Your modified system classes can replace those in rt.jar with -Xbootclasspath/p: on the command line (on Oracle "Sun" JRE, others will presumably have something similar). Run your program and tests and see what comes up.
(Possibly you might have to hack around issues with class loading order in the system classes.)
I doubt any such utility is readily available, but I could be wrong.
This is quite complex, considering that dynamically loaded classes (via reflection) can themselves load other classes dynamically and that the names of loaded classes may come from variables or some runtime input.
Your codebase probably does neither of these. If this a one time effort searching strings might be a good option. Or you look for calls to reflection methods.
As the other posters have mentioned, this cannot be done with static analysis due to the dynamic nature of Reflection. If you are using Eclipse, you might find this coverage tool to be useful, and it's very easy to work with. It's called EclEmma
Related
For Java source files, I would like to find out:
Which classes use which other classes (fully qualified names)?
Which methods call which other methods (fully qualified names)?
What would be a reasonable way to achieve that?
EDIT:
To clarify: I want a list of source code files as input. The output should be (as specified above) which class uses which other class and which method calls which other method. I do not want to inspect other loaded classes at runtime, like when using reflection.
You need to use static analysis tool as STAN standalone mode:
The standalone application is targeted to architects and project managers who are typically not using the IDE.
Or JArchitect (available also using command line)
JArchitect is a powerful tool for static code analysis. It can provide a lot of insight into complex code bases. Using custom code queries you are able to build your own rule sets in a very comfortable way.
In the Class Browser right-click menu, JArchitect proposes to explore the graph of dependencies between members (methods + fields) of a type.
Another option is SourceTrail
The graph visualization provides a quick overview of any class, method, field, etc., of interest and all its relations. The graph is fully interactive. Use it to move through the codebase by focusing on related nodes and edges.
(source: sourcetrail.com)
Unfortunately, reflection doesn't give you all the information you need to do this.
I've done it with ASM (https://asm.ow2.io/).
It provides the ability to walk the byte code of all of your classes using the visitor pattern, including the actual method implementations, from which you can extract the references to other classes.
I'm sorry that I cannot provide the implementation, because it's proprietary.
Note that this works from your .jar files, not your sources. If you really need to work from sources, then have a look at https://github.com/javaparser . Really, though, it's better to use the byte code, since the java language changes frequently, while the byte code specification does not.
I am not sure how to get a listing, but for identifying refactoring opportunities, you might try IntelliJ IDEA. It will dull out the signature line of any methods that are not accessed in the project. It will also detect code segments that are repeated elsewhere in the project, so you can extract common code.
I am writing a plugin of another large java program .
I want to modify some byte code of some java method of the java program during runtime, so that I can intercept the method calls (namely, inject some hooking code into the method).
Any way can achieve this?
PS:
I've checked the following approaches:
1.change the classloader of the java program. (we CANNOT change it)
2.use java proxy. (We CANNOT use java proxy, because java proxy would create a new proxy object. We DON'T use the proxy object. We need to hook the java program's object, and Use that object)
3. use -javaagent option ( we CANNOT add the commandline option for the java program.)
PS more [Edited again]:
My classes was loaded by ext class loader (I put my jar files in JAVA_HOME\lib\ext folder).
The large java program is an applet program loaded by Browser. When the browser start the applet, it also loads my classes.
PS more more [Edited again]:
Although it's running in Applet. I can have full permission. Because I can modify java.policy and java.security file.
Thanks,
Calvin
Just use -javaagent opiton, which is used to modify the bytecode at runtime. You can find more about -javaagent from This Link or from This Link
There are several libraries which you can use. See for example here.
Once a class was already loaded/initialized by the VM it will be impossible to manipulate, though.
By the way, in principle you can also just replace the class to be 'hooked' with your own proxy class file. As long as the class' visible interface does not change this may work. (Sub-classes of the class may horribly fail at runtime though.) This replacement can be as easy as changing the classpath so that your class of the same name will be found first, before the original one. Delegating to the original class of the same name may be a little more complex in this case.
Yes, you can, but the process would be a bit tricky, as you would operate directly with memory. For this purpose, you'd look at unofficial documentation on sun.misc package and its Unsafe class.
Warning 1: the Unsafe class would be removed in JDK 9 according to official sources.
Warning 2: the Sun company would not take responsibility for your code to work correctly, as this class should not be used at all, and exists for system usage only.
Sorry, but this is not possible. First off, bytecode is immutable after classloading. The JVM provides several APIs that can be used to do something like this, but they are obviously highly privileged.
If you're running in a low privilege environment like a browser Applet, then you're obviously not going to be allowed to do this, and any method you could should be considered a security vulnerability.
But the question is why you are using applets in the first place, and why you want to modify code after loading. There's almost certainly a better way to do what you're trying to do.
I'm working on a sandbox feature for my java antivirus, and I've come into a question: Does the specified package on a class matter for compilation?
Example:
I'm running a program that wants to use Runtime.getRuntime().exec(), when the classloader attempts to load that to run a method, does it check the package qualified in the file, if they exist? I would prefer not to try and change files in the JVM, but to simply load ones from a different package. I can accomplish the loading and such, but my only dilemma, will it crash and burn? Inside the java, it would be registered as say, java.lang.Runtime, but the compiled code will say for example pkg.pkg.Runtime and will it need to extend the old runtime? My guess is that extending the old runtime would just break it. Does anyone know anything about this? I'm working on making a testable example, but I'm still a bit away and wanted to get some answers, as well as this might benefit some people.
Does the specified package on a class matter for compilation?
Yes it does matter. A class called pkg.pkg.Runtime() cannot be loaded as if it was java.lang.Runtime.
Furthermore, if my memory is correct, the JVM has some additional security measures in it to prevent normal applications from injecting classes into core packages such as java.lang.
If you need to change the behaviour of the java.lang.Runtime class (for experimental purposes!) then I think you will need to put your modified version on the boot classpath, ahead of the "rt.jar" file.
However:
This level of tinkering can easily result in JVM instability; i.e. hard JVM crashes that are difficult to diagnose.
If your aim is to produce a "production quality" tool, then you will find that things that involve tinkering with the JVM are not considered acceptable. People are going to be very suspicious of installation instructions that say things like "add this to your installed JVM's bootclasspath".
Distributing a "tinkered with" JVM may fall foul of Oracle's Java licensing agreement.
My advice would be to look for a less intrusive way of doing what you are trying to do. For instance, if you are trying to do virus checking, either do it outside of the JVM, or in a custom application classloader.
You commented:
I have a custom classloader, my question is: If I compile a class that is labelled as say, pkg.pkg.Runtime, can I register in my classloader as java.lang.Runtime?
As I said above, no you can't. A bytecode file has the classname embedded in it. If you attempt to "pull a swifty" by loading a class with a different name, the JVM will throw an Error.
And:
If not, then how can I replace the class? If the compiled package name has to equal the request referenced naming, then can I modify the .class file to to match, or perhaps compile it as if it were in the java.lang package?
That's what you would have to do. You need to name the class java.lang.Runtime in the source code and compile it as such.
But what I meant by my advice above is that you should use do the virus checking in the class loader. Forget about trying to replace / modify the behaviour of Runtime. It is a bad idea for the reasons I listed above.
I need to diagnose all invoked methods in a class(either declared in the class or not) using it's source code. Means that give the class source code to a method as an input and get the invoked method by the class as the output. In fact I need a class/method which operates same as java lexical analyzer .
Is there any method to diagnose all invoked methods ?
of course I tried to use Runtime.traceMethodCalls(); to solve the problem but there was no output. I've read I need to run java debug with java -g but unfortunately when I try to run java -g it makes error. Now what should I do ? Is there any approach ?
1) In the general case, no. Reflection will always allow the code to make method calls that you won't be able to analyze without actually running the code.
2) Tracing the method calls won't give you the full picture either, since a method is not in any way guaranteed (or even likely) to make all the calls it can every time you call it.
Your best bet is some kind of "best effort" code analysis. You may want to try enlisting the compiler's help with that. For example, compile the code and analyze the generated class file for all emitted external symbols. It won't guarantee catching every call (see #1), but it will get you close in most cases.
You can utilize one of the open source static analyzers for Java as a starting point. Checkstyle allows you to build your own modules. Soot has a pretty flexible API and a good example of call analysis. FindBugs might also allow you too write a custom module. AFAIK all three are embeddable in the form of a JAR, so you can incorporate whatever you come up with into your own custom program.
From your question it is hard to determine what is exactly problem you're trying to solve.
But in case:
If you want to analyze source code, to see which parts of it are redundant and may be removed, then you could use some IDE (Eclipse, IntelliJ IDEA Community Edition etc.) In IDE's you have features to search for usages of method and also you have functionality to analyze code and highlight unused methods as warnings/errors.
If you want to see where during runtime some method is called, then you could use profiling tool to collect information on those method invocations. Depending on tool you could see also from where those methods were called. But bare in mind, that when you execute program, then it is not guaranteed that your interesting method is called from every possible place.
if you are developing an automated tool for displaying calling graphs of methods. Then you need to parse source and start working with code entities. One way would be to implement your own compiler and go on from there. But easier way would be to reuse opensourced parser/compiler/analyzer and build your tool around it.
I've used IntelliJ IDEA CE that has such functionalitys and may be downloaded with source http://www.jetbrains.org/display/IJOS/Home
Also there is well known product Eclipse that has its sources available.
Both of these products have enormous code base, so isolating interesting part would be difficult. But it would still be easier than writing your own java compiler and werifying that it works for every corner case.
For analyzing the bytecode as mentioned above you could take a look at JBoss Bytecode. It is more for testing but may also be helpful for analyzing code.
sven.malvik.de
You may plug into the compiler.
Have a look the source of Project Lombok for instance.
There is no general mechanism, so they have one mechanism for javac and one for eclipse's compiler.
http://projectlombok.org/
It may not be best practice but are there ways of removing unsused classes from a third party's jar files. Something that looks at the way in which my classes are using the library and does some kind of coverage analysis, then spits out another jar with all of the untouched classes removed.
Obviously there are issues with this. Specifically, the usage scenario I put it though may not use all classes all the time.
But neglecting these problems, can it be done in principle?
There is a way.
The JarJar project does this AFAIR. The first goal of the JarJar project is to allow one to embed third party libraries in your own jar, changing the package structure if necessary. Doing so it can strip out the classes that are not needed.
Check it out at http://code.google.com/p/jarjar/.
Here is a link about shrinking jars: http://sixlegs.com/blog/java/jarjar-keep.html
There is a tool in Ant called a classfileset. You specify the list of root classes that you know you need, and then the classfileset recursively analyzes their code to find all dependencies.
Alternatively, you could develop a good test suite that exercises all of the functions that you need, then run your tests under a test coverage tool. The tool will tell you which classes (and statement in them) were actually utilized. This could give you an even smaller set of code than what you'd find with static analysis.
I use ProGuard for this. As well as being an excellent obfuscator, it has a code shrinking phase which can combine multiple JARs and then strip out any unused classes or class members. It does an excellent job at shrinking.
At a previous job, I used a Java obfuscator that as well as obfuscating the code, also removed classes and methods that weren't being used. If you were doing "Class.byName" or any other type of reflection stuff, you needed to tell the obfuscator because it couldn't tell by inspecting the code what classes or methods called by reflection.
The problem, of course, is that you don't know if other parts of the third party library are doing any reflection, and so removing an "unused" class might cause things to break in an obscure case that you haven't tested.
jar is just a zip file, so I guess you can. If you could get to the source, it's cleaner. Maybe try disassembling the class?
Adding to this question, can that improve performance? Since the classes not used would not be JIT compiled improving startup time or does the java automatically detect that while compiling to bytecode and do not even deal with the code that is not used?
This would be an interesting project (has anyone done it already?)
I presume you'd give the tool your jar(s) as a starting point, and the library jar to clean up. It could use reflection to determine which classes your jar(s) reference directly, and which are used indirectly down the call tree (this is not trivial at all, but doable). If it encounters any reflection code in any of the two places, it should give a very loud warning.