For Java source files, I would like to find out:
Which classes use which other classes (fully qualified names)?
Which methods call which other methods (fully qualified names)?
What would be a reasonable way to achieve that?
EDIT:
To clarify: I want a list of source code files as input. The output should be (as specified above) which class uses which other class and which method calls which other method. I do not want to inspect other loaded classes at runtime, like when using reflection.
You need to use static analysis tool as STAN standalone mode:
The standalone application is targeted to architects and project managers who are typically not using the IDE.
Or JArchitect (available also using command line)
JArchitect is a powerful tool for static code analysis. It can provide a lot of insight into complex code bases. Using custom code queries you are able to build your own rule sets in a very comfortable way.
In the Class Browser right-click menu, JArchitect proposes to explore the graph of dependencies between members (methods + fields) of a type.
Another option is SourceTrail
The graph visualization provides a quick overview of any class, method, field, etc., of interest and all its relations. The graph is fully interactive. Use it to move through the codebase by focusing on related nodes and edges.
(source: sourcetrail.com)
Unfortunately, reflection doesn't give you all the information you need to do this.
I've done it with ASM (https://asm.ow2.io/).
It provides the ability to walk the byte code of all of your classes using the visitor pattern, including the actual method implementations, from which you can extract the references to other classes.
I'm sorry that I cannot provide the implementation, because it's proprietary.
Note that this works from your .jar files, not your sources. If you really need to work from sources, then have a look at https://github.com/javaparser . Really, though, it's better to use the byte code, since the java language changes frequently, while the byte code specification does not.
I am not sure how to get a listing, but for identifying refactoring opportunities, you might try IntelliJ IDEA. It will dull out the signature line of any methods that are not accessed in the project. It will also detect code segments that are repeated elsewhere in the project, so you can extract common code.
Related
I'm looking for different ways to prevent internals leaking into an API. This is a huge problem because once these internals leak into the API; you can run either into unexpected incompatibility issues or into frozen internals.
One of the simplest ways to do so is just make use of different Maven modules; one module with API and one module with implementation. This way it is impossible to expose the implementation from the API.
Unfortunately not everyone agrees this is the best approach; But are there other alternatives? E.g using checkstyle or other 'architecture checking' tools?
PS: Java 9 for us is not usable, since we are about to upgrade to Java 8 and this will be the lowest supporting version for quite some time to come.
Following your checkstyle idea, it should be possible to set up rules which examine import statements in source files.
Checkstyle has built-in support for that, specifically the IllegalImport and ImportControl rules.
This of course works best if public and internal classes can be easily separated by package names.
The idea for IllegalImport would be that you configure a TreeWalker in checkstyle which only looks at your API-sources, and which excludes imports from internal packages.
With the ImportControl rule on the other hand you can define very detailed access rules for the whole application/module in a separate XML file.
It is standard in Java to define an API using interfaces and implement them using classes. That way you can change the "internals" however you want and nothing changes for the user(s) of the API.
One alternative is to have one module (Jar file) for API and implementation (but then again, is it an API or just any kind of library?). Inside one separates classes and interfaces by using packages, e.g. com.acme.stuff.api and com.acme.stuff.impl. It is important to make classes inside the latter package protected or just package-protected.
Not only does the package name show the consuming developer "hey, this is the implementation", it is also not possible to use anything inside (let's omit reflections at this point for the sake of simplicity).
But again: This is against the idea of an API, because usually the implementation can be changed. With this approach one cannot separate API from implementation, because both are inside the same module.
If it is only about hiding internals of a library, then this is one (not the one) feasible approach.
And just in case you meant a library instead of an API, which only exposes its "frontend" (by using interfaces or abstract classes and such), use different package names, e.g. com.acme.stuff and com.acme.stuff.internal. The same visibility rules apply of course.
Also: This way one does not need Checkstyle and other burdens.
Here is a good start : http://wiki.netbeans.org/API_Design
Key point : Do not expose more than you want Obviously the less of the implementation is expressed in the API, the more flexibility one can have in future. There are some tricks that one can use to hide the implementation, but still deliver the desired functionality
I think you don't need any checkstyle or anything like that, just a good old solid design and architecture should be enough. Polymorphism is all you need here.
One of the simplest ways to do so is just make use of different Maven
modules; one module with API and one module with implementation. This
way it is impossible to expose the implementation from the API.
Yes, I totally agree, hide as much as possible, separate your interface in a standalone project.
I need to diagnose all invoked methods in a class(either declared in the class or not) using it's source code. Means that give the class source code to a method as an input and get the invoked method by the class as the output. In fact I need a class/method which operates same as java lexical analyzer .
Is there any method to diagnose all invoked methods ?
of course I tried to use Runtime.traceMethodCalls(); to solve the problem but there was no output. I've read I need to run java debug with java -g but unfortunately when I try to run java -g it makes error. Now what should I do ? Is there any approach ?
1) In the general case, no. Reflection will always allow the code to make method calls that you won't be able to analyze without actually running the code.
2) Tracing the method calls won't give you the full picture either, since a method is not in any way guaranteed (or even likely) to make all the calls it can every time you call it.
Your best bet is some kind of "best effort" code analysis. You may want to try enlisting the compiler's help with that. For example, compile the code and analyze the generated class file for all emitted external symbols. It won't guarantee catching every call (see #1), but it will get you close in most cases.
You can utilize one of the open source static analyzers for Java as a starting point. Checkstyle allows you to build your own modules. Soot has a pretty flexible API and a good example of call analysis. FindBugs might also allow you too write a custom module. AFAIK all three are embeddable in the form of a JAR, so you can incorporate whatever you come up with into your own custom program.
From your question it is hard to determine what is exactly problem you're trying to solve.
But in case:
If you want to analyze source code, to see which parts of it are redundant and may be removed, then you could use some IDE (Eclipse, IntelliJ IDEA Community Edition etc.) In IDE's you have features to search for usages of method and also you have functionality to analyze code and highlight unused methods as warnings/errors.
If you want to see where during runtime some method is called, then you could use profiling tool to collect information on those method invocations. Depending on tool you could see also from where those methods were called. But bare in mind, that when you execute program, then it is not guaranteed that your interesting method is called from every possible place.
if you are developing an automated tool for displaying calling graphs of methods. Then you need to parse source and start working with code entities. One way would be to implement your own compiler and go on from there. But easier way would be to reuse opensourced parser/compiler/analyzer and build your tool around it.
I've used IntelliJ IDEA CE that has such functionalitys and may be downloaded with source http://www.jetbrains.org/display/IJOS/Home
Also there is well known product Eclipse that has its sources available.
Both of these products have enormous code base, so isolating interesting part would be difficult. But it would still be easier than writing your own java compiler and werifying that it works for every corner case.
For analyzing the bytecode as mentioned above you could take a look at JBoss Bytecode. It is more for testing but may also be helpful for analyzing code.
sven.malvik.de
You may plug into the compiler.
Have a look the source of Project Lombok for instance.
There is no general mechanism, so they have one mechanism for javac and one for eclipse's compiler.
http://projectlombok.org/
We have huge codebase and some classes are often used via reflection all over the code. We can safely remove classes and compiler is happy, but some of them are used dynamically using reflection so I can't locate them otherwise than searching strings ...
Is there some reflection explorer for Java code?
No simple tool to do this. However you can use code coverage instead. What this does is give you a report of all the line of code executed. This can be even more useful in either improving test code or removing dead code.
Reflections is by definition very dynamic and you have to run the right code to see what it would do. i.e. you have to have reasonable tests. You can add logging to everything Reflection does if you can access this code, or perhaps you can use instrumentation of these libraries (or change them directly)
I suggest, using appropriately licensed source for your JRE, modifying the reflection classes to log when classes are used by reflection (use a map/WeakHashMap to ignore duplicates). Your modified system classes can replace those in rt.jar with -Xbootclasspath/p: on the command line (on Oracle "Sun" JRE, others will presumably have something similar). Run your program and tests and see what comes up.
(Possibly you might have to hack around issues with class loading order in the system classes.)
I doubt any such utility is readily available, but I could be wrong.
This is quite complex, considering that dynamically loaded classes (via reflection) can themselves load other classes dynamically and that the names of loaded classes may come from variables or some runtime input.
Your codebase probably does neither of these. If this a one time effort searching strings might be a good option. Or you look for calls to reflection methods.
As the other posters have mentioned, this cannot be done with static analysis due to the dynamic nature of Reflection. If you are using Eclipse, you might find this coverage tool to be useful, and it's very easy to work with. It's called EclEmma
I have what must surely be a fairly common documentation need...
I'm implementing a rather sizable Java library code base that has, among other things, various classes intended to be exposed to a caller/implementor at the appropriate level of abstraction. At the same time, the code base contains, of course, various internal classes, interfaces, and other abstractions that the user of the library doesn't need to know about in order to use the API.
Lots of other API libraries out there make the mistake of simply throwing everything into the Javadocs, and leaving it up to the user to figure out which objects and entities they actually need to deal with as a caller through some combination of guesswork, inference, and, if you're lucky, example code.
I don't want to be in that same position. I would like to have an "internal" set of Javadocs that expose the entire extent of the codebase, and an "external" set of Javadocs intended to clearly communicate to the developers the characteristics of the classes that they actually need to use to get their work done. I don't need or want to muddy the waters with various internal abstractions that they don't need to see or know about - there's no need for them to know how it all works under the hood, and it would just confuse and misdirect them, making for a very inefficient API learning process.
How can I accomplish this? Is there a well-known combination of arguments to 'javadoc' and perhaps some annotations that can make this happen?
Thanks very much for your consideration!
Assuming that you have followed best-practice and put your internal classes in different packages to your public APIs, you can run javadoc with the public API package names as command line arguments.
Refer to the javadoc command line synopsis for more details.
(If you haven't organized your packages to keep internal classes out of API packages, you may be in for a bit of pain ...)
In addition to Stephen C's answer and using the javadoc tool, you can specify exactly which packages appear in the javadoc (hence Stephen C's comment about 'pain' if they aren't organised logically) using something like this:
Say you have 5 classes and you want only the classes in the org.notprivate package to appear in the Javadoc:
org.notprivate.Foo
org.notprivate.Bar
org.notprivate.Stuff
org.notpublic.Things
org.notpublic.More
You can use something like:
javadoc -d target/api -source 1.6 -sourcepath src/main/java org.notprivate
That's just a quick example, if you need to specify each class you'll need to look at the link Stephen C provided in more detail
Posted here for clarity:
Javadoc Documentation
I would like to have ... an "external" set of Javadocs intended to clearly communicate to the developers the characteristics of the classes that they actually need to use to get their work done. I don't need or want to muddy the waters with various internal abstractions that they don't need to see or know about - there's no need for them to know how it all works under the hood, and it would just confuse and misdirect them, making for a very inefficient API learning process.
Given this desire, perhaps Javadoc isn't the best method of documenting the overall system view or for giving a "here's what you need to know"-type info to new developers?
I would recommend supplementing your Javadoc files with a separate guide/document/wiki/something to give the meta-view.
You can use some extra arguments when invoking the javadoc tool :
-public : Shows only public classes and members.
-protected : Shows only protected and public classes and members. This is the default.
-package : Shows only package, protected, and public classes and members.
-private : Shows all classes and members.
So, with these options you can generate a full documentation for internal usage, and give a 'light' documentation with only the public interface to your customers.
If you're using Eclipse, the Javadoc wizard shows radio buttons to help you choose the documentation level - which is "public fields only" by default.
Is there a tool to deobfuscate java obfuscated codes?
The codes is extracted from a compiled class but they are obfuscated and non-readable.
First step would be to learn with which tool it was obfuscated. Maybe there's already a "deobfuscator" around for the particular obfuscator.
On the other hand, you can also just run an IDE and use its refactoring powers. Rename the class, method and variable names to something sensitive. Use your human logical thinking powers to figure what the code actually represents and name them sensitively. And the picture would slowly but surely grow.
Good luck.
Did you try to make the code less obscure with Java Deobfuscator (aka JDO), a kind of smart decompiler?
Currently JDO does the following:
renames obfuscated methods, variables, constants and class names
to be unique and more indicative of
their type
propogates changes throughout the entire source tree (beta)
has an easy to use GUI
allow you to specify the name for a field, method and class (new feature!)
Currently JDO does not do the
following (but it might one day)
modify method bytecode in any way
Not to gravedig but I wrote a tool that works on most commercial obfuscators
https://github.com/Contra/JMD
I used Java Deobfuscator (aka JDO) but it has a few bugs. It can't work with case sensitive file names.
So I've changed the source and uploaded a patch for that in sourceforge.
The patch, Download
Most likely only human mindpower to make sense of it. Get the best decompiler available and ponder on its output.
Maybe it will work on Unix/Linux/MacOS?
If so, you could move one step of your process to a VM, in where you unpack the code, before you rename the too long names. How long is the file name limit on Windows?