I need a Java source code parsing library for Java to programmatically extract method definitions and annotations.
Specifically, given code like :
#WebMethod(operationName = "MyOperation")
public String myOperation(String param1,int param2) { ....
}
I have the following requirements:
1. Extract the name, return type and names and types of the method parameters
2. Extract the annotations associated with the method
3. Finally, create a new source file by removing the annotations
I am currently using JaxMeJS http://ws.apache.org/jaxme/js/jparser.html which satisfies 1. but not 2. or 3.
Could you recommended a parsing library that can fulfill all 3 requirements ?
JavaCC comes with a 1.5 parser.
Annotation processing is part of the Annotation processing tool in Java which runs as part of the Java compiler. I'm not sure how you can remove the annotations from the source code. This is not supported by APT. (It might be easier to remove the annotations from the compiler byte code than from the source code.)
I've implemented a source code generator based on annotations using APT in the Quickcheck project.
Another root could be to use ASM (or any other byte code manipulation tool) to read the annotations.
I think APT, as suggested by Thomas Jung is what you need. Still, if you want to look into other options, do check Java 6 grammar for antlr
How about using JDT parser?
JDT means Java Development Tools.
Eclipse use this.
I also did my project with it.
It is really nice. And there is much info about it.
Related
I am generating Java source files for my project using a source generation tool (antlr). However, I am writing most, if not all of my code, in Kotlin.
Kotlin already offers great Java interop, so using the generated sources is not a problem. However, because of how Kotlin brings Java's nullable types into a null-safe system, I lose most of the null-safety that I use Kotlin for. At the very best I have warnings of platform types (make type explicit to avoid subtle bugs); at the worst I have unexpected crashes and subtle bugs.
Kotlin does, however, respect nullability annotations, such as JSR-305, FindBugs, Lombok, Eclipse, and JetBrains's respective forms of #Nullable/#NonNull, bringing those in as the appropriate non-null type or optional.
Because the code is generated and I have access to the source (and understand how it works), I know which functions can/not return null, and want to annotate them as such so they include neatly into my null-safe code. However, I cannot add annotations directly into the code, as it is generated during the build step and would overwrite any manual changes.
Is it possible to / what is the best way to annotate the nullability of generated java sources for the purpose of use in null-safe code?
The best way would be to modify the source code generator so that it includes the annotations that you require.
If you can't modify the generator (e.g. because the generator is proprietary and you don't have its source code) then you can do this using bytecode engineering. For example, this page on the ASM site gives one way to do it:
http://asm.ow2.org/doc/tutorial-annotations.html
Of course, in either case you need some way to tell the tool (the generator, the bytecode rewriter, whatever) which methods should be annotated.
I would like to manipulate Java classes (with java extension not .class) so that I could :
Delete all methods of a class (keeping the constructor)
Add unimplemented methods
Remove unused imports
...
Is there an API that could accomplish this ?
What I've done so far is trying to manipulate the .java files like text files (with regex,FileUtils, etc.).
Regards.
I
You could look at using the AST (Abstract Syntax Tree) tools from the Eclipse JDT project.
There is a tutorial to get you started at Vogella: Eclipse JDT - Abstract Syntax Tree (AST) and the Java Model - Tutorial
If you only want to temporarily modify the classes (i.e. within the scope of the jvm) then you could do this with reflection:
What is reflection and why is it useful?
If you're taking about permanently altering/creating source code then this is maybe best done using an IDE. Most IDE will tell you about unimplemented methods and provide auto completion to create them. They will also format the source code, remove unused imports etc.
You can use a regular expression, the question then is then what regular expression (And what other options are there!)
Regular expressions maybe aren't ideally suited to this, and for example, when it comes to another task they're not ideally suited to, such as parsing XML, people say don't do it, use an XML parser, but in this case, if you find that there is an absence of a tool built for parsing java source code, then regular expressions may be the best option.
Yes, you can use java reflection api. Please check here
Later edit: To update the class structure you can use javassist. Here you have an example.
I'm working with an API that generate a lot of java code to me.
But this API does not handle import correctly, so it write full qualified name of every class. eg:
public class Foo{
com.my.company.Bar bar;
public com.my.company.Bar getBar(){
return bar;
}
}
I would like to find an API to post process this generated code and write something like that:
import com.my.company.Bar;
public class Foo{
Bar bar;
public Bar getBar(){
return bar;
}
}
Is there any known API able to do that?
Filtering imports from existing code isn't trivial; imagine you have two classes with the same name but different package.
My usual approach is to have a helper class which manages the imports for me. In the generator, I can
String type = importSet.add(Foo.class);
type is then used in the method to access the type. The import set collects all imports and handles duplicates.
For this to work, you need this "main loop":
importSet = new ImportSet();
String body = generateClass();
out.write(importSet);
out.write(body);
i.e. you need to generate all the code for the class itself (collecting the imports as you go) first. Then you write the imports to the file and after that the generated class body.
If you want to change the sources, I suggest to use the Eclipse Java compiler because it can give you the AST of the code. You can then apply various transformations on this tree. I have an example in my blog how to get the AST.
As far as I see you expect to generate Java code right? If yes . We are using Eclipse JDT in our project which is from Eclipse IDE and they use it for Java code generation. And I encourage to use it, however depending on need you might go for simple solution like QDox or even other solution.
You can consider either one of the solution
Eclipse JDT
Javaparser
Qdox
Eclipse JDT
Pros
Impressive functionality
Very rich API
Support for Java 7 features and they have a plan to also support Java 8 features
Localizable syntax error messages
Con's
Steep learning curve
Resources
Intro to Eclipse JDT
http://www.eclipse.org/articles/article.php?file=Article-JavaCodeManipulation_AST/index.html
How to use JDT API outside eclipse
How can I use the java Eclipse Abstract Syntax Tree in a project outside Eclipse? (ie not an eclipse plugin)
Nice example and project on AST
http://www.ibm.com/developerworks/opensource/library/os-ast/
Access Eclipse jar plugin sources
http://www.vogella.de/articles/EclipseCodeAccess/article.html
Test samples
http://git.eclipse.org/c/jdt/eclipse.jdt.core.git/tree/org.eclipse.jdt.core.tests.model/src/org/eclipse/jdt/core/tests
Qdox
Pro's
build from scratch and modify
lightweight
FQN type based searching
clean and readable API
extensive querying possibility like isFinal(), isPrivate()
object oriented constructs, almost everything in a java file can be represented in terms of objects like, JavaClass, JavaField, JavaMethod, JavaParmeter
Con's
last public release was 1.10 on 2009-09-04, how ever 2.0 snapshot is available
1.10 is not supporting static import;
writing to a file is pre-formatted, don't have much control over it, can't specify the line numbers
no article or tutorial, source code is the only reference
Resources
- Qdox http://qdox.codehaus.org/changes-report.html
I am looking for ways to remove all the annotations from existing Java Source Code. I am looking for an ant task or any other approach. I have seen some solutions that do this at the class level, but I am looking to do this at the source code to source code level.
I have done this through Java Parser code available in Lombok.
Look at these methods which has the logic
lombok.javac.handlers.JavacHandlerUtil#deleteAnnotationIfNecessary
lombok.javac.handlers.JavacHandlerUtil#deleteImportFromCompilationUnit
I ended up using JEdit which has brilliant regular expression support.
I wanted to replace specific annotations (I wanted to keep stuff like #Override). You can easily do that for all buffers or a directory tree.
Just write some simple expressions for the annotations you want to remove. For example
^\s*#NamedQueries\(\n\{[^\}]+\}\)\n
To automate certain manual tasks in an legacy project, I need to modify existing java files from within java or groovy code.
I don´t want to use RegEx, because it would be neither quick nor clean in my opinion.
I found javassist and srcgen4javassist. The first one lets me modify my sources as I wish, but only writes bytecode, loosing all comments and annotations. And with the second one I didnt manage to read an existing Class not created with srcgen4javassist itself.
Is there an elegant solution, or do i need to bite the bullet and use Regex?
you could really parse the code using something like eclipse's ASTParser at which point you coudl locate your replacement targets xpath-style, but its a lot of work.
you could also consider marking replacement areas with annotation and writing an annotation processor to generate/alter sources at runtime, but (at least in my opinion) the API is cumbersome.
you can combine regexp with some marker in the source code, something like
//START REPLACEMENT-TARGET
...code to be edited/replaced
//END REPLACEMENT TARGET
which would make your regexp targeting a lot safer.