Develop Reference

Proper way for getting annotations of Java-Methods in custom Sonar rules

I try to write a custom Sonar-rule (Java) and struggle with the proper way for getting annotation of invoked Java methods.
This rule should detect illegal access to methods and fields marked as #VisibleForTesting. Accesing such elements from production code is illegal, but access from the same class is legal.
My first approach is to implement BaseTreeVisitor.visitMethodInvocation(MethodInvocationTree) and use the metadata of the belonging symbol:
get annotations: methodInvocationSymbol.metadata().annotations()
analyse the names of the symbols of each AnnotationInstance (annotations class) and its owner (annotations package)
This works great as long other classes invoke the method. When the call is made inside the same class - the annotations name and package are set to !unknownSymbol!.
I also tried an alternative approach: to analyse the IdentifierTree of the AnnotationTree (inside of BaseTreeVisitor.visitMethodInvocation(MethodInvocationTree)), but I could not find the way how to get the package name of the annotation.
I use sonar-java-plugin 3.7.1 and sonar-plugin-api 5.1.
The code is commited in this mvn / Eclipse project.
The UnexpectedAccessCheckTest contains 2 use-cases. The "InvokedFromOtherClass" works well. The InvokedFromSameClass passes the test, but it happens only by accident - the annotation is not correctly detected.
It produces this output:
[main] DEBUG d.t.s.p.vft.checks.IsAnAnnotation - Checking Annotation.
Expected [com.google.common.annotations.VisibleForTesting]
got [.!unknownSymbol!]
The correct running use-case produces this:
[main] DEBUG d.t.s.p.vft.checks.IsAnAnnotation - Checking Annotation.
Expected [com.google.common.annotations.VisibleForTesting]
got [com.google.common.annotations.VisibleForTesting]
Do you have any hint for me?

The first way is prefered and should work for every method call, even within the same class.
If it does not work it probably means that the method invoked is not resolved from semantic (which is why the symbol is set to unknown, the analyzer has some bugs regarding method calls when generics are involved).
In order to point you out to the correct ticket, an example would be required.
Regarding your comment about the syntax tree : the identifier in the syntax tree won't contain information about the package (as its name gives it away, it concerns only syntax ;) ). However, you can use the following to find the type of the annotation from the source :
annotationTree.annotationType().symbolType().is("com.mypackage.MyAnnotation")
Make sure that the expected binary is in the Classpath while runnnig the scan (see sonar.java.binaries property, or usage of maven-dependency-plugin if you use Maven).

Does Reflections library ignore the RetentionPolicy

For understanding of the Java annotations I tried some hands on and got few doubts, even though looking at execution I am still confused. Here is what I am doing.
Define a Annotation
#Retention(RetentionPolicy.CLASS)
#Target(value=ElementType.TYPE)
public #interface Command {
}
Now I initialize the commands
Reflections reflections = new Reflections(CMDS_PACKAGE);
Set<Class<?>> allClasses = reflections.getTypesAnnotatedWith(Command.class); // line 2
for (Class clazz : allClasses) {
MYCommand cmd = (MYCommand) clazz.newInstance();
System.out.println(cmd.getClass().getAnnotation(Command.class));// line 6
log.info("loading Command [ {} ]", clazz.getCanonicalName());
}
when I run the program line 6 displays null.
When the policy is RetentionPolicy.RUNTIME line 6 displays the correct Command.
During this process the line 2 is still giving me correct Annotated class irrespective of policy. So does it mean that the Reflection Library is ignoring the RetentionPolicy
I am really confused even though reading most of tutorials.
The question for me actually is that , why is this different behaviour? When annotated with RetentionPolicy.CLASS policy It should not have given me at runtime. Is my understanding wrong or can anyone please share there valuable inputs on the understanding of these both.

Yes, the Reflections library (not Reflection, but Reflection*s*) does ignore the visibility of annotations by default.
This can be changed using the org.reflections.adapters.JavassistAdapter#includeInvisibleTag flag. Something like:
JavassistAdapter mdAdapter = new JavassistAdapter();
mdAdapter.includeInvisibleTag = false;
new Reflections(new ConfigurationBuilder()
...
.setMetadataAdapter(mdAdapter)
...
Another option would be to use the JavaReflectionAdapter instead.
HTH

In the first place, the RetentionPolicy dictates what a conforming compiler has to do. Annotations with RetentionPolicy.SOURCE do not make it into the class file while the other two, RetentionPolicy.CLASS and RetentionPolicy.RUNTIME, are stored within the class file but using different attributes to allow to make a distinction between them when reading the class file.
The documentation of RetentionPolicy.CLASS says:
Annotations are to be recorded in the class file by the compiler but need not be retained by the VM at run time. This is the default behavior.
Here, the responsibility is clearly documented, the VM shall not retain them and the Reflection API built into the JRE conforms to it. Though “need not be retained” does not sound like a strong requirement.
But 3rd party libraries like the Reflection Library you are using are free to implement whatever they want when parsing a class file. Since the documentation for the method you have called simply says: “get types annotated with a given annotation”, the behavior isn’t wrong as the type has that annotation.
And you are able to find out the RetentionPolicy of that Annotation even before invoking that method by analyzing the Annotations of the Annotation. So it makes no sense invoking the method when you already know that the annotation has the RetentionPolicy.CLASS and then bother because the method does something instead of nothing.
But, of course, it would be better if that behavior was documented completely. So you might ask the author of that 3rd party library to improve the documentation.

What is the purpose of annotations in Java?

I understand that annotations serve a purpose to modify code without actually BEING code, such as:
#Author(
name = "Benjamin Franklin",
date = "3/27/2003"
)
But I don't understand how using the annotation is any better/ clearer/ more concise than just saying name = "Benjamin Franklin" ? How does the addition of annotations strengthen the code?
EDIT: Sorry for another questoin, but I know that #Override can help prevent/ track spelling mistakes when calling methods or classes, but how does it do that? Does it help the actual program at all?

Annotations are just metadata. On their own they serve little to no purpose. There must be an annotation processor, either at the compiler or run time level that uses them for something.
With an annotation like
#Author(
name = "Benjamin Franklin",
date = "3/27/2003"
)
for example, some annotation processor might read it with reflection at run time and create some log file that this author wrote whatever it's annotating on that date.

Annotations are metadata.
#Override annotation is used to make sure that you are overriding method of a superclass and not just making a method with the same name. Common mistakes here consist of:
spelling the method's name wrong
equal(Object o) instead of equals(Object o)
putting different set of arguments
MyString extends String { public boolean equals(MyString str) {} }
equals(MyString str) is not overriding the method equals(Object o) and therefore will not be used by standard Java comparators (which is used in some standard functions, such as List.contains() and this is prone to error situation).
This annotation helps compiler to ensure that you code everything correctly and in this way it helps program.
#Deprecated annotation doesn't make program not to compile but it makes developers think about using the code that can and/or will be removed in a future releases. So they (developers) would think about moving onto another (updated) set of functions. And if you compile your program with the flag -Xlint compilation process will return with an error unless you remove all usages of deprecated code or explicitly mark them with annotation #SuppressWarnings("deprecation").
#SuppressWarnings is used to suppress warnings (yes, I know it's Captain Obvious style :)). There is a deprecation suppression with #SuppressWarnings("deprecation"), unsafe type casting with #SuppressWarnings("unchecked") and some others. This is helpfull when your project compiler have a compilation flag -Xlint and you cannot (or don't want to) change that.
There are also annotation processors that you integrate into your program build process to ensure that program code meets some sort of criteria. For example with IntelliJ Idea IDE annotation processor you can use #Nullable and #NotNull annotations. They show other programmers when they use your code so that can transfer null as a certain parameter to a method or not. If they transfer null it will cause exception during compilation or before executing a single line method's code.
So annotations are quite helpful if you use them to their full potential.

Annotations are most likely used by other programs. Examples include:
#Override
IDE (compiler?) ensures that the signatures match
#Deprecated
IDE marks occurences, compiler warning
#FXML
JavaFX can use these annotations initialize variables in a controller class when an .fxml File is inflated (see http://docs.oracle.com/javafx/2/get_started/fxml_tutorial.htm). They are also used by JavaFX Scene Builder.

Annotations works as a way to marking up the code. Several frameworks uses it, and some others make a great use of it producing your own.
Besides, is important to understand that annotations are the equivalent to meta-data, but is much more than that, since it works as a tag language for the code.

Java #Annotation
#Annotation(from Java 5) adds a metadata which are used for instruction in compile, deployment and run time. It is defined by RetentionPolicy
RetentionPolicy defines a lifetime
RetentionPolicy.SOURCE: It is visible only in compile time(#Override, #SuppressWarnings, #StringDef). For example it can be used by apt to generate some code
RetentionPolicy.CLASS: It is visible in compile and deployment time(.class). For example it can be used by ASM or Java AOP paradigm like AspectJ
RetentionPolicy.RUNTIME: It is visible in deployment and run time. For example it can be used java reflection using getAnnotations(). Dagger 2 uses #Scope annotation
Create a custom Annotation
#Retention(<retention_policy>) //optional
#Target(<element_type>) //optional to specify Java element like, field, method...
#Inherited // optional will be visible by subclass
#Documented // optional will be visible by JavaDoc
#interface MyAnnotation {
//attributes:
String someName();
}
using
#MyAnnotation(someName = "Alex")
public class SomeClass {
}

Search for all instances of certain annotation type

Suppose I have this annotation
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface Name
{
String value();
}
This is going to be used as follows
#Name("name1")
public static Foo foo = new Foo();
I have multiples of these across my project source files. Is there an fairly simple way to search and collect all those "foo"s that're preceded by #Name?
In other words, I'd like to write a method that would return a Set<Foo> containing these.
Thanks!!!

I am not really familiar with the classpath scanners that others are suggesting. They seem like a robust - if not ideal - solution.
If you have control over the source, then you could use annotation processing.
Create an annotation processor that will create a class - MapClass with a static member Map<String,Foo>. Every time the annotation processor encounters the #Name annotation, it adds that to the source code of MapClass. When it finishes processing the annotations, it will have the same effect as if you hard coded the map.
Annotation processing happens during compile time. If some of the classes in your project are not compiled by you. For example, if someone else compiles some classes and gives a jar to you, then it won't work as easily. But if all the classes are compiled by you then it should not be a problem.
To create an annotation processor, extend AbstractProcessor. You will want to annotate your class with a # SupportedAnnotationTypes ( "Name" ) annotation (make sure name is the fully qualified name of your annotation.
Override the process method. process has two parameters: annotations and roundEnv. annotations is just the set of annotations that this particular processor supports - in your case it should be (Name). roundEnv is a useful utility class.
Iterate through the one annotation in annotations. Use roundEnv to getElementsAnnotatedWith. This should give you the set of all elements that carry the #Name annotation.
AbstractProcessor has another utility member - processingEnv. Use its getFiler method to createSourceFile.
Then you have to modify your compilation a little bit. You must compile your processor separately and before the other classes. After the processor is compiled and you are compiling the other classes you must tell the compiler about your processor. If you are using the command line you would add -processorpath /path/to/processor/class[es] and -processor qualified.name.of.processor.
The advantages of this approach over the class path scanner is that everything happens at compile time. So for example, if you accidentally add a #Name annotation to a Bar element, then you can have the processor throw a compile time error (if you wish the processor can ignore it). Then you can fix it before the product ships. With a class path scanner, any error thrown is a run time error - which the user will see.
The disadvantage of this approach is also that everything happens at compile time. This makes it harder to dynamically add classes to the project.

What you need is a Classpath scanner. I have used Metapossum Scanner (it won out because it is in the mvn repo) to scan for annotated Classes, but I do not think it would scan for annotated Fields.
The other option I looked into was Reflections. I have not used Reflections, only researched it. The documentation has a getFieldsAnnotatedWith query that seems like what you need.
Be forewarned, the Classpath scanners are slooow and get slooower the more you have in your Classpath.

No not really (not from code).
A solution would be to put them all in a class, and then iterate on the Fields (getFields()) of the class and check for Annotations (getAnnotation())

You may want to have a look at Scannotation! It may solve your problem!!!
Scannotation is a Java library that creates an annotation database from a set of .class files. This database is really just a set of maps that index what annotations are used and what classes are using them.
PS.: VRaptor framework uses it internally!

Annotation concepts in Java

To quote this link :
Some developers think that the Java compiler understands the tag and
work accordingly. This is not right. The tags actually have no meaning
to the Java compiler or runtime itself. There are tools that can
interpret these tags
.
If the information contained in the annotation is only metadata, why wont my code compile if I annotate wrongly ? That particular annotation should be simply ignored right ?
Edit :
Just to provide an example... A simple JAX-RS web service on Jersey uses an annotation like :
#Path("mypath")
Now, if I change this to :
#Paths("mypath")
OR
#Path(123)
it should NOT stop me from compiling the code according to the above link...

The article is wrong for at least some annotations. Thinks like #SuppressWarnings and #Override the compiler does have very specific knowledge. In fact, the article points this out itself:
Metadata is used by the compiler to perform some basic compile-time checking. For example there is a override annotation that lets you specify that a method overrides another method from a superclass.
Quite how it can be used by the compiler if "the tags actually have no meaning to the Java compiler", I don't know...
Additionally, even for annotations that the compiler doesn't attach any semantic meaning to, it will still verify that when you try to specify particular arguments etc, that those arguments have sensible names and types for the annotation you're using.

Annotations are basically a special form of interface, so the compiler has to be able to load the annotation definition in order to encode the information so it can be included in the class file. Once it's in the class file, the class loader will load it as part of the class, so that annotation-processing tools can access the information. The compiler will verify that only defined arguments are used, as well as supplying default values for attributes that aren't specified (and have defaults defined).