Develop Reference

Where are the rules for annotation classes written for java

I am trying to understand how the annotation understands what to do when we use the annotation. I am not talking about the behaviors like when to execute
that is covered by Retention, values etc. I want to understand how annotations understand the rules for that annotation. For example how does #Override annotation knows how to check if the function overrides a method in super class and so on. I tried digging a lot and I reached here but I don't get where the rules for the annotations are written. It feels like magic to me.

As already commented, processors (e.g. the compiler) must interpret the annotations, but the running program can also read/use the annotations (e.g by reflection).
For example the #Override annotation is used by the compiler, see it's documentation:
Indicates that a method declaration is intended to override a method declaration in a supertype. If a method is annotated with this annotation type compilers are required to generate an error message unless at least one of the following conditions hold:
The method does override or implement a method declared in a supertype.
The method has a signature that is override-equivalent to that of any public method declared in Object.
The #Override annotation is part of the standard API, other annotations may be part of some framework (e.g .JUnit #Test) or additional annotation processors, see Annotation Processing in javac. The developer can also declare annotations, see Annotation Interfaces.
In other words:
an Annotation is just that, an annotation. It is like a tag or mark that can be added to some elements (e.g. to a method, class, ...). There is no rule or anything similar directly attached to it. But some tools, like the compiler, or even normal Java code (some framework/library or written by you) can read and handle that annotation as desired.
There are a couple of annotations in the Java Language Specification (JLS) which compilers are required to handle. The action for the #Override (as example) is coded in the compiler, to do as specified in the JLS. Same for #Deprecated.

Java annotation example in official docs

While reading the Oracle documentation on annotations (quite new to this concept), I came across the following snippet in the beginning (link at the bottom). I am not clear on what the example is illustrating. Is the public #interface definition an enhanced version of a normal interface definition? id(), engineer() etc are methods that return default values if not specified in the interface implementation? But then the instantiation is confusing, is it providing an implementation of an interface where id() returns 2868724 etc? Also not clear what the function travelThroughTime() is for. Any clarifications appreciated:
/**
* Describes the Request-For-Enhancement(RFE) that led
* to the presence of the annotated API element.
*/
public #interface RequestForEnhancement {
int id();
String synopsis();
String engineer() default "[unassigned]";
String date(); default "[unimplemented]";
}
#RequestForEnhancement(
id = 2868724,
synopsis = "Enable time-travel",
engineer = "Mr. Peabody",
date = "4/1/3007"
)
public static void travelThroughTime(Date destination) { ... }
http://docs.oracle.com/javase/1.5.0/docs/guide/language/annotations.html

To break down your questions:
Is #interface just an enhancement of interface?:
No, #interface is declaring something quite different from a standard interface- you are essentially declaring an annotation type. Making this declaration enables the declared thing to be used as an annotation in other code. So the declaration:
public #interface RequestForEnhancement
enables the annotation #RequestForEnhancement to be used in later code.
An annotation describes metadata for a method or a class. The #RequestForEnhancement annotation, for example, might be placed in front of a method in another class to indicate that some developer wants that method to be changed in some way.
Declaring an interface, by contrast, is declaring the signature of a group of functions. Classes which later implement an interface must then provide implementations of those functions.
What are the "methods" (synopsis(), engineer(), etc.) in the annotation body for? These are not really methods like you would be used to seeing in a class or interface definition. Instead, these represent fields that the annotation you've just declared has. A #RequestForEnhancement annotation on a method should indicate what the requested change to the method is, and possibly who is expected to implement the enhancement to the method. Thus the fields synopsis and engineer are fields that can be included in the annotation.
What does this section mean?:
#RequestForEnhancement(
id = 2868724,
synopsis = "Enable time-travel",
engineer = "Mr. Peabody",
date = "4/1/3007"
)
public static void travelThroughTime(Date destination) { ... }
This is an example of using the annotation that we've declared in the block starting with #RequestForEnhancement. Usages like this will likely occur all over your codebase, in many different classes, once the annotation has been defined. In this particular example, there is a method travelThroughTime(Date destination) in some class which apparently doesn't work very well. Some developer coming across the method thought it should be improved by making it do what it appears to claim to do (travel through time). That developer decided to reflect his request by putting an #RequestForEnhancement annotation on the method with some information about when the request was made, who was expected to make the enhancement, etc.
Sure, but how do you use the contents of an annotation for anything useful? (A question I'll ask for you :-) )
So let's say I want to write a tool which looks through all of my code for methods annotated with #RequestForEnhancement and send an e-mail to the engineers listed in the request, along with information about the annotation and the request for enhancement. How would I get started?
The basic mechanism to find out what methods have an annotation and the way to get values from the annotation is through Java reflection. A tutorial which includes an example of annotations and reflection is here (it's actually a good tutorial on annotations in general).
So sure, you can use reflection to get info out of these annotations, but when would you run a tool to use the info from the annotations? (another one I'll ask for you) Java provides the ability for you to define annotation processors which use annotation information when your code is compiled. Here's what looks like a reasonable tutorial. You can also use the information in your annotations at runtime. If you've ever used JavaFX, for example, you may have noticed that annotations can affect runtime behavior (adding #FXML to a field helps JavaFX fill that field with a value defined in your fxml).

Does Reflections library ignore the RetentionPolicy

For understanding of the Java annotations I tried some hands on and got few doubts, even though looking at execution I am still confused. Here is what I am doing.
Define a Annotation
#Retention(RetentionPolicy.CLASS)
#Target(value=ElementType.TYPE)
public #interface Command {
}
Now I initialize the commands
Reflections reflections = new Reflections(CMDS_PACKAGE);
Set<Class<?>> allClasses = reflections.getTypesAnnotatedWith(Command.class); // line 2
for (Class clazz : allClasses) {
MYCommand cmd = (MYCommand) clazz.newInstance();
System.out.println(cmd.getClass().getAnnotation(Command.class));// line 6
log.info("loading Command [ {} ]", clazz.getCanonicalName());
}
when I run the program line 6 displays null.
When the policy is RetentionPolicy.RUNTIME line 6 displays the correct Command.
During this process the line 2 is still giving me correct Annotated class irrespective of policy. So does it mean that the Reflection Library is ignoring the RetentionPolicy
I am really confused even though reading most of tutorials.
The question for me actually is that , why is this different behaviour? When annotated with RetentionPolicy.CLASS policy It should not have given me at runtime. Is my understanding wrong or can anyone please share there valuable inputs on the understanding of these both.

Yes, the Reflections library (not Reflection, but Reflection*s*) does ignore the visibility of annotations by default.
This can be changed using the org.reflections.adapters.JavassistAdapter#includeInvisibleTag flag. Something like:
JavassistAdapter mdAdapter = new JavassistAdapter();
mdAdapter.includeInvisibleTag = false;
new Reflections(new ConfigurationBuilder()
...
.setMetadataAdapter(mdAdapter)
...
Another option would be to use the JavaReflectionAdapter instead.
HTH

In the first place, the RetentionPolicy dictates what a conforming compiler has to do. Annotations with RetentionPolicy.SOURCE do not make it into the class file while the other two, RetentionPolicy.CLASS and RetentionPolicy.RUNTIME, are stored within the class file but using different attributes to allow to make a distinction between them when reading the class file.
The documentation of RetentionPolicy.CLASS says:
Annotations are to be recorded in the class file by the compiler but need not be retained by the VM at run time. This is the default behavior.
Here, the responsibility is clearly documented, the VM shall not retain them and the Reflection API built into the JRE conforms to it. Though “need not be retained” does not sound like a strong requirement.
But 3rd party libraries like the Reflection Library you are using are free to implement whatever they want when parsing a class file. Since the documentation for the method you have called simply says: “get types annotated with a given annotation”, the behavior isn’t wrong as the type has that annotation.
And you are able to find out the RetentionPolicy of that Annotation even before invoking that method by analyzing the Annotations of the Annotation. So it makes no sense invoking the method when you already know that the annotation has the RetentionPolicy.CLASS and then bother because the method does something instead of nothing.
But, of course, it would be better if that behavior was documented completely. So you might ask the author of that 3rd party library to improve the documentation.

What is the purpose of annotations in Java?

I understand that annotations serve a purpose to modify code without actually BEING code, such as:
#Author(
name = "Benjamin Franklin",
date = "3/27/2003"
)
But I don't understand how using the annotation is any better/ clearer/ more concise than just saying name = "Benjamin Franklin" ? How does the addition of annotations strengthen the code?
EDIT: Sorry for another questoin, but I know that #Override can help prevent/ track spelling mistakes when calling methods or classes, but how does it do that? Does it help the actual program at all?

Annotations are just metadata. On their own they serve little to no purpose. There must be an annotation processor, either at the compiler or run time level that uses them for something.
With an annotation like
#Author(
name = "Benjamin Franklin",
date = "3/27/2003"
)
for example, some annotation processor might read it with reflection at run time and create some log file that this author wrote whatever it's annotating on that date.

Annotations are metadata.
#Override annotation is used to make sure that you are overriding method of a superclass and not just making a method with the same name. Common mistakes here consist of:
spelling the method's name wrong
equal(Object o) instead of equals(Object o)
putting different set of arguments
MyString extends String { public boolean equals(MyString str) {} }
equals(MyString str) is not overriding the method equals(Object o) and therefore will not be used by standard Java comparators (which is used in some standard functions, such as List.contains() and this is prone to error situation).
This annotation helps compiler to ensure that you code everything correctly and in this way it helps program.
#Deprecated annotation doesn't make program not to compile but it makes developers think about using the code that can and/or will be removed in a future releases. So they (developers) would think about moving onto another (updated) set of functions. And if you compile your program with the flag -Xlint compilation process will return with an error unless you remove all usages of deprecated code or explicitly mark them with annotation #SuppressWarnings("deprecation").
#SuppressWarnings is used to suppress warnings (yes, I know it's Captain Obvious style :)). There is a deprecation suppression with #SuppressWarnings("deprecation"), unsafe type casting with #SuppressWarnings("unchecked") and some others. This is helpfull when your project compiler have a compilation flag -Xlint and you cannot (or don't want to) change that.
There are also annotation processors that you integrate into your program build process to ensure that program code meets some sort of criteria. For example with IntelliJ Idea IDE annotation processor you can use #Nullable and #NotNull annotations. They show other programmers when they use your code so that can transfer null as a certain parameter to a method or not. If they transfer null it will cause exception during compilation or before executing a single line method's code.
So annotations are quite helpful if you use them to their full potential.

Annotations are most likely used by other programs. Examples include:
#Override
IDE (compiler?) ensures that the signatures match
#Deprecated
IDE marks occurences, compiler warning
#FXML
JavaFX can use these annotations initialize variables in a controller class when an .fxml File is inflated (see http://docs.oracle.com/javafx/2/get_started/fxml_tutorial.htm). They are also used by JavaFX Scene Builder.

Annotations works as a way to marking up the code. Several frameworks uses it, and some others make a great use of it producing your own.
Besides, is important to understand that annotations are the equivalent to meta-data, but is much more than that, since it works as a tag language for the code.

Java #Annotation
#Annotation(from Java 5) adds a metadata which are used for instruction in compile, deployment and run time. It is defined by RetentionPolicy
RetentionPolicy defines a lifetime
RetentionPolicy.SOURCE: It is visible only in compile time(#Override, #SuppressWarnings, #StringDef). For example it can be used by apt to generate some code
RetentionPolicy.CLASS: It is visible in compile and deployment time(.class). For example it can be used by ASM or Java AOP paradigm like AspectJ
RetentionPolicy.RUNTIME: It is visible in deployment and run time. For example it can be used java reflection using getAnnotations(). Dagger 2 uses #Scope annotation
Create a custom Annotation
#Retention(<retention_policy>) //optional
#Target(<element_type>) //optional to specify Java element like, field, method...
#Inherited // optional will be visible by subclass
#Documented // optional will be visible by JavaDoc
#interface MyAnnotation {
//attributes:
String someName();
}
using
#MyAnnotation(someName = "Alex")
public class SomeClass {
}

What happens when two different annotations have the same name?

I never encountered the case and now I'm wondering: what happens when/if two different annotations have the same name? (I'll give here an example using an annotation provided by IntelliJ but the question is really generic so I'm not tagging it 'IntelliJ')
For example the first sentence here:
http://youtrack.jetbrains.net/issue/IDEABKL-4959
says:
Many libraries have their own #NotNull
annotations (intellij,
hibernate-validation, ..).
What happens exactly if I wanted to use, say, both IntelliJ's #NotNull and Hibernate's #NotNull? (once again #NotNull is just an example where I happen to find a clash)
Are they incompatible? If they're incompatible, is it for the whole project?
This is something I'm really not familiar with...

In such a case you need to specify the full qualified name, e.g
#bar.baz.Foo
#org.fubar.Foo
void myMethod() {...}

There is no ambiguity, because the annotation's package name will be specified in an import or on the annotation itself.
JSR-305 addresses your specific example. It seeks a standard set of annotations, and refers specifically to FindBugs' and IntelliJ's nullability annotations.
Nullness annotations (e.g., #NonNull
and #CheckForNull). Both FindBugs and
IntelliJ already support their own
versions of nullness annotations.

They don't as the full package is part of the name. The effect is that you can only import one and will have to refer to any other with its fully qualified name. Like so
#NotNull
#com.jetbrains.annotations.NotNull
public Object ...

That won't matter since each annotation's full qualified name won't be the same. You can declare the qualified name on the import section.

The issue is the same for two classes/interafces/enums/annotations with the same name. They should appear in different packages. If they are in the same package (e.g. different versions) but different jar/directories, then the first one found in the classpath is chosen.