Develop Reference

Generic working : compile time vs runtime

What actually happens in a generic class when we compile the file and what happens at runtime? how does T behave at compile time vs runtime? What was the main purpose of introducing generics? Since we can do the same thing with the Object class. I am very much confused and have spent 3 months understanding this generic topic. Kindly anyone here, explain it in every detail. Thanks
//demo class, basically what is happening here ?
class Generic<T>{
Generic(){
T[] arr = (T[]) new Object[5];
}
public static void main(String [] args) {
new Generic();
}
} // class
// another demo class , let say i have a Student class
class AnotherGeneric<T extends Student> {
T fun(){
T data = (T)new Object();
return data;
}
public static void main(String[] args) {
Student std = new AnotherGeneric<Student>().fun();
}
}// class

Mostly, generics just disappear entirely at runtime. Generics is, in essence, "compiler checked documentation". It's a way to get both of these things at the same time:
You have a method that returns, say, a List. You'd like to document that the list only contains strings.
You'd like for the compiler to be aware of this and tell users who treat that list as if it contains something other than strings to go: "Hey, there - hang on. I don't think you understand how this method works, given that you appear to be treating it as if it has non-strings in it, which it won't, as the documentation says that it won't". Or vice versa: "Hey there - hang on. You documented that the list you return only contain strings but you appear to be attempting to stuff a number in there. That doesn't make sense. I shall not compile this inconsistency until you fix it".
And very much in last place, generics makes your code very slightly shorter, as the compiler will inserts casts for you silently. When the method is documented to return only a list of strings, when you call that method and then call .get(0) on the result, the compiler "pre-casts" it to a String for you.
That's it. It doesn't change anything at runtime. Those casts are even generated by the compiler.
So, how does it work:
In signatures, generics is compiled into the class file, but the JVM treats these as effectively 'a comment' - the JVM completely ignores them. The point of this is solely for the benefit of javac, who can read these comments and act accordingly. In other words, the fact that ArrayList has generics needs to be known by javac in order to properly compile the line new ArrayList<String>() - and how does javac know? By checking the class file that contains the ArrayList code. Signatures are:
The name of a class.
The extends and implements clauses of a class.
The type of every field, and the name of every field.
The return type of every method, and the name of every method.
The type (not name) of every parameter of a method.
The throws clause of a method.
Everywhere else, generics just disappear. So, if you write inside a method: List<String> list = new ArrayList<String>();, the code you end up with is JUST new ArrayList() in the class file. That string is just gone. It also explains why given a List<?> x; there is simply no way to ask this list: What is your component type. Because it is no longer available at runtime.
Javac uses this information to figure out what to do.
For the purposes of compilation, ALL generics-typed stuff is compiled as if they are their lower bound.
What about generic casts?
The closest other java language feature to a generic cast is #SuppressWarnings. A generic cast does literally nothing. It's just you telling the compiler: Shut up, I know what I'm doing (hence, you best really know what you are doing to use them!!).
For example, given:
void foo(List<?> x) {
List<String> y = (List<String>) x;
}
The compiler does nothing. There is no way for the compiler to generate code that actually checks if x really is a List. The above code cannot throw an exception, even if there are non-strings in that list. As I said before, generics also cause the compiler to inject casts. So, if you later write:
x.get(0).toLowerCase();
That will compile (there is no need to cast x.get(0) to String, however, it is compiled that way!) - and if you pass a list to this method that has a non-string object as first item, that line throws a ClassCastException even though you didn't write any casts on that line. That's because the compiler inserted a cast for you.
Think about it like this: Generics are for linking types in signatures.
Imagine you want to write a method that logs its argument and then just returns it. That's all it does.
You want to now 'link' the type of the argument to the return type: You want to tell the compiler and all users of this method: Whatever type you feed into this method is identical to the type that rolls of it.
In normal java you cannot do this:
Object log(Object o) {
log.mark("Logged: {}", o);
return o;
}
The above works fine but is annoying to use. I can't do this:
String y = scanner.next();
new URL(log(y)).openConnection();
The reason I can't do that, is the log(y) expression is of type Object, and the URL constructor requires a String. Us humans can clearly see that log(y) is obviously going to return a string, but the signature of the log method doesn't indicate this at all. We have to look at the implementation of log to know this, and perhaps tomorrow this implementation changes. The log method does not indicate that any future updates will continue to just 'return the parameter' like this. So javac does not let you write this code.
But now we add generics:
public <T> T log(T o) {
log.mark("Logged: {}", o);
return o;
}
And now it works fine. We've told the compiler that there exists a link between the 2 places we used T in this code: The caller gets to choose what T ends up being, and the compiler ensures that no matter what the caller chose, your code works.
Hence, if you define a type parameter and use it exactly 0 or 1 times, it's virtually always either a bug or a weird hack. The point is to link things and '0 or 1 times' is obviously not linking things.
Generics goes much further than this, your question's scope is far too broad. If you want to know every detail, read the Java Lang Spec, which gets into hopeless amounts of detail that will take your 6 months to even understand. There's no real point to this. You don't need to know the chemical composition of brake fluid to drive a car either.

This is the way I was taught the importance of generics.
Imagine that you were blindfolded, then told to do some basic task, such as move boxes from one side of the room to the other. Now also imagine that the room is full of other blindfolded people doing exactly the same thing as you.
Programming without generics would be tell all of these people to do their tasks, and then run the risk of them accidentally crashing into each other and damaging the boxes.
Programming with generics would be to sit down with each blindfolded person, and give all of them a very specific plan beforehand. For example, tell one of them to go forward 10 feet, grab the box on the floor in front of them, turn 180 degress, then go 10 feet, then put the box down. Then (and this is the important part) you draw a map of all of the plans and make sure that each of the blindfolded people's paths CANNOT cross each other. That is what generics give you. If you can prove that none of paths cross each other, then it doesn't matter if they are blindfolded - they cannot bump into each other - by design!
Once you can prove that they cannot bump into each other, you can start doing something more complex, like telling one blindfolded person to hand a box to another blindfolded person. And if you get really good at it, you can have paths that actually do cross, but only one person is crossing the intersection at the time.
That is the power of generics in Java - you can perform unsafe actions safely by planning it all ahead of time - at compile time! Then, when you are at runtime, it doesn't matter that you are blind - you know exactly what to do, and you have proven that you cannot crash into anyone else. As a result, when you actually do the task, you don't slowly shuffle forwards, putting your hands in front of you, constantly checking in fear that you will bump into someone else. You sprint headfirst forwards, blindly, but confident that you cannot fail, because the entire path has been mapped out for you.
Now, I should mention, the only way Java generics work is by ensuring none of the paths cross. You do this by turning on warnings when you compile your java code. If you get warnings about unchecked or raw, then that means your code is not safe, and you need to fix your plan. Once you compile with no warnings related to generics, you can be certain that your types are safe and will not crash into each other unexpectedly.
And finally, generics are powerful, but they do not play well with nulls. If you let nulls sneak into your code, that is a blindspot which generics cannot protect you from. Be very certain to limit, if not remove, the nulls in your code, otherwise your generics may not be bulletproof. If you avoid nulls and compile without warnings, you can guarantee that your code will never run into a type error unexpectedly.

shouldn't this code produce a ClassCastException

The following code compiles and runs successfully without any exception
import java.util.ArrayList;
class SuperSample {}
class Sample extends SuperSample {
#SuppressWarnings("unchecked")
public static void main(String[] args) {
try {
ArrayList<Sample> sList = new ArrayList<Sample>();
Object o = sList;
ArrayList<SuperSample> ssList = (ArrayList<SuperSample>)o;
ssList.add(new SuperSample());
} catch (Exception e) {
e.printStackTrace();
}
}
}
shouldn't the line ArrayList<SuperSample> ssList = (ArrayList<SuperSample>)o; produce a ClassCastException ?
while the following code produces a compile time error error to prevent heap pollution, shouldn't the code mentioned above hold a similar prevention at runtime?
ArrayList<Sample> sList = new ArrayList<Sample>();
ArrayList<SuperSample> ssList = (ArrayList<SuperSample>) sList;
EDIT:
If Type Erasure is the reason behind this, shouldn't there be additional mechanisms to prevent an invalid object from being added to the List? for instance
String[] iArray = new String[5];
Object[] iObject = iArray;
iObject[0]= 5.5; // throws ArrayStoreException
then why,
ssList.add(new SuperSample());
is not made to throw any Exception?

No it should not, at run time both lists have the same type ArrayList. This is called erasure. Generic parameters are not part of compiled class, they all are erased during compilation. From JVM's perspective your code is equal to:
public static void main(String[] args) {
try {
ArrayList sList = new ArrayList();
Object o = sList;
ArrayList ssList = (ArrayList)o;
ssList.add(new SuperSample());
} catch (Exception e) {
e.printStackTrace();
}
}
Basically generics only simplify development, by producing compile time errors and warnings, but they don't affect execution at all.
EDIT:
Well, the base concept behind this is Reifiable Type. Id strongly recomend reading this manual:
A reifiable type is a type whose type information is fully available
at runtime. This includes primitives, non-generic types, raw types,
and invocations of unbound wildcards.
Non-reifiable types are types where information has been removed at
compile-time by type erasure
To be short: arrays are rifiable and generic collections are not. So when you store smth in the array, type is checked by JVM, because array's type is present at runtime. Array represents just a piece of memmory, while collection is an ordinary class, which might have any sort of implementation. For example it can store data in db or on the disk under the hood. If you'd like to get deeper, I suggest reading Java Generics and Collections book.

In your code example,
class SuperSample { }
class Sample extends SuperSample { }
...
ArrayList<Sample> sList = new ArrayList<Sample>();
Object o = sList;
ArrayList<SuperSample> ssList = (ArrayList<SuperSample>)o;
Shouldn't the last line produce a ClassCastException?
No. That exception is thrown by the JVM when it detects incompatible types being cast at runtime. As others have noted, this is because of erasure of generic types. That is, generic types are known only to the compiler. At the JVM level, the variables are all of type ArrayList (the generics having been erased) so there is no ClassCastException at runtime.
As an aside, instead of assigning to an intermediate local variable of type Object, a more concise way to do this assignment is to cast through raw:
ArrayList<SuperSample> ssList = (ArrayList)sList;
where a "raw" type is the erased version of a generic type.
Shouldn't there be additional mechanisms to prevent an invalid object from being added to the List?
Yes, there are. The first mechanism is compile-time checking. In your own answer you found the right location in the Java Language Specification where it describes heap pollution which is the term for an invalid object occurring in the list. The money quote from that section, way down at the bottom, is
If no operation that requires a compile-time unchecked warning to be issued takes place, and no unsafe aliasing occurs of array variables with non-reifiable element types, then heap pollution cannot occur.
So the mechanism you're looking for is in the compiler, and the compiler notifies you of this via compilation warnings. However, you've disabled this mechanism by using the #SuppressWarnings annotation. If you were to remove this annotation, you'd get a compiler warning at the offending line. If you absolutely want to prevent heap pollution, don't use #SuppressWarnings, and add the options -Xlint:unchecked -Werror to your javac command line.
The second mechanism is runtime checking, which requires use of one of the checked wrappers. Replace the initialization of sList with the following:
List<Sample> sList = Collections.checkedList(new ArrayList<Sample>(), Sample.class);
This will cause a ClassCastException to be thrown at the point where a SuperSample is added to the list.

The key here to answer your question is Type Erasure in java
You have a warning at compile time for your first case and not in the second because of your indirection by an object which prevent the compiler to raise you a warning (I'm guessing that this warning is raised when casting a parametrized type to another one which is not done on your second case, if anyone can confirm that I would be glad to here about it).
And your code run because, in the end sList ssList et o are all ArrayList

I think that this cant produce ClassCastException because of backward compatibility issue in Java.
Generic information is not included in bytecode (compiler get rids of it during compilation).
Imagine scenario that you use in your project some old legacy code (some old library writen in java 1.4) and you pass generic List to some method in this legacy code.
You can do this.
In time before generics legacy code was allowed to put anything at all (except primitives) into a collection.
So this legacy code cant get ClassCastException even if it try to put String to List<Integer>.
From the legacy code perspective it is just List.
So this strange behaviour is a consequence of type erasure and to allow backward compatibility in Java.
EDIT:
You get ArrayStoreException for arrays because at runtime the JVM KNOWS the type of arrays, and you dont get any exception for collections because of type erasure and this backward compatibility issue JVM doesnt know the type of collection at runtime.
You can read about this topic in "SCJP Sun® Certified Programmer for Java™ 6 Study Guide" book in chapter 7 "Generics and Collections"

From the JLS (4.12.2)
It is possible that a variable of a parameterized type refers to an object that is not
of that parameterized type. This situation is known as heap pollution. This situation
can only occur if the program performed some operation that would give rise
to an unchecked warning at compile-time.
For example, the code:
List l = new ArrayList<Number>();
List<String> ls = l; // unchecked warning
gives rise to an unchecked warning, because it is not possible to ascertain, either at compile-
time (within the limits of the compile-time type checking rules) or at run-time, whether
the variable l does indeed refer to a List<String>.
If the code above is executed, heap pollution arises, as the variable ls, declared to be a
List<String>, refers to a value that is not in fact a List<String>.
The problem cannot be identified at run-time because type variables are not reified,
and thus instances do not carry any information at run-time regarding the actual type
parameters used to create them.

No error with this collection declared with generics?

I have below code snippet and this works fine. Shouldn't it throw compile time error because I have defined c as ArrayList which will contain String object but I am adding Integer object. So why it did not throw compile time/Run time error?
Collection c = new ArrayList<String>();
c.add(123);
I know below will throw compile time error but why not above. Whats the logical difference between both these code snippet?
Collection<String>() c = new ArrayList();
c.add(123);

The first code snippet does not result in a compile time error, because at the line
c.add(123)
the compiler inspects the type of c. Since you declared c as Collection, the compiler treats it as such. Since Collection offers a method add(Object), it is perfectly reasonable to add any object to the c, especially an integer. Note that this program will however result in a runtime-error, if you attempt to read back the collection values as Strings.
In your second code snippet you provide more information for the compiler to work with. In this snippet it knows that the Collection it deals with is an Collection<String>, which can only accept Strings. Thus, there is no method add(int) or add(Object), only add(String). This leads to a compile-time error.

why it did not throw compile time error?
Because it's not syntactically or semantically invalid, it's just unwise.
Note that most modern IDEs (e.g. Eclipse) can be configured to warn you about the unparameterised Collection c, and optionally to fail to compile.

In the first example, the collection is "raw". This will usually result in a warning but not an error (depending on your exact set-up). This is primary in order to be able to compile all the pre-Java 5 legacy code around.
The the second example, you assign a "raw" object to a parameterized version, which only can be done with an explicit cast.

1) What is the logical difference?
Above: A Collection can be declared without a generic type. This is called a raw type. The collection can then hold any kind of collection. Since, with a raw typed collection, at runtime you might use a collection of strings as a collection of integers causing a runtime exception the compiler will usually throw a warning. Since you have not typed the collection in the above example the compiler can not prevent these runtime exceptions. The warning can be ignored if you know what it is for and know what you are doing.
Below: But a variable declared as a Collection<String> cannot hold any kind of collection. It has to be a collection of the type String. It is strong typed. The compiler is correct to see this as an error.
2) Why does the above snippet not cause a compiler error?
Java is strong typed, which ensures type safety. The above snippet is not type safe, but allowed by Java nonetheless. This is probably for historical reasons: Generics were only introduced with Java 1.5, so if the above snippet would have caused a compile error then most Java 1.4 code would have been broken in the Java 1.5 compiler.
Not every programming language evolves in such a backward compatible manner (PHP for instance). Apparently backward compatibility was valued over type safety when introducing Java 1.5.

What does the unsafe operations compiler error mean?

What does this error mean?
Note: Main.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Any suggestions how to avoid that kind of error?

That's the error you get when you use Collections without specifying a type. You probably have something like:
ArrayList myList = new ArrayList(); // or some other Collection class
If that's the case, you need to change that to specify what type of objects you want to store. For example:
ArrayList<String> myList = new ArrayList<String>();
Read up on Java Generics for more information.
This is my best guess without seeing your code and the full error message. There could be other causes for that message, this is just the problem that I've seen accompany that message before.

First, recompile with -Xlint:unchecked to see what the problem is. Then fix those problems. There are a number of potential high-level causes for unchecked warnings. One is that you didn't provide type parameters where you should have. There are some situations where they are unavoidable, and then you can suppress the specific warning, but these are the exception, and care must be taken that you aren't suppressing warnings that are really important.
So recompile with -Xlint:unchecked and post additional questions if you have trouble with any of the specific issues that are revealed.

What does it mean?
Java generics allow you to write something like this:
List<String> l = ...;
String s = l.get(0); // note there is no explicit typecast.
But if the compiler tells you that your code has "unchecked or unsafe operations", it is saying that you have broken the rules for using generics safely, and your code may give runtime class cast exceptions at unexpected places; e.g. in the statement above where we left out the typecast.
There are a few things that will cause the compiler to complain about unchecked or unsafe operations, and each one requires a different remediation. Do what the compiler is telling you and run it with the -Xlint option.

Does Java casting introduce overhead? Why?

Is there any overhead when we cast objects of one type to another? Or the compiler just resolves everything and there is no cost at run time?
Is this a general things, or there are different cases?
For example, suppose we have an array of Object[], where each element might have a different type. But we always know for sure that, say, element 0 is a Double, element 1 is a String. (I know this is a wrong design, but let's just assume I had to do this.)
Is Java's type information still kept around at run time? Or everything is forgotten after compilation, and if we do (Double)elements[0], we'll just follow the pointer and interpret those 8 bytes as a double, whatever that is?
I'm very unclear about how types are done in Java. If you have any reccommendation on books or article then thanks, too.

There are 2 types of casting:
Implicit casting, when you cast from a type to a wider type, which is done automatically and there is no overhead:
String s = "Cast";
Object o = s; // implicit casting
Explicit casting, when you go from a wider type to a more narrow one. For this case, you must explicitly use casting like that:
Object o = someObject;
String s = (String) o; // explicit casting
In this second case, there is overhead in runtime, because the two types must be checked and in case that casting is not feasible, JVM must throw a ClassCastException.
Taken from JavaWorld: The cost of casting
Casting is used to convert between
types -- between reference types in
particular, for the type of casting
operation in which we're interested
here.
Upcast operations (also called
widening conversions in the Java
Language Specification) convert a
subclass reference to an ancestor
class reference. This casting
operation is normally automatic, since
it's always safe and can be
implemented directly by the compiler.
Downcast operations (also called
narrowing conversions in the Java
Language Specification) convert an
ancestor class reference to a subclass
reference. This casting operation
creates execution overhead, since Java
requires that the cast be checked at
runtime to make sure that it's valid.
If the referenced object is not an
instance of either the target type for
the cast or a subclass of that type,
the attempted cast is not permitted
and must throw a
java.lang.ClassCastException.

For a reasonable implementation of Java:
Each object has a header containing, amongst other things, a pointer to the runtime type (for instance Double or String, but it could never be CharSequence or AbstractList). Assuming the runtime compiler (generally HotSpot in Sun's case) cannot determine the type statically a some checking needs to be performed by the generated machine code.
First that pointer to the runtime type needs to be read. This is necessary for calling a virtual method in a similar situation anyway.
For casting to a class type, it is known exactly how many superclasses there are until you hit java.lang.Object, so the type can be read at a constant offset from the type pointer (actually the first eight in HotSpot). Again this is analogous to reading a method pointer for a virtual method.
Then the read value just needs a comparison to the expected static type of the cast. Depending upon instruction set architecture, another instruction will need to branch (or fault) on an incorrect branch. ISAs such as 32-bit ARM have conditional instruction and may be able to have the sad path pass through the happy path.
Interfaces are more difficult due to multiple inheritance of interface. Generally the last two casts to interfaces are cached in the runtime type. IN the very early days (over a decade ago), interfaces were a bit slow, but that is no longer relevant.
Hopefully you can see that this sort of thing is largely irrelevant to performance. Your source code is more important. In terms of performance, the biggest hit in your scenario is liable to be cache misses from chasing object pointers all over the place (the type information will of course be common).

For example, suppose we have an array of Object[], where each element might have a different type. But we always know for sure that, say, element 0 is a Double, element 1 is a String. (I know this is a wrong design, but let's just assume I had to do this.)
The compiler does not note the types of the individual elements of an array. It simply checks that the type of each element expression is assignable to the array element type.
Is Java's type information still kept around at run time? Or everything is forgotten after compilation, and if we do (Double)elements[0], we'll just follow the pointer and interpret those 8 bytes as a double, whatever that is?
Some information is kept around at run time, but not the static types of the individual elements. You can tell this from looking at the class file format.
It is theoretically possible that the JIT compiler could use "escape analysis" to eliminate unnecessary type checks in some assignments. However, doing this to the degree you are suggesting would be beyond the bounds of realistic optimization. The payoff of analysing the types of individual elements would be too small.
Besides, people should not write application code like that anyway.

The byte code instruction for performing casting at runtime is called checkcast. You can disassemble Java code using javap to see what instructions are generated.
For arrays, Java keeps type information at runtime. Most of the time, the compiler will catch type errors for you, but there are cases where you will run into an ArrayStoreException when trying to store an object in an array, but the type does not match (and the compiler didn't catch it). The Java language spec gives the following example:
class Point { int x, y; }
class ColoredPoint extends Point { int color; }
class Test {
public static void main(String[] args) {
ColoredPoint[] cpa = new ColoredPoint[10];
Point[] pa = cpa;
System.out.println(pa[1] == null);
try {
pa[0] = new Point();
} catch (ArrayStoreException e) {
System.out.println(e);
}
}
}
Point[] pa = cpa is valid since ColoredPoint is a subclass of Point, but pa[0] = new Point() is not valid.
This is opposed to generic types, where there is no type information kept at runtime. The compiler inserts checkcast instructions where necessary.
This difference in typing for generic types and arrays makes it often unsuitable to mix arrays and generic types.

In theory, there is overhead introduced.
However, modern JVMs are smart.
Each implementation is different, but it is not unreasonable to assume that there could exist an implementation that JIT optimized away casting checks when it could guarantee that there would never be a conflict.
As for which specific JVMs offer this, I couldn't tell you. I must admit I'd like to know the specifics of JIT optimization myself, but these are for JVM engineers to worry about.
The moral of the story is to write understandable code first. If you're experiencing slowdowns, profile and identify your problem.
Odds are good that it won't be due to casting.
Never sacrifice clean, safe code in an attempt to optimize it UNTIL YOU KNOW YOU NEED TO.