Develop Reference

Java strange warning "suspicious toArray() call"

When implementing a Stream-class using a delegate stream, I get strange warning by IntelliJ:
Array of type java.lang.Object[] expected, A[] found
The code which triggers the warning:
public class MyStream<T> implements Stream<T> {
private final Stream<T> delegate;
public MyStream(Stream<T> delegate) {
this.delegate = delegate;
}
*snip*
#Override
public <A> A[] toArray(IntFunction<A[]> generator) {
return delegate.toArray(generator);
}
*snip*
}
The full warning:
Array of type java.lang.Object[] expected, A[] found
Inspection info: Reports two types of suspicious calls to Collection.toArray(). The first type is any calls where the type of the specified array argument is not of the same type as the array type to which the result is casted. Example:
void m(List list) {
Number[] ns = (Number[])
list.toArray(new String[0]);
}
The second type is any calls where the type of the specified array argument does not match the type parameter of the collection declaration. Example:
void m(List<Number> list) {
Number[] ns =
list.toArray(new String[0]);
}
I somewhat assume this is a false positive, caused by type erasure, mostly because I cannot find any sane reason why this would cause a problem, and also because the examples from the warning do not match very well with my code.
However, when thinking about it, I wondered: If this is a false positive triggered because of type erasure, why does it know about A at all? Also, Java does usually cast a A[] to a Object[] implicitly, so why doesn't it here?
So: What is happening here?

Here's some different code that can serve to illustrate the problem your IDE is detecting:
Stream<Integer> intStream = Stream.of(1, 2, 3);
String[] stringArray = intStream.toArray(i -> new String[3]);
Or, using your own class (I didn't test this):
new MyStream<>(Stream.of(1, 2, 3)).toArray(i -> new String[3]);
That code throws a java.lang.ArrayStoreException on the second line (the second one would be expected to, as well). But it compiles.
That is exactly the problem with delegate.toArray(generator);. You're effectively calling
Stream<T>.toArray(IntFunction<A[]>)
Where A is not guaranteed to be the same as or compatible with T. To relate this to the example code above, T has the place of Integer and A has the place of String. See the problem?
The JavaDocs of Stream.toArray warns about this:
Throws: ArrayStoreException - if the runtime type of any element of this stream is not assignable to the runtime component type of the generated array
That is the problem that IntelliJ is detecting. You cannot assume that <A> and the type-wide parameter <T> are going to coincide.
In other words, it's not a false positive and you can use the example above to reproduce it.

IntelliJ thinks that this is suspicious because your stream is a stream of Ts, but you are producing an array of As. Note that A is the type parameter of the toArray method, which is independent of the type parameter T for the class. Not much to do with type erasure.
This is indeed a false positive. You are implementing the method for Stream, which requires an A[] to be returned after all. The method in Stream is indeed designed rather unsafely like this, allowing you to pass in anything for A, no matter what type of stream it is. It is documented that it will throw an ArrayStoreException if A is the wrong type.
Throws
ArrayStoreException - if the runtime type of the array returned from the array generator is not a supertype of the runtime type of every element in this stream
By delegating to delegate, you will be throwing an ArrayStoreException, which you should, (according to the docs) if someone passed a wrong A to toArray too, assuming the instance stored in delegate is correctly implemented. There's nothing wrong here
You can suppress the inspection for this one line by writing this comment above it:
//noinspection SuspiciousToArrayCall

varargs heap pollution : what's the big deal?

I was reading about varargs heap pollution and I don't really get how varargs or non-reifiable types would be responsible for problems that do not already exist without genericity. Indeed, I can very easily replace
public static void faultyMethod(List<String>... l) {
Object[] objectArray = l; // Valid
objectArray[0] = Arrays.asList(42);
String s = l[0].get(0); // ClassCastException thrown here
}
with
public static void faultyMethod(String... l) {
Object[] objectArray = l; // Valid
objectArray[0] = 42; // ArrayStoreException thrown here
String s = l[0];
}
The second one simply uses the covariance of arrays, which is really the problem here. (Even if List<String> was reifiable, I guess it would still be a subclass of Object and I would still be able to assign any object to the array.) Of course I can see there's a little difference between the two, but this code is faulty whether it uses generics or not.
What do they mean by heap pollution (it makes me think about memory usage but the only problem they talk about is potential type unsafetiness), and how is it different from any type violation using arrays' covariance?

You're right that the common (and fundamental) problem is with the covariance of arrays. But of those two examples you gave, the first is more dangerous, because can modify your data structures and put them into a state that will break much later on.
Consider if your first example hadn't triggered the ClassCastException:
public static void faultyMethod(List<String>... l) {
Object[] objectArray = l; // Valid
objectArray[0] = Arrays.asList(42); // Also valid
}
And here's how somebody uses it:
List<String> firstList = Arrays.asList("hello", "world");
List<String> secondList = Arrays.asList("hello", "dolly");
faultyMethod(firstList, secondList);
return secondList.isEmpty()
? firstList
: secondList;
So now we have a List<String> that actually contains an Integer, and it's floating around, safely. At some point later — possibly much later, and if it's serialized, possibly much later and in a different JVM — someone finally executes String s = theList.get(0). This failure is so far distant from what caused it that it could be very difficult to track down.
Note that the ClassCastException's stack trace doesn't tell us where the error really happened; it just tells us who triggered it. In other words, it doesn't give us much information about how to fix the bug; and that's what makes it a bigger deal than an ArrayStoreException.

The difference between an array and a List is that the array checks it's references. e.g.
Object[] array = new String[1];
array[0] = new Integer(1); // fails at runtime.
however
List list = new ArrayList<String>();
list.add(new Integer(1)); // doesn't fail.

From the linked document, I believe what Oracle means by "heap pollution" is to have data values that are technically allowed by the JVM specification, but are disallowed by the rules for generics in the Java programming language.
To give you an example, let's say we define a simple List container like this:
class List<E> {
Object[] values;
int len = 0;
List() { values = new Object[10]; }
void add(E obj) { values[len++] = obj; }
E get(int i) { return (E)values[i]; }
}
This is an example of code that is generic and safe:
List<String> lst = new List<String>();
lst.add("abc");
This is an example of code that uses raw types (bypassing generics) but still respects type safety at a semantic level, because the value we added has a compatible type:
String x = (String)lst.values[0];
The twist - now here is code that works with raw types and does something bad, causing "heap pollution":
lst.values[lst.len++] = new Integer("3");
The code above works because the array is of type Object[], which can store an Integer. Now when we try to retrieve the value, it'll cause a ClassCastException - at retrieval time (which is way after the corruption occurred), instead of at add time:
String y = lst.get(1); // ClassCastException for Integer(3) -> String
Note that the ClassCastException happens in our current stack frame, not even in List.get(), because the cast in List.get() is a no-op at run time due to Java's type erasure system.
Basically, we inserted an Integer into a List<String> by bypassing generics. Then when we tried to get() an element, the list object failed to uphold its promise that it must return a String (or null).

Prior to generics, there was absolutely no possibility that an object's runtime type is inconsistent with its static type. This is obviously a very desirable property.
We can cast an object to an incorrect runtime type, but the cast would fail immediately, at the exact site of casting; the error stops there.
Object obj = "string";
((Integer)obj).intValue();
// we are not gonna get an Integer object
With the introduction of generics, along with type erasure (the root of all evils), now it is possible that a method returns String at compile time, yet returns Integer at runtime. This is messed up. And we should do everything we can to stop it from the source. It is why the compiler is so vocal about every sight of unchecked casts.
The worst thing about heap pollution is that the runtime behavior is undefined! Different compiler/runtime may execute the program in different ways. See case1 and case2.

They are different because ClassCastException and ArrayStoreException are different.
Generics compile-time type checking rules should ensure that it's impossible to get a ClassCastException in a place where you didn't put an explicit cast, unless your code (or some code you called or called you) did something unsafe at compile-time, in which case you should (or whatever code did the unsafe thing should) receive a compile-time warning about it.
ArrayStoreException, on the other hand, is a normal part of how arrays work in Java, and pre-dates Generics. It is not possible for compile-time type checking to prevent ArrayStoreException because of the way the type system for arrays is designed in Java.

Why can't I create an array of a type parameter in Java?

Well, I have read a lot of answers to this question, but I have a more specific one. Take the following snippet of code as an example.
public class GenericArray<E>{
E[] s= new E[5];
}
After type erasure, it becomes
public class GenericArray{
Object[] s= new Object[5];
}
This snippet of code seems to work well. Why does it cause a compile-time error?
In addition, I have known from other answers that the following codes work well for the same purpose.
public class GenericArray<E>{
E[] s= (E[])new Object[5];
}
I've read some comments saying that the piece of code above is unsafe, but why is it unsafe? Could anyone provide me with a specific example where the above piece of code causes an error?
In addition, the following code is wrong as well. But why? It seems to work well after erasure, too.
public class GenericArray<E>{
E s= new E();
}

Array declarations are required to have a reifiable type, and generics are not reifiable.
From the documentation: the only type you can place on an array is one that is reifiable, that is:
It refers to a non-generic class or interface type declaration.
It is a parameterized type in which all type arguments are unbounded wildcards (§4.5.1).
It is a raw type (§4.8).
It is a primitive type (§4.2).
It is an array type (§10.1) whose element type is reifiable.
It is a nested type where, for each type T separated by a ".", T itself is reifiable.
This means that the only legal declaration for a "generic" array would be something like List<?>[] elements = new ArrayList[10];. But that's definitely not a generic array, it's an array of List of unknown type.
The main reason that Java is complaining about the you performing the cast to E[] is because it's an unchecked cast. That is, you're going from a checked type explicitly to an unchecked one; in this case, a checked generic type E to an unchecked type Object. However, this is the only way to create an array that is generic, and is generally considered safe if you have to use arrays.
In general, the advice to avoid a scenario like that is to use generic collections where and when you can.

This snippet of code seems to work well. Why does it cause a compile-time error?
First, because it would violate type safety (i.e. it is unsafe - see below), and in general code that can be statically determined to do this is not allowed to compile.
Remember that, due to type erasure, the type E is not known at run-time. The expression new E[10] could at best create an array of the erased type, in this case Object, rendering your original statement:
E[] s= new E[5];
Equivalent to:
E[] s= new Object[5];
Which is certainly not legal. For instance:
String[] s = new Object[10];
... is not compilable, for basically the same reason.
You argued that after erasure, the statement would be legal, implying that you think this means that the original statement should also be considered legal. However this is not right, as can be shown with another simple example:
ArrayList<String> l = new ArrayList<Object>();
The erasure of the above would be ArrayList l = new ArrayList();, which is legal, while the original is clearly not.
Coming at it from a more philosophical angle, type erasure is not supposed to change the semantics of the code, but it would do so in this case - the array created would be an array of Object rather than an array of E (whatever E might be). Storing a non-E object reference in it would then be possible, whereas if the array were really an E[], it should instead generate an ArrayStoreException.
why is it unsafe?
(Bearing in mind we are now talking about the case where E[] s= new E[5]; has been replaced with E[] s = (E[]) new Object[5];)
It is unsafe (which in this instance is short for type unsafe) because it creates at run-time a situation in which a variable (s) holds a reference to an object instance which is not a sub-type of the variable's declared type (Object[] is not a subtype of E[], unless E==Object).
Could anyone provide me with a specific example where the above piece of code causes an error?
The essential problem is that it is possible to put non-E objects into an array that you create by performing a cast (as in (E[]) new Object[5]). For example, say there is a method foo which takes an Object[] parameter, defined as:
void foo(Object [] oa) {
oa[0] = new Object();
}
Then take the following code:
String [] sa = new String[5];
foo(sa);
String s = sa[0]; // If this line was reached, s would
// definitely refer to a String (though
// with the given definition of foo, this
// line won't be reached...)
The array definitely contains String objects even after the call to foo. On the other hand:
E[] ea = (E[]) new Object[5];
foo(ea);
E e = ea[0]; // e may now refer to a non-E object!
The foo method might have inserted a non-E object into the array. So even though the third line looks safe, the first (unsafe) line has violated the constraints that guarantee that safety.
A full example:
class Foo<E>
{
void foo(Object [] oa) {
oa[0] = new Object();
}
public E get() {
E[] ea = (E[]) new Object[5];
foo(ea);
return ea[0]; // returns the wrong type
}
}
class Other
{
public void callMe() {
Foo<String> f = new Foo<>();
String s = f.get(); // ClassCastException on *this* line
}
}
The code generates a ClassCastException when run, and it is not safe. Code without unsafe operations such as casts, on the other hand, cannot produce this type of error.
In addition, the following code is wrong as well. But why? It seems to work well after erasure, too.
The code in question:
public class GenericArray<E>{
E s= new E();
}
After erasure, this would be:
Object s = new Object();
While this line itself would be fine, to treat the lines as being the same would introduce the semantic change and safety issue that I have described above, which is why the compiler won't accept it. As an example of why it could cause a problem:
public <E> E getAnE() {
return new E();
}
... because after type erasure, 'new E()' would become 'new Object()' and returning a non-E object from the method clearly violates its type constraints (it is supposed to return an E) and is therefore unsafe. If the above method were to compile, and you called it with:
String s = <String>getAnE();
... then you would get a type error at runtime, since you would be attempting to assign an Object to a String variable.
Further notes / clarification:
Unsafe (which is short for "type unsafe") means that it could potentially cause a run-time type error in code that would otherwise be sound. (It actually means more than this, but this definition is enough for purposes of this answer).
it's possible to cause a ClassCastException or ArrayStoreException or other exceptions with "safe" code, but these exceptions only occur at well defined points. That is, you can normally only get a ClassCastException when you perform a cast, an operation that inherently carries this risk. Similarly, you can only get an ArrayStoreException when you store a value into an array.
the compiler doesn't verify that such an error will actually occur before it complains that an operation is unsafe. It just knows that that certain operations are potentially able to cause problems, and warns about these cases.
that you can't create a new instance of (or an array of) a type parameter is both a language feature designed to preserve safety and probably also to reflect the implementation restrictions posed by the use of type erasure. That is, new E() might be expected to produce an instance of the actual type parameter, when in fact it could only produce an instance of the erased type. To allow it to compile would be unsafe and potentially confusing. In general you can use E in place of an actual type with no ill effect, but that is not the case for instantiation.

A compiler can use a variable of type Object to do anything a variable of type Cat can do. The compiler may have to add a typecast, but such typecast will either throw an exception or yield a reference to an instance of Cat. Because of this, the generated code for a SomeCollection<T> doesn't have to actually use any variables of type T; the compiler can replace T with Object and cast things like function return values to T where necessary.
A compiler cannot use an Object[], however, to do everything a Cat[] can do. If a SomeCollection[] had an array of type T[], it would not be able to create an instance of that array type without knowing the type of T. It could create an instance of Object[] and store references to instances of T in it without knowing the type of T, but any attempt to cast such an array to T[] would be guaranteed to fail unless T happened to be Object.

Let's say generic arrays are allowed in Java. Now, take a look at following code,
Object[] myStrs = new Object[2];
myStrs[0] = 100; // This is fine
myStrs[1] = "hi"; // Ambiguity! Hence Error.
If user is allowed to create generic Array, then user can do as I've shown in above code and it will confuse compiler. It defeats the purpose of arrays (Arrays can handle only same/similar/homogeneous type of elements, remember?). You can always use array of class/struct if you want heterogeneous array.
More info here.

Why does Java's type erasure not break this?

public class Test {
public static class Nested<T> {
public T val;
Nested(T val) { this.val = val; }
}
public static void main(String[] args) {
Nested<Integer> a = new Nested<Integer>(5);
Nested<Integer> b = new Nested<Integer>(2);
Integer diff = a.val - b.val;
}
}
The above code works fine. However, if I add a method to Nested:
T diff(Nested<T> other) { return this.val - other.val; }
I get a compilation error:
operator - cannot be applied to T,T
This makes sense to me. The type of T gets erased at runtime, so Java can't apply an operator that's only defined for certain classes like Integer. But why does a.val - b.val work?
Edit:
Lots of good answers. Thanks everyone. The gist of it, if I understand correctly, is that the compiler can add casts to Integer in a.val - b.val because it knows a and b were instantiated as as Nested<Integer>. However, since this.val - other.val occurs inside the body of a generic function definition (where T still could be anything), the compiler cannot add the casts that would be necessary to make "-" work. This leads to a more interesting question, namely, if the Java compiler were capable of inlining, would it be possible for a generic function like diff to work?

The difference between the two is whether you are inside a generic method or you are outside of it.
You got it absolutely right that inside the method T is not known to be an Integer, so operator minus - cannot be applied. However, when you are in main(), outside the generic method, the compiler knows that you've instantiated Nested with Integer, so it knows very well how to apply the operator. Even though the implementation of the generic has erased the type to produce the code for Nested<T>, the compiler does not think of a and b in terms of Nested<T>: it has enough knowledge to insert an appropriate cast, unbox the results, and apply the minus - operator.

You are getting a compile-time error, not a runtime one.
public static void main(String[] args) {
Nested<Integer> a = new Nested<Integer>(5);
Nested<Integer> b = new Nested<Integer>(2);
Integer diff = a.val - b.val;
}
Here, compiler knows that both T are Integer. You just declared <Integer>.
T diff(Nested<T> other) { return this.val - other.val; }
Here, compiler is not certain about T. It could be anything. And, numeric only operator - is not allowed for just anything.

a.val - b.val works because it is validated by the compiler, not in runtime. The compiler "sees" that you're using <Integer> and it compiles and runs Ok, in runtime there is no problem even with erasure because the compiler already validated that.

Because the code doesn't live within Nested, the type is known. The compiler can clearly see that a.val - b.val is an Integer minus an Integer, which can be auto-boxed. The compiler essentially rewrites it to
Integer diff = Integer.valueOf(((Integer) a.val).intValue() - ((Integer) b.val).intValue())
The .intValue and .valueOf calls are from the auto-boxing and auto-unboxing.
The type casts are safe for the compiler to insert because you used a parameterized type Nested.
True, technically, a could be something else, like a Calendar object, since the type is unknown at runtime. But if you are using generics, the compiler trusts that you aren't doing anything dumb to circumvent it. Therefore, if a.val or b.val were anything other than Integers, a ClassCastException would be thrown at runtime.

Because method call is at runtime and a.val - b.val is checked at compile time.
In first case, compiler knows that the type is Integer and -
operation is allowed for integers.
In second case, the type of T is not known to the compiler in advance, hence it is not sure whether - operation is valid or not. Hence the compiler error.
Consider we use the method as diff(Nested<Book> other) so there is no way a book can be subtracted from other.

Java use generics to set the primitive type of an array later

I am trying to write some simple numerical code in Java where one can choose between a float and double later. A simplified version of my class looks like the example below:
public class UniformGrid<T> {
public T[] data;
public UniformGrid(int arrayDim) {
data = new T[arrayDim];
}
}
This didn't work I got a generic array creation error when trying to compile. Googling and reading some SO answers I learned about java.lang.reflect.Array and tried to use
data = (T[]) Array.newInstance(T.class, arrayDim);
Which also didn't work, since T is (probably) a primitive type. My Java knowledge is quite rusty (especially when it comes to generics) and I would like to know why the new operator cannot be used with a generic array type. Also of course I am interested in how one would solve this problem in Java.

You cannot create a generic array in Java because of type erasure. The easiest way to get around this would be to use a a List<T>. But if you must use an array, you can use an Object[] for your array and ensure that only T objects are put into it. (This is the strategy ArrayList takes.)
Ex:
private Object[] data = new Object[10];
private int size = 0;
public void add(T obj) {
data[size++] = obj;
}
public T get(int i){
return (T) data[i];
}
Of course you'll get an unchecked warning from your compiler, but you can suppress that.

Generics can't be used when creating an array because you don't know at runtime what type T is. This is called type erasure.
The solution is simple: use List<T> data.

Sorry, you'll have to take another approach:
Type parameters must be reference types, they can't be primitive types.
Only reference types support polymorphism, and only for instance methods. Primitive types do not. float and double don't have a common supertype; you can not write an expression like a + b and choose at runtime whether to perform float addition or double addition. And since Java (unlike C++ or C#, which emit new code for each type parameter) uses the same bytecode for all instances of a generic type, you'd need polymorphism to use a different operator implementation.
If you really need this, I'd look into code generation, perhaps as part of an automated build. (A simple search & replace on the source ought to be able to turn a library operating on double into a library operating on float.)

This is possible, as long as you use Float and Double instead of float and double, as primitive types are not allowed in Java Generics. Of course, this will probably be quite slow. And, you won't be able to (safely) allow direct public access to the array. So this answer is not very useful, but it might be theoretically interesting. Anyway, how to construct the array ...
data = (T[]) new Object[arrayDim];
This will give you a warning, but it's not directly anything to worry about. It works in this particular form - it's inside a generic constructor and data is the only reference to this newly constructed object. See this page about this.
You will not be able to access this array object publicly in the way you might like. You'll need to set up methods in UniformGrid<T> to get and set objects. This way, the compiler will ensure type-safety and the runtime won't give you any problems.
private T[] data;
public void set(int pos, T t) {
data[pos] = t;
}
public T get(int pos) {
return data[pos];
}
In this case, the interface to set will (at compile-time) enforce the correct type is passed. The underlying array is of type Object[] but that's OK as it can take any reference type - and all generic types are effectively List<Object> or something like that at runtime anyway.
The interesting bit is the getter. The compiler 'knows' that the type of data is T[] and hence the getter will compile cleanly and promises to return a T. So as long as you keep the data private and only access it through get and set then everything will be fine.
Some example code is on ideone.
public static void main(String[] args) {
UniformGrid<A> uf = new UniformGrid<A>(1);
//uf.insert(0, new Object()); // compile error
uf.insert(0, new A());
uf.insert(0, new B());
Object o1= uf.get(0);
A o2= uf.get(0);
// B o2= uf.get(0); // compiler error
System.out.println(o1);
System.out.println(o2);
System.out.println("OK so far");
// A via_array1 = uf.data[0]; // Exception in thread "main" java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [LA;
}
As you would desire, there are compilation errors with uf.insert(0, new Object()) and B o2= uf.get(0);
But you shouldn't make the data member public. If you did, you could write and compile A via_array1 = uf.data[0];. That line looks like it should be OK, but you get a runtime exception: Ljava.lang.Object; cannot be cast to [LA;.
In short, the get and set interface provide a safe interface. But if you go to this much trouble to use an array, you should just use an ArrayList<T> instead. Moral of the story: in any language (Java or C++), with generics or without generics, just say no to arrays. :-)

Item 25 in Effective Java, 2nd Edition talks about this problem:
Arrays are covariant and reified; generics are invariant and erased.
As a consequence, arrays provide run-time type safety but not compile-time type safety and vice versa for generics. Generally speaking arrays and generics don't mix well.