Develop Reference

Question about Java generics and design of java.util.function.Function

question about Wildcard
Example：Student extends Person
Person person = new Person();
Student student = new Student();
List<? super Student> list = new ArrayList<>();
list.add(student); // success
list.add(person); // compile error
List<? extends Person> list2 = new ArrayList<>();
list2.add(person); // compile error
list2.add(student);// compile error
I have read the answer below a question "capture#1-of ? extends Object is not applicable"
You are using generic wildcard. You cannot perform add operation as class type is not determinate. You cannot add/put anything(except null) -- Aniket Thakur
Official doc：The wildcard is never used as a type argument for a generic method invocation, a generic class instance creation, or a supertype
But why could list.add(student) compile successfully ?
Design of java.util.function.Function
public interface Function<T, R>{
//...
default <V> Function<V, R> compose(Function<? super V, ? extends T> before) {
Objects.requireNonNull(before);
return (V v) -> apply(before.apply(v));
}
}
Why before is designed to Function<? super V, ? extends T> rather than Function<V,T> when the type of return is Function<V,R> and type of the input is V ? (It still can pass compile and use flexibly)

To understand these questions, you have to understand how generics work with subtyping (which is explicitly denoted in Java using the extends keyword). Andreas mentioned the PECS rules, which are their representations in Java.
First of all, I want to point out that the codes above can be corrected by a simple cast
ArrayList<? super Student> list = new ArrayList<>();
list.add(new Student());
ArrayList<Person> a = (ArrayList<Person>) list; // a covariance
a.add(new Person());
And compiles & runs well (rather than raising any exceptions)
The reason is simple, when we have a consumer (which takes some objects and consume them, such as the add method), we expect it to take objects of type no more than(superclasses) the type T we specified, because the process of consuming needs possibly any member(variables, methods etc.) of the type it wants, and we want to ensure that type T satisfy all the members the consumer requires.
On the contrary, a producer, which produces objects for us (like the get method), has to supply objects of type no less than the specified type T so that we can access any member that T has on the object produced.
These two are closely related to subtyping forms called covariance and contravariance
As for the second question, you can refer to the implementation of Consumer<T> as well (which is somewhat simpler):
default Consumer<T> andThen(Consumer<? super T> after) {
Objects.requireNonNull(after);
return (T t) -> { accept(t); after.accept(t); };
}
the reason why we need this ? super T is that: when we are combining two Consumers using the method andThen, suppose that the former Consumer takes an object of type T, we expect the later to take a object of type no more than T so it would not try to access any member that T doesn't have.
Therefore, rather than simply writing Consumer<T> after but Consumer<? super T> after, we allow the former consumer (of type T) to be combined with a consumer that takes an object not exactly of type T, but maybe smaller then T, by the convenience of covariance. That makes the following codes sound:
Consumer<Student> stu = (student) -> {};
Consumer<Person> per = (person) -> {};
stu.andThen(per);
The compose method of type Function also applies, by the same consideration.

IMO This is probably the most complex concept in vanilla Java. So let's break this down a bit. I'll start with your second question.
Function<T, R> takes an instance t of type T and returns an instance r of type R. With inheritance that means that you could supply an instance foo of type Foo if Foo extends T and similarly return bar of type Bar if Bar extends R.
As a library maintainer who wants to write a flexible generic method, it's hard, and actually impossible, to know in advance all the classes which might be used with this method which extend T and R. So how are we going to write a method that handles them? Further, the fact that these instances have types which extend the base class is none of our concern.
This is where the wildcard comes in. During the method call we say that you can use any class which meets the envelope of the required class. For the method in question, we have two different wildcards using upper and lower bounded generic type parameters:
public interface Function<T, R>{
default <V> Function<V, R> compose(Function<? super V, ? extends T> before)
Lets now say that we want to take advantage of this method... for the example lets define some basic classes:
class Animal{}
class Dog extends Animal{}
class Fruit{}
class Apple extends Fruit{}
class Fish{}
class Tuna extends Fish{}
Imagine our function and transformation is defined as below:
Function<Animal, Apple> base = ...;
Function<Fish, Animal> transformation = ...;
We can combine these functions using compose to create a new function:
Function<Fish, Apple> composed = base.compose(transformation);
This is all fine and dandy, but now imagine that in the desired output function we actually only want to use Tuna as the input. If we did not use the lower-bounded ? super V as the input type parameter for the Function we pass to compose then we would get a compiler error:
default <V> Function<V, R> compose(Function<V, ? extends T> before)
...
Function<Tuna, Apple> composed = base.compose(transformation);
> Incompatible types:
> Found: Function<Fish, Apple>, required: Function<Tuna, Apple>
This happens because the return type for the call to compose specifies V as Tuna while transformation on the other hand specifies its "V" as Fish. So now when we try to pass transformation to compose the compiler requires transformation to accept a Tuna as its V and of course Tuna does not identically match Fish.
On the other hand, the original version of the code (? super V) allows us to treat V as a lower bound (i.e. it allows "contravariance" vs. "invariance" over V). Instead of encountering a mismatch between Tuna and Fish the compiler is able to successfully apply the lower bound check ? super V which evaluates to Fish super Tuna, which is true since Tuna extends Fish.
For the other case, imagine our call is defined as:
Function<Animal, Apple> base = ...;
Function<Fish, Dog> transformation = ...;
Function<Fish, Apple> composed = base.compose(transformation);
If we did not have the wildcard ? extends T then we would get another error:
default <V> Function<V, R> compose(Function<? super V, T> before)
Function<Fish, Apple> composed = base.compose(transformation);
// error converting transformation from
// Function<Fish, Dog> to Function<Fish, Animal>
The wildcard ? extends T allows this to work as T is resolved to Animal and the wildcard resolves to Dog, which can satisfy the constraint Dog extends Animal.
For your first question; these bounds really only work in the context of a method call. During the course of the method, the wildcard will be resolved to an actual type, just as ? super V was resolved to Fish and ? extends T was resolved to Dog. Without the information from the generic signature, we would have no way for the compiler to know what class can be used on the type's methods, and therefore none are allowed.

Java Generics - difference in method declaration

What is the difference between the following two method declarations:
1. <R> Stream<R> myFunc(Function<? super T, ? extends R> mapper);
2. Stream<R> myFunc(Function<? super T, ? extends R> mapper);
For the 2nd declaration to compile, I need to add type parameter to class like this.
public class MyGenericsTest<T, R>
In this case compiler is ensuring that the return type of myFunc is determined at the compile time. The compiler could have known that from the method signature as well. I am confused on why these 2 declarations are treated differently by compiler.

By writing <R> Stream<R> myFunc(Function<? super T, ? extends R> mapper) you are telling the compiler that:
R is any class and is local to the method (by starting with<R> at the beginning)
The return type is a Stream of R
T is a class specified in the type parameter of MyGenericsTest<T> (if you dont specify it, it wont work as the compiler will not know T)
If you change to Stream<R> myFunc(Function<? super T, ? extends R> mapper), R and T are not local (no <R, T> at the beginning of the method) and the compiler expects them to be defined at a class level as MyGenericsTest<T, R>

The second form:
Stream<R> myFunc(Function<? super T, ? extends R> mapper);
really isn't any different from this:
Stream<String> myFunc(Function<? super T, ? extends String> mapper);
to the compiler, except that it uses a different type. The second one returns a stream of String. The first one returns a stream of R, whatever R is. The compiler already knows what String is. For the first one, the compiler has to know what R is, which means it has to be defined somewhere. It can be a generic parameter on an outer class, but it could also be a non-generic class that you import from somewhere else (written by someone who is very bad at coming up with meaningful names).
Keep in mind that although we often use single upper-case letters as generic parameters, that's just a human convention. The compiler just treats it like any other identifier.
But that's why your two examples are so different. The first one is a syntax that defines the method as a generic method with a type parameter that you're calling R. The second one is the exact same syntax as a method that returns Stream<String> or List<Integer> or something.

Why is it not possible to pass a <?> or a <? extends T> generic class instance to a method expecting a <? super T>?

The PECS principle is about what kind of argument you select in a function, depending on how you will use that parameter.
My question is about the fact that, once you chose to use super (because your function possibly is a Consumer), you cannot pass to that function certain generic class instances.
Let's consider the following program:
public class Main{
public static void main(String[] args){
List<?> unbound = new ArrayList<Long>();
List<? extends Long> extendsBound = new ArrayList<Long>();
List<? super Long> superBound = new ArrayList<Long>();
takeExtend(unbound);
takeExtend(extendsBound);
takeExtend(superBound);
takeSuper(unbound);
takeSuper(extendsBound);
takeSuper(superBound);
}
static <T> void takeExtend(List<? extends T> l){}
static <T> void takeSuper(List<? super T> l){}
}
The compiler gives the following error:
error: method takeSuper in class Main cannot be applied to given
types;
takeSuper(unbound);
^ required: List<? super T> found: List<CAP#1> reason: cannot infer type-variable(s) T
(argument mismatch; List<CAP#1> cannot be converted to List<? super T>) where T is a type-variable:
T extends Object declared in method <T>takeSuper(List<? super T>) where CAP#1 is a fresh type-variable:
CAP#1 extends Object from capture of ?
I was trying to find a sort of symmetry, such as the one we have for the PECS rule, but I didn't find any.
So:
Why is it not possible to pass a <?> or a <? extends T> to a function expecting <? super T>?
Why is it possible to pass a <?> to a function expecting a <? extends T>?
Why is it possible to pass a <? super T> to a function expecting a <? extends T>?

You can think of a generic method (i.e. a method with its own type parameters) as making a set of statements about its type parameters.
For example, the method <T> someMethod(List<T>) is saying:
There exists some type T for which the type parameter of this list equals T.
Easy. All lists fit this criterion, so any list is a valid input to the method.
Everything inside the method can now make use of the knowledge that that statement is true. How is that knowledge useful? Well, if we get an item from the list, we know it matches the list's type parameter. Because we captured the type as T, we can hold a reference to the object and put it back in the list later, all still without knowing what type it really is.
The method declaration <T> someMethod(List<? extends T>) makes a slightly different claim:
There exists some type T for which the type parameter of this list is a subtype of T.
This is also trivially true for all lists. However, you might notice that it makes <T> someMethod(List<? extends T>) a bit useless. You're telling the compiler to capture the fact that items that come out of the list share some common supertype with other items that come out of the list. Unless you have, inside the method, some other consumer that is known to accept <? super T>, there's nothing you can do with that information. It's much less useful than the knowledge that all the items that come from the list are of the same type.
So, why doesn't <T> takeSuper(List<? super T>) work the same way?
The method declaration <T> takeSuper(List<? super T>) can be interpreted as claiming that:
There exists some type T for which the type parameter of this list is a supertype of T.
If we have a List<? super Long> and we pass it to a method that captures <T> takeSuper(List<? super T>) it's easy to see that a reference of type Long would satisfy T. We could pass a Long to the method as a parameter of type T, and then add it to the list from inside the method.
But what about if we have a List<?> and we capture its type using the method <T> takeSuper(List<? super T>)? By declaring the list as a List<?> we're saying that we don't currently know what its type parameter is. In doing that, we're telling the compiler that we have absolutely no way to get a reference of a type that matches the type parameter of the list. In other words, the compiler knows with certainty that no object can satisfy the type parameter T. Remember, for this method, an object is of type T if it is known to be of a type that is a subtype of the type parameter of the list. If we don't know anything about the type parameter of the list, that's impossible.
The same is true for a List<? extends Long>. We know that the items we fetch from the list will be a subtype of Long, but we don't have a lower bound for their type. We can never prove that any type is a subtype of the list's type parameter. So, for the method <T> takeSuper(List<? super T>), there is again provably no way to get a reference of type T.
Interestingly, my compiler (Java 8) doesn't complain about calling the method <T> takeSuper(List<? super T>) with a List<?> as input. I suppose it recognizes that since there's no way to get a reference of type T, there's no harm in just ignoring the useless type parameter.

Why is it not possible to pass a <?> or a <? extends T> to a function expecting <? super T>?
Consider this method:
static <T> void sillyAdd(List<? super T> l, T t){
l.add(t);
}
Now look how we could use it if this were possible:
List<Integer> onlyIntegers = new ArrayList<>();
List<? extends Number> anyNumber = onlyIntegers;
sillyAdd(anyNumber, Float.valueOf(0)); /* Not allowed... */
Integer i = onlyIntegers.get(0);
The last line would throw a ClassCastException because we were allowed to put a Float into a List<Integer>.
Why is it possible to pass a <?> to a function expecting a <? extends T>?
takeExtend(unbound);
There are no bounds on T; it could be any type. While the type parameter of unbound is unknown, it does have some type, and since T can be any type, it's a match.
Methods like this are sometimes used as a helper in some odd corners of generics. Logically, we know that the following should be no problem:
public static void rotate(List<?> l) {
l.add(l.remove(0));
}
But the rules of generics don't imply type-safety here. Instead, we can use a helper like this:
public static void rotate(List<?> l) {
helper(l);
}
private static <T> void helper(List<T> l) {
l.add(l.remove(0));
}
Why is it possible to pass a <? super T> to a function expecting a <? extends T>?
It's not, in a way. Given this example:
static <T> void takeExtend(List<? extends T> l)
List<? super Long> superBound = new ArrayList<Long>();
takeExtend(superBound);
You might think that T is inferred to be Long. That would mean that you passed a List<? super Long> to a method declared like void takeExtend(List<? extends Long> l), which seems wrong.
But T isn't inferred to be Long. If you specify Long explicitly as the type parameter, you'll see that it won't work:
Main.<Long>takeExtend(superBound);
What's actually happening is that T is inferred to be ? super Long, so the generic type of the method is something like void takeExtend(List<? extends ? super Long>). That means that everything in the list is something that extends an unknown super class of Long.

Why is it not possible to pass a <?> or a <? extends S> to a function expecting <? super T>? (Question corrected for clarity: ? extends S).
The compiler wants to infer the T, which is the bottom limit, and it can't. Every class extends Object, but there is no universal inheritance tree bottom class, which the compiler could default to.
Why is it possible to pass a <?> to a function expecting a <? extends T>?
It's about inference. You can pass a <?> because the compiler can make something out of it. Namely, it can make Object out of T. The same happens with the following:
List<?> unbound = new ArrayList<>();
Why is it possible to pass a <? super T> to a function expecting a <? extends T>?
Again, it's about inferring the upper limit now, so the compiler can default to Object. It's not interested in what ? is super to, nor whether it is super to anything at all.

I've been thinking this stuff all night long, with the amazing help of your comments, trying to find an easy-to-remember solution.
This is what I got.
A method accepting a <? extends T>, to get T, must infer the upper limit of that hierarchy. Since in Java every class is extending Object, the compiler can rely on the fact that at least the upper limit is Object. So:
if we pass a <? extends T>, the upper bound is already defined and it
will use that one
if we pass <?>, we don't know anything, but we know
that every class extends Object, so the compiler will infer Object
if we pass <? super T>, we know the lower bound, but since every class
extends Object, the compiler infer Object.
A method accepting a <? super T>, to get T, must infer the lower limit of that hierarchy. In Java by default we can count on an upper bound, but not on a lower bound. So the considerations made in the previous case are not valid anymore. So:
if we pass a <? extends T>, there is no lower bound, because the
hierarchy can be extended as much as we want
if we pass a <?>,
there is no lower bound, because the hierarchy can be extended as
much as we want
if we pass a <? super T>, a lower bound is already
defined, so we can use that one.
Recapping:
a method accepting a <? extends T> can always rely on the fact that
an upper bound exists and that is Object -> we can pass every bound to it
a method accepting a <? super T> can infer a lower bound only if it is explicitly defined in the parameter -> we can only pass a <? super T>

Hamcrest Matchers contains with List of matchers

I am trying to use org.hamcrest.Matchers.contains(java.util.List<Matcher<? super E>>), but the compiler tells me that it cannot resolve the method.
I even tried the example given by Hamcrest here, but I get the same compilation error:
assertThat(Arrays.asList("foo", "bar"), contains(Arrays.asList(equalTo("foo"), equalTo("bar"))));
Error:(13, 9) java: no suitable method found for assertThat(java.util.List<java.lang.String>,org.hamcrest.Matcher<java.lang.Iterable<? extends java.util.List<org.hamcrest.Matcher<java.lang.String>>>>)
method org.hamcrest.MatcherAssert.<T>assertThat(T,org.hamcrest.Matcher<? super T>) is not applicable
(actual argument org.hamcrest.Matcher<java.lang.Iterable<? extends java.util.List<org.hamcrest.Matcher<java.lang.String>>>> cannot be converted to org.hamcrest.Matcher<? super java.util.List<java.lang.String>> by method invocation conversion)
method org.hamcrest.MatcherAssert.<T>assertThat(java.lang.String,T,org.hamcrest.Matcher<? super T>) is not applicable
(cannot instantiate from arguments because actual and formal argument lists differ in length)
method org.hamcrest.MatcherAssert.assertThat(java.lang.String,boolean) is not applicable
(actual argument java.util.List<java.lang.String> cannot be converted to java.lang.String by method invocation conversion)
I tried to cast the second argument to Matcher<? super List<String>>
assertThat(Arrays.asList("foo", "bar"), (Matcher<? super List<String>>)contains(Arrays.asList(equalTo("foo"), equalTo("bar"))));
but then I get another compilation error:
Error:(16, 88) java: inconvertible types
required: org.hamcrest.Matcher<? super java.util.List<java.lang.String>>
found: org.hamcrest.Matcher<java.lang.Iterable<? extends java.util.List<org.hamcrest.Matcher<java.lang.String>>>>
Is there a way to properly use this method?

The problem is that Arrays.asList(equalTo("foo"), equalTo("bar")); will give you the type List<Matcher<String>>, but you really want List<Matcher<? super String>>. You have to explicitly specify the type:
assertThat(str,
contains(Arrays.<Matcher<? super String>>asList(
equalTo("foo"),
equalTo("bar"))));

Understanding method signature

I am trying to understand this method signature:
public final <T> FluentIterable<T> transform(Function<? super E,T> function)
(taken from http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/collect/FluentIterable.html#transform%28com.google.common.base.Function%29)
I understand that transform returns a FluentIterable<T>, but I don't understand what the <T> before FluentIterable<T> means?
Further, in Function<? super E,T>, I understand what ? super E means, but I can't imagine how the compiler can check this - isn't E unknown at compile time, so how shall the compiler know, if ? is a supertype of E?

<T> means that this is a method with the generic type parameter T. This is similiar to <T> after a class name (like List<T>), just for methods.
E is not unknown at compile time. E is also a generic type parameter defined in the class definition:
public abstract class FluentIterable<E> extends Object implements Iterable<E>
When you create an instance of this class the compiler knows the type of E.