Develop Reference

Why Arrays.sort(T[] a, Comparator<? super T> c) infers T as Object for a 2d array?

Say if I want to sort a 2d array. (just reorder the rows, don't touch data within each row).
In following snippet: all 3 cases use the same Arrays.sort(T[] a, Comparator<? super T> c) method signature. Case (a) works fine. However, just by adding a if condition to the second argument, the inference of T changes. I couldn't comprehend why.
// array contains 3 tuples, sort it by the first element, then second element
int[][] array1 = new int[3][2];
array1[0] = new int[]{1,2};
array1[1] = new int[]{2,3};
array1[2] = new int[]{2,4};
// Case (a): compiles good, tuple is inferred as int[]
Arrays.sort(array1, Comparator.comparingInt(tuple -> tuple[0])); // Arrays.sort(T[] a, Comparator<? super T> c) correctly infers that T refers to int[]
// Case (b.1): compile error: incompatible types
// tuple is now inferred as Object, why?
Arrays.sort(array1,
(a1, a2) -> a1[0] == a2[0] ?
Comparator.comparingInt(tuple -> tuple[1]) : Comparator.comparingInt(tuple -> tuple[0]));
// Case (b.2): compile error: incompatible types
Arrays.sort(array1, Comparator.comparingInt(tuple -> tuple[0]).thenComparingInt(tuple -> tuple[1]));
// Case (c): if downcast tuple[0] to ((int[])tuple)[0], then (b) works fine.
Update:
Enlightened by the comments, I soon realized that case (b.1) is actual not valid.
The lambda in (b.1) suppose to return an integer, not a comparator. E.g.
Arrays.sort(array1, (a1, a2) -> a1[0] == a2[0] ? 0 : 1);
In all other scenarios, I see Comparator.<int[]>comparingInt(...) forces the inference correctly.

Short answer: the compiler is not smart enough to infer through such complex expressions. It needs some help inferring the type:
Arrays.sort(array1, Comparator.<int[]>comparingInt(tuple -> tuple[0]).thenComparingInt(tuple -> tuple[1]));
Related JEP: http://openjdk.java.net/jeps/101
As for the ternary expression case, I think it needs further adaptation, since you need to return an int in the lambda, not a Comparator:
Arrays.sort(array1,
(a1, a2) -> a1[0] == a2[0] ?
Comparator.<int[]>comparingInt(tuple -> tuple[1]).compare(a1, a2) :
Comparator.<int[]>comparingInt(tuple -> tuple[0]).compare(a1, a2));

After digging a bit further, here's an intuitive explanation on this behavior:
public interface Comparator<T> {
...
default Comparator<T> thenComparingInt(ToIntFunction<? super T> keyExtractor)
public static <T> Comparator<T> comparingInt(ToIntFunction<? super T> keyExtractor)
...
}
The public static method needs the caller to define T (as goes by the first <T> in the second method) when invoking the method. Otherwise it defaults to Object.
This is unlike the non-static method, where T is already defined during instantiation.
So looking at Comparator.comparingInt(tuple -> tuple[0]);, compiler won't know what T is and hence defaults to Object.
To properly feed compiler the actual type T, we can explicitly define T in different places.
// define T when calling the method
Comparator.<int[]>comparingInt(tuple -> tuple[0]);
// infer by return type, this is samilar to placing the rhs to Arrays.sort(int[][] a, rhs)
Comparator<int[]> c = Comparator.comparingInt(tuple -> tuple[0]);
// explicit target type for lambda expression
Comparator.comparingInt((ToIntFunction<int[]>) tuple -> tuple[0]);
// explicit type for lambda parameter
Comparator.comparingInt((int[] tuple) -> tuple[0]);
// This works but it's doesn't change the inference of T as comments suggests below.
Comparator.comparingInt(tuple -> ((int[])tuple)[0]);
Once it returns a Comparator<int[]> object (thus defined T as int[]), the instance method thenComparingInt is called upon knowing what T is.

Getting the stream object in Collectors toMap method using Method References in Java 8

I am trying to iterate a list using stream() and putting in a map, where the key is the steam element itself, and the value is an AtomicBoolean, true.
List<String> streamDetails = Arrays.asList("One","Two");
toReplay = streamDetails.stream().collect(Collectors.toMap(x -> x.toString(), new AtomicBoolean(true)));
I get the below errors at compile time.
Type mismatch: cannot convert from String to K
The method toMap(Function<? super T,? extends K>, Function<? super T,? extends U>) in the type Collectors is not applicable for the arguments ((<no type> x) -> {},
AtomicBoolean)
What could I be doing wrong, what shall I replace my x -> x.toString() with?

new AtomicBoolean(true) is an expression that is not valid for the second parameter to Collectors.toMap.
toMap here would want a Function<? super String, ? extends AtomicBoolean> (intended to convert a stream element (or type String) to a map value of your intended type, AtomicBoolean), and a correct argument could be:
Collectors.toMap(x -> x.toString(), x -> new AtomicBoolean(true))
Which can also be written using Function.identity:
Collectors.toMap(Function.identity(), x -> new AtomicBoolean(true))

Why can't Java typecheck this code?

I have some stream handling code that takes a stream of words and performs some operations on them, then reduces them to a Map containing the words as keys and the number of occurrences of the word as a Long value. For the sake of the brevity of the code, I used the jOOL library's Seq class, which contains a number of useful shortcut methods.
The code compiles just fine if I write it like this:
item.setWordIndex (
getWords (item) // returns a Seq<String>
.map (this::removePunctuation) // String -> String
.map (stemmer::stem) // String -> String
.groupBy(str -> str, Collectors.counting ()));
However, if I attempt to replace the str -> str lambda with the more self-documenting Function::identity, I get the following errors:
The method setWordIndex(Map<String,Long>) in the type MyClass is not applicable for the arguments (Map<Object,Long>)
The type Function does not define identity(String) that is applicable here
Why does Function::identity behave any differently to str -> str, which I (perhaps naively) assumed was directly equivalent, and why can't the compiler handle it when it is used?
(And yes, I'm aware I could remove the identity function by moving the previous map application into the groupBy operation, but I find the code clearer like this, because it follows the application logic more directly)

You want Function.identity() (which returns a Function<T, T>), not Function::identity (which matches the SAM type Supplier<Function<T, T>>).
The following code compiles fine:
static String removePunctuation(String x) { return x; }
static String stem(String x) { return x; }
// ...
final Map<String, Long> yeah = Seq.of("a", "b", "c")
.map(Test::removePunctuation)
.map(Test::stem)
.groupBy(Function.identity(), Collectors.counting());

There is a slight difference between the two types; they are not directly equivalent:
Function.identity() has to return the input type, because its type is Function<T, T>;
str -> str can return a wider type; effectively it is Function<? extends T, T>.

Equivalent of Scala's foldLeft in Java 8

What is the equivalent of of Scala's great foldLeft in Java 8?
I was tempted to think it was reduce, but reduce has to return something of identical type to what it reduces on.
Example:
import java.util.List;
public class Foo {
// this method works pretty well
public int sum(List<Integer> numbers) {
return numbers.stream()
.reduce(0, (acc, n) -> (acc + n));
}
// this method makes the file not compile
public String concatenate(List<Character> chars) {
return chars.stream()
.reduce(new StringBuilder(""), (acc, c) -> acc.append(c)).toString();
}
}
The problem in the code above is the accumulator: new StringBuilder("")
Thus, could anyone point me to the proper equivalent of the foldLeft/fix my code?

There is no equivalent of foldLeft in Java 8's Stream API. As others noted, reduce(identity, accumulator, combiner) comes close, but it's not equivalent with foldLeft because it requires the resulting type B to combine with itself and be associative (in other terms, be monoid-like), a property that not every type has.
There is also an enhancement request for this: add Stream.foldLeft() terminal operation
To see why reduce won't work, consider the following code, where you intend to execute a series of arithmetic operations starting with given number:
val arithOps = List(('+', 1), ('*', 4), ('-', 2), ('/', 5))
val fun: (Int, (Char, Int)) => Int = {
case (x, ('+', y)) => x + y
case (x, ('-', y)) => x - y
case (x, ('*', y)) => x * y
case (x, ('/', y)) => x / y
}
val number = 2
arithOps.foldLeft(number)(fun) // ((2 + 1) * 4 - 2) / 5
If you tried writing reduce(2, fun, combine), what combiner function could you pass that combines two numbers? Adding the two numbers together clearly does not solve it. Also, the value 2 is clearly not an identity element.
Note that no operation that requires a sequential execution can be expressed in terms of reduce. foldLeft is actually more generic than reduce: you can implement reduce with foldLeft but you cannot implement foldLeft with reduce.

Update:
Here is initial attempt to get your code fixed:
public static String concatenate(List<Character> chars) {
return chars
.stream()
.reduce(new StringBuilder(),
StringBuilder::append,
StringBuilder::append).toString();
}
It uses the following reduce method:
<U> U reduce(U identity,
BiFunction<U, ? super T, U> accumulator,
BinaryOperator<U> combiner);
It may sound confusing but if you look at the javadocs there is a nice explanation that may help you quickly grasp the details. The reduction is equivalent to the following code:
U result = identity;
for (T element : this stream)
result = accumulator.apply(result, element)
return result;
For a more in-depth explanation please check this source.
This usage is not correct though because it violates the contract of reduce which states that the accumulator should be an associative, non-interfering, stateless function for incorporating an additional element into a result. In other words since the identity is mutable the result will be broken in case of parallel execution.
As pointed in the comments below a correct option is using the reduction as follows:
return chars.stream().collect(
StringBuilder::new,
StringBuilder::append,
StringBuilder::append).toString();
The supplier StringBuilder::new will be used to create reusable containers which will be later combined.

The method you are looking for is java.util.Stream.reduce, particularly the overload with three parameters, identity, accumulator, and binary function. That is the correct equivalent to Scala's foldLeft.
However, you are not allowed to use Java's reduce that way, and also not Scala's foldLeft for that matter. Use collect instead.

It can be done by using Collectors:
public static <A, B> Collector<A, ?, B> foldLeft(final B init, final BiFunction<? super B, ? super A, ? extends B> f) {
return Collectors.collectingAndThen(
Collectors.reducing(Function.<B>identity(), a -> b -> f.apply(b, a), Function::andThen),
endo -> endo.apply(init)
);
}
Usage example:
IntStream.rangeClosed(1, 100).boxed().collect(foldLeft(50, (a, b) -> a - b)); // Output = -5000
For your question, this does what you wanted:
public String concatenate(List<Character> chars) {
return chars.stream()
.collect(foldLeft(new StringBuilder(), StringBuilder::append)).toString();
}

Others are correct there's no equivalent though. Here's a util that comes close-
<U, T> U foldLeft(Collection<T> sequence, U identity, BiFunction<U, ? super T, U> accumulator) {
U result = identity;
for (T element : sequence)
result = accumulator.apply(result, element);
return result;
}
your case using the above method would look like-
public String concatenate(List<Character> chars) {
return foldLeft(chars, new StringBuilder(""), StringBuilder::append).toString();
}
Or without the lambda method ref sugar,
public String concatenate(List<Character> chars) {
return foldLeft(chars, new StringBuilder(""), (stringBuilder, character) -> stringBuilder.append(character)).toString();
}

How to do additional processing in Java Stream API near the "::" method reference operator

Using the Java Stream API, is there a way to do additional processing to adjust the value of whatever is passed to a method reference?
I'll give two examples.
Example 1.
In the first example, I start with a Stream<Path>, and I want to return a Map<String, Path> in which the keys in the map are processed version of the filename using another function that takes a String filename (not a Path). Specifically:
public Map<String, Path> createMap(Path sourceFolder, PathMatcher filter) {
return stream.filter(filter::matches)
.collect(Collectors.toMap(FilenameHelper::parseFilename, Function.identity()));
parseFilename(String filename) takes a String filename, but of course the method reference gets a Path. I'd like to say something like, FilenameHelper::parseFilename(((Path)Function.identity()).toFile().getName()) but that doesn't work (Eclipse says: "The left-hand side of an assignment must be a variable"). I can work around it by creating a new method that takes a Path and just does return parseFilename(path.toFile().toName()) but that's not cool.
Example 2.
In the second example, I have rows, a List<List<String>>> that represents a data table (rows, then columns). I have a method that should return a List<String> consisting of a specific column in that table for every nth row. I want to do something like:
public List<String> getDataFromColumn(String columnName, int nth) {
/// Without a clause at ???, this returns a List<List<String>>
return IntStream.range(0, rows.size())
.filter(n -> n % nth == 0) // Get every nth row
.mapToObj(rows::get)
.???
.collect(Collectors.toList());
}
Where "???" should be something like map(ArrayList::get(headers.indexOf(columnName))) (where headers is a List<String> containing the column headers) but if I put that in, I get an AssignmentOperator syntax error in the get part of this clause. Replacing map with forEach doesn't help here. In other words, I don't want rows.get(n), I want rows.get(n).get(headers.indexOf(columnName).
Question
In both of these examples, I want to do something additional to the value that is being passed to the method pointed to with the method reference operator (::). Is there a "Java Stream-ic" way to do additional processing to the thing being passed to the method reference?

Method references are essentially a convenient substitute for lambdas where the function signature is an exact match to the method signature. In your case you can just use regular lambdas:
public Map<String, Path> createMap(Path sourceFolder, PathMatcher filter) {
return stream.filter(filter::matches)
.collect(Collectors.toMap(path -> FilenameHelper.parseFilename(path.toFile().getName()), Function.identity()));
}
public List<String> getDataFromColumn(String columnName, int nth) {
return IntStream.range(0, rows.size())
.filter(n -> n % nth == 0)
.mapToObj(rows::get)
.map(row -> row.get(headers.indexOf(columnName)))
.collect(Collectors.toList());
}

How about Function.compose? Of course you cannot use FilenameHelper::parseFilename.compose, but you can easily write a static helper method to work around it:
static <T, V, R> Function<T, R> compose(Function<T, V> f, Function<V, R> g) {
return g.compose(f);
}
Now we can compose method references:
return stream.filter(filter::matches)
.collect(Collectors.toMap(
compose(
compose(Path::getFileName, Path::toString),
FilenameHelper::parseFilename),
Function.identity()));
This is actually not very readable but an alternative to writing a full lambda.

No, this functionality is currently not provided.
The usual way would be to just not use a method reference and instead call the method the "usual" way using a lambda expression:
stream.filter(filter::matches)
.collect(Collectors.toMap(p -> FilenameHelper.parseFilename(p.getFileName()), Function.identity()));

No, there is not. There is no syntax to do that.
And if you wanted such a thing then lambda expression is what you want.
Method reference or lambda, under the hood you are still going to get a class that actually implements the Predicate/Function so it does not matter.
And that argument but that's not cool, to me under the conditions that there is no syntax for that, it's the best option you have.
Underneath the actual calls that you there is a MethodHandle (introduced in jdk-7) and MethodHandles do not have a way to achieve what you want. I think the same restriction exists in C++ with method pointers.