Develop Reference

Where is the reference to the lambda function?

I'm trying to understand exactly how lambdas and higher order functions work at the JVM level in modern Java. I wrote this simple test class:
public final class Main {
public static void main(String[] args) {
var s = new Object[] { 1.0, 2.0, 3.0 };
System.out.println(sum(s, x -> 1000000.0));
}
public static double sum(Object[] s, Function<Object, Double> f) {
var r = 0.0;
for (var a : s) {
r += f.apply(a);
}
return r;
}
}
that compiles to this:
Compiled from "Main.java"
public final class prover.Main {
public prover.Main();
Code:
0: aload_0
1: invokespecial #1 // Method java/lang/Object."<init>":()V
4: return
public static double sum(java.lang.Object[], java.util.function.Function<java.lang.Object, java.lang.Double>);
Code:
0: dconst_0
1: dstore_2
2: aload_0
3: astore 4
5: aload 4
7: arraylength
8: istore 5
10: iconst_0
11: istore 6
13: iload 6
15: iload 5
17: if_icmpge 50
20: aload 4
22: iload 6
24: aaload
25: astore 7
27: dload_2
28: aload_1
29: aload 7
31: invokeinterface #7, 2 // InterfaceMethod java/util/function/Function.apply:(Ljava/lang/Object;)Ljava/lang/Object;
36: checkcast #13 // class java/lang/Double
39: invokevirtual #15 // Method java/lang/Double.doubleValue:()D
42: dadd
43: dstore_2
44: iinc 6, 1
47: goto 13
50: dload_2
51: dreturn
public static void main(java.lang.String[]);
Code:
0: iconst_3
1: anewarray #2 // class java/lang/Object
4: dup
5: iconst_0
6: dconst_1
7: invokestatic #19 // Method java/lang/Double.valueOf:(D)Ljava/lang/Double;
10: aastore
11: dup
12: iconst_1
13: ldc2_w #23 // double 2.0d
16: invokestatic #19 // Method java/lang/Double.valueOf:(D)Ljava/lang/Double;
19: aastore
20: dup
21: iconst_2
22: ldc2_w #25 // double 3.0d
25: invokestatic #19 // Method java/lang/Double.valueOf:(D)Ljava/lang/Double;
28: aastore
29: astore_1
30: getstatic #27 // Field java/lang/System.out:Ljava/io/PrintStream;
33: aload_1
34: invokedynamic #33, 0 // InvokeDynamic #0:apply:()Ljava/util/function/Function;
39: invokestatic #36 // Method sum:([Ljava/lang/Object;Ljava/util/function/Function;)D
42: invokevirtual #42 // Method java/io/PrintStream.println:(D)V
45: return
private static java.lang.Double lambda$main$0(java.lang.Object);
Code:
0: ldc2_w #48 // double 1000000.0d
3: invokestatic #19 // Method java/lang/Double.valueOf:(D)Ljava/lang/Double;
6: areturn
}
Now, the lambda function itself gets compiled to the private static method at the end, that much is clear enough. But where is it referred to? The code to call sum seems to be:
33: aload_1
34: invokedynamic #33, 0 // InvokeDynamic #0:apply:()Ljava/util/function/Function;
39: invokestatic #36 // Method sum:([Ljava/lang/Object;Ljava/util/function/Function;)D
Is that somehow referring to the lambda function? If so, how? What's the reference?

Using javap -p -v will produce a section labeled BootstrapMethods that lists all bootstrap methods used to initialize lambdas:
BootstrapMethods:
0: #41 REF_invokeStatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
Method arguments:
#42 (Ljava/lang/Object;)Ljava/lang/Object;
#43 REF_invokeStatic Scratch.lambda$main$0:(Ljava/lang/Object;)Ljava/lang/Double;
#44 (Ljava/lang/Object;)Ljava/lang/Double;
This one contains references to the methods implementing the actual code (Scratch.lambda$main$0 in my case, the exact name will vary depending on compiler-vendor/-version/-flags).
Note that the representation in the Class files has intentionally been kept at a fairly high level (there are bootstrap methods that return the actual code to be executed at run time). This means that the JVM doesn't have a lot of restrictions as to how to implement and optimize this. That also means that studying the bytecode will only ever tell you so much, because the JVM can pretty freely interpret what it sees there.

Does String concatenation get optimized to use existing StringBuilders?

I have the following code:
StringBuilder str = new StringBuilder("foo");
for(Field f : fields){
str.append("|" + f);
}
str.append("|" + bar);
String result = str.toString();
I know compiler will optimize string concatenation "|" + f and replace it with StringBuilder. However will a new StringBuilder be created or the existing str will be used in Java 8? How about Java 9?

By default in java-9 there will be no StringBuilder for string concatenation; it is a runtime decision how it's made via the invokedynamic. And the default policy is not a StringBuilder::append one.
You can also read more here.
Under java-8 a new one will be created (really easy to spot two occurrences of invokespecial // Method java/lang/StringBuilder."<init>":()V in the de-compiled bytecode.
Also, you have a suggestion about append.append...; just notice that this is much better than sb.append ... sb.append, and here is why.

As String concatenation optimization is performed by the Java compiler, you can see what it does by decompiling the byte code:
$ cat Test.java
interface Field {}
public class Test {
static String toString(Field[] fields, Object bar) {
StringBuilder str = new StringBuilder("foo");
for(Field f : fields){
str.append("|" + f);
}
str.append("|" + bar);
return str.toString();
}
}
$ javac Test.java
$ javap -c Test.class
Compiled from "Test.java"
public class stackoverflow.Test {
public stackoverflow.Test();
Code:
0: aload_0
1: invokespecial #8 // Method java/lang/Object."<init>":()V
4: return
static java.lang.String toString(stackoverflow.Field[], java.lang.Object);
Code:
0: new #16 // class java/lang/StringBuilder
3: dup
4: ldc #18 // String foo
6: invokespecial #20 // Method java/lang/StringBuilder."<init>":(Ljava/lang/String;)V
9: astore_2
10: aload_0
11: dup
12: astore 6
14: arraylength
15: istore 5
17: iconst_0
18: istore 4
20: goto 53
23: aload 6
25: iload 4
27: aaload
28: astore_3
29: aload_2
30: new #16 // class java/lang/StringBuilder
33: dup
34: ldc #23 // String |
36: invokespecial #20 // Method java/lang/StringBuilder."<init>":(Ljava/lang/String;)V
39: aload_3
40: invokevirtual #25 // Method java/lang/StringBuilder.append:(Ljava/lang/Object;)Ljava/lang/StringBuilder;
43: invokevirtual #29 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
46: invokevirtual #32 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
49: pop
50: iinc 4, 1
53: iload 4
55: iload 5
57: if_icmplt 23
60: aload_2
61: new #16 // class java/lang/StringBuilder
64: dup
65: ldc #23 // String |
67: invokespecial #20 // Method java/lang/StringBuilder."<init>":(Ljava/lang/String;)V
70: aload_1
71: invokevirtual #25 // Method java/lang/StringBuilder.append:(Ljava/lang/Object;)Ljava/lang/StringBuilder;
74: invokevirtual #29 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
77: invokevirtual #32 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
80: pop
81: aload_2
82: invokevirtual #29 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
85: areturn
}
As you can see, the code invokes StringBuilder constructors (Method java/lang/StringBuilder."<init>":) in 3 places, so new StringBuilders would be created in each iteration (unless the just-in-time compiler performs fancy optimization).
This is very unlikely to be a significant performance problem, but in the unlikely case it is you can easily fix this by rewriting to
str.append("|").append(f);

As per the Java 9 API documentation
Implementation Note:
The implementation of the string concatenation operator is left to the discretion of a Java compiler, as long as the compiler ultimately conforms to The Java™ Language Specification. For example, the javac compiler may implement the operator with StringBuffer, StringBuilder, or java.lang.invoke.StringConcatFactory depending on the JDK version. The implementation of string conversion is typically through the method toString, defined by Object and inherited by all classes in Java.
According to this it will create a new String builder each iteration in your case. So, as mentioned by several people here, using the below code is more optimized
append("|").append(f)
You can find API Documentation here

When would a compiler choose StringBuffer over StringBuilder for String concatenation

I was looking at the String Javadoc when I noticed this bit about String concatenation:
The Java language provides special support for the string
concatenation operator ( + ), and for conversion of other objects to
strings. String concatenation is implemented through the
StringBuilder(or StringBuffer)
From the Java 8 JLS 15.8.1, it is a choice given to the compiler (emphasis mine):
An implementation may choose to perform conversion and concatenation
in one step to avoid creating and then discarding an intermediate
String object. To increase the performance of repeated string
concatenation, a Java compiler may use the StringBuffer class or a
similar technique to reduce the number of intermediate String objects
that are created by evaluation of an expression.
I made a small program to see what it compiles down to
public class Tester {
public static void main(String[] args) {
System.out.println("hello");
for (int i = 1; i < 5; i++) {
String s = "hi " + i;
System.out.println(s);
}
String t = "me";
for (int i = 1; i < 5; i++) {
t += i;
System.out.println(t);
}
System.out.println(t);
}
}
And the output when running javap -c Tester shows that StringBuilder is being used:
Compiled from "Tester.java"
public class Tester {
public Tester();
Code:
0: aload_0
1: invokespecial #1 // Method java/lang/Object."<init>":()V
4: return
public static void main(java.lang.String[]);
Code:
0: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #3 // String hello
5: invokevirtual #4 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
8: iconst_1
9: istore_1
10: iload_1
11: iconst_5
12: if_icmpge 48
15: new #5 // class java/lang/StringBuilder
18: dup
19: invokespecial #6 // Method java/lang/StringBuilder."<init>":()V
22: ldc #7 // String hi
24: invokevirtual #8 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
27: iload_1
28: invokevirtual #9 // Method java/lang/StringBuilder.append:(I)Ljava/lang/StringBuilder;
31: invokevirtual #10 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
34: astore_2
35: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
38: aload_2
39: invokevirtual #4 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
42: iinc 1, 1
45: goto 10
48: ldc #11 // String me
50: astore_1
51: iconst_1
52: istore_2
53: iload_2
54: iconst_5
55: if_icmpge 90
58: new #5 // class java/lang/StringBuilder
61: dup
62: invokespecial #6 // Method java/lang/StringBuilder."<init>":()V
65: aload_1
66: invokevirtual #8 // Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
69: iload_2
70: invokevirtual #9 // Method java/lang/StringBuilder.append:(I)Ljava/lang/StringBuilder;
73: invokevirtual #10 // Method java/lang/StringBuilder.toString:()Ljava/lang/String;
76: astore_1
77: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
80: aload_1
81: invokevirtual #4 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
84: iinc 2, 1
87: goto 53
90: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
93: aload_1
94: invokevirtual #4 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
97: return
}
I've looked at a handful of questions that tell the story that StringBuilder is generally faster because of synchronization in StringBuffer, and that is substituted for these string concatenations:
When is StringBuffer/StringBuilder not implicitly used by the compiler?
What happens when Java Compiler sees many String concatenations in one line?
Java: String concat vs StringBuilder - optimised, so what should I do?
Best practices/performance: mixing StringBuilder.append with String.concat
StringBuilder vs String concatenation in toString() in Java
StringBuilder and StringBuffer
So, considering the things I've read that show that StringBuilder is generally the better choice leads me to wonder a couple things:
When and why would a compiler choose to use StringBuffer instead of StringBuilder?
Wouldn't it make more sense if the compiler had the choice of using any AbstractStringBuilder implementation?

The specification’s wording you have cited origins from older specifications. The simple answer is, before Java 1.5 aka Java 5 there was no StringBuilder. So older compilers didn’t have to choose between StringBuffer and StringBuilder and the specification at that time simply recommended using what is available.
With Java 5, StringBuilder was introduced which doesn’t use synchronized methods which is perfect for the use case of String concatenation as that’s a pure local operation. So for compilers targeting 1.5 or higher, there is a choice (which is still covered by the specification’s words “or a similar technique”) and they will choose StringBuilder as there is no reason for using StringBuffer when targeting 1.5 or higher.
Starting with Java 9, there is a new technology using an invokedynamic bytecode instruction with a bootstrap method from the StringConcatFactory class, but it’s still covered by the specification’s words “or a similar technique”.

It depends on the compiler implementation (javac is not the only compiler). However, there is never a good case for using StringBuffer over StringBuilder for these types of uses. It is always a single use narrowly scoped object where the synchronization of StringBuffer provides no functional value.
So the short answer is that no compiler /should/ ever use StringBuffer.

What is the performance differences between if and logical operators

I was wondering would there be a performance differences while i use logical operators instead of several if statements. I saw a nice link, does this apply to java also?

I just created class like
class Test{
static java.util.Random r=new java.util.Random();
boolean test(){
return r.nextBoolean();
}
void test1(){
if (test() && test() && test())
System.out.println("3x yes");
}
void test2(){
if (test())
if (test())
if (test())
System.out.println("3x yes");
}
}
compiled it then decompiled by javap -c Test and got these result
class Test {
static java.util.Random r;
Test();
Code:
0: aload_0
1: invokespecial #1 // Method java/lang/Object."<init>":
()V
4: return
boolean test();
Code:
0: getstatic #2 // Field r:Ljava/util/Random;
3: invokevirtual #3 // Method java/util/Random.nextBoole
an:()Z
6: ireturn
void test1();
Code:
0: aload_0
1: invokevirtual #4 // Method test:()Z
4: ifeq 29
7: aload_0
8: invokevirtual #4 // Method test:()Z
11: ifeq 29
14: aload_0
15: invokevirtual #4 // Method test:()Z
18: ifeq 29
21: getstatic #5 // Field java/lang/System.out:Ljava/
io/PrintStream;
24: ldc #6 // String 3x yes
26: invokevirtual #7 // Method java/io/PrintStream.printl
n:(Ljava/lang/String;)V
29: return
void test2();
Code:
0: aload_0
1: invokevirtual #4 // Method test:()Z
4: ifeq 29
7: aload_0
8: invokevirtual #4 // Method test:()Z
11: ifeq 29
14: aload_0
15: invokevirtual #4 // Method test:()Z
18: ifeq 29
21: getstatic #5 // Field java/lang/System.out:Ljava/
io/PrintStream;
24: ldc #6 // String 3x yes
26: invokevirtual #7 // Method java/io/PrintStream.printl
n:(Ljava/lang/String;)V
29: return
static {};
Code:
0: new #8 // class java/util/Random
3: dup
4: invokespecial #9 // Method java/util/Random."<init>":
()V
7: putstatic #2 // Field r:Ljava/util/Random;
10: return
}
As you can see bytecodes of test1 and test2 are same, so there is no difference in using
if (test() && test() && test())
or
if (test())
if (test())
if (test())

It's implementation dependent - so you'll need to benchmark to be sure.
Having said that, most JIT compilers are smart enough to optimise boolean comparisons very effectively so you are unlikely to see any difference.
The only area where logical operators are likely to offer a substantial advantage are in cases where you are using them to perform bitwise computations. This can be very efficient since it can result in branchless code that exploits hardware instructions.

Both forms compile to the same code. Contrary to the suggestions in other answers, this is not an 'optimization', and it would be astonishing if different compilers did different things. There is only one sensible way to compile && and that is by treating it the same as another 'if'. I can't even think of a non-sensible way.

JVM/Java, are method accessibility rules enforced at runtime?

I was curious about how the JVM works. Does the JVM acknowledge method accesibility rules like 'private' protected or is that only done at compile time?
For example, is it possible at around line37 to do some bytecode manipulation and call a protected method, say test3? Normally the compiler would not let me call that method because it is declared protected. But I was curious if that protected rule is enforced at runtime?
u.test1();
// Is it possible at runtime, to call 'test3' through bytecode manipulation
// #line37
package org.berlin.algo.basic.test;
public class RunX {
private String zzz = "rrrrr";
public void test1() {
// Note: I am intentionally use 'new' here as part of my test, not a
// good practice I know but allowed by the language.
Object x = new String("Test1 -----[1.1] " + zzz);
x = new String("Test1 --- [1.2]" + x.toString());
System.out.println(x);
this.test2();
this.test3();
}
/**
* Here, I noticed that the compiler removed my 'test2' code block.
* Does that always happen?
*/
private void test2() {
Object x = new String("Test2#line21--->>> [2.1]");
System.out.println(x);
}
protected void test3() {
Object x = new String("Test3#line27 {Will the JVM enforce the 'protected' method rule for test3? --->>> [3.1]");
x = new String("Test3#line28--->>> [3.2]");
System.out.println(x);
}
public static void main(final String [] args) {
System.out.println("Running");
RunX u = new RunX();
u.test1();
// Is it possible at runtime, to call 'test3' through bytecode manipulation
// #line37
System.out.println("Done");
}
} // End of the Class //
/*
JVM bytecode: javap -v RunX
Compiled from "RunX.java"
public class org.berlin.algo.basic.test.RunX extends java.lang.Object
SourceFile: "RunX.java"
minor version: 0
major version: 50
Constant pool:
const #1 = class #2; // org/berlin/algo/basic/test/RunX
const #2 = Asciz org/berlin/algo/basic/test/RunX;
...
...
const #84 = Asciz SourceFile;
const #85 = Asciz RunX.java;
{
public org.berlin.algo.basic.test.RunX();
Code:
Stack=2, Locals=1, Args_size=1
0: aload_0
1: invokespecial #10; //Method java/lang/Object."<init>":()V
4: aload_0
5: ldc #12; //String rrrrr
7: putfield #14; //Field zzz:Ljava/lang/String;
10: return
LineNumberTable:
line 3: 0
line 5: 4
line 3: 10
LocalVariableTable:
Start Length Slot Name Signature
0 11 0 this Lorg/berlin/algo/basic/test/RunX;
public void test1();
Code:
Stack=5, Locals=2, Args_size=1
0: new #21; //class java/lang/String
3: dup
4: new #23; //class java/lang/StringBuilder
7: dup
8: ldc #25; //String Test1 -----[1.1]
10: invokespecial #27; //Method java/lang/StringBuilder."<init>":(Ljava/lang/String;)V
13: aload_0
14: getfield #14; //Field zzz:Ljava/lang/String;
17: invokevirtual #30; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
20: invokevirtual #34; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
23: invokespecial #38; //Method java/lang/String."<init>":(Ljava/lang/String;)V
26: astore_1
27: new #21; //class java/lang/String
30: dup
31: new #23; //class java/lang/StringBuilder
34: dup
35: ldc #39; //String Test1 --- [1.2]
37: invokespecial #27; //Method java/lang/StringBuilder."<init>":(Ljava/lang/String;)V
40: aload_1
41: invokevirtual #41; //Method java/lang/Object.toString:()Ljava/lang/String;
44: invokevirtual #30; //Method java/lang/StringBuilder.append:(Ljava/lang/String;)Ljava/lang/StringBuilder;
47: invokevirtual #34; //Method java/lang/StringBuilder.toString:()Ljava/lang/String;
50: invokespecial #38; //Method java/lang/String."<init>":(Ljava/lang/String;)V
53: astore_1
54: getstatic #42; //Field java/lang/System.out:Ljava/io/PrintStream;
57: aload_1
58: invokevirtual #48; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
61: aload_0
62: invokespecial #54; //Method test2:()V
65: aload_0
66: invokevirtual #57; //Method test3:()V
69: return
LocalVariableTable:
Start Length Slot Name Signature
0 70 0 this Lorg/berlin/algo/basic/test/RunX;
27 43 1 x Ljava/lang/Object;
protected void test3();
Code:
Stack=3, Locals=2, Args_size=1
0: new #21; //class java/lang/String
3: dup
4: ldc #66; //String Test3#line27 {Will the JVM enforce the 'protected' method rule for test3? --->>> [3.1]
6: invokespecial #38; //Method java/lang/String."<init>":(Ljava/lang/String;)V
9: astore_1
10: new #21; //class java/lang/String
13: dup
14: ldc #68; //String Test3#line28--->>> [3.2]
16: invokespecial #38; //Method java/lang/String."<init>":(Ljava/lang/String;)V
19: astore_1
20: getstatic #42; //Field java/lang/System.out:Ljava/io/PrintStream;
23: aload_1
24: invokevirtual #48; //Method java/io/PrintStream.println:(Ljava/lang/Object;)V
27: return
LocalVariableTable:
Start Length Slot Name Signature
0 28 0 this Lorg/berlin/algo/basic/test/RunX;
10 18 1 x Ljava/lang/Object;
public static void main(java.lang.String[]);
Code:
Stack=2, Locals=2, Args_size=1
0: getstatic #42; //Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #72; //String Running
5: invokevirtual #74; //Method java/io/PrintStream.println:(Ljava/lang/String;)V
8: new #1; //class org/berlin/algo/basic/test/RunX
11: dup
12: invokespecial #76; //Method "<init>":()V
15: astore_1
16: aload_1
17: invokevirtual #77; //Method test1:()V
20: getstatic #42; //Field java/lang/System.out:Ljava/io/PrintStream;
23: ldc #79; //String Done
25: invokevirtual #74; //Method java/io/PrintStream.println:(Ljava/lang/String;)V
28: return
LocalVariableTable:
Start Length Slot Name Signature
0 29 0 args [Ljava/lang/String;
16 13 1 u Lorg/berlin/algo/basic/test/RunX;
}
*/

To the JLS!
15.12.4 Runtime Evaluation of Method Invocation
At run time, method invocation requires five steps. First, a target reference may be computed. Second, the argument expressions are evaluated. Third, the accessibility of the method to be invoked is checked. Fourth, the actual code for the method to be executed is located. Fifth, a new activation frame is created, synchronization is performed if necessary, and control is transferred to the method code.
The wording of the JLS indicates that the accessibility would be checked at runtime.

The JVM does acknowledge these. They can be overridden, by calling setAccessible(true) as Prashant Bhate does, but by default they are enforced. (See http://download.oracle.com/javase/6/docs/api/java/lang/reflect/AccessibleObject.html#setAccessible%28boolean%29.)
By the way, you write that "the compiler doesn't encode type method visibility rules into the Java bytecde file"; but it does. In addition to the above, it has to encode these, for a number of reasons. For example:
you can compile a class A that references class B even if you only have the compiled version of class B.
you can inspect a method's visibility via reflection (the getModifiers() method).
private methods aren't virtual -slash- can't be overridden by subclasses.

If you want to call this method from outside current class you could call private & protected methods using reflection.
Method m = RunX.class.getDeclaredMethod("test3");
m.setAccesible(true);
m.invoke(u);
however you can call this protected (and also private) method directly from main() without any issues.

Oli mentioned it rightly that ultimately you can do anything if you come to extent of byte code manipulation (if done correctly !!!).
Although I will like answer your question of accessibility honor at runtime in Java. If you have any doubts then please go ahead and use reflection to call the private method of one class from other class and you will get your answer. Java creates the function table of class at runtime when loading it and allow the refererence to the functions in limit of accessibility rule. However Java provides facility where you can call the private methods via reflection using setAccessible(true) on the method reference before invoking it.