Incompatible argument to function with ASM bytecode instrumentation - java

I am having some troubles running a simple main program with Guava libraries.
I have instrumented the classes to get the methods parameters using my code from here : Java method parameters values in ASM
The issue is, that while the code works for small projects (aka Tower of Hanoi), with Guava I have errors and exceptions.
In particular, when testing the Joiner.join method, I have this error:
Exception in thread "Jalen Agent" java.lang.VerifyError: (class: com/google/common/base/Joiner, method: withKeyValueSeparator signature: (Ljava/lang/String;)Lcom/google/common/base/Joiner$MapJoiner;) Incompatible argument to function
at Main.joinBench(Main.java:42)
at Main.main(Main.java:20)
And when running the example using -noverify, I have an exception:
Exception in thread "Jalen Agent" java.lang.ArrayIndexOutOfBoundsException: 1
at com.google.common.base.Joiner.<init>(Joiner.java)
at com.google.common.base.Joiner.on(Joiner.java:71)
at Main.joinBench(Main.java:42)
at Main.main(Main.java:20)
The bytecode of the method is consistent:
public static com.google.common.base.Joiner on(java.lang.String);
Code:
0: bipush 1
2: anewarray #4 // class java/lang/Object
5: astore_1
6: aload_1
7: bipush 0
9: aload_0
10: aastore
11: ldc #20 // int 369
13: ldc #21 // String com/google/common/base/Joiner
15: ldc #22 // String on
17: aload_1
18: invokestatic #28 // Method jalen/MethodStats.onMethodEntry:(ILjava/lang/String;Ljava/lang/String;[Ljava/lang/Object;)V
21: new #2 // class com/google/common/base/Joiner
24: dup
25: aload_0
26: invokespecial #32 // Method "<init>":(Ljava/lang/String;)V
29: ldc #20 // int 369
31: invokestatic #36 // Method jalen/MethodStats.onMethodExit:(I)V
34: areturn
I understand that the error may be related to libraries version, but the main java program was compiled against the instrumented library and run using the same jar of the library.
Any ideas on why this is happening? And how it can be solved?
Thanks :)
EDIT
Here are the bytecode of the method withKeyValueSeparator before and after instrumentation
Original bytecode:
public com.google.common.base.Joiner$MapJoiner withKeyValueSeparator(java.lang.String);
Code:
0: new #33 // class com/google/common/base/Joiner$MapJoiner
3: dup
4: aload_0
5: aload_1
6: aconst_null
7: invokespecial #34 // Method com/google/common/base/Joiner$MapJoiner."<init>":(Lcom/google/common/base/Joiner;Ljava/lang/String;Lcom/google/common/base/Joiner$1;)V
10: areturn
Instrumented bytecode:
public com.google.common.base.Joiner$MapJoiner withKeyValueSeparator(java.lang.String);
Code:
0: bipush 1
2: anewarray #4 // class java/lang/Object
5: astore_1
6: aload_1
7: bipush 1
9: aload_1
10: aastore
11: ldc #199 // int 390
13: ldc #21 // String com/google/common/base/Joiner
15: ldc #200 // String withKeyValueSeparator
17: aload_1
18: invokestatic #28 // Method jalen/MethodStats.onMethodEntry:(ILjava/lang/String;Ljava/lang/String;[Ljava/lang/Object;)V
21: new #8 // class com/google/common/base/Joiner$MapJoiner
24: dup
25: aload_0
26: aload_1
27: aconst_null
28: invokespecial #203 // Method com/google/common/base/Joiner$MapJoiner."<init>":(Lcom/google/common/base/Joiner;Ljava/lang/String;Lcom/google/common/base/Joiner$1;)V
31: ldc #199 // int 390
33: invokestatic #36 // Method jalen/MethodStats.onMethodExit:(I)V
36: areturn
Here are the full bytecode of the joiner class :
Original : http://pastebin.com/VsccVX18
Instrumented : http://pastebin.com/xtke1a8y

The original code of withKeyValueSeparator passes a bunch of its arguments to the MapJoiner constructor. You're adding instrumentation code that stores an array in the second slot of the local variable table (using astore_1). This overwrites the first argument to withKeyValueSeparator, which is a String. (The first slot of the local variable table is a MapJoiner instance itself, a.k.a this.) So when the original function's code tries to pass the object in the second slot of the local var table to the constructor, there is that "Incompatible argument" error.
To fix this, you should allocate a new slot in the local variable table for your array; this answer outlines how.

First, I do not see why this should be related to libraries versions. It seems that the bytecode is not instrumented correctly, which causes the verification to fail and the exception if you use -noverify.
Regarding the verification error, it indicates that there is an error in Joiner.withKeyValueSeparator(). The code of this method tries to invoke another method with incompatible method arguments. Could you give the instrumented bytecode of the withKeyValueSeparator() method? (and perferrably the non-instrumented as well)
The error you see with -noverify occurs in the constructor of the Joiner, there seems to be nothing wrong with the Joiner.on() method. Again, could you post the bytecode of the Joiner. method? (instrumented and non-instrumented)

Related

Why does a Java string comparison behave different in Java 15 and Java 11?

Please consider the following class:
class Eq {
public static void main(String[] args) {
System.out.println("" == ".".substring(1));
}
}
The example is supposed to show that multiple copies of the empty string may exist in memory. I still have an old OpenJDK 11 where the program outputs false as expected. Under OpenJDK 15, the program outputs true. The generated bytecode for the class files looks similar (even though they differ in register values):
Java 11:
public static void main(java.lang.String[]);
Code:
0: getstatic #7 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #13 // String
5: ldc #15 // String .
7: iconst_1
8: invokevirtual #17 // Method java/lang/String.substring:(I)Ljava/lang/String;
11: if_acmpne 18
14: iconst_1
15: goto 19
18: iconst_0
19: invokevirtual #23 // Method java/io/PrintStream.println:(Z)V
22: return
Java 15:
public static void main(java.lang.String[]);
Code:
0: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #3 // String
5: ldc #4 // String .
7: iconst_1
8: invokevirtual #5 // Method java/lang/String.substring:(I)Ljava/lang/String;
11: if_acmpne 18
14: iconst_1
15: goto 19
18: iconst_0
19: invokevirtual #6 // Method java/io/PrintStream.println:(Z)V
22: return
I tried to exclude static compiler optimizations by reading "." from stdin but this does not change the outcome. I have tried to disable the JIT via -Djava.compiler=NONE and played around with adjusting the string table size via -XX:StringTableSize=100000. I now have the following questions:
Can someone reproduce the issue (i.e. did I do it correctly? I can provide the class files if that helps)
How do I find out the exact reason for the different behaviour?
What (in your opinion) is the source for the different behaviour?
I think just strategies to approach how to find the reason for the behaviour that don't answer the question might also be interesting.
This is mentioned in the JDK 15 Release Notes.
It was changed as requested by JDK-8240094:
JDK-8240094 : Optimize empty substring handling
String.substring return "" in some cases, but could be improved to do so in all cases when the substring length is zero.
Related:
JDK-8240225 : Optimize empty substring handling
Optimize String.substring and related operations like stripLeading, stripTrailing to avoid redundantly creating a new empty String.
Sub Task:
JDK-8251556 : Release Note: Optimized Empty Substring Handling
The implementation of String.substring and related methods stripLeading and stripTrailing have changed in this release to avoid redundantly creating a new empty String. This may impact code that depends on unspecified behaviour and the identity of empty sub-strings.

Byte Buddy Advice breaks Eclipse debugger

I have used net.bytebuddy.asm.Advice to add code before and after suitably annotated methods, to start and stop timers. The modified classes are manually loaded into the target class loader before their originals can be referenced, thereby supplanting them. I'm using OSGi (Equinox).
Pretty sweet but when I stop the Eclipse (Photon 4.8.0) debugger on a breakpoint in the target method, the Variables view shows only:
com.sun.jdi.InternalException: Got error code in reply:35 occurred
retrieving 'this' from stack frame.
Is this inevitable and unavoidable? Kinda ruins my use-case if this renders instrumented code undebuggable :(
(I have disabled the option "Show method result after a step operation (if supported by the VM; may be slow".)
Example
I believe I may have found some issues with the generated bytecode.
Class to be instrumented:
1 package com.tom.test;
2
3 import com.tom.instrument.Instrumented;
4 import com.tom.instrument.Timed;
5
6 #Instrumented(serviceType = "blah")
7 public class Test {
8
9 #Timed
10 public void writeName() {
11 final String myLocal = "Tom";
12 System.out.println(myLocal);
13 }
14
15 }
"Advice":
package com.tom.instrument;
import net.bytebuddy.asm.Advice.OnMethodEnter;
public class Instrumentation {
#OnMethodEnter
public static void onMethodEnter() {
System.out.println("Enter");
}
}
Call to Byte Buddy:
new ByteBuddy()
.redefine(type, ClassFileLocator.ForClassLoader.of(this.classLoader))
.visit(Advice.to(Instrumentation.class)
.on(isAnnotatedWith(Timed.class)))
.make().saveIn(new File("instrumented"));
Result in javap:
Compiled from "Test.java"
...
public void writeName();
Code:
0: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #40 // String Enter
5: invokevirtual #25 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
8: goto 11
11: ldc #17 // String Tom
13: astore_1
14: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
17: ldc #17 // String Tom
19: invokevirtual #25 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
22: return
LineNumberTable:
line 11: 0
line 12: 14
line 13: 22
LocalVariableTable:
Start Length Slot Name Signature
11 12 0 this Lcom/tom/test/Test;
14 9 1 myLocal Ljava/lang/String;
}
If I set a break point on line 11 of Test.java then the Eclipse Debug view says: <unknown receiving type>(Test).writeName() line: 11
And the Variables view says: com.sun.jdi.InternalException: Got error code in reply:35 occurred retrieving 'this' from stack frame.
If I hack the bytecode changing 00 to 0B at 0x2A2 so the line number table looks like:
LineNumberTable:
line 11: 11
line 12: 14
line 13: 22
Then everything is fine! And that kinda seems correct to me but I'm no expert here.
If I also use #OnMethodExit too then it's a bit more complicated. Add the following to Instrumentation.class:
#OnMethodExit
public static void onMethodExit() {
System.out.println("Exit");
}
javap gives:
Compiled from "Test.java"
...
public void writeName();
Code:
0: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #40 // String Enter
5: invokevirtual #25 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
8: goto 11
11: aload_0
12: astore_1
13: ldc #17 // String Tom
15: astore_2
16: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
19: ldc #17 // String Tom
21: invokevirtual #25 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
24: goto 27
27: getstatic #19 // Field java/lang/System.out:Ljava/io/PrintStream;
30: ldc #42 // String Exit
32: invokevirtual #25 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
35: goto 38
38: return
LineNumberTable:
line 11: 0
line 12: 16
line 13: 24
LocalVariableTable:
Start Length Slot Name Signature
13 14 1 this Lcom/tom/test/Test;
16 11 2 myLocal Ljava/lang/String;
}
To fix this I have to update the line number table and the local variable table. Like this:
LineNumberTable:
line 11: 13
line 12: 16
line 13: 24
LocalVariableTable:
Start Length Slot Name Signature
13 14 0 this Lcom/tom/test/Test;
16 11 1 myLocal Ljava/lang/String;
Diff:
Maybe it's a bug the the Eclipse debugger expects this to always be in slot 0? Or maybe that's the way it should be. The error code 35 comes from the JVM though.
The reason why adding the exit advice changes the slots seem to be because it causes ForInstrumentedMethod.Default.Copying to be used instead of Simple. And they have different implementations of variable().
See Eclipse bug 531706:
The issue occurs when not all classes have been instrumented, see comment #4 by Tobias Hirning:
...
Now I am also getting a clearer picture: The errors only appeared in
method invocations where the methods are in jar files. And I think
they were not instrumented.
...
The error happens in the VM, not in Eclipse. When Eclipse requests the variables via the debugging interface error code 35 is returned instead of the values. The change made due to the mentioned bug report is to ignore it, see comment #7 by Till Brychcy (who made the change):
...
I've been able to reproduce the problem and simply ignoring the
InternalException in this codepath improves the situation.
You'll sometimes see a message about the error code 35 in the
variables view, but in general it seems to work.
To avoid this issue, you have to instrument all classes.
Workaround
Aha! All my fishing around looking to make a patch has lead me to find something that I wish I had found sooner.
I can use #OnMethodEnter(prependLineNumber = false) to avoid the issue with the line numbers.
And using #OnMethodExit(backupArguments = false) avoids the problem with the slots.
This is good news for me! However, presumably these are not the defaults for good reason. I don't yet understand if there are important negative consequences of using these options.

Copy Bytecode of Method to Another Method Using Javassist

I am using Javassist to manipulate the bytecode of a .class file. i have to retrieve & modify the bytecode of existing method and write the new bytecode to new method.
Suppose i have a method Method_old() in my class file having the following bytecode
0: bipush 10
2: istore_1
3: bipush 20
5: istore_2
6: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
9: iload_1
10: invokevirtual #11 // Method java/io/PrintStream.println:(I)V
13: return
and i want to modify it as given below,
0: bipush 10
2: istore_1
3: getstatic #2 // Field java/lang/System.out:Ljava/io/PrintStream;
6: iload_1
7: invokevirtual #11 // Method java/io/PrintStream.println:(I)V
8: return
and Store it to a new Method named method_new(). How Can I do so using the Javassist.

Eclipse EcLemma number of covered instructions more than lines of code

In the coverage result, it shows that I've covered 9 instructions while there are only 5 lines highlighted green. Which are the other 4 instructions?
Click the dropdown arrow at the top right of the Coverage box. It'll give you a couple different ways to measure your coverage. The default seems to be instructions (bytecode instructions), but you can manually select lines.
The reason you are seeing 9 instructions is because there are 9 bytecode instructions in Foo:
$ javap -c Foo.class
Compiled from "Foo.java"
public class Foo {
public Foo();
Code:
0: aload_0
1: invokespecial #8 // Method java/lang/Object."<init>":()V
4: return
public static void main(java.lang.String[]);
Code:
0: getstatic #16 // Field java/lang/System.out:Ljava/io/PrintStream;
3: ldc #22 // String Test
5: invokevirtual #24 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
8: new #1 // class Foo
11: invokespecial #30 // Method "<init>":()V
14: return
}
As #schmosel says, it is counting bytecode instructions.
You can verify this by reading the EMMA reference documentation (EclEMMA is an Eclipse GUI wrapped around EMMA), in which the phrase "bytecode instructions" is used throughout.

Why doesn't the Java 7 byteode verifier choke on this?

I'm working on code that calculates entries in the StackFrameMap (SFM). The goal is to be able to generate (SFM) entries that make the Java 7 bytecode verifier happy. Following a TDD methodology, I started by creating bogus SMF entries for the verifier to complain about; I would the replace these with my properly-calculated entries to see that I was doing it correctly.
The problem is: I can't get the bytecode verifier to complain. Here is an example, starting with the original Java code (this code is not supposed to do anything useful):
public int stackFrameTest(int x) {
if (x > 0) {
System.out.println("positive x");
}
return -x;
}
This generates the following bytecode (with SFM):
public int stackFrameTest(int);
flags: ACC_PUBLIC
Code:
stack=2, locals=2, args_size=2
0: iload_1
1: ifle 12
4: getstatic #47 // Field java/lang/System.out:Ljava/io/PrintStream;
7: ldc #85 // String positive x
9: invokevirtual #55 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
12: iload_1
13: ineg
14: ireturn
StackMapTable: number_of_entries = 1
frame_type = 12 /* same */
Now, I change the SFM to contain this:
StackMapTable: number_of_entries = 1
frame_type = 255 /* full_frame */
offset_delta = 12
locals = [ double, float ]
stack = [ double ]
As you can see, that is completely bogus, but it loads without error. I read the JVM spec, and I couldn't see any reason why this would work. I'm not using the SplitBytecodeVerifier option.
EDIT: Per the accepted answer below, Eclipse had been set to emit Java 6 class files (version 50.0). Classfiles of this this version will quietly ignore issues with the StackFrameMap. After changing the setting to use the default Java 7 classfile format (51.0), it worked as expected.
I am unable to reproduce your results. I tried modifying the stack frame and it failed to load as expected. If you want, I can post my modified classfile.
It's not clear what happened, but you've almost certainly made a mistake somewhere. The most likely explanation is that your classfile has version 50.0, in which case the JVM will fall back to normal verification when the stackmap is invalid. You need to set the version to 51.0 to force stackmap validation. Another possibility is that you simply messed up editing the file and didn't actually save the changes or didn't make the changes you thought you did.
Here's the assembly for my modified classfile.
.version 51 0
.class super public StackFrameTest4
.super java/lang/Object
.method public <init> : ()V
.limit stack 1
.limit locals 1
aload_0
invokespecial java/lang/Object <init> ()V
return
.end method
.method static public main : ([Ljava/lang/String;)V
.limit stack 2
.limit locals 1
new StackFrameTest
dup
invokespecial StackFrameTest <init> ()V
bipush 42
invokevirtual StackFrameTest stackFrameTest (I)I
pop
return
.end method
.method public stackFrameTest : (I)I
.limit stack 2
.limit locals 2
iload_1
ifle L12
getstatic java/lang/System out Ljava/io/PrintStream;
ldc 'positive x'
invokevirtual java/io/PrintStream println (Ljava/lang/String;)V
L12:
.stack full
locals Double Float
stack Double
.end stack
iload_1
ineg
ireturn
.end method

Categories

Resources