SpringBoot openJ9 Kubernetes Crash InterpreterEmulator.cpp:208 - java

I'm randomly getting this crash recently and I'm not sure how to debug this. Can anyone please point me in the right direction.
Tools I have used are,
Spring Boot 2 + Kubernetes + adopt openj9 jdk13
Assertion failed at /home/jenkins/workspace/build-scripts/jobs/jdk13u/jdk13u-linux-x64-openj9/workspace/build/src/build/linux-x86_64-server-release/vm/compiler/../compiler/optimizer/InterpreterEmulator.cpp:208: 0
VMState: 0x000501ff
unexpected bytecode in thunk archetype 0x7f98f6483ee0 at bcIndex 40 JBdload2 (41)
compiling java/lang/invoke/AsTypeHandle.invokeExact_thunkArchetype_X(Ljava/lang/Object;Ljava/lang/Object;)V at level: warm
0:/opt/java/openjdk/lib/compressedrefs/libj9jit29.so(+0x798015) [0x7f9962316015]
1: /opt/java/openjdk/lib/compressedrefs/libj9jit29.so(+0x7a2dc0) [0x7f9962320dc0]
JVMDUMP039I Processing dump event “abort”, detail “” at 2020/05/12 16:35:10 - please wait.
JVMDUMP032I JVM requested System dump using ‘/home/springboot/core.20200512.163510.1.0001.dmp’ in response to an event

TLDR: This is fixed in the v.0.20.0 release (April 2020) of OpenJ9. The latest release binaries from AdoptOpenJDK for JDK 8, 11 & 14 contain the fix.
The assertion failure is an assertion JIT compiling the MethodHandle (MH) code. A new abstract interpreter was added to give the JIT a better view of the MH "thunkArchetype" code and it missed some of the bytecodes that could occur in MH thunkArchetypes.
See https://github.com/eclipse/openj9/pull/8598 for more details of the fix.

Related

JavaFX-11/macOS: DirectoryChooser crashes 'NSObjectNotAvailableException'

I develop a java application using openjfx modules.
I now want to use DirectoryChooser to let the user select several files as an input (code below). Everytime I start the application from the IDE it crashes with the exception below. By crashing I mean totally terminates (not freezing).
I also tried some sample code like this to figure out if there is anything wrong with my implementation but each sample I found gave me the same issue.
I'm unsure if this is a general issue/bug because I saw old bugs, in the oracle bug tracker but they where all marked as fixed.
I would be happy if someone could provide some help on this. I would even be happy with a workaround that just avoids DirectoryChooser if there is any.
Steps to reproduce:
create a small Application with the demo code from here.
run the application
try to open directory chooser
Background information:
OS: macOS 10.15.2 Catalina
IDE: intelliJ IDEA Ultimate 2019.3
Exception
2020-01-30 17:18:31.288 java[49749:1319417] *** Terminating app due to uncaught exception 'NSObjectNotAvailableException', reason: 'GlassOpenPanel is not a supported subclass for sandboxing'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff364ea8ab __exceptionPreprocess + 250
1 libobjc.A.dylib 0x00007fff6c75b805 objc_exception_throw + 48
2 CoreFoundation 0x00007fff364ea701 +[NSException raise:format:] + 189
3 AppKit 0x00007fff33d43aa9 +[NSSavePanel(Instantiation) _crunchyRawUnbonedPanel] + 368
4 libglass.dylib 0x00000001370a2ba6 Java_com_sun_glass_ui_mac_MacCommonDialogs__1showFolderChooser + 358
5 ??? 0x0000000117695950 0x0 + 4687747408
6 ??? 0x000000011768fa00 0x0 + 4687723008
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Solution
As suggested by mipa increasing the OpenJFX version number solved the problem.
At the moment, it seems working for OpenJFX version 14-ea+7 and 15-ea+1 from MavenCentral
Why do you use an outdated Version of JavaFX? The current version is 13 and 14 is coming out shortly. EA releases are already available. You can also use them with Java 11.
I tried two of the examples you mentioned with JavaFX 13 and they worked without problem. This may be due to the fact that a bug has been fixed or due to the fact that I am still using MacOS 10.14.6 Mojave.
Give JavaFX 13 a try and then report back.

Error linking JNA library on AdoptOpenJDK on MacOS

I am trying to run some JUnit tests over cassandra. But I get the following error:
[08/12/19 10:48:40:411](main)([]) INFO - c.h.c.c.e.EmbeddedCassandra - Starting embedded Cassandra server.
8/12/19 10:48:41:497](main)([]) ERROR - o.a.c.u.NativeLibraryDarwin - Failed to link the C library against JNA. Native methods will be unavailable.
java.lang.UnsatisfiedLinkError: /private/var/folders/ty/wl4gxf352m328101m101hwh40000gn/T/jna--321969061/jna10641195286884112036.tmp: dlopen(/private/var/folders/ty/wl4gxf352m328101m101hwh40000gn/T/jna--321969061/jna10641195286884112036.tmp, 1): no suitable image found. Did find:
/private/var/folders/ty/wl4gxf352m328101m101hwh40000gn/T/jna--321969061/jna10641195286884112036.tmp: code signature in (/private/var/folders/ty/wl4gxf352m328101m101hwh40000gn/T/jna--321969061/jna10641195286884112036.tmp) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.
It was running well until I changed to mac mojave and re-setup everything. I think it is an issue related to permissions may be or JNA?
IDE: IntelliJ
Java: AdoptOpenJDK 11.0.4
JNA: 4.2.2
Any kind of help will be highly appreciated!
This is the result of a bug in AdoptOpenJDK jdk-11.0.4+11 on macOS, persisting through 11.2.
It will be fixed in the jdk-11.0.4+11.3 release.
If you can't wait for the new release you can temporarily resolve by downgrading to 11.0.3+7
What version of cassandra unit? Support for java 11 (https://issues.apache.org/jira/browse/CASSANDRA-9608) isn't in until cassandra 4 and I dont think embedded cassandra is setup for that yet

Extracting information from a a java core dump with jmap(1.5)

Long story short, some coworkers are running a pretty old setup(oc4j jdk1.5.6 in x86_64) with an application which happens to be mission critical. They recently have tried to deploy a new version of the application, but as soon as they do the java process(es) throw a core dump and die.
The problem is, the core dumps seem to be fine, gdb can open them, but jmap and other tools refuse to process them:
# /usr/java/jdk1.5.0_06/bin/jmap /usr/java/jdk1.5.0_06/bin/java core
Attaching to core core from executable /usr/java/jdk1.5.0_06/bin/java, please wait...
Error attaching to core file: Can't attach to the core file
And newer versions throw a exception:
# jdk1.6.0_45/bin/jmap /usr/java/jdk1.5.0_06/bin/java core
Attaching to core core from executable /usr/java/jdk1.5.0_06/bin/java, please wait...
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.tools.jmap.JMap.runTool(JMap.java:179)
at sun.tools.jmap.JMap.main(JMap.java:110)
Caused by: sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 20.45-b01. Target VM is 1.5.0_06-b05
at sun.jvm.hotspot.runtime.VM.checkVMVersion(VM.java:224)
at sun.jvm.hotspot.runtime.VM.<init>(VM.java:287)
at sun.jvm.hotspot.runtime.VM.initialize(VM.java:357)
at sun.jvm.hotspot.bugspot.BugSpotAgent.setupVM(BugSpotAgent.java:594)
at sun.jvm.hotspot.bugspot.BugSpotAgent.go(BugSpotAgent.java:494)
at sun.jvm.hotspot.bugspot.BugSpotAgent.attach(BugSpotAgent.java:348)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:169)
at sun.jvm.hotspot.tools.PMap.main(PMap.java:67)
... 6 more
gdb offers little information without symbols:
Reading symbols from /usr/java/jdk1.5.0_06/bin/java...(no debugging symbols found)...done.
[New Thread 9841]
[New Thread 31442]
[New Thread 31441]
...
Core was generated by `/usr/java/jdk1.5.0_06/bin/java -server -XX:+UseConcMarkSweepGC -XX:MaxHeapFreeR'.
Program terminated with signal 6, Aborted.
#0 0x0000003bbf030285 in ?? ()
(gdb) bt
#0 0x0000003bbf030285 in ?? ()
#1 0x0000003bbf031d30 in ?? ()
#2 0x0000000000000000 in ?? ()
The only valuable information I've gathered from the core is that most threads
are blocked(I'm far from being a gdb guru):
35 Thread 10093 0x0000003bbfc0b1c0 in pthread_cond_timedwait##GLIBC_2.3.2 ()
from /lib64/libpthread.so.0
34 Thread 10097 0x0000003bbfc0b1c0 in pthread_cond_timedwait##GLIBC_2.3.2 ()
from /lib64/libpthread.so.0
33 Thread 10099 0x0000003bbfc0b1c0 in pthread_cond_timedwait##GLIBC_2.3.2 ()
from /lib64/libpthread.so.0
Besides, I don't know if it's really relevant. The app is almost always heavily loaded, and my bet is that there were some lock contention already but since it's another's team app my knowledge about it it's pretty shallow.
I guess this is a long shot, but is there something that we can do to get a java thread dump or something like that? Do Sun used to offer debuginfo of the jdk as I guess is avalaible now with openjdk?
Thanks in advance.
UPDATE: The other team has resolved the issue without getting info from the core dump, just by trial and error after successfully replicating the problem in a test system. I'm still intrigued about the thing: how to debug an ancient java core dump which jmap can't process, it might be valuable info for the future, althought it seems is that there is no solution to that problem. Probably the JVM memory got corrupted and that's why jmap can't process it.
You can add the following JVM option when starting your application, that will allow you to run any command you specify if a fatal JVM error occurs:
-XX:OnError="<cmd args>"
For instance, you could run a command (or a script) that will perform certain actions like get a heap or thread dump.
Jmap and other JVM utilities are extremely version sensitive. From your error, it is self explanatory that hopefully the same jvm is not used in your case.
Java VisualVM can load core dumps directly. But you must use the same jvm that created the core file.
Resource Link:
https://stackoverflow.com/a/9981498/2293534
Suggestion#1:
kjkoster has given a solution here in this tutorial.
You need to use the jmap that comes with the JVM. From your error
message I gather that you are using a different version of jmap than
of the JVM.
Please check what JVMs are installed on your machine and ensure than
when you run jmap, you use the right version.
To solve such issues I never rely on path. Instead I set JAVA_HOME to
be the one that the JVM uses and then invoke both the JVM and jmap
like so:
Code:
$ JAVA_HOME=/usr/local/jdk1.6.0
$ export JAVA_HOME
$ ${JAVA_HOME}/bin/java ...
...
$ ${JAVA_HOME}/bin/jmap ...
Hope this helps.
Suggestion#2:
It is a full solution step by step is given by Chamilad. Hope it will clarify your root cause and solution procedure.
Almost every Java developer knows about jmap and jstack tools that come with the JDK. These provide functionality to extract heap and thread information of a running JVM instance. Easy.
What if there’s a running JVM that has produced a deadlock and you want to take a thread dump while the process is running? You go in and run the following.
jstack pid >> thread_dump.txt
Turns out the system doesn’t know what jstack is. You don’t panic, but you get a tiny sensation at the back of your head saying you’re not leaving early this Friday.
What has happened is the running JVM is based on a JRE and not a JDK. The JRE is a minimal runtime that doesn’t pack the monitoring and analysis tools that the JDK packs.
So what are our options here?
Stop the process. Download JDK, start the process again on top of
JDK and hope the deadlock happens again. Nope.
Start JVisualVM on your laptop and hope the process has JMX enabled. Nope.
tools.jar TO THE RESCUE!
Functionalities such as jstack are implemented in the tools.jar file which is packed inside <JDK_HOME>/lib folder. We can use this to invoke the JStack class and get a thread dump of the running process.
So we march on to download and extract the JDK, and then to run the following.
java -classpath <JDK_HOME>/lib/tools.jar sun.tools.jstack.JStack <pid> >> thread_dump.txt
..and come across the following error.
Exception in thread "main" java.lang.UnsatisfiedLinkError: no attach in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at sun.tools.attach.LinuxVirtualMachine.<clinit>(LinuxVirtualMachine.java:342)
at sun.tools.attach.LinuxAttachProvider.attachVirtualMachine(LinuxAttachProvider.java:63)
at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:208)
at sun.tools.jstack.JStack.runThreadDump(JStack.java:163)
at sun.tools.jstack.JStack.main(JStack.java:116)
Darn it! Spoiled again!
How do we solve this? The above error is caused when the process can’t find libattach.so file which is related to the Dynamic Attach function related to JStack. Setting the following environment variable will help the JVM to find the libattach.so file.
export LD_LIBRARY_PATH=<JDK_HOME>/jre/lib/amd64/
Now let’s run JStack again, this time with results!
java -classpath <JDK_HOME>/lib/tools.jar sun.tools.jstack.JStack <pid> >> thread_dump.txt
Now that we have the thread dump, we move on to the heap dump. The tool we normally use is jmap but that too is not available on the JRE. So what? We can use the binary in the JDK’s bin directory right? right?
root#snowflake1 latest]# <JDK_HOME>/bin/jmap -heap <pid>
Attaching to process ID <pid>, please wait…
Error attaching to process: sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 25.102-b14. Target VM is 25.91-b14
sun.jvm.hotspot.debugger.DebuggerException: sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 25.102-b14. Target VM is 25.91-b14
at sun.jvm.hotspot.HotSpotAgent.setupVM(HotSpotAgent.java:435)
at sun.jvm.hotspot.HotSpotAgent.go(HotSpotAgent.java:305)
at sun.jvm.hotspot.HotSpotAgent.attach(HotSpotAgent.java:140)
at sun.jvm.hotspot.tools.Tool.start(Tool.java:185)
at sun.jvm.hotspot.tools.Tool.execute(Tool.java:118)
at sun.jvm.hotspot.tools.HeapSummary.main(HeapSummary.java:49)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.tools.jmap.JMap.runTool(JMap.java:201)
at sun.tools.jmap.JMap.main(JMap.java:130)
Caused by: sun.jvm.hotspot.runtime.VMVersionMismatchException: Supported versions are 25.102-b14. Target VM is 25.91-b14
at sun.jvm.hotspot.runtime.VM.checkVMVersion(VM.java:227)
at sun.jvm.hotspot.runtime.VM.<init>(VM.java:294)
at sun.jvm.hotspot.runtime.VM.initialize(VM.java:370)
at sun.jvm.hotspot.HotSpotAgent.setupVM(HotSpotAgent.java:431)
… 11 more
Nope! Unless you match the JDK version with the exact version the JRE is at you get the above issue (which is pretty self-explanatory). So we download the JDK of the JRE of on top our process is running and run jmap again.
<JDK_HOME>/bin/jmap -dump:file=heap_dump.hprof <pid>
Resource Link:
Extracting memory and thread dumps from a running JRE based JVM
jmap is not going to help you debug a core dump. The JVM dumps core when either it has a bug or you have JNI code with a problem. Organizations with mission-critical applications should, sadly, see upgrading from unsupported versions of the JVM as mission-critical, or be prepared to pay Oracle a fortune for help.

Gradle aapt error: syntax ")" not found on Android Studio 1.5.1 Linux

Unburden
First of all, I would like to unburden: Android Studio completely sucks and it is by far the worst big IDE I've used all my life long! I got stuck on errors just for opening that thing! And when I have hardly fixed one, seventeen new ones come rightly from the hell to my screen! I am (trying) to use that piece of [you know] for 4 days now, and till now I was not able to code one single line of code! What does an IDE is supposed to be useful for again?
-- Edit1: I just installed the full 1.4.1 version right now and first thing it showed me was: WizardException: Ignoring unknown package filter 'source-25' and others errors.
-- Edit2: I did a new FRESH Mint 18 install in a completely clean partition, installed only Java 7 (Oracle) and tried to run the AS-1.5.1 but... the same error on install and a subtle different log of error messages (please, see the log here) when gradleing.
The problem
Ok, the last problem (1.5.1) is that now I am trying to make gradle to build the "project" (it is just an empty activity created by the wizard, default code untouched) and I get this error:
~/AndroidStudioProjects/MyApplication $ gradle clean build --stacktrace
Incremental java compilation is an incubating feature.
:clean UP-TO-DATE
:app:clean
//...[other succesfull msgs here]...
:app:mergeDebugResourcesAAPT err(Facade for 16120822): /home/sidney/Android/Sdk/build-tools/25.0.0/aapt: 3: /home/sidney/Android/Sdk/build-tools/25.0.0/aapt: Syntax error: Unterminated quoted string
AAPT err(Facade for 22545775): /home/sidney/Android/Sdk/build-tools/25.0.0/aapt: 3: /home/sidney/Android/Sdk/build-tools/25.0.0/aapt: Syntax error: Unterminated quoted string
//...[other repeated errors here]...
Exception in thread "png-cruncher_8" java.lang.RuntimeException: Timed out while waiting for slave aapt process, make sure the aapt execute at /home/sidney/Android/Sdk/build-tools/25.0.0/aapt can run successfully (some anti-virus may block it) or try setting environment variable SLAVE_AAPT_TIMEOUT to a value bigger than 60 seconds
at com.android.builder.png.AaptProcess.waitForReady(AaptProcess.java:108)
at com.android.builder.png.QueuedCruncher$1.creation(QueuedCruncher.java:110)
at com.android.builder.tasks.WorkQueue.run(WorkQueue.java:203)
at java.lang.Thread.run(Thread.java:745)
//...[other repeated errors here]...
--- UPDATE: Please, see this other log of the external terminal used to run AS ---
What I've tried to solve that:
Install aapt system-wide;
Switch from Java 8 to Java 7;
Change gradle to off-line;
Change JDK configs to switch from the AS one to the system one;
Set SLAVE_AAPT_TIMEOUT to 30 (I changed back as you can see in the log link)
Set ANDROID_EMULATOR_FORCE_32BIT to true;
Install via ubuntu-make and also via ppa;
And so many others things I could find over Internet.
Environment specs
Linux Mint 18 (32 bits)
AS 1.5.1 (now (trying to use) 2.1.0)
Java Oracle 8 (Already tried the 7)
Why don't I just switch to Eclipse with plug-in
I have a course in mind (in Udacity) and they use that "thing" and it is a prerequisite. I've sent 3 e-mails to them questioning about the possibility of use other IDEs but without answer after 4 days, unfortunately.
Thank you so much for any help!
That error looks like your build tools aren't properly installed/set up. The aapt distributed with the build tools is versioned/linked with the tools, so studio probably isn't going to grab the random system level one. Have you attempted to install them, or did the older Studio version fail to prompt you to do so?
Beyond that, I'm afraid that Studio 1.4 is very deprecated and was missing many, many fixes that are in the newer versions, so it's somewhat likely that you would run into random errors with that version :-( What errors were you encountering with 2.2.1? It would be easier for the Studio team to help address those versus anything you find in 1.4.
My solution? Buy a 64 bits laptop! No installation error, no gradle error, no java error, no error anymore.

After SonarQube 5.2 upgrade java analysis fails with java.lang.IllegalArgumentException: There's no changeset on line

This is an analysis that I have been running for a while.
The SCM is Git.
I upgraded SonarQube to 5.2 (from http://downloads.sourceforge.net/project/sonar-pkg/deb as I am using Ubuntu LTS 14.04.3)
sonar.log now gives:
2015.11.17 10:43:00 ERROR web[o.s.s.c.t.CeWorkerRunnableImpl] Executed task | project=energy:energy | id=AVEVClvzTc_W2Q8I5ipV | time=647ms
Through the web interface I tracked down:
http://localhost:9000/api/ce/logs?taskId=AVEVClvzTc_W2Q8I5ipV
which gave:
java.lang.IllegalArgumentException: There's no changeset on line 170
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:125) ~[guava-17.0.jar:na]
at org.sonar.server.computation.scm.ScmInfoImpl.getChangesetForLine(ScmInfoImpl.java:64) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.scm.ReportScmInfo.getChangesetForLine(ReportScmInfo.java:71) ~[sonar-server-5.2.jar:na]
at org.sonar.server.computation.step.NewCoverageMeasuresStep$NewCoverageCounter.initialize(NewCoverageMeasuresStep.java:391) ~[sonar-server-5.2.jar:na]
(It would be an improvement if the error message told you the file name in which it thinks line 170 is wrong.)
Is there anything to be done other than wait for version 5.3 ?
I had the same problem for the Java plugin. I solved it by executing an analysis with SCM support disabled:
sonar.scm.disabled = true
Note that I didn't have to disable it permanently, only for one analysis. After that it worked fine again.
I had the same error but the problem turned out to be the javascript plugin. I was running the latest version (2.8) installed through the update centre. After removing the plugin, the analysis succeeded but now I have no metrics on the Javascript code, which is roughly have the project code base.
Incidentally, the maven sonar plugin (2.7.1) reports "ANALYSIS SUCCESSFUL" even though the analysis actually failed.
EDIT: I noticed a new version (2.9) of the Javascript plugin. Analysis succeeded after the upgrade.

Categories

Resources