I am running built in sample example which comes as part of Spark installation, and running in Hadoop 2.7 + Spark with JDK 8. However it is giving me the following error:
Exception in thread "main" java.lang.OutOfMemoryError: Cannot allocate
new DoublePointer(10000000): totalBytes = 363M, physicalBytes = 911M
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.OutOfMemoryError: Physical memory usage is too
high: physicalBytes = 911M > maxPhysicalBytes = 911M
at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:572)
at org.bytedeco.javacpp.Pointer.init(Pointer.java:121)
I followed the following SO question as well, and did the configuration changes.
In addition to this, I referred to these links as well: YARN-4714 , HADOOP-11090
Are there any issues in running Spark in JDK 8.
The below are the versions of the softwares that I am running in my simple cluster:
jdk-8u131-linux-x64
scala-2.12.2
spark-2.1.1-bin-without-hadoop
hadoop-2.7.0
One thing when I run the program in JDK 7, it is working fine, but failing with JDK 8.
Has anyone encountered this problem and if so, what is the fix? Isn't hadoop , spark, scala not yet compatible with JDK 8?
Can anyone please help me?
You are receiving OOM error is the indication of shortage of memory for java to launch. As you have mentioned JDK7 it worked fine. Upgrading to JDK8 require more memory compare to JDK7. Please check JDK8 memory requirements here - https://dzone.com/articles/java-8-permgen-metaspace
Spark is not yet released for Scala 2.12. I don't know if it solves the original problem but you should switch to Scala 2.11 anyway.
Related
Using journey browser project I have set up a simple maven project in eclipse, using the dependency provided on the page, I have tried to run the code example (also provided on the page)
And I get an NPE:
Exception in thread "main" java.lang.ExceptionInInitializerError
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:588)
at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:583)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/java.awt.Toolkit.getDefaultToolkit(Toolkit.java:582)
at java.desktop/java.awt.Toolkit.getEventQueue(Toolkit.java:1494)
at java.desktop/java.awt.EventQueue.isDispatchThread(EventQueue.java:1086)
at java.desktop/javax.swing.SwingUtilities.isEventDispatchThread(SwingUtilities.java:1493)
at com.codebrig.journey.JourneyBrowserView.<init>(JourneyBrowserView.java:78)
at com.codebrig.journey.JourneyBrowserView.<init>(JourneyBrowserView.java:71)
at JourneyBrowser.main(JourneyBrowser.java:13)
Caused by: java.lang.NullPointerException
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2646)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
at java.base/java.lang.System.loadLibrary(System.java:1870)
at java.desktop/sun.awt.windows.WToolkit$1.run(WToolkit.java:118)
at java.desktop/sun.awt.windows.WToolkit$1.run(WToolkit.java:115)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/sun.awt.windows.WToolkit.loadLibraries(WToolkit.java:114)
at java.desktop/sun.awt.windows.WToolkit.<clinit>(WToolkit.java:129)
... 12 more
Can anyone explain what is happening and why here, I have tried this on 64bit windows with java Coretto 11(jdk11.0.7_10),I initially found a bug here, relating to loadLibray in open JDK and thought that maybe the problem, I just don't have a good enough understanding to work out how to get around it?
I'm not sure, but I think that Corretto bug is probably the one that is causing the problem. As noted, it is from their "upstream"; i.e the OpenJDK codebase. It was due to regression that appeared in jdk11.0.7 due to a backport of a fix for another problem. Apparently, the fix changes some internal JDK fields and that breaks application code. As JDK-8240521 puts it:
The backport of the JDK-8231584 changes internal JDK fields processing. The problem is that the many third-party applications copy-pasted a hack that depends on particular JDK implementation.
If I am reading the Oracle bug entries correctly, the reversion of the broken fix should be in JDK11.0.8. Alternatively, an earlier JDK 11 release than 11.0.7 shouldn't have the broken fix.
Let me know if changing your Java 11 install solves the problem. (If not, I'll see if I can get the line numbers to match up.)
I am using Windows 10 and I have installed Cassandra 3.9.0 and my updated JDK version is java version "1.8.0_211"
When I run the command cassandra -f
it stops giving me this error:
Exception (java.lang.AbstractMethodError) encountered during startup: org.apache.cassandra.utils.JMXServerUtils$Exporter.exportObject(Ljava/rmi/Remote;ILjava/rmi/server/RMIClientSocketFactory;Ljava/rmi/server/RMIServerSocketFactory;Lsun/misc/ObjectInputFilter;)Ljava/rmi/Remote;
java.lang.AbstractMethodError: org.apache.cassandra.utils.JMXServerUtils$Exporter.exportObject(Ljava/rmi/Remote;ILjava/rmi/server/RMIClientSocketFactory;Ljava/rmi/server/RMIServerSocketFactory;Lsun/misc/ObjectInputFilter;)Ljava/rmi/Remote;
at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:150)
at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:135)
at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:405)
at org.apache.cassandra.utils.JMXServerUtils.createJMXServer(JMXServerUtils.java:106)
at org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:145)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:219)
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730)
ERROR 12:54:22 Exception encountered during startup
java.lang.AbstractMethodError: org.apache.cassandra.utils.JMXServerUtils$Exporter.exportObject(Ljava/rmi/Remote;ILjava/rmi/server/RMIClientSocketFactory;Ljava/rmi/server/RMIServerSocketFactory;Lsun/misc/ObjectInputFilter;)Ljava/rmi/Remote;
at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:150) ~[na:1.8.0_161]
at javax.management.remote.rmi.RMIJRMPServerImpl.export(RMIJRMPServerImpl.java:135) ~[na:1.8.0_161]
at javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:405) ~[na:1.8.0_161]
at org.apache.cassandra.utils.JMXServerUtils.createJMXServer(JMXServerUtils.java:106) ~[apache-cassandra-3.9.0.jar:3.9.0]
at org.apache.cassandra.service.CassandraDaemon.maybeInitJmx(CassandraDaemon.java:145) [apache-cassandra-3.9.0.jar:3.9.0]
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:219) [apache-cassandra-3.9.0.jar:3.9.0]
at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:601) [apache-cassandra-3.9.0.jar:3.9.0]
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:730) [apache-cassandra-3.9.0.jar:3.9.0]
I have gone through many posts but none of them solves my problem. Please suggest. and let me know if more information is required.
see: https://issues.apache.org/jira/browse/CASSANDRA-14173
org.apache.cassandra.utils.JMXServerUtils which is used to
programatically configure the JMX server and RMI registry
(CASSANDRA-2967, CASSANDRA-10091) depends on some JDK internal
classes/interfaces. A change to one of these, introduced in Oracle JDK
1.8.0_162 is incompatible, which means we cannot build using that JDK version. Upgrading the JVM on a node running 3.6+ will result in
Cassandra being unable to start.
Upgrade to newer version of Cassandra (3.11.4 to date) or downgrade jdk first and the problem will be resolved
I got this error message in the console when I tried to run, I just switched laptops and I've wiped my old one. This project has always been stored in a USB so I use that to run it on my new laptop. (So I had to re-download all the applications), I need this to fix so bad due to its my project for class and I haven't record it and I'm supposed to do that and I don't want to fail the class please could u help thank u:
java.lang.NoClassDefFoundError: com/apple/eawt/QuitHandler
at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3119)
at java.base/java.lang.Class.getMethodsRecursive(Class.java:3260)
at java.base/java.lang.Class.getMethod0(Class.java:3246)
at java.base/java.lang.Class.getMethod(Class.java:2065)
at processing.core.PApplet.runSketch(PApplet.java:10751)
at processing.core.PApplet.main(PApplet.java:10548)
at processing.core.PApplet.main(PApplet.java:10530)
at robothead.Robothead.main(Robothead.java:8)
Caused by: java.lang.ClassNotFoundException: com.apple.eawt.QuitHandler
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499)
... 9 more
I at first download JDK 10 and I read it only works with JDK 8 so I download that and delete JDK 10 in hopes that stuff will return back to normal and it does not works. Also the project, I did it in java oxygen and I downloaded photon on my new laptop and click on the yes I want to convert the files thing so I decided to make a new workspace and copy paste the code but still got that error?Before it was MacBook Air and I switch to the MacBook Pro.
You are mixing up a lot of things.
The basic problem is that starting with Java 9 oracle made changes that lead to such errors ( see https://docs.oracle.com/javase/9/migrate/toc.htm#JSMIG-GUID-C86F7403-7D7C-456F-8873-18C7F1BEDE2F).
So, in order to get this to work, you have to ensure that Java 8 is used. And that your Java 10 installation was really, fully removed. You should then check your eclipse settings to also ensure that it is pointing to Java 8.
I use ScrumWorks Pro on my Mac OS X,but an error occurred and the main error message is：
java.lang.ClassNotFoundException: apple.laf.AquaTableUI
I installed the oracle JDK8 and java version is 1.8.0_111.
I searched for a long time and hardly had this solution.But I found AquaTableUI class in openjdk8.
I want to know if apple.laf.AquaTableUI is only available in openjdk,but not in Oracle.If not, is there a solution?
Looking forward to your answer,Thanks.
I am running JavaExample.java provided with Z3 4.4.2 but I get the following output and then error:
Z3 Major Version: 4
Z3 Full Version: 4.3.2.0
SimpleExample
Opt
Exception in thread "main" java.lang.UnsatisfiedLinkError: com.microsoft.z3.Native.INTERNALmkOptimize(J)J
at com.microsoft.z3.Native.INTERNALmkOptimize(Native Method)
at com.microsoft.z3.Native.mkOptimize(Native.java:5208)
at com.microsoft.z3.Optimize.<init>(Optimize.java:262)
at com.microsoft.z3.Context.mkOptimize(Context.java:3043)
at Z3Example.optimizeExample(Z3Example.java:2323)
at Z3Example.main(Z3Example.java:2362)
To be fair, I am using the 64-bit libz3java.dll provided with 4.3.2, while using the jar file com.microsoft.z3.jar from version 4.4.2, because that was the only combination that I managed to get working (this details those problems). Could that be the version difference the reason for this error, or is there something else?
Yes, the version difference is responsible for this issue: 4.3.2 did not support optimization and thus doesn't come with mkOptimize. I'll take a look at the other issue separately.