I have two Apache Flink clusters: 1.1.3 in production and 1.3.2 in staging.
I'm interested in sending metrics to a Graphite server, so I set it up as explained in https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html.
I got it to work in my 1.1.3 cluster, but not in 1.3.2. The jar files I added to the Flink lib directory are:
In 1.1.3:
http://central.maven.org/maven2/io/dropwizard/metrics/metrics-graphite/3.1.0/metrics-graphite-3.1.0.jar
http://central.maven.org/maven2/org/apache/flink/flink-metrics-dropwizard/1.1.1-hadoop1/flink-metrics-dropwizard-1.1.1-hadoop1.jar
http://central.maven.org/maven2/org/apache/flink/flink-metrics-graphite/1.1.4/flink-metrics-graphite-1.1.4.jar
http://central.maven.org/maven2/org/apache/flink/flink-metrics-jmx/1.1.4/flink-metrics-jmx-1.1.4.jar
In 1.3.2:
http://central.maven.org/maven2/io/dropwizard/metrics/metrics-graphite/3.1.0/metrics-graphite-3.1.0.jar
http://central.maven.org/maven2/org/apache/flink/flink-metrics-dropwizard/1.1.1-hadoop1/flink-metrics-dropwizard-1.1.1-hadoop1.jar
http://central.maven.org/maven2/org/apache/flink/flink-metrics-graphite/1.3.2/flink-metrics-graphite-1.3.2.jar
http://central.maven.org/maven2/org/apache/flink/flink-metrics-jmx/1.3.2/flink-metrics-jmx-1.3.2.jar
The settings I added is the same on both (except they send to a different Graphite server):
metrics.reporters: grph
metrics.reporter.grph.class: org.apache.flink.metrics.graphite.GraphiteReporter
metrics.reporter.grph.host: 10.x.x.x
metrics.reporter.grph.port: 2003
metrics.reporter.grph.prefix: flink
metrics.reporter.grph.protocol: TCP
The error message I'm seeing on the staging cluster (1.3.2) is:
java.lang.NoClassDefFoundError: com/codahale/metrics/Reporter
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.flink.runtime.metrics.MetricRegistry.<init>(MetricRegistry.java:123)
at org.apache.flink.runtime.taskexecutor.TaskManagerServices.fromConfiguration(TaskManagerServices.java:188)
at org.apache.flink.runtime.taskmanager.TaskManager$.startTaskManagerComponentsAndActor(TaskManager.scala:1921)
at org.apache.flink.runtime.taskmanager.TaskManager$.runTaskManager(TaskManager.scala:1819)
at org.apache.flink.runtime.taskmanager.TaskManager$.selectNetworkInterfaceAndRunTaskManager(TaskManager.scala:1673)
at org.apache.flink.runtime.taskmanager.TaskManager$$anon$2.call(TaskManager.scala:1574)
at org.apache.flink.runtime.taskmanager.TaskManager$$anon$2.call(TaskManager.scala:1572)
at org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:43)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:40)
at org.apache.flink.runtime.taskmanager.TaskManager$.main(TaskManager.scala:1572)
at org.apache.flink.runtime.taskmanager.TaskManager.main(TaskManager.scala)
Caused by: java.lang.ClassNotFoundException: com.codahale.metrics.Reporter
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 40 common frames omitted
Any help would be greatly appreciated!
You have to add io.dropwizard.metrics:metrics-core 3.1.0 to the /lib folder as well. The reason is that in 1.1 the Flink runtime itself was using metrics-core, which is no longer the case in 1.3.
Alternatively, you can also use the flink-metrics-graphite jar-with-dependencies which should contain everything you need.
Related
I'm a newbie to both Java and Gate so I'll give my detailed steps. I'm trying to use Stanford pos tagger within my java application but it gave me this error:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Reading POS tagger model from file:/home/khaled/smartgit/projects/MicroblogsAnnotator/GATE_Developer_8.4.1/plugins/Stanford_CoreNLP/resourcs/english-left3words-distsim.tagger ... done [2.6 sec].
following another question here i downloaded "slf4j-log4j12.jar" and tried to include it in my class path. I'm not sure if that's the correct way to add it but to do so, from netbeans i clicked on
project->Properties->Libraries->add Jar
running again now it gives me another error:
java.lang.NoClassDefFoundError: org/slf4j/spi/LoggerFactoryBinder
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at gate.util.GateClassLoader.loadClass(GateClassLoader.java:183)
at gate.util.GateClassLoader.loadClass(GateClassLoader.java:142)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:141)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:120)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:331)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:283)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:304)
at edu.stanford.nlp.io.IOUtils.<clinit>(IOUtils.java:42)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(MaxentTagger.java:765)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.<init>(MaxentTagger.java:298)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.<init>(MaxentTagger.java:263)
at gate.stanford.Tagger.init(Tagger.java:129)
at gate.Factory.createResource(Factory.java:432)
at gate.Factory.createResource(Factory.java:139)
at gate.Factory.createResource(Factory.java:121)
at MicroblogsAnnotator.GateTool.CreateController(GateTool.java:110)
at MicroblogsAnnotator.GateTool.TwitIE(GateTool.java:185)
at MicroblogsAnnotator.GateTool.RunTwitIE(GateTool.java:207)
at MicroblogsAnnotator.MicroblogsAnnotator.main(MicroblogsAnnotator.java:36)
Caused by: java.lang.ClassNotFoundException: org.slf4j.spi.LoggerFactoryBinder
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 31 more
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/spi/LoggerFactoryBinder
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at gate.util.GateClassLoader.loadClass(GateClassLoader.java:183)
at gate.util.GateClassLoader.loadClass(GateClassLoader.java:142)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:141)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:120)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:331)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:283)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:304)
at edu.stanford.nlp.io.IOUtils.<clinit>(IOUtils.java:42)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(MaxentTagger.java:765)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.<init>(MaxentTagger.java:298)
at edu.stanford.nlp.tagger.maxent.MaxentTagger.<init>(MaxentTagger.java:263)
at gate.stanford.Tagger.init(Tagger.java:129)
at gate.Factory.createResource(Factory.java:432)
at gate.Factory.createResource(Factory.java:139)
at gate.Factory.createResource(Factory.java:121)
at MicroblogsAnnotator.GateTool.CreateController(GateTool.java:110)
at MicroblogsAnnotator.GateTool.TwitIE(GateTool.java:185)
at MicroblogsAnnotator.GateTool.RunTwitIE(GateTool.java:207)
at MicroblogsAnnotator.MicroblogsAnnotator.main(MicroblogsAnnotator.java:36)
Caused by: java.lang.ClassNotFoundException: org.slf4j.spi.LoggerFactoryBinder
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 31 more
You are almost there, you need to include also the slf4j-api.jar.
And then maybe a log4j config file will be needed in case you are not having any yet...
I have a code that runs fine in Eclipse. But when I try to run through native packaging using Gradle, I get this message:
Caused by: java.lang.NoClassDefFoundError: com/embraer/rw4j/output/OutputEditorTags (wrong name: com/embraer/rw4j/view/report/OutputEditorTags)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.embraer.rw4j.output.HeaderEditor.handleAddTagContent(HeaderEditor.java:71)
... 86 more
The strange thing is that I don't have any package com/embraer/rw4j/view/report. I don't know where the JVM is getting this reference.
I am trying to get data from hbase database (1.2.5) using Spark 2.0.2 with the code in this page :
https://github.com/apache/hbase/blob/master/hbase-spark/src/main/java/org/apache/hadoop/hbase/spark/example/hbasecontext/JavaHBaseBulkGetExample.java
But I get this error :
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/spark/Logging at java.lang.ClassLoader.defineClass1(Native
Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467) at
java.net.URLClassLoader.access$100(URLClassLoader.java:73) at
java.net.URLClassLoader$1.run(URLClassLoader.java:368) at
java.net.URLClassLoader$1.run(URLClassLoader.java:362) at
java.security.AccessController.doPrivileged(Native Method) at
java.net.URLClassLoader.findClass(URLClassLoader.java:361) at
java.lang.ClassLoader.loadClass(ClassLoader.java:424) at
java.lang.ClassLoader.loadClass(ClassLoader.java:357) at
com.cloudera.spark.hbase.JavaHBaseContext.(JavaHBaseContext.scala:23)
at org.hipi.examples.Test.main(Test.java:71) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:736)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:185)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:210)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused
by: java.lang.ClassNotFoundException: org.apache.spark.Logging at
java.net.URLClassLoader.findClass(URLClassLoader.java:381) at
java.lang.ClassLoader.loadClass(ClassLoader.java:424) at
java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 22 more
I have seen on the internet that Logging class exist only in spark 1.5.2 and lower versions.
Is there is any solution to make that work even with Spark 2.0.2 ?
There are few BUGs related with this:
https://issues.apache.org/jira/browse/SPARK-13928
https://issues.apache.org/jira/browse/HBASE-16428
https://mahout.apache.org/users/algorithms/intro-cooccurrence-spark.html
Using this reference, I was trying mahout spark-itemsimilarity but getting an error when running the following command
mahout spark-itemsimilarity --input in-file --output out-dir
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: scopt/OptionParser
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: scopt.OptionParser
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 19 more
I referred this for installing mahout:
https://github.com/apache/mahout#building-from-source
I followed the guide exactly.
What could be the possible reason for this error? Am I missing anything?
Exception in thread "main" java.lang.IncompatibleClassChangeError: Implementing class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructor0(Class.java:3075)
at java.lang.Class.newInstance(Class.java:412)
at play.plugins.PluginCollection.loadPlugins(PluginCollection.java:168)
I got the above error when i executed Play run (Play framework 1.2.7) iam trying to upgrade the Elastic search version to 1.3.0. Plz guide me through this