sc.newAPIHadoopRDD is continuously giving me the error.
val hBaseRDD = sc.newAPIHadoopRDD(hbase_conf, classOf[TableInputFormat], classOf[org.apache.hadoop.hbase.io.ImmutableBytesWritable], classOf[org.apache.hadoop.hbase.client.Result]);
java.lang.NoSuchMethodError: ava.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer$.handledType()Ljava/lang/Class;
at com.fasterxml.jackson.module.scala.deser.NumberDeserializers$.<init>(ScalaNumberDeserializersModule.scala:49)
at com.fasterxml.jackson.module.scala.deser.NumberDeserializers$.<clinit>(ScalaNumberDeserializersModule.scala)
at com.fasterxml.jackson.module.scala.deser.ScalaNumberDeserializersModule$class.$init$(ScalaNumberDeserializersModule.scala:61)
at com.fasterxml.jackson.module.scala.DefaultScalaModule.<init>(DefaultScalaModule.scala:20)
at com.fasterxml.jackson.module.scala.DefaultScalaModule$.<init>(DefaultScalaModule.scala:37)
at com.fasterxml.jackson.module.scala.DefaultScalaModule$.<clinit>(DefaultScalaModule.scala)
at org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:82)
at org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:701)
at org.apache.spark.SparkContext.newAPIHadoopRDD(SparkContext.scala:1132)
I am trying to fetch values from Hbase.it is working perfectly fine in my local system already gone through many other answers related to this topic but nothing has helped me out yet.
But whenever I try to run it on my cluster it gives me error as mentioned above.
Already done all these imports
import org.apache.hadoop.hbase.HBaseConfiguration
import org.apache.hadoop.hbase.mapreduce.TableInputFormat
import org.apache.hadoop.hbase.client.HBaseAdmin
import org.apache.hadoop.hbase.{HTableDescriptor,HColumnDescriptor}
import org.apache.hadoop.hbase.util.Bytes
import org.apache.hadoop.hbase.client.{Put,HTable}
import org.apache.hadoop.fs.Path
import org.apache.hadoop.hbase._
import org.apache.hadoop.hbase.client._
import org.apache.hadoop.hbase.util._
import org.apache.spark._
import org.apache.hadoop.hbase.client.{HBaseAdmin, Result}
import org.apache.hadoop.hbase.{ HBaseConfiguration, HTableDescriptor }
import org.apache.hadoop.hbase.mapreduce.TableInputFormat
import org.apache.hadoop.hbase.io.ImmutableBytesWritable
Followed all the steps of installation as well as classpath from
https://acadgild.com/blog/apache-spark-hbase/
Please Help me.
I got my problem after searching and exploring other jars
My Hadoop Version 2.7.3
My Hbase Version 1.4.2
Libraries I was using was of version 1.4.2 only but straight away using them as
--driver-class-path $HBASE_HOME
As mentioned in the Link which I referred from.
But the issue was it was giving me some JARS incompatibility and also multiple occurrences of JARS with same name.
Actually the only Dependencies that are needed to run Hbase successfully are
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>1.3.1</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-common</artifactId>
<version>1.3.1</version>
</dependency>
Updated Libraries of Hbase doesnt have the required classes to implement Hbase.
After using these version of libraries it worked for me perfectly fine.
Related
I have installed and setup graphqldgs on a spring boot project following the guide here: https://netflix.github.io/dgs/getting-started/
When i try to write a test case - following the guide here: https://netflix.github.io/dgs/query-execution-testing/ - I am unable to import one of the values, as shown below
import com.netflix.graphql.dgs.autoconfig.DgsAutoConfiguration
I cannot locate DgsAutoConfiguration. The only thing i have is an asterix (*) after dot notation below:
import com.netflix.graphql.dgs.autoconfig.
I have these graphqldgs pom dependencies below:
<dependency>
<groupId>com.netflix.graphql.dgs</groupId>
<artifactId>graphql-dgs-spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>com.netflix.graphql.dgs</groupId>
<artifactId>graphql-dgs-platform-dependencies</artifactId>
<!-- The DGS BOM/platform dependency. This is the only place you set version of DGS -->
<version>4.9.16</version>
<type>pom</type>
<scope>import</scope>
</dependency>
Any help is appreciated. Thanks!
Hi iam working on the flink introduction but iam not able to import properly
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.walkthrough.common.sink.AlertSink;
import org.apache.flink.walkthrough.common.entity.Alert;
import org.apache.flink.walkthrough.common.entity.Transaction;
import org.apache.flink.walkthrough.common.source.TransactionSource;
these above packages are not able to import
i have used from the following site link
in this link i have imported the project via mvn command
please help me resolve this
I just tried this myself, and found that in order to run the application in my IDE (IntelliJ) I had to select the option under Run / Edit Configurations to 'Include dependencies with "Provided" scope'. Once I did that, everything worked fine.
I was able to build the application from the command line (via "mvn clean package") without making any changes.
Can you please verify the below artifact in your maven pom.xml?
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.12</artifactId>
<version>1.11.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-walkthrough-table-java</artifactId>
<version>1.11.1</version>
</dependency>
I have added the dependency to my pom.xml:
<dependency>
<groupId>org.jetbrains.xodus</groupId>
<artifactId>xodus-openAPI</artifactId>
<version>1.3.124</version>
</dependency>
<dependency>
<groupId>org.jetbrains.xodus</groupId>
<artifactId>xodus-environment</artifactId>
<version>1.3.124</version>
</dependency>
But it's the only one that doesn't seem to work when I import it in a Java class. I tried many different ways to import it, such as import org.jetbrains.exodus.env.Environments; but none of them work.
I am using JDK 11 and Netbeans 11.2.
Right package for Environments API is jetbrains.exodus.env, so import jetbrains.exodus.env.Environments;should work.
I'm using a program that relies on the following two imports:
import org.lwjgl.opencl.CLDevice;
import org.lwjgl.opencl.CLPlatform;
Eclipse is reporting that the "import cannot be resolved" even though I've added LWJGL OpenCL as a dependency to my project.
Here's a snapshot of my POM file:
<dependency>
<groupId>org.lwjgl</groupId>
<artifactId>lwjgl-opencl</artifactId>
<version>3.1.6</version>
</dependency>
I've read somewhere that these classes only exist in an earlier version so I tried changing the version to all of the versions found here ( from 3.1.0 to 3.1.6) but none of them resolved the issue.
Is there an earlier/different version that is not on the Maven repository page? If not where could I find the said class?
Thanks
It seems that you are using the abandoned lwjgl v2 library. It can be found in another Maven repository:
<dependency>
<groupId>org.lwjgl.lwjgl</groupId>
<artifactId>lwjgl</artifactId>
<version>2.9.3</version>
</dependency>
In https://www.tutorialspoint.com/neo4j/neo4j_cypher_api_example.htm and How to load CSV file with cypher in java?, there are always these two import lines:
import org.neo4j.cypher.javacompat.ExecutionEngine;
import org.neo4j.cypher.javacompat.ExecutionResult;
However, there are errors when I tried to import these two items. For the class ExecutionEngine, it seems I could import from org.neo4j.cypher.internal or org.neo4j.cypher.internal.javacompat, which one should I import (given I did nothing wrong so that I could not import from org.neo4j.cypher.javacompat)?
What is a reason why I could not import from org.neo4j.cypher.javacompat?
The relevant dependency of the driver in maven repository I am using:
<dependency>
<groupId>org.neo4j.driver</groupId>
<artifactId>neo4j-java-driver</artifactId>
<version>1.2.1</version>
</dependency>
Your dependency is incorrect- org.neo4j.cypher.javacompat does not belong to the Neo4j Java Driver, rather:
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j</artifactId>
<version>${neo.version}</version>
</dependency>
See https://neo4j.com/docs/java-reference/current/#_add_neo4j_as_a_dependency for more info