Persisting or writing spark dataset to OrientDB using java - java

How can I write a Spark dataset to OrientDB using java?
I already read data from OrientDB using Orient Java JDBC driver, but I am unable to persist the same dataset into OrientDB.
Code:
public void readAndWriteData(SparkSession spark, Map<String, String> dbProps, Properties destinationDb) {
Dataset<Row> tableDataset = spark.read().format("jdbc").options(dbProps).load();
tableDat‌​aset.show();
tableDat‌​aset.createOrReplace‌​TempView("TEMP_V");
D‌​ataset<Row> tableDataset1 = spark.sql("SELECT NAME FROM TEMP_V");
tableDataset1.show();
tableDataset1.write().format("‌​org.apache.spark.ori‌​entdb.documents")
.option("dburl", "jdbc:orient:REMOTE:localhost/testdb")
.option("user", "root")
.option("password", "root")
.option("class", "Test")
.mode(SaveMode.Append).save();
}
Here i am getting error as
Exception in thread "main" java.lang.RuntimeException: Connection Exception Occurred: Error on opening database 'jdbc:orient:REMOTE:localhost/testdb'
can you please help me on this error. I have used same db connection for reading and persisting

Related

SQLServerBulkCopy in jdbc error SybResultSet.isClosed()Z is abstract

i am new to java, i am trying to connect do bulkcopy from sybase to SQL server database.
i used the jdbc3, i am getting the below error
Exception in thread "main" java.lang.AbstractMethodError: Method com/sybase/jdbc3/jdbc/SybResultSet.isClosed()Z is abstract
at com.sybase.jdbc3.jdbc.SybResultSet.isClosed(Unknown Source)
at com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeResultSet(SQLServerBulkCopy.java:574)
at com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(SQLServerBulkCopy.java:533)
at SQLServer.dbConnect(SQLServer.java:36)
at SQLServer.main(SQLServer.java:46)
i have follwoing questinos
which jdbc driver works with com.microsoft.sqlserver.jdbc.SQLServerBulkCopy
is there any other alternative

Cannot start a transaction for OLE DB provider "Microsoft.ACE.OLEDB.12.0"

Two days ago when I executed the procedure through Java coding I got following error message:
com.microsoft.sqlserver.jdbc.SQLServerException: Cannot start a transaction for OLE DB provider "Microsoft.ACE.OLEDB.12.0" for linked server "(null)"
This is the snap of SQL procedure code
set #sql1='INSERT OPENROWSET(
''Microsoft.ACE.OLEDB.12.0'','D:\Users\media\workspacekepler\.metadata\.plugins\org.eclipse.wst.server.core\tmp5\wtpwebapps\NPODownload\download\dump.mdb';
'admin';'',Allocation) SELECT * FROM Allocation'
exec(#sql1)

Cassandra connection has been closed error when querying via Spark

I'm trying to access remote Cassandra using Spark in Java. However, when I'm trying to execute an aggregation function (count), the following error:
Exception in thread "main" com.datastax.driver.core.exceptions.TransportException: [/192.168.1.103:9042] Connection has been closed
at com.datastax.driver.core.exceptions.TransportException.copy(TransportException.java:38)
at com.datastax.driver.core.exceptions.TransportException.copy(TransportException.java:24)
at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
I already set the timeout in the Cassandra.yml to big value.
Here is my code:
SparkConf conf = new SparkConf();
conf.setAppName("Test");
conf.setMaster("local[*]");
conf.set("spark.cassandra.connection.host", "host");
Spark app = new Spark(conf);
app.run();
.
.
.
CassandraConnector connector = CassandraConnector.apply(sc.getConf());
// Prepare the schema
try (Session session = connector.openSession()) {
session.execute("USE keyspace0");
ResultSet results = session.execute("SELECT count(*) FROM table0");

Java Apache Phoenix 4.1.0/4.2.2. Not able to create connection

I am not able to connect to HBase through Apache phoenix driver.
Env info:
hadoop-2.6.0.
hbase-0.98.9-hadoop2.
phoenix-4.1.0-server-hadoop2(Kept on all region servers).
phoenix-4.1.0-client-hadoop2(Using this jar to create a jdbc connection).
Java Client side , I am getting exception
Caused by: org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: java.io.IOException: Class org.apache.phoenix.coprocessor.MetaDataRegionObserver cannot be loaded
at ...
Caused by: java.io.IOException: Class org.apache.phoenix.coprocessor.MetaDataRegionObserver cannot be loaded
at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.testTableCoprocessorAttrs...
At HBase Master node logs I am getting this error:
2015-02-02 12:48:11,550 DEBUG [FifoRpcScheduler.handler1-thread-14] util.FSTableDescriptors: Exception during readTableDecriptor. Current table name = SYSTEM.CATALOG
org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor file under hdfs://HadoopNode:9000/home/hduser/Data/hbase/data/default/SYSTEM.CATALOG
Code which I am using to create phoenix connection:
String zkQuorum = "HbaseMasterNode:2222";
try
{
Class.forName("org.apache.phoenix.jdbc.PhoenixDriver");
String connectionURL = "jdbc:phoenix:" + zkQuorum;
Connection connection = DriverManager.getConnection(connectionURL);
System.out.println(connection);
}
catch (Exception e)
{
throw new IllegalArgumentException("Create phoenix connection(" + zkQuorum + ") throw exception", e);
}
With the help of Basic Hbase java APIs I am able to connect but this issue I am facing only If I try to use Phoenix driver for HBase.

Basics of Hector & Cassandra

I'm working with Cassandra-0.8.2.
I am working with the most recent version of Hector &
My java version is 1.6.0_26
I'm very new to Cassandra & Hector.
What I'm trying to do:
1. connect to an up & running instance of cassandra on a different server. I know it's running b/c I can ssh through my terminal into the server running this Cassandra instance and run the CLI with full functionality.
2. then I want to connect to a keyspace & create a column family and then add a value to that column family through Hector.
I think my problem is that this running instance of Cassandra on this server might not be configured to get commands that are not local. I think my next step will be to add a local instance of Cassandra on the cpu I'm working on and try to do this locally. What do you think?
Here's my Java code:
import me.prettyprint.cassandra.serializers.StringSerializer;
import me.prettyprint.cassandra.service.CassandraHostConfigurator;
import me.prettyprint.hector.api.Cluster;
import me.prettyprint.hector.api.Keyspace;
import me.prettyprint.hector.api.ddl.ColumnFamilyDefinition;
import me.prettyprint.hector.api.ddl.ComparatorType;
import me.prettyprint.hector.api.factory.HFactory;
import me.prettyprint.hector.api.mutation.Mutator;
public class MySample {
public static void main(String[] args) {
Cluster cluster = HFactory.getOrCreateCluster("Test Cluster", "xxx.xxx.x.41:9160");
Keyspace keyspace = HFactory.createKeyspace("apples", cluster);
ColumnFamilyDefinition cf = HFactory.createColumnFamilyDefinition("apples","ColumnFamily2",ComparatorType.UTF8TYPE);
StringSerializer stringSerializer = StringSerializer.get();
Mutator<String> mutator = HFactory.createMutator(keyspace, stringSerializer);
mutator.insert("jsmith", "Standard1", HFactory.createStringColumn("first", "John"));
}
}
My ERROR is:
16:22:19,852 INFO CassandraHostRetryService:37 - Downed Host Retry service started with queue size -1 and retry delay 10s
16:22:20,136 INFO JmxMonitor:54 - Registering JMX me.prettyprint.cassandra.service_Test Cluster:ServiceType=hector,MonitorType=hector
Exception in thread "main" me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:Keyspace apples does not exist)
at me.prettyprint.cassandra.connection.HThriftClient.getCassandra(HThriftClient.java:70)
at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:226)
at me.prettyprint.cassandra.service.KeyspaceServiceImpl.operateWithFailover(KeyspaceServiceImpl.java:131)
at me.prettyprint.cassandra.service.KeyspaceServiceImpl.batchMutate(KeyspaceServiceImpl.java:102)
at me.prettyprint.cassandra.service.KeyspaceServiceImpl.batchMutate(KeyspaceServiceImpl.java:108)
at me.prettyprint.cassandra.model.MutatorImpl$3.doInKeyspace(MutatorImpl.java:222)
at me.prettyprint.cassandra.model.MutatorImpl$3.doInKeyspace(MutatorImpl.java:219)
at me.prettyprint.cassandra.model.KeyspaceOperationCallback.doInKeyspaceAndMeasure(KeyspaceOperationCallback.java:20)
at me.prettyprint.cassandra.model.ExecutingKeyspace.doExecute(ExecutingKeyspace.java:85)
at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:219)
at me.prettyprint.cassandra.model.MutatorImpl.insert(MutatorImpl.java:59)
at org.cassandra.examples.MySample.main(MySample.java:25)
Caused by: InvalidRequestException(why:Keyspace apples does not exist)
at org.apache.cassandra.thrift.Cassandra$set_keyspace_result.read(Cassandra.java:5302)
at org.apache.cassandra.thrift.Cassandra$Client.recv_set_keyspace(Cassandra.java:481)
at org.apache.cassandra.thrift.Cassandra$Client.set_keyspace(Cassandra.java:456)
at me.prettyprint.cassandra.connection.HThriftClient.getCassandra(HThriftClient.java:68)
... 11 more
Thank you in advance for your help.

The exception you are getting is,
why:Keyspace apples does not exist
In your code, this line does not actually create the keyspace,
Keyspace keyspace = HFactory.createKeyspace("apples", cluster);
As described here, this is the code you need to define your keyspace,
ColumnFamilyDefinition cfDef = HFactory.createColumnFamilyDefinition("MyKeyspace", "ColumnFamilyName", ComparatorType.BYTESTYPE);
KeyspaceDefinition newKeyspace = HFactory.createKeyspaceDefinition("MyKeyspace", ThriftKsDef.DEF_STRATEGY_CLASS, replicationFactor, Arrays.asList(cfDef));
// Add the schema to the cluster.
// "true" as the second param means that Hector will block until all nodes see the change.
cluster.addKeyspace(newKeyspace, true);

We also have a getting started guide up on the wiki as well which might be of some help.

Categories

Resources