Inserting Into Cassandra using batch statements - java

I am inserting data into Cassandra coming from a csv file using batch statements. My table looks like this
create table exposures(expoid bigint,fileid bigint,studyid text, projname text, w text, x text, y text, z text)
System.out.println(colDataMap);
String keyspace = "orchtablespaces";
String tabName = fileName;
//String tableFile = "/home/blr-lt-202/empAccount.txt";
String tableFile = fname;
Set<String> colNamesSet = colDataMap.keySet();
String[] colNames = colNamesSet.toArray(new String[colNamesSet.size()]);
System.out.println("ColNames ::" +colNames);
String makeStatement = makeSt(keyspace,tabName,colNames);
System.out.println("makeStatement ::"+makeStatement);
if(count==65534)
{
session.executeAsync(bs);
count = 0;
bs = new BatchStatement();
}
PreparedStatement statement = session.prepare(makeStatement);
//bcz expo id and seq_n
String expoid =(String) colDataMap.get("expoid");
String Seq_No = (String) colDataMap.get("Seq_No");
colDataMap.put(expoid, Long.valueOf(expoid));
colDataMap.put("Seq_No", Long.valueOf(Seq_No));
BoundStatement query = statement.bind(colDataMap.values().toArray(new Object[colDataMap.size()]));
//BoundStatement query = statement.bind(colDataMap.get("Seq_No"),colDataMap.get("fileId"),colDataMap.get("studyId"),colDataMap.get("projectName"),colDataMap.get("dosetxt"),colDataMap.get("sdurtunit"),colDataMap.get("durtunit"));
System.out.println("query "+query);
bs.add(query);
count++;
}
session.executeAsync(bs);
session.close();
The Map ColDataMap has all the Cassandra Table col Names as key and their values In the map as shown below
{Seq_No=0, fileId=123, studyId=786, w=PreCLinic, x=Tasq 30mg/kg, y=12 days, z=12 days}
Set<String> colNamesSet =colDataMap.keySet();
String[] colNames = colNamesSet.toArray(new String[colNamesSet.size()]);
System.out.println("ColNames ::" +colNames);
String makeStatement = makeSt(keyspace,tabName,colNames);
System.out.println("makeStatement ::"+makeStatement);
The method makeSt prepares the prepared Statement
makeStatement ::INSERT INTO orchtablespaces.EXPOSURES ( Seq_No,fileId,studyId,w,x,y,z ) values ( ?,?,?,?,?,?,? )
This is all fine. Then I do the following
PreparedStatement statement = session.prepare(makeStatement);
//bcz expoid and seq_no are of type bigint int the cassandra table
String expoid =(String) colDataMap.get("expoid");
String Seq_No = (String) colDataMap.get("Seq_No");
colDataMap.put(expoid, Long.valueOf(expoid));
colDataMap.put("Seq_No", Long.valueOf(Seq_No));
BoundStatement query = statement.bind(colDataMap.values().toArray(new Object[colDataMap.size()]));
When i run the program I am getting the following error
com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:289)com.test.load.microarr.CopyOfLoadMicroArr.transformSourceFile(CopyOfLoadMicroArr.java:486)com.test.load.microarr.CopyOfLoadMicroArr.loadData(CopyOfLoadMicroArr.java:149)
com.test.load.microarr.CopyOfLoadMicroArr.main(CopyOfLoadMicroArr.java:114)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
com.datastax.driver.core.RequestHandler.sendRequest(RequestHandler.java:107)
com.datastax.driver.core.SessionManager.execute(SessionManager.java:538)
com.datastax.driver.core.SessionManager.prepareAsync(SessionManager.java:124)
com.datastax.driver.core.AbstractSession.prepare(AbstractSession.java:77)
... 3 more
com.datastax.driver.core.AbstractSession.prepare(AbstractSession.java:79)
The exact line where the program is breaking is line 486 and that is at
PreparedStatement statement = session.prepare(makeStatement);
I am using DataStax Cassandra 2.1.8 .. any suggestions and solutions would be really helpful !! Thanks !

You definitely have problem with the connection to the cassandra.
Form the Datastax Java driver documentation:
NoHostAvailableException - if no host in the cluster can be contacted successfully to prepare this statement.
And form the error description
Exception thrown when a query cannot be performed because no host are
available. This exception is thrown if
either there is no host live in
the cluster at the moment of the query
all host that have been tried
have failed due to a connection problem
For debugging purpose, the
list of hosts that have been tried along with the failure cause can be
retrieved using the errors method.
Try to catch this error and output list of tried host - this could help you detect is there are wrong host or wrong port. Use getErrors() method - http://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/exceptions/NoHostAvailableException.html#getErrors()

Related

MySQL command: Search for sub-string in database -- return matches (Java)

// Use case 1.
String authorName = "Wei She";
String query = "SELECT * FROM articles WHERE authors LIKE %?%";
PreparedStatement getAuthors = newConn.prepareStatement(query);
getAuthors.setString(1, authorName);
ResultSet resultSet = getAuthors.executeQuery();
while (resultSet.next()) {
String authors = resultSet.getString("authors");
System.out.println(authors);
}
Here is a subset of my code. The data is already in my local MySQL database. The rest of the connection code is present.
My question is: how should I format the %?% section in Java code?
Is there something I am missing? Here is the error output.
Exception in thread "main" java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '/%'Wei She'/%' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:975)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeQuery(ClientPreparedStatement.java:1025)
at src.TestSQLQuery.main(TestSQLQuery.java:34)
I have a similar query used in MySQL Workbench: SELECT * FROM articles WHERE authors LIKE '%Wei She%';
The above query works fine in Workbench and returns multiple entries with other authors along with "Wei She".
%?% is not valid. You need to just use ? and have the % signs in the value passed to PreparedStatement:
String query = "SELECT * FROM articles WHERE authors LIKE ?";
PreparedStatement getAuthors = newConn.prepareStatement(query);
getAuthors.setString(1, "%" + authorName + "%");

How to pass dashed strings to oracle from a query written in java

I have a problem with this query when I pass it to an oracle dbms
SELECT * FROM RD_RBF WHERE REQUEST_ID = 'S2N-F01-000000000001'
because of the dashes in the string the jvm return me this exception
java.sql.SQLException: Fail to convert to internal representation
How can I pass this query to oracle correctly? Thanks a lot
P.S. I'm not shure of the code because I'm using Talend software that generates automatically the code of components but I can post part of the code above
String dbquery_tOracleInput_1 = "SELECT * FROM RD_RBF WHERE REQUEST_ID = 'S2N-F01-000000000001'";
java.sql.ResultSet rs_tOracleInput_1 = null;
try{
rs_tOracleInput_1 = stmt_tOracleInput_1.executeQuery(dbquery_tOracleInput_1);
java.sql.ResultSetMetaData rsmd_tOracleInput_1 = rs_tOracleInput_1.getMetaData();
int colQtyInRs_tOracleInput_1 = rsmd_tOracleInput_1.getColumnCount();
Use toraclerow component.
Query as follows in component:
"SELECT * FROM RD_RBF WHERE REQUEST_ID = ?"
Go to advance setting, select use prepared statement and add 1 paarmeter index.
also select the propagate Query's record set.
tOracleRow_1 -----> tParseRecordSet----->

Java ResultSet get nothing but sql get result in mysql

I am using MySql database which has one table 'tradeinfo'.
Table structure:
Date TradeCode
2017.01.01 0001
2017.02.05 0002
2017.03.05 0001
My sql to find lastest trading day of the one tradecode is
SELECT TradeCode, MAX(date) most_recent_time FROM tradeinfo WHERE TradeCode = '0001'
I test the sql in Mysql db and can get right result which is "2017.03.05 0001"
But for my java code which is "lastestdbrecordsdate = rs.getDate("MOST_RECENT_TIME"); ", It ever return right result. But few days later, when run it again, I always get NULL.
My java code is:
Connection con = DriverManager.getConnection("jdbc:mysql://...",user,password);
String sqlstatement = "SELECT TradeCode, MAX(date) most_recent_time FROM tradeinfo WHERE TradeCode = '0001' ";
PreparedStatement sqlstat = con.prepareStatement(sqlstatement);
ResultSet rsquery = sqlstat.executeQuery(sqlstatement);
CachedRowSetImpl cachedRS = new CachedRowSetImpl();
cachedRS.populate(rsquery);
while(cachedRS.next() ) {
System.out.println(cachedRS.getMetaData().getColumnCount());
Date lastestdbrecordsdate = cachedRS.getDate("MOST_RECENT_TIME");
}
Is the problem that I config the mysql wrongly or I write wrong java code?
Thanks all!
You have several problems here. First, you should be using the following query:
SELECT MAX(date) most_recent_time FROM tradeinfo WHERE TradeCode = '0001'
Adding TradeCode to the select list doesn't make any sense because it is not an aggregate, but rather each record has a value for this column.
With regard to why you are getting null results, you need to call ResultSet#next() to advance the cursor to the first line:
Connection con = DriverManager.getConnection("jdbc:mysql://...", user, password);
Statement sqlstat = con.prepareStatement(sqlstatement);
ResultSet rsquery = sqlstat.executeQuery(); // DON'T pass anything to executeQuery()
if (rsquery.next()) {
Date lastestdbrecordsdate = rs.getDate("most_recent_time");
}
Another problem I just noticed is that you were passing in the query string to your call to Statement#executeQuery(). This is wrong, and you should not be passing anything to this method.

JDBC: Create table query syntax error

I am trying to execute a create query using JDBC. I have a method which creates the query and then I execute it but its showing me syntax error. Below is the stack trace :
com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'supplierId varchar,supplierUrl varchar,totalActivities varchar,activityName varc' at line 1
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2941)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1623)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1715)
at com.mysql.jdbc.Connection.execSQL(Connection.java:3243)
at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1343)
at com.mysql.jdbc.Statement.executeUpdate(Statement.java:1260)
Now the query generated is this :
create table demo ( ID INTEGER PRIMARY KEY AUTO_INCREMENT,supplierName varchar,supplierId varchar,supplierUrl varchar,totalActivities varchar,activityName varchar,activityPrice varchar,tourCode varchar,starRating varchar,totalReviews varchar,geography varchar,duration varchar,category varchar,subCategory varchar);
And below is the method which is generating this query :
private static String getCreateTableQuery(String tableName, String columnData) {
StringBuilder sqlStatement = new StringBuilder("");
sqlStatement.append("create table " + tableName + " ( ID INTEGER PRIMARY KEY AUTO_INCREMENT,");
String[] columns = columnData.split(">"); // columns are separated by >
for (int i = 0; i < columns.length; i++) {
sqlStatement.append(columns[i] + " varchar");
if (i != columns.length - 1) { // no commas after last column
sqlStatement.append(",");
}
}
sqlStatement.append(");");
return sqlStatement.toString();
}
And this is how am executing the query :
SessionImpl sessionImpl = (SessionImpl) getSessionFactory().openSession();
Connection conn = (Connection) sessionImpl.connection();
Statement statement = (Statement) conn.createStatement();
statement.executeUpdate(query);
sessionImpl.close();
conn.close();
Am unable to understand the syntax error. Can someone please explain?
I think you have to pass max length for varchar fields:
Please check this your query will be like that:
create table demo ( ID INTEGER PRIMARY KEY AUTO_INCREMENT,supplierName varchar(255),supplierId varchar(255),supplierUrl varchar(255),totalActivities varchar(255),activityName varchar(255),activityPrice varchar(255),tourCode varchar(255),starRating varchar(255),totalReviews varchar(255),geography varchar(255),duration varchar(255),category varchar(255),subCategory varchar(255));
Here is insert Query:
insert into demo
( supplierName, supplierId, supplierUrl, totalActivities, activityName,
activityPrice, tourCode, starRating, totalReviews, geography, duration,
category, subCategory)
values
(supplierName, supplierId, supplierUrl, totalActivities, activityName,
activityPrice, tourCode, starRating, totalReviews, geography, duration,
category, subCategory)
In Mysql you need to define a length to the varchar. Take a look here:
Why does VARCHAR need length specification?
I don't see a problem with your Java code. Fix your create table statement and you'll probably be fine.

Fetch data from mysql using EBean

I am trying to make a simple program to fetch data from a table.
I am following http://www.avaje.org/ebean/getstarted_props.html#iud but am unable to get data. I have created new Entity Class from Database from Netbeans (which creates classes from relations). Here is what I am using:
ebean.properties
ebean.ddl.generate=true
ebean.ddl.run=true
ebean.debug.sql=true
ebean.debug.lazyload=false
ebean.logging=all
ebean.logging.logfilesharing=all
ebean.logging.directory=D:\\logs
ebean.logging.iud=sql
ebean.logging.query=sql
ebean.logging.sqlquery=sql
ebean.logging.txnCommit=none
datasource.default=h2
datasource.h2.username=sa
datasource.h2.password=
datasource.h2.databaseUrl=jdbc:h2:mem:tests;DB_CLOSE_DELAY=-1
datasource.h2.databaseDriver=org.h2.Driver
datasource.h2.minConnections=1
datasource.h2.maxConnections=25
datasource.h2.heartbeatsql=select 1
datasource.h2.isolationlevel=read_committed
datasource.mysql.username=root
datasource.mysql.password=kalsym#123
datasource.mysql.databaseUrl=jdbc:mysql://127.0.0.1:3306/wsp
datasource.mysql.databaseDriver=com.mysql.jdbc.Driver
datasource.mysql.minConnections=1
datasource.mysql.maxConnections=25
datasource.mysql.isolationlevel=read_committed
Table Data
Insert into routing_algo_type (name, description) values ('LCR', 'Least Cost Routing');
Code to fetch data
RoutingAlgoType routingObj = new RoutingAlgoType();
routingObj.setName("LCR");
RoutingAlgoType routingObj2 = Ebean.find(RoutingAlgoType.class, routingObj);
System.out.println("Got "+routingObj2.getDescription());
Now find returns null, which means it cant find the data?
I used following code to test connection
String sql = "select count(*) as count from dual";
SqlRow row = Ebean.createSqlQuery(sql).findUnique();
Integer i = row.getInteger("count");
System.out.println("Got " + i + " - DataSource good.");
Result from above code is
Got 1 - DataSource good.
Is there any way to check the connection?

Categories

Resources