I Insert rows into BigQuery with the InsertAll method using JAVA. It is working always fine. But when we try to update the same row from the JAVA code am getting the below error,
com.google.cloud.bigquery.BigQueryException UPDATE or DELETE DML statements over table project123:mydataset.test would affect rows in the streaming buffer, which is not supported
So I tried from BigQueryConsole.
I inserted a row using the INSERT query then immediately UPDATE the same row. It worked fine.
When I read the articles of BIGQUERY, they are mentioning both InsertAll from JAVA and INSERT query from Console using Streaming Buffer. In that case, the console query execution should be got failed.
Why Console query is working fine? But from Java InsertAll it is throwing me an exception.
It will be really helpful if anyone helps me to know the exact details.
If any suggestions to use Native insert query insertion from Java instead of InsertAll to BigQuery, It will be a great help.
Please find the code snippet
First am inserting the values to the BigQuery using the below code snippet
Map<String, Object> map = new HashMap<>();
map.put("1", "name");
map.put("2", "age");
BigQuery bQuery = BigQueryOptions.newBuilder().setCredentials(credentials).setProjectId(id)
.build().getService();
InsertAllResponse response = bQuery .insertAll(InsertAllRequest.newBuilder(tableId).addRow(map).build());
Once it is getting inserted, am trying to update the row in that table with the following code snippet
String updateQuery = String.format( "UPDATE `%s` SET name = \"%s\" WHERE age = \"%s\")", name, age);
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query).build();
bQuery.query(queryConfig);
Insert is working fine. when I tried to update the same inserted row am getting the streaming buffer error.
Thanks in advance.
When you read the documentation, it's clear that the insertAll perform a stream write into BigQuery.
When you use INSERT DML (INSERT INTO <table> [VALUES....|SELECT...]), you perform a query, not a stream write. So, the data management isn't the same. The performance are also different (Stream write can write up to 1 million of rows per seconds, the DML is query by query, and took more time for less data).
So, I don't know your code and what you want to achieve. But if you want to use usual query (INSERT, UPDATE, DELETE), use query API.
EDIT
I tried to adapt your code (but it was wrong, I took some assumptions) and I can propose you this. Simply perform a QUERY, not a Load Job or a Streaming write.
String tableName = "YOUR_TABLE_NAME";
String insertQuery = String.format("INSERT INTO %s(name, age) VALUES (1,2)", tableName);
QueryRequest queryRequest = QueryRequest.builder(insertQuery).build();
bQuery.query(queryRequest);
String updateQuery = String.format( "UPDATE `%s` SET name = \"%s\" WHERE age = \"%s\")", tableName, name, age);
queryRequest = QueryRequest.builder(updateQuery).build();
bQuery.query(queryRequest);
Related
I am new in the database side. My Question is, how to append a piece of query as param arguments. In my logic we are trying to append query using below
GET_DATA= "SELECT [:metrics] from table name"
am passing metrics as argument like below
paramMap.put("metrics", "name,age");
when am executing the query using query runner.
ResultSet rs = queryRunner.runQuery(context, GET_DATA, paramMap, RESULT_SET_HANDLER);
The output of the query like below
SELECT 'name,age' from table name
How can i avoid single quotes from the query?
I have tried these two changes but getting SQL error
[metrics]
metrics
If we put similar syntax with a condition like where colum_name=[:abc],its work fine for me.
Expecting a better solution.
Try this:
String columns = paramMap.get("metrics");
GET_DATA = "Select" +
columns +
" FROM" +
" name";
I can connect to my access database and select, insert records etc. I am now trying to delete records and as far as I can see I am using the correct syntax. I have followed just about every tutorial I can find and they are not doing anything different that I can see.
String deleteSql = "DELETE FROM table1 WHERE sometext=? or sometext=?";
ps = module.getSupportConnection().prepareStatement(deleteSql);
ps.setString(1,"four");
ps.setString(2,"five");
int rs = ps.executeUpdate();
System.out.println(rs);
I have tried it without using int rs = .. but I used it just to see what the output was and it returns '2' which is what I was expecting as there are two records that meet the criteria used. It just wont delete the records and I cant see why. I dont get any errors when running the code. I appreciate this may not be a ucanaccess issue per se.
I'm having an odd problem with SimpleJdbcInsert.executeAndReturnKey with Sybase (jTDS driver) and certain data.
Take the following example:
SimpleJdbcInsert insert = new SimpleJdbcInsert(jdbcTemplate)
.withTableName("TABLE_NAME")
.usingGeneratedKeyColumns("ID");
List<String> columns = new ArrayList<String>();
columns.add("SOME_NUMERIC_DATA");
columns.add("SOME_STRING_DATA");
Map<String, Object> params = new HashMap<String, Object>();
params.put("SOME_NUMERIC_DATA", 10.02);
params.put("SOME_STRING_DATA", "AAAA");
Number insertId = insert.executeAndReturnKey(params);
The above will fail with
DataIntegrityViolationException: Unable to retrieve the generated key for the insert
The insert itself is fine as if I do an insert.execute(params) the insert will work correctly (but I need the generated column value).
If I insert null instead of 10.02 for the SOME_NUMERIC_DATA column then it will work correctly and return the generated column value. Also if all of the fields are VARCHAR/String then it will work correctly.
Can anyone see anything here that might be causing this with a combination of string and numeric fields.
I should also add that when I use the exact same code with an H2 database it works all of the time - this seems to be related to Sybase/jTDS
I had the same problem with SQL Server and fixed it by calling this configuration method right before the call to executeAndReturnKey():
mySimpleJdbcInsert.setAccessTableColumnMetaData(false);
I suspect the error has to do with database metadata : as explained in the spring reference http://docs.spring.io/spring-framework/docs/current/spring-framework-reference/html/jdbc.html, SimpleJdbcInsert uses database metadata to construct the actual insert statement.
One could also use the SQL OUTPUT clause such as
INSERT INTO myTable (Name, Age)
OUTPUT Inserted.Id
VALUES (?,?)
And use some more generic JdbcTemplate.execute() to handle the insert.
Is it possible to create a sqlite prepared statement in OrmLite?
If so, how to bind the query values which may change across different queries.
Is it possible to create a sqlite prepared statement in OrmLite?
You need to RTFM since ORMLite's online documentation is pretty extensive. If you look in the index for "prepared statement" you find out about the QueryBuilder which #Egor pointed out.
how to bind the query values which may change across different queries.
A little further in that section you learn about select arguments which is how you bind query values that change across queries. This is in the index under "arguments to queries".
To quote from the docs here's how you prepare a custom query:
QueryBuilder<Account, String> queryBuilder = dao.queryBuilder();
Where<Account, String> where = queryBuilder.where();
SelectArg selectArg = new SelectArg();
// define our query as 'name = ?'
where.eq("name", selectArg);
// prepare it so it is ready for later query or iterator calls
PreparedQuery<Account> preparedQuery = queryBuilder.prepare();
When you are ready to run the query you set the select argument and issue the query:
selectArg.setValue("foo");
List<Account> accounts = dao.query(preparedQuery);
Later, you can set the select argument to another value and re-run the query:
selectArg.setValue("bar");
accounts = accountDao.query(preparedQuery);
I've detected a performance problem with hibernate and native queries on Oracle. When I execute a complex SQL query with several parameters on TOAD I get the result in miliseconds. However, when I execute the same query using Hibernate this time is incremented hugely (up to four seconds or even more).
My SQL query is rather complex, return an unique value (so, the problem is not related with the time necessary to instation classes) and it contains several parameters with the the format ':nameParameter'. This query is stored in a String. For example,
String myNamedNativeQuery = "select count(*) from tables "+
"where column1 = :nameParameter1 "+
"and column2 = :nameParameter2";
//actually my sentence is much more complex!!
When I execute the sentence on TOAD it is resolved in few miliseconds. But using this sentence with Hibernate
SQLQuery query = session.createSQLQuery("myNamedNativeQuery");
query.setParameter(nameParameter1, value1);
query.setParameter(nameParameter2, value2);
query.uniqueResult();
are necessary several seconds to get the same result.
I realized if I replaced the parameters directly on the native query and then I execute the sentence using Hibernate the time decreases drastically. It would be something like that:
String strQuery = session.getNamedQuery("myNamedNativeQuery").getQueryString();
myNamedNativeQuery = myNamedNativeQuery.replace("nameParameter1", value1);
myNamedNativeQuery = myNamedNativeQuery.replace("nameParameter2", value2);
SQLQuery query = session.createSQLQuery("myNamedNativeQuery");
query.uniqueResult();
Anybody knows what's happening??
Thanks in advance.
PS: The Oracle version is 9i and Hibernate 3.2
I think what's happening with this code :
SQLQuery query = session.createSQLQuery("myNamedNativeQuery");
query.setParameter(nameParameter1, value1);
query.setParameter(nameParameter2, value2);
query.uniqueResult();
is this:
at line 1 : a query plan is created based on some expected values for your named parameters.
at line 4 : the query is executed with value1 and value2, but those values are not "good values" for the query plan that was elaborate at line 1 and so, the database is executing a very inappropriate plan for the actual values and it takes a lot of time.
Why ?
Looking at the source code of HibernateSessionImpl.createSQLQuery(...) I found this line of code:
SQLQueryImpl query = new SQLQueryImpl(
sql,
this,
factory.getQueryPlanCache().getSQLParameterMetadata( sql )
);
which is calling getQueryPlanCache() with some parameterMetaData. I assume that this metadata is not good enough.
My answer to you is:
Remove all bind parameters and use StatelessSession instead of Session
Use SQLQuery instead of query with full SQL including parameter values
StatelessSession session = sessionFactory.openStatelessSession();
I had similar problem and till I get better solution,this is what I managed to make it work.
See Hibernate parameterized sql query slow and active oracle sessions
<property name = "hibernate.temp.use_jdbc_metadata_defaults">false</property>
Add this to your hibernate.cfg.xml or update your application properties file.