At the moment I only know that you can do this in JPA
SELECT art_bezeichnung FROM table WHERE art_nr = :parameter
.setParameter("parameter", "H03")
But can you also add a whole where clause via a parameter
SELECT art_bezeichnung FROM table :parameter
.setParameter("parameter", "WHERE art_nr = H03")
I searched the Internet for it, but couldn´t find a clear answer.
That's not possible because of possible SQL-Injection.
Use a if statement instead:
if (Parameter is set) add where to SQL Statement and Parameter.
I'm trying to move a large number of records from one MySQL instance two another inside RDS. They are on different VPCs and different AWS accounts, so I can't create a data pipeline that would do the copy for me.
I've written a quick java program that connects to both the import database and the export database and does the following:
query the import database for the highest in table.primary_key with SELECT MAX(primary_key) FROM table
get a result set from the export table with SELECT * FROM table WHERE(primary_key > max_from_import) LIMIT 1000000
create a PreparedStatement object from the import connection and set the queryString to INSERT INTO table (col1....coln) VALUES (?....n?)
iterate over the result set and set the prepared statement columns to the ones from the result cursor (with some minor manipulations to the data), call execute on the PreparedStatement object, clear its' parameters, then move to the next result.
With this method I'm able to see around 100000 records being imported an hour, but I know that from this question that a way to optimize inserts is not to create a new query each time, but to append more data with each insert. i.e.
INSERT INTO table (col1...coln) VALUES (val1...valn), (val1...valn)....(val1...valn);
Does the jdbc driver know to do this, or is there some sort of optimization I can make on my end to improve insert run time?
UPDATE:
Both answers recommended using the add and execute batch, as well as removing auto commit. Removing auto commit saw a slight improvement (10%), doing the batch yielded a run time of less than 50% of the individual inserts.
You need to use batch insert. Internally, Connector/J (MySQL JDBC driver) can rewrite batch inserts into multi values insert statements.
(Note that this is the default Connector/J behavior. You can add
the option useServerPrepStmts=true to the JDBC url to enable server side prepared statements)
The code looks like the following:
try(PreparedStatement stmt = connection.prepareStatement(sql)) {
for(value : valueList) {
stmt.clearParameters();
stmt.setParameter(1, value);
stmt.addBatch();
}
stmt.executeBatch();
}
The code above will generate a multi value insert:
INSERT tablename(field) VALUES(value1), (value2), (value3) ...
First create a JDBC connection to Destination database and make its auto commit property to false.
After that in a loop do the following
Read N(for example 1000) number of rows from Source database and write that to destination database.
After some inserts commit destination database connection.
Sample code to get more idea is given below
Connection sourceCon = getSourceDbConnction();
Connection destCon = getDestinationDbConnction();
destCon.setAutoCommit(false);
int i=0;
String query;
while((query=getInsertQuery()!=null)
{
statement.executeUpdate(query);
i++;
if(i%10 == 0)
{
destCon.commit();
i=0;
}
}
destCon.commit();
The getInsertQuery function should give string in INSERT INTO table (col1...coln) VALUES (val1...valn), (val1...valn)....(val1...valn); format.
Also it should return null, if all tables are processed.
If you are using Prepared Statements, you can use addBatch and executeBatch functions. Inside loop add values using addBatch function. After some inserts call executeBatch.
I'm trying to convert a MySQL stored procedure to java code. It's pretty simple SQL. It creates three temporary tables and then does a select based on them. However, I'm having trouble with SQLWarnings related to Data Truncation being thrown while creating the last temp table.
I'm not looking for an explanation of the warnings themselves. I was getting the same warnings during execution of the stored procedure (due to improperly formatted datetime fields), only there it didn't cause me any problems. In the java version, after Statement.executeUpdate() throws an SQLWarning, the temp table I was attempting to create doesn't exist.
How can I suppress the SQLWarnings? I'd prefer to be able to log a message noting the warning text, but I want my temp table to successfully be created as well.
My code looks something like this:
sqlStatement = "create temporary table mytemptable as (select ...) ;";
try {
stmt.executeUpdate(sqlStatement);
// Throws SQLWarning about Data Truncation
} catch (SQLWarning sqlW) {
logInfo("SQLWarning Caught: " + sqlW.getMessage());
}
stmt.executeQuery("select * from mytemptable ");
// Throws SQLException because mytemptable doesn't exist
To me it's a little strange but the equivalent of this worked:
create temporary table mytemptable ignore select ...;
This way, no exception is thrown by jdbc, but you can still call getWarnings() on the Statement object to get a list of warnings that were ignored.
Thanks Norbert van Nobelen for helping me figure this out in the comments of the OP.
I recently encountered the following problem with buiding queries in jooq (version 3.1.0):
I want to build delete statement with order and limit constraints. So, my aim is to build something like this:
DELETE FROM table ORDER BY field DESC LIMIT 1 (this is MySql syntax)
But i haven't found nesessary methods in result delete query object:
DSLContext context = createContext();
DeleteWhereStep delete = context.delete(createTable(table));
DeleteConditionStep whereStep = delete.where(condition);
whereStep.orderBy(...)//and no such method here
There are all nesessary methods in select statements and none for delete.
Is it possible to set order and limit for delete request in jooq?
As of jOOQ 3.2, these sorts of extensions are currently not implemented yet. Chances are, that #203 could be implemented in jOOQ 3.3, though.
In the mean time, you have two options:
Resort to plain SQL
i.e. write something like:
context.execute("DELETE FROM {0} ORDER BY {1} DESC LIMIT 1",
createTable(table),
field);
Manually transform your SQL statement into something equivalent
I suspect that the ORDER BY .. LIMIT extension to the MySQL DELETE statement is just sugar for:
DELETE FROM table t
WHERE t.id IN (
SELECT id FROM table
ORDER BY field LIMIT 1
)
Or with jOOQ:
context.delete(TABLE)
.where(TABLE.ID.in(
select(TABLE.ID)
.from(TABLE)
.orderBy(TABLE.FIELD)
.limit(1)
))
I am getting below exception, when trying to insert a batch of rows to an existing table
ORA-00942: table or view does not exist
I can confirm that the table exists in db and I can insert data to that table using oracle
sql developer. But when I try to insert rows using preparedstatement in java, its throwing table does not exist error.
Please find the stack trace of error below
java.sql.SQLException: ORA-00942: table or view does not exist
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
at oracle.jdbc.ttc7.TTIoer.processError(TTIoer.java:289)
at oracle.jdbc.ttc7.Oall7.receive(Oall7.java:573)
at oracle.jdbc.ttc7.TTC7Protocol.doOall7(TTC7Protocol.java:1889)
at oracle.jdbc.ttc7.TTC7Protocol.parseExecuteFetch(TTC7Protocol.java:1093)
at oracle.jdbc.driver.OracleStatement.executeNonQuery(OracleStatement.java:2047)
at oracle.jdbc.driver.OracleStatement.doExecuteOther(OracleStatement.java:1940)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout>>(OracleStatement.java:2709)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:589)
at quotecopy.DbConnection.insertIntoDestinationDb(DbConnection.java:591)
at quotecopy.QuoteCopier.main(QuoteCopier.java:72)
Can anyone suggest the reasons for this error ?
Update : Issue solved
There was no problem with my database connection properties or with my table or view name. The solution to the problem was very strange. One of the columns that I was trying insert was of Clob type. As I had a lot of trouble handling clob data in oracle db before, gave a try by replacing the clob setter with a temporary string setter and the same code executed with out any problems and all the rows were correctly inserted!!!.
ie. peparedstatement.setClob(columnIndex, clob)
was replaced with
peparedstatement.setString(columnIndex, "String")
Why an error table or view does exist error was throws for error in inserting clob data. Could anyone of you please explain ?
Thanks a lot for your answers and comments.
Oracle will also report this error if the table exists, but you don't have any privileges on it. So if you are sure that the table is there, check the grants.
There seems to be some issue with setCLOB() that causes an ORA-00942 under some circumstances when the target table does exist and is correctly privileged. I'm having this exact issue now, I can make the ORA-00942 go away by simply not binding the CLOB into the same table.
I've tried setClob() with a java.sql.Clob and setCLOB() with an oracle.jdbc.CLOB but with the same result.
As you say, if you bind as a string the problem goes away - but this then limits your data size to 4k.
From testing it seems to be triggered when a transaction is open on the session prior to binding the CLOB. I'll feed back when I've solved this...checking Oracle support.
There was no problem with my database connection properties or with my table or view name. The solution to the problem was very strange. One of the columns that I was trying insert was of Clob type. As I had a lot of trouble handling clob data in oracle db before, gave a try by replacing the clob setter with a temporary string setter and the same code executed with out any problems and all the rows were correctly inserted!!!.
ie. peparedstatement.setClob(columnIndex, clob)
was replaced with
peparedstatement.setString(columnIndex, "String")
#unbeli is right. Not having appropriate grants on a table will result in this error. For what it's worth, I recently experienced this. I was experiencing the exact problem that you described, I could execute insert statements through sql developer but would fail when using hibernate. I finally realized that my code was doing more than the obvious insert. Inserting into other tables that did not have appropriate grants. Adjusting grant privileges solved this for me.
Note: Don't have reputation to comment, otherwise this may have been a comment.
We experienced this issue on a BLOB column. Just in case anyone else lands on this question when encountering this error, here is how we resolved the issue:
We started out with this:
preparedStatement.setBlob(parameterIndex, resultSet.getBlob(columnName)); break;
We resolved the issue by changing that line to this:
java.sql.Blob blob = resultSet.getBlob(columnName);
if (blob != null) {
java.io.InputStream blobData = blob.getBinaryStream();
preparedStatement.setBinaryStream(parameterIndex, blobData);
} else {
preparedStatement.setBinaryStream(parameterIndex, null);
}
I found how to solve this problem without using JDBC's setString() method which limits the data to 4K.
What you need to do is to use preparedStatement.setClob(int parameterIndex, Reader reader). At least this is what that worked for me. Thought Oracle drivers converts data to character stream to insert, seems like not. Or something specific causing an error.
Using a characterStream seems to work for me. I am reading tables from one db and writing to another one using jdbc. And i was getting table not found error just like it is mentioned above. So this is how i solved the problem:
case Types.CLOB: //Using a switch statement for all columns, this is for CLOB columns
Clob clobData = resultSet.getClob(columnIndex); // The source db
if (clobData != null) {
preparedStatement.setClob(columnIndex, clobData.getCharacterStream());
} else {
preparedStatement.setClob(columnIndex, clobData);
}
clobData = null;
return;
All good now.
Is your script providing the schema name, or do you rely on the user logged into the database to select the default schema?
It might be that you do not name the schema and that you perform your batch with a system user instead of the schema user resulting in the wrong execution context for a script that would work fine if executed by the user that has the target schema set as default schema. Your best action would be to include the schema name in the insert statements:
INSERT INTO myschema.mytable (mycolums) VALUES ('myvalue')
update: Do you try to bind the table name as bound value in your prepared statement? That won't work.
It works for me:
Clob clob1;
while (rs.next()) {
rs.setString(1, rs.getString("FIELD_1"));
clob1 = rs.getClob("CLOB1");
if (clob1 != null) {
sta.setClob(2, clob1.getCharacterStream());
} else {
sta.setClob(2, clob1);
}
clob1 = null;
sta.setString(3, rs.getString("FIELD_3"));
}
Is it possible that you are doing INSERT for VARCHAR but doing an INSERT then an UPDATE for CLOB?
If so, you'll need to grant UPDATE permissions to the table in addition to INSERT.
See https://stackoverflow.com/a/64352414/1089967
Here I got the solution for the question. The problem is on glass fish if you are using it. When you create JNDI name make sure pool name is correct and pool name is the name of connection pool name that you are created.