I have a Spring CRUD application, with an Oracle SQL database.
I have several integration tests and use the #Sql annotation to run .sql files which put the database in a desired state before running the test. I have had no problem with any of my .sql scripts so far, although they have all be very simple INSERT and DROP statements.
I am now trying to set up a scenario in which the database holds thousands of records in a particular table. I am not bothered about the content, only the number of records.
In order to replicate this scenario, I have written the following SQL script:
DECLARE
id integer := 1;
BEGIN
WHILE id <= 3000
LOOP
INSERT INTO USER (ID, FIRST_NAME, LAST_NAME, AGE)
VALUES (id, 'John', 'Smith', 32);
id := id + 1;
END LOOP;
END;
I have ran this script using IntelliJ and it works as expected.
However, when I put the script into a #Sql annotation, I run into the following error:
org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement #1 of class path resource [sql/setUp.sql]: DECLARE id integer := 1; nested exception is java.sql.SQLException: ORA-06550: line 1, column 23:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
* & = - + ; < / > at in is mod remainder not rem
<an exponent (**)> <> or != or ~= >= <= <> and or like like2
like4 likec between || multiset member submultiset
I am at a loss as to the solution; I have encountered similar issues where the solution was just a missing semi-colon, but that doesn't seem to be the case here.
What's even more mystifying is that I tried running the following script via the #Sql annotation:
BEGIN
INSERT INTO USER (ID, FIRST_NAME, LAST_NAME, AGE)
VALUES (1, 'John', 'Smith', 32);
END;
This produced the following error, which makes even less sense:
org.springframework.jdbc.datasource.init.ScriptStatementFailedException: Failed to execute SQL script statement #1 of class path resource [sql/test.sql]: BEGIN INSERT INTO USER (ID, FIRST_NAME, LAST_NAME, AGE) VALUES (1, 'John', 'Smith', 32); nested exception is java.sql.SQLException: ORA-06550: line 1, column 87:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
;
Most interesting to note is that removing the BEGIN and END statements (leaving just the INSERT INTO statement) works.
So does Spring JDBC simply not support BEGIN/END blocks? Are there limitations to the pre test #Sql annotation that I'm not aware of? Or have I missed something painfully obvious?
Thanks to #MarkRotteveel for his suggestion, I have discovered the solution to this issue.
JDBC was indeed running each line of the script as if it was a separate statement, which is incompatible with statement blocks. This is because the separator is set to the ; symbol by default, causing JDBC to treat each block of code ended with a ; as if it was a separate script. I set this separator to the EOF symbol, which caused JDBC to treat the whole file as a single script.
My original test code looked something like this:
#Sql(scripts = {"sql/cleanup.sql", "sql/setUp.sql"})
public void testWithThousandsOfRecords() {
// do tests
}
My test code now looks like this:
#Sql(scripts = "sql/cleanup.sql")
#Sql(scripts = "sql/setUp.sql", config = #SqlConfig(separator = ScriptUtils.EOF_STATEMENT_SEPARATOR))
public void testWithThousandsOfRecords() {
// do tests
}
Note that I had to separate my cleanup and setUp scripts, as changing the separator may break some scripts that worked before.
Related
I am working on a project where I have to use Oracle Database 12c and I have to write all queries manually (so I can't use Spring Data).
For creating all tables and relationships, I use schema.sql and for template data I use data.sql.
And I have a problem with checking if table or data already exists.
In MySQL creating table would be like "create table if not exists".
In PL/SQL unfortunately, there is no equivalent for "if not exists". I replaced this functionality by:
begin
execute immediate
'CREATE TABLE user_data (
some data
)';
exception when others then if SQLCODE = -955 then null; else raise; end if;
end;
And it works when I run this script in SQL Developer or in Intellij's SQL console but the problem occurs when I want to run an application and Spring Boot tries to execute a script from schema.sql.
Output in terminal tells that:
nested exception is java.sql.SQLException: ORA-06550: line 8, column 4:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
* & = - + ; < / > at in is mod remainder not rem return
returning <an exponent (**)> <> or != or ~= >= <= <> and or
like like2 like4 likec between into using || multiset bulk
member submultiset
So it looks like Spring Boot doesn't know that it should run statement between "begin" and "end".
Any idea how can I manage the problem with database initialization ?
As a workaround, I could drop tables with every application run but it is not an optimal solution (and it wouldn't work when someone run the application for the first time).
Firstly, I would like to share two topics that seem to be relevant to this problem:
Unable to use "DROP TABLE IF EXISTS" in schema.sql for a Spring Boot application
executeSqlScript fails with Spring for PL/SQL block
There you will find a solution that should work: create a stored procedure and use in your schema.sql statement like
call recreate_table('USER_DATA','CREATE TABLE USER_DATA (SOME DATA)');
CALL statement is widely used across different databases, shortened to statement with only one semicolon and thus works well.
Secondly, I may only suppose, that the main problem is that anonymous blocks in PL/SQL (as well as other complex enough statements that may contain more than one semicolon) should be finished by a / character. I would recommend you to try to append this character to the end of your script, take a look at this and this answers, and if it does not work, create a stored procedure.
Also note that there is another way to check existence of the table (that comes over this wait for an exception pattern):
select count(*)
from user_tables t
where t.table_name = 'USER_DATA'
and rownum < 2
I have a strange problem. I'm executing insert using prepared statement like this:
try (Connection connection = connectionPool.getConnection();
PreparedStatement ps = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)) { //TODO: caching of PS
int i = 1;
ParameterMetaData pmd = ps.getParameterMetaData();
...
} catch (SQLException e) {
throw new TGFIOException("Error executing SQL command " + sql, e);
}
Insert statement is like this:
insert into dbo.CurrencyRates(RateDate, CurrencyID, Rate) values ( ?, ?, ? )
Unfortunately it fails with following exception:
com.microsoft.sqlserver.jdbc.SQLServerException: com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near the keyword 'WHERE'.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)
at com.microsoft.sqlserver.jdbc.SQLServerParameterMetaData.<init>(SQLServerParameterMetaData.java:426)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.getParameterMetaData(SQLServerPreparedStatement.java:1532)
at com.jolbox.bonecp.PreparedStatementHandle.getParameterMetaData(PreparedStatementHandle.java:246)
There is no WHERE in the statement, so I am puzzled why it fails on metadata extraction...
EDIT:
SQL Server = 10.50.2500.0 Express Edition,
Driver = sqljdbc4.jar from 4.0 package
Also, I am using getParameterMetaData because I need to set some params to null and the preferred method is to use setNull() where you need SQLType.
EDIT2:
I've tested with Driver sqljdbc41 from newest 6.0 package - results are the same
EDIT3:
I've removed call to getParameterMetaData() and it worked, unfortunately it is a generic part that should max portable, yet it does not work with this single table (inserts to other tables on the same database works fine !!!)
EDIT4:
I've tried with different insert statements for this table and all of them works fine if I skip ps.getParameterMetaData() and fail when I call it. If I try with 2 or more params I get usual near WHERE error. If I try one column insert I get an error stating that the column name is incorrect, even if it is correct and without the meta data call it works perfectly fine. I will try to trace what driver tries to do underneath...
After some tracing on what actually the driver does (many thanks a_horse_with_no_name), I've come to some funny conclusion.
The solution for my question is to:
Replace following insert statement
INSERT INTO CurrencyRates(RateDate, CurrencyID, Rate) VALUES ( ?, ?, ? )
With this statement
INSERT INTO CurrencyRates (RateDate, CurrencyID, Rate) VALUES ( ?, ?, ? )
Logic behind that is that SQL driver does some metadata extraction in the background, and it creates a query with following fragment: ... FROM CurrencyRates(RateDate WHERE ... if you do not put space after table name, yet for the ordinary call this is perfectly possible!
EDIT:
This is obviously an inconsistency as (putting aside what actually is a valid insert) it should consistently accept or reject this query no matter if I call for meta data or not.
I have an anonymous pl/sql block in an sql file and I want to execute and assign values to it in Java. My sql block looks like this
DECLARE
someInput1 NUMBER(1);
someInput2 NUMBER(2);
someString1 VARCHAR(100);
BEGIN
someInput1 := ‘&1’;
someInput2 := ‘&2’;
--get name in table A
BEGIN
SELECT a.value INTO someString1
FROM TABLE_A a
WHERE a.id = someInput1;
END;
UPDATE TABLE_B b
SET b.someStringRow = someString1
WHERE b.someIntRow = someInput2;
COMMIT;
END;
/
exit;
What I am planning to do is load the sql file in a Java String, change ‘&1’ to ?1 and execute it as a CallableStatement. However, I am getting
PLS-00103: Encountered the symbol “” when expecting one of the following
begin function package pragma procedure subtype us <an identifier> <a double quoted delimited identifier> form current cursor
Error occured at lines in DECLARE section
Is my approach an acceptable solution, if yes, what may be wrong in my approach?
Are there other better solution for my problem? Thanks
you have to remove quote marks, and replace &1 with :1, and you should be fine, also, you will need remove '/' and exit; and they relate to sqlplus and not to pl/sql directly
I am using Oracle 11g, I am executing Oracle sql script through java code. My SQL script may contain SQL statements(DDL or DML) or PL/SQL blocks, so I don't want to parse the script in my java code but used This solution to execute complete script at once. Following is the sample code, where SQLExec class is in ant jar.
This solution worked for most cases except that if sql script contains create or replace trigger it fails with java.sql.SQLSyntaxErrorException: ORA-00900: invalid SQL statement. I have also specified snippet of sql script which fails.
Please note that if I run same script through SQL Developer, it runs fine.
Following is the Java code:
private void executeSql(String sqlFilePath) {
final class SqlExecuter extends SQLExec {
public SqlExecuter() {
Project project = new Project();
project.init();
setProject(project);
setTaskType("sql");
setTaskName("sql");
}
}
SqlExecuter executer = new SqlExecuter();
executer.setSrc(new File(sqlFilePath));
executer.setDriver(args.getDriver());
executer.setPassword(args.getPwd());
executer.setUserid(args.getUser());
executer.setUrl(args.getUrl());
executer.execute();
}
SQL Script snippet:
......
......
CREATE OR REPLACE TRIGGER MY_TRG
BEFORE INSERT ON MY_TABLE
FOR EACH ROW
BEGIN
:NEW.MYNUMBER := MY_SEQUENCENUM.NEXTVAL;
END;
Following is the Exception trace:
Exception in thread "main" java.sql.SQLSyntaxErrorException: ORA-00900: invalid SQL statement
at org.apache.tools.ant.taskdefs.SQLExec.execute(SQLExec.java:398)
at com.kuldeep.OracleConnectionTest.executeSql(OracleConnectionTest.java:160)
at com.kuldeep.OracleConnectionTest.main(OracleConnectionTest.java:25)
Caused by: java.sql.SQLSyntaxErrorException: ORA-00900: invalid SQL statement
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:395)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:802)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:436)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:186)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:521)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:194)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1000)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1307)
at oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1882)
at oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1847)
at oracle.jdbc.driver.OracleStatementWrapper.execute(OracleStatementWrapper.java:301)
at org.apache.tools.ant.taskdefs.SQLExec.execSQL(SQLExec.java:499)
at org.apache.tools.ant.taskdefs.SQLExec.runStatements(SQLExec.java:470)
at org.apache.tools.ant.taskdefs.SQLExec$Transaction.runTransaction(SQLExec.java:664)
at org.apache.tools.ant.taskdefs.SQLExec$Transaction.access$000(SQLExec.java:627)
at org.apache.tools.ant.taskdefs.SQLExec.execute(SQLExec.java:370)
In the documentation it says:
Multiple statements can be provided, separated by semicolons (or the defined delimiter).
Therefore, using the semicolon character (;) as the default delimiter, SQLEXEC interprets the CREATE TRIGGER statement of your script as two statements, giving this error message as the result.
As #Reza Goodarzi mentioned the cause of invalid SQL statement is semicolon being used as the statement separator. So to solve my issue I am separating each statement with slash(/) as delimiter and followed these rules which I created myself:
Each SQL statement (not part of PL/SQL block) and PL/SQL block must end with a forwarded slash (/) in a new line.
SQL statement (not part of PL/SQL blocks) should not end with semicolon (;). I just removed semicolon from the end of statements.
For PL/SQL block do not remove the semicolon(;) from end of the block as well as from any statement contained within the block.
And by making these changes in my SQL Scripts I executed (using jdbc) each PL/SQL block and each SQL statement (not part of PL/SQL block) at a time by parsing the file myself instead of using SQLExec or any other external api/library.
I think you need to change your trigger to set your new ID
create or replace trigger MY_TRG
BEFORE insert MY_TABLE
for each row
begin
if (:new.MYNUMBER is null) then
select MY_SEQUENCENUM.nextval
into :new.MYNUMBER
from DUAL;
end if;
end;
/
or this:
create or replace trigger TG_BIU_TABLE1
before insert or update on TABLE1
for each row
begin
if (:new.ID1 is null) then
select SQ_TABLE1.nextval
into :new.ID1
from DUAL
end if
end
/
You can also add a delimiter in the execute statement, as so:
......
......
DELIMITER $$
CREATE OR REPLACE TRIGGER MY_TRG
BEFORE INSERT ON MY_TABLE
FOR EACH ROW
BEGIN
:NEW.MYNUMBER := MY_SEQUENCENUM.NEXTVAL;
END; $$
......
I also left the final part of the script just for the triggers and procedures, as the delimiter is used onward.
That did the trick for me. Courtesy of SQL Developer´s Migration tool.
My clienet(android) sends user details in the user form and my servlet just enters those details into to the database(postgre sql). I tried to give to do a sql injection attack by giving ;DELETE FROM tbl_name; in the username field.
But postgresql just treats it as a value and enters it as the username. How do I do the SQLINjection attack. (I have not done any sort of checking in the postgre sql or the servlet).Does it mean that postgresql is SQLInjection attack resistant?
I am using the following statements to insert the data:
String insert ="insert into userdetail(username,id,sno) values('"+username+"','"+userid+"','"+no+"')";
Statement stmt = conn.createStatement();
stmt.executeUpdate(insert);
The username contains ;DELETE FROM userdetail;.
I have tried the following also:
');DELETE FROM userdetail;
But it fives the following error:
org.postgresql.util.PSQLException: ERROR: unterminat
ed quoted string at or near "');"
Position: 1
I have also tried this:
','',');DELETE FROM userdetail;
This gives the following error:
17:36:46,828 INFO [STDOUT] org.postgresql.util.PSQLException: ERROR: unterminat
ed quoted string at or near "''');"
Position: 38
but does not delete the records of the table. How do I make it delete the tables records?
The key trick is that the complete statement string must
do something innovative
be still valid SQL
So far the answers have omitted the second part. But an invalid SQL statement will abort the transaction and hence most likely does nothing at all. If you set autoCommit to true that attack may be easier.
But this string should to the trick in a "clean" way:
foo', '42', '42'); delete from userdetail; --
Note: The resulting string is this (line breaks only for better reading):
insert into userdetail(username,id,sno) values('foo', '42', '42');
delete from userdetail;
-- ','21','21')
Both the INSERT part is complete and correct (assuming no unique index collisions of course) and also the DELETE is correct. The potentially offending rest is masked by the trailing SQL comment --.
hat type of attack that you are describing - even if you get the use of single quotes "correct" - will not work with PostgreSQL
It does not work because the JDBC driver does not allow to run more than one statement in a single Statement.execute() call.
It will throw an error ("invalid character" pointing to the ;)
(Sorry, this is only true for Oracle)
There are other scenarios that would work with a badly written application.
Assuming the application is checking the username/password like this (note that this is a very simplified example!)
String sql = "SELECT count(*) FROM users WHERE username = '";
sql += username;
sql += "' AND password = '";
sql += pwd;
sql += "'"
then a possible attack could be to enter the value:
' or 1=1 or '' = '
into the password field.
This would wind up with the following generated SQL
SELECT count(*) FROM users WHERE username = 'arthur'
AND password = '' or 1=1 or ''=''
Which would always be true and one could login without a password.
Try ', '', '');DELETE FROM tbl_name; to delete your table.
In simplest case, you can give ' as user name and your server should give you an error.
Besides, there is no SQL injection resistant database because SQL injection happens because of poorly coded server-side (CGI) script and not because of database weaknesses.