I am working on a project where I have to use Oracle Database 12c and I have to write all queries manually (so I can't use Spring Data).
For creating all tables and relationships, I use schema.sql and for template data I use data.sql.
And I have a problem with checking if table or data already exists.
In MySQL creating table would be like "create table if not exists".
In PL/SQL unfortunately, there is no equivalent for "if not exists". I replaced this functionality by:
begin
execute immediate
'CREATE TABLE user_data (
some data
)';
exception when others then if SQLCODE = -955 then null; else raise; end if;
end;
And it works when I run this script in SQL Developer or in Intellij's SQL console but the problem occurs when I want to run an application and Spring Boot tries to execute a script from schema.sql.
Output in terminal tells that:
nested exception is java.sql.SQLException: ORA-06550: line 8, column 4:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
* & = - + ; < / > at in is mod remainder not rem return
returning <an exponent (**)> <> or != or ~= >= <= <> and or
like like2 like4 likec between into using || multiset bulk
member submultiset
So it looks like Spring Boot doesn't know that it should run statement between "begin" and "end".
Any idea how can I manage the problem with database initialization ?
As a workaround, I could drop tables with every application run but it is not an optimal solution (and it wouldn't work when someone run the application for the first time).
Firstly, I would like to share two topics that seem to be relevant to this problem:
Unable to use "DROP TABLE IF EXISTS" in schema.sql for a Spring Boot application
executeSqlScript fails with Spring for PL/SQL block
There you will find a solution that should work: create a stored procedure and use in your schema.sql statement like
call recreate_table('USER_DATA','CREATE TABLE USER_DATA (SOME DATA)');
CALL statement is widely used across different databases, shortened to statement with only one semicolon and thus works well.
Secondly, I may only suppose, that the main problem is that anonymous blocks in PL/SQL (as well as other complex enough statements that may contain more than one semicolon) should be finished by a / character. I would recommend you to try to append this character to the end of your script, take a look at this and this answers, and if it does not work, create a stored procedure.
Also note that there is another way to check existence of the table (that comes over this wait for an exception pattern):
select count(*)
from user_tables t
where t.table_name = 'USER_DATA'
and rownum < 2
Related
I am trying to write an in-memory database in order to test my DAO objects. I get the following stack trace :
Caused by: org.h2.jdbc.JdbcSQLException: Syntax error in SQL statement " CREATE SEQUENCE CE0AOFQ AS[*] BIGINT START WITH 1 INCREMENT BY 1"; SQL statement:
CREATE SEQUENCE CE0AOFQ AS BIGINT START WITH 1 INCREMENT BY 1 [42000-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
at org.h2.message.DbException.get(DbException.java:179)
at org.h2.message.DbException.get(DbException.java:155)
at org.h2.message.DbException.getSyntaxError(DbException.java:191)
at org.h2.command.Parser.getSyntaxError(Parser.java:534)
at org.h2.command.Parser.prepareCommand(Parser.java:261)
at org.h2.engine.Session.prepareLocal(Session.java:578)
at org.h2.engine.Session.prepareCommand(Session.java:519)
at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1204)
at org.h2.jdbc.JdbcStatement.executeUpdateInternal(JdbcStatement.java:132)
at org.h2.jdbc.JdbcStatement.executeUpdate(JdbcStatement.java:121)
at org.springframework.jdbc.datasource.init.ResourceDatabasePopulator.executeSqlScript(ResourceDatabasePopulator.java:169)
When trying to execute this command into a file called init.sql:
CREATE SEQUENCE CE0AOFQ AS BIGINT START WITH 1 INCREMENT BY 1;
As far as I checked (here, for example), this query seems correct.
Any idea why this is not working ?
You cannot specify the datatype of a sequence in h2. It is always BIGINT.
Use:
CREATE SEQUENCE CE0AOFQ START WITH 1 INCREMENT BY 1;
For anyone struggling (like me) with H2 databse syntax, follow lance-java's advice and read the actual H2 grammar documentation. You may find good advices, for example:
Creates a new sequence. The data type of a sequence is BIGINT. Used values are never re-used, even when the transaction is rolled back.
Which means you cannot change it.
I am using schema.sql file to CREATE/DROP tables in my Spring Boot application and it works fine.
But when I have added procedure for altering table:
DELIMITER $$
CREATE PROCEDURE Alter_Table()
BEGIN
IF NOT EXISTS( SELECT NULL
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'test_table'
AND table_schema = 'test'
AND column_name = 'cc_test_id') THEN
alter table test_table add cc_test_id VARCHAR(128) NOT NULL;
END IF;
END $$
call Alter_Table;
I received:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException exception.
However, execution of this procedure in MySQL workbench finishes with successful results.
So, should anyone know what is the reason for this issue, let me know?
Here is the solution I found that works well enough, though it is not ideal as you have to change your SQL script.
In your application.properties file change the DataSource separator property:
spring.datasource.separator=^;
Then update your schema.sql file to look as follows:
CREATE PROCEDURE Alter_Table()
BEGIN
IF NOT EXISTS( SELECT NULL
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'test_table'
AND table_schema = 'test'
AND column_name = 'cc_test_id') THEN
alter table test_table add cc_test_id VARCHAR(128) NOT NULL;
END IF;
END ^;
call Alter_Table ^;
The DELIMITER command only works with the MySQL CLI client and Workbench and will not work for Spring Boot database initialization. Once you have removed the DELIMITER commands, Spring Boot will still throw an exception as it will not understand the ; characters in the stored procedures are not separate statements, so you have to change the datasource separator property as a workaround.
Here's the fix:
Set your spring.datasource.separator property to ^^^ END OF SCRIPT ^^^, and Spring Boot will execute your entire schema.sql script as a single statement.
Here's why:
Spring Boot splits your schema.sql script into statements, then sends each statement to the database individually for execution. By default, the script is split on semicolons, as that is the default value of the spring.datasource.separator property (per the documentation). This causes your schema.sql to be split, and the first statement executed is the following:
DELIMITER $$
CREATE PROCEDURE Alter_Table()
BEGIN
IF NOT EXISTS( SELECT NULL
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'test_table'
AND table_schema = 'test'
AND column_name = 'cc_test_id') THEN
alter table test_table add cc_test_id VARCHAR(128) NOT NULL
This is invalid SQL because the dollar-quote is never terminated in the statement.
The javadoc for org.springframework.jdbc.datasource.init.ScriptUtils.EOF_STATEMENT_SEPARATOR provides a good explanation of how my suggested solution works:
End of file (EOF) SQL statement separator: "^^^ END OF SCRIPT ^^^".
This value may be supplied as the separator to executeSqlScript(Connection, EncodedResource, boolean, boolean, String, String, String, String) to denote that an SQL script contains a single statement (potentially spanning multiple lines) with no explicit statement separator. Note that such a script should not actually contain this value; it is merely a virtual statement separator.
Customizing database populator with the appropriate separator EOF_STATEMENT_SEPARATOR resolves this issue:
#Bean
public ConnectionFactoryInitializer initializer(ConnectionFactory connectionFactory) {
var resource = new ResourceDatabasePopulator(new ClassPathResource("schema.sql"));
resource.setSeparator(ScriptUtils.EOF_STATEMENT_SEPARATOR);
var populator = new CompositeDatabasePopulator();
populator.addPopulators(resource);
}
None of the above answers were working for me in 2022, but they provide a point of focus to come up with my own working solution(s). I posted my answer under the similar question, but for completeness to save you some extra clicks , I'm providing it here too.
I had been facing this issue too and just now I found a solution. Actually 3! So you can choose any of them which suits your needs. But the key to it is the correctly formatted String which represents the DB scripts to create a SP.
I haven't figure out how to combine dropping and creation of a SP in one SQL script/String, but I'm happy with those solutions anyway.
FYI:
No special annotation or setting on the class nor method level is needed
I'm using Spring FW, which simplifies things, so don't need to do low level implementation of everything
After searching and trying various definitely working 'tips', the following resources helped me to arrive to those for sure working solutions:
Execute SQL file from Spring JDBC Template
similar question Spring and MySQL stored procedure on stackoverflow, where I have actually posted my answer there
but most useful was official Oracle tutorial on JavaSE & JDBC
and other Java/JDBC tutorial where you can see the most important bit of formatting the sql Java String
Solution 1: using JdbcTemplate and Java String
It is very important to have trailing spaces in your nicely formatted String sql script
String sqlSP = "CREATE PROCEDURE test_stored_proc (IN pInstanceId varchar(255)) " +
"BEGIN " +
"SELECT * FROM vw_subscriptions WHERE instanceId = pInstanceId; " +
"END";
jdbcTemplate.execute(sqlSP);
Solution 2: using ResourceDatabasePopulator & ClassPathResource to load SQL script in a file
ClassPathResource resource = new ClassPathResource("/data/create-sp_TEST_STORED_PROC.sql");
jdbcTemplate.execute("DROP procedure IF EXISTS `test_stored_proc`;");
ResourceDatabasePopulator databasePopulator = new ResourceDatabasePopulator(resource);
databasePopulator.setSeparator(ScriptUtils.EOF_STATEMENT_SEPARATOR);
databasePopulator.execute(testSetupDataSource);
Solution 3: using ScriptUtils & ClassPathResource to load SQL script in a file
ClassPathResource resource = new ClassPathResource("/data/create-sp_TEST_STORED_PROC.sql");
jdbcTemplate.execute("DROP procedure IF EXISTS `test_stored_proc`;");
ScriptUtils.executeSqlScript(Objects.requireNonNull(testSetupDataSource).getConnection(), new EncodedResource(resource),
false, false,
ScriptUtils.DEFAULT_COMMENT_PREFIX, ScriptUtils.EOF_STATEMENT_SEPARATOR,
ScriptUtils.DEFAULT_BLOCK_COMMENT_START_DELIMITER, ScriptUtils.DEFAULT_BLOCK_COMMENT_END_DELIMITER);
Content of the create-sp_TEST_STORED_PROC.sql script
For solution #1 and #2 the same rule as for the 1st solution applies:
The formatting and presence of whitespace characters is important especially an extra trailing space at the end of each line.
So the below code is archived by my vim setup of representing whitespace characters
CREATE·PROCEDURE·test_stored_proc·(IN·pInstanceId·varchar(255))~¬
BEGIN~¬
–→SELECT·*·FROM·vw_subscriptions·WHERE·instanceId·=·pInstanceId;~¬
END;~¬
I believe it is internally represented as one line piece of string:
CREATE PROCEDURE test_stored_proc (IN pInstanceId varchar(255)) BEGIN SELECT * FROM vw_subscriptions WHERE instanceId = pInstanceId; END
Almost full source-code can be found at my GitHub
If you try to load scripts for SpringBootTest, #Sql and #SqlConfig will work.
#Sql(
"/db/integration/stored-procedures.sql",
config = SqlConfig(separator = "$$")
)
In sql scripts, you should remove the DELIMITER $$ statement. Like this:
CREATE PROCEDURE <sp_name>()
<your owesome definition>
END $$
If the script file contains only one CREATE PROCEDURE statement, separator=EOF_STATEMENT_SEPARATOR will be fine.
I try to set up the sql file from changeset to run a store procedure that is saved into my project but it returns sql error.
Here I declared the sqlfile:
<sqlFile path="/db/FindAllFriends.sql" endDelimiter="//" relativeToChangelogFile="true" splitStatements="true" stripComments="true" />
Here is my procedure:
DELIMITER //
CREATE PROCEDURE aaa()
BEGIN
END //
DELIMITER ;
If I let the "delimiter" in script it will return an sql error (you have an error in your sql syntax; check the manual....). If I remove it, it will be ok but it works only with ";" delimiter.
If I tried to drop and then create the procedure (or multiple statements):
Drop procedure if exists aaa;
CREATE PROCEDURE aaa()
BEGIN
END;
I receive: Error executing sql drop procedure if exists aaa;
My problem is that I need first to "drop procedure if exists" and after to create it, and I'm forced to use "delimiter" inside of script.
I don't know if it is a problem with the mysql parser, liquibase error or I made a mistake in my script.
I'm struggling with procedures (on Oracle) for some time now, and I start to believe that Liquibase was not made with procedures in mind ;)
Anyway - maybe dropProcedure and createProcedure changes would be sufficient for you?
I find executeCommand change with sqlplus to be the best solution to run scripts on Oracle...
Finally I was able run all scripts; I don't know what I made wrong when I tried first time, but now works. I had to declare the endDelimiter in sqlFile and to use this delimiter into script, also I deleted the declaration of "Delimiter //" from script
I'm trying to track the amount of redo being generated during a database session with the following query:
SELECT a.name, b.VALUE
FROM v$statname a, v$mystat b
WHERE a.statistic# = b.statistic# AND a.name = 'redo size';
This query works directly in SQL*Plus and Toad, but I get an ORA-00911 exception using JDBC, and I've narrowed it down to the "statistic#" column name.
How do I get around this?
The column name statistic# is not the problem.
My bet is that you also send the terminating ; from inside your Java program.
But you may not include the the ; when executing a SQL statement through JDBC (at least not in Oracle and some other DBMS).
Remove the ; from your SQL String and it should be fine.
put it in double quotes - that should let you call a field anything in Oracle
Switch on JDBC logging, check your driver documentation for how to do this. In the JDBC log you see the actual statement prepared and executed in the DB. This eliminates one possible cause for the error.
I am creating and testing a stored procedure through remote using JUnit , my user name is test and this is my code,
String sql = "create procedure test.firstProc3 AS select GETDATE()";
cursor.execute(sql);
cursor.executeQuery("firstProc3");
ANd to drop the procedure:
String dropSQL = "DROP PROCEDURE test.firstProc3";
cursor.execute(dropSQL);
Since the Drop Procedure is not working, I am unable to run the same SP for the second time ( new session) My error is:
There is already an object named 'firstProc3' in the database.
When I gave sp_help on the server side,
the table had the row with the value
firstProc3 test stored procedure
However when I give
DROP PROCEDURE test.firstProc3 in the Query analyzer, the row is getting deleted from the table.
What could be the issue in trying to do the same operation through Junit?
Are there any permissions to be set?
PS - the user test has the db_ddladmin enabled.
I managed to finally solve the problem by using the following :
declare #object_id int
select #object_id = object_id('test.firstProc5')
EXEC sp_MSdrop_object #object_id
GO
This is removing the data from the table too.And I am able to successfully run the Testcase multiple times !
Thanks for your help :)
The link I referred to is here, hope it helps someone:
http://www.mssqlcity.com/Articles/Undoc/SQL2000UndocSP.htm
Based upon the error message, I doubt it is a permissions issue. It looks like your cursor object is running the create again when you're trying to drop, or you're never really dropping the proc before running your unit test again, or the tests are running out of logical order in some fashion. Can you instantiate a new cursor object and try the drop with that, or verify the SQL being run against the database using profiler or some debug output?
Beyond that, I would be curious to see if changing your SQL commands to check for proc existence makes the error go away (not that this is the right answer):
if object_id('test.firstProc3') is null begin
-- Dynamic SQL because create must be first in a batch
exec ('create procedure test.firstProc3 as select getdate()')
end
if object_id('test.firstProc3') is not null begin
drop procedure test.firstProc3
end
Maybe something small, and maybe it is even not the case, but have you tried to use escaping in your drop statement?
something like
drop procedure [test].[firstProc3]
I can remember that i had an issue with this when i called a drop table from c# code...