I am using schema.sql file to CREATE/DROP tables in my Spring Boot application and it works fine.
But when I have added procedure for altering table:
DELIMITER $$
CREATE PROCEDURE Alter_Table()
BEGIN
IF NOT EXISTS( SELECT NULL
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'test_table'
AND table_schema = 'test'
AND column_name = 'cc_test_id') THEN
alter table test_table add cc_test_id VARCHAR(128) NOT NULL;
END IF;
END $$
call Alter_Table;
I received:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException exception.
However, execution of this procedure in MySQL workbench finishes with successful results.
So, should anyone know what is the reason for this issue, let me know?
Here is the solution I found that works well enough, though it is not ideal as you have to change your SQL script.
In your application.properties file change the DataSource separator property:
spring.datasource.separator=^;
Then update your schema.sql file to look as follows:
CREATE PROCEDURE Alter_Table()
BEGIN
IF NOT EXISTS( SELECT NULL
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'test_table'
AND table_schema = 'test'
AND column_name = 'cc_test_id') THEN
alter table test_table add cc_test_id VARCHAR(128) NOT NULL;
END IF;
END ^;
call Alter_Table ^;
The DELIMITER command only works with the MySQL CLI client and Workbench and will not work for Spring Boot database initialization. Once you have removed the DELIMITER commands, Spring Boot will still throw an exception as it will not understand the ; characters in the stored procedures are not separate statements, so you have to change the datasource separator property as a workaround.
Here's the fix:
Set your spring.datasource.separator property to ^^^ END OF SCRIPT ^^^, and Spring Boot will execute your entire schema.sql script as a single statement.
Here's why:
Spring Boot splits your schema.sql script into statements, then sends each statement to the database individually for execution. By default, the script is split on semicolons, as that is the default value of the spring.datasource.separator property (per the documentation). This causes your schema.sql to be split, and the first statement executed is the following:
DELIMITER $$
CREATE PROCEDURE Alter_Table()
BEGIN
IF NOT EXISTS( SELECT NULL
FROM INFORMATION_SCHEMA.COLUMNS
WHERE table_name = 'test_table'
AND table_schema = 'test'
AND column_name = 'cc_test_id') THEN
alter table test_table add cc_test_id VARCHAR(128) NOT NULL
This is invalid SQL because the dollar-quote is never terminated in the statement.
The javadoc for org.springframework.jdbc.datasource.init.ScriptUtils.EOF_STATEMENT_SEPARATOR provides a good explanation of how my suggested solution works:
End of file (EOF) SQL statement separator: "^^^ END OF SCRIPT ^^^".
This value may be supplied as the separator to executeSqlScript(Connection, EncodedResource, boolean, boolean, String, String, String, String) to denote that an SQL script contains a single statement (potentially spanning multiple lines) with no explicit statement separator. Note that such a script should not actually contain this value; it is merely a virtual statement separator.
Customizing database populator with the appropriate separator EOF_STATEMENT_SEPARATOR resolves this issue:
#Bean
public ConnectionFactoryInitializer initializer(ConnectionFactory connectionFactory) {
var resource = new ResourceDatabasePopulator(new ClassPathResource("schema.sql"));
resource.setSeparator(ScriptUtils.EOF_STATEMENT_SEPARATOR);
var populator = new CompositeDatabasePopulator();
populator.addPopulators(resource);
}
None of the above answers were working for me in 2022, but they provide a point of focus to come up with my own working solution(s). I posted my answer under the similar question, but for completeness to save you some extra clicks , I'm providing it here too.
I had been facing this issue too and just now I found a solution. Actually 3! So you can choose any of them which suits your needs. But the key to it is the correctly formatted String which represents the DB scripts to create a SP.
I haven't figure out how to combine dropping and creation of a SP in one SQL script/String, but I'm happy with those solutions anyway.
FYI:
No special annotation or setting on the class nor method level is needed
I'm using Spring FW, which simplifies things, so don't need to do low level implementation of everything
After searching and trying various definitely working 'tips', the following resources helped me to arrive to those for sure working solutions:
Execute SQL file from Spring JDBC Template
similar question Spring and MySQL stored procedure on stackoverflow, where I have actually posted my answer there
but most useful was official Oracle tutorial on JavaSE & JDBC
and other Java/JDBC tutorial where you can see the most important bit of formatting the sql Java String
Solution 1: using JdbcTemplate and Java String
It is very important to have trailing spaces in your nicely formatted String sql script
String sqlSP = "CREATE PROCEDURE test_stored_proc (IN pInstanceId varchar(255)) " +
"BEGIN " +
"SELECT * FROM vw_subscriptions WHERE instanceId = pInstanceId; " +
"END";
jdbcTemplate.execute(sqlSP);
Solution 2: using ResourceDatabasePopulator & ClassPathResource to load SQL script in a file
ClassPathResource resource = new ClassPathResource("/data/create-sp_TEST_STORED_PROC.sql");
jdbcTemplate.execute("DROP procedure IF EXISTS `test_stored_proc`;");
ResourceDatabasePopulator databasePopulator = new ResourceDatabasePopulator(resource);
databasePopulator.setSeparator(ScriptUtils.EOF_STATEMENT_SEPARATOR);
databasePopulator.execute(testSetupDataSource);
Solution 3: using ScriptUtils & ClassPathResource to load SQL script in a file
ClassPathResource resource = new ClassPathResource("/data/create-sp_TEST_STORED_PROC.sql");
jdbcTemplate.execute("DROP procedure IF EXISTS `test_stored_proc`;");
ScriptUtils.executeSqlScript(Objects.requireNonNull(testSetupDataSource).getConnection(), new EncodedResource(resource),
false, false,
ScriptUtils.DEFAULT_COMMENT_PREFIX, ScriptUtils.EOF_STATEMENT_SEPARATOR,
ScriptUtils.DEFAULT_BLOCK_COMMENT_START_DELIMITER, ScriptUtils.DEFAULT_BLOCK_COMMENT_END_DELIMITER);
Content of the create-sp_TEST_STORED_PROC.sql script
For solution #1 and #2 the same rule as for the 1st solution applies:
The formatting and presence of whitespace characters is important especially an extra trailing space at the end of each line.
So the below code is archived by my vim setup of representing whitespace characters
CREATE·PROCEDURE·test_stored_proc·(IN·pInstanceId·varchar(255))~¬
BEGIN~¬
–→SELECT·*·FROM·vw_subscriptions·WHERE·instanceId·=·pInstanceId;~¬
END;~¬
I believe it is internally represented as one line piece of string:
CREATE PROCEDURE test_stored_proc (IN pInstanceId varchar(255)) BEGIN SELECT * FROM vw_subscriptions WHERE instanceId = pInstanceId; END
Almost full source-code can be found at my GitHub
If you try to load scripts for SpringBootTest, #Sql and #SqlConfig will work.
#Sql(
"/db/integration/stored-procedures.sql",
config = SqlConfig(separator = "$$")
)
In sql scripts, you should remove the DELIMITER $$ statement. Like this:
CREATE PROCEDURE <sp_name>()
<your owesome definition>
END $$
If the script file contains only one CREATE PROCEDURE statement, separator=EOF_STATEMENT_SEPARATOR will be fine.
Related
I am working on a project where I have to use Oracle Database 12c and I have to write all queries manually (so I can't use Spring Data).
For creating all tables and relationships, I use schema.sql and for template data I use data.sql.
And I have a problem with checking if table or data already exists.
In MySQL creating table would be like "create table if not exists".
In PL/SQL unfortunately, there is no equivalent for "if not exists". I replaced this functionality by:
begin
execute immediate
'CREATE TABLE user_data (
some data
)';
exception when others then if SQLCODE = -955 then null; else raise; end if;
end;
And it works when I run this script in SQL Developer or in Intellij's SQL console but the problem occurs when I want to run an application and Spring Boot tries to execute a script from schema.sql.
Output in terminal tells that:
nested exception is java.sql.SQLException: ORA-06550: line 8, column 4:
PLS-00103: Encountered the symbol "end-of-file" when expecting one of the following:
* & = - + ; < / > at in is mod remainder not rem return
returning <an exponent (**)> <> or != or ~= >= <= <> and or
like like2 like4 likec between into using || multiset bulk
member submultiset
So it looks like Spring Boot doesn't know that it should run statement between "begin" and "end".
Any idea how can I manage the problem with database initialization ?
As a workaround, I could drop tables with every application run but it is not an optimal solution (and it wouldn't work when someone run the application for the first time).
Firstly, I would like to share two topics that seem to be relevant to this problem:
Unable to use "DROP TABLE IF EXISTS" in schema.sql for a Spring Boot application
executeSqlScript fails with Spring for PL/SQL block
There you will find a solution that should work: create a stored procedure and use in your schema.sql statement like
call recreate_table('USER_DATA','CREATE TABLE USER_DATA (SOME DATA)');
CALL statement is widely used across different databases, shortened to statement with only one semicolon and thus works well.
Secondly, I may only suppose, that the main problem is that anonymous blocks in PL/SQL (as well as other complex enough statements that may contain more than one semicolon) should be finished by a / character. I would recommend you to try to append this character to the end of your script, take a look at this and this answers, and if it does not work, create a stored procedure.
Also note that there is another way to check existence of the table (that comes over this wait for an exception pattern):
select count(*)
from user_tables t
where t.table_name = 'USER_DATA'
and rownum < 2
At the first stage i'm managing my app via JDBC, so i'm the resposible to build and validate all the SQL. But i was wondering if JPA could give me a hand on these tasks.
So, at this moment i've already have a DbaUser model, which was generated from the DBA_USERS table on the OracleDB, and i can actually list all of them.
However, i'm trying to manage to create or update more, but whenever i try to create using
em.createQuery("CREATE USER C##ANTONIO IDENTIFIED BY Antionio123").executeUpdate();
An syntax exception is launched: The query does not start with a valid identifier, has to be either SELECT, UPDATE or DELETE FROM.
Could you guys enlighten me a bit more or pointing me to some proper tutorials? I've been googling but nothing concrete apprears on Oracle DBs system tables.
Update1 (Thanks to #JB Nizet)
After replacing the execution of the query from JPQL to Native SQL, i've got an error such as:
Query: DataModifyQuery(sql="CREATE USER C? IDENTIFIED BY ANTONIO123").
I've replaced the hashtags with a scape character "...C##..." with "...C\#\#..." but the issue earns a different flavour:
Query: DataModifyQuery(sql="CREATE USER C\? IDENTIFIED BY ANTONIO123")
... and i really need to send the "##" to the Oracle DB. How can i force these special characters?
Update 2
So...after googling a bit more, i've found out positional parameters, and i've also discovered that we cannot have named paramteres on JPA native queries. After this, i've tried:
em.createNativeQuery("CREATE USER ?1 IDENTIFIED BY ANTONIO123").setParameter(1, "C##ANTONIO").executeUpdate();
Which triggers: ORA-01935: missing user or role name
alongside with
Error Code: 1935
Call: CREATE USER ? IDENTIFIED BY ANTONIO123
Which tells me that this binding doesn't work. Is there another way to do it?
Kind regards and thanks in advance,
Sammy
createQuery() expects a JPQL query. What you passed is not JPQL. It's SQL.
Use createNativeQuery() to execute SQL.
To create a common user (prefixed with C##) you (i.e. your JPA connection pool user) need a specific priviledges.
CREATE ROLE and
SET CONTAINER
Those are not a typical privileges granted to a JPA connection, so I'm guessing you will fail with the creation of a new common user.
Additionally you need to be connected to the root container.
The further example are plain JDBC called from Groovy, it should be easy possible to pass it to JPA if you get the DB connection.
def stmt = con.prepareStatement("SELECT SYS_CONTEXT('USERENV', 'CON_NAME') CON_NAME FROM dual")
def rs = stmt.executeQuery()
while(rs.next())
{
println "container name= ${rs.getString('CON_NAME')}"
}
gives
container name= CDB$ROOT
Note that if you are connected to a local DB, you get an error while trying to create a user prefixed with C##
ORA-65094: invalid local user or role name
If both conditions are fulfilled, it is possible to create / drop the common user:
String cu = "create user \"C##TEST\" identified by password123 profile \"DEFAULT\" account unlock"
con.createStatement().execute(cu)
resp.
cu = "drop user \"C##TEST\""
con.createStatement().execute(cu)
Finally should be stated, that this exercise was done for the aim of completeness only. I do not see a real use case for a JPA pool connection to be granted such privileges and connecting the root container. The database maintenance is typically done not using JPA.
Another option would be to create a stored procedure in the database and then invoke stored procedure from JPA, thus you do not need to bother about caveats and syntax.
E.g.assume that JPA provider is EclipseLink
Database stored procedure
CREATE OR REPLACE PROCEDURE p_user_creation (p_username IN VARCHAR2,
p_password IN VARCHAR2,
p_return OUT NUMBER)
IS
v_syntax VARCHAR2 (256);
BEGIN
IF (p_username IS NOT NULL)
THEN
v_syntax :=
'CREATE USER '''
|| p_username
|| ''' IDENTIFIED BY '''
|| p_password
|| '''';
EXECUTE IMMEDIATE v_syntax;
p_return := 0;
END IF;
EXCEPTION
WHEN OTHERS
THEN
raise_application_error (-20002, 'An error has occurred!');
END;
Java code snippet to invoke stored procedure
try {
Integer returnValue = null;
StoredProcedureQuery storedProcedureQuery =
getEntityManager().createStoredProcedureQuery("p_user_creation");
storedProcedureQuery.registerStoredProcedureParameter("p_username", String.class, ParameterMode.IN);
storedProcedureQuery.registerStoredProcedureParameter("p_password", String.class, ParameterMode.IN);
storedProcedureQuery.registerStoredProcedureParameter("p_return", Integer.class, ParameterMode.OUT);
storedProcedureQuery.setParameter("p_username", "SCOTT");
storedProcedureQuery.setParameter("p_password", "tiger");
storedProcedureQuery.execute();
returnValue = (Integer) storedProcedureQuery.getOutputParameterValue("p_return");
} catch (Exception e) {
log.error("Error " + e.getMessage());
}
I try to set up the sql file from changeset to run a store procedure that is saved into my project but it returns sql error.
Here I declared the sqlfile:
<sqlFile path="/db/FindAllFriends.sql" endDelimiter="//" relativeToChangelogFile="true" splitStatements="true" stripComments="true" />
Here is my procedure:
DELIMITER //
CREATE PROCEDURE aaa()
BEGIN
END //
DELIMITER ;
If I let the "delimiter" in script it will return an sql error (you have an error in your sql syntax; check the manual....). If I remove it, it will be ok but it works only with ";" delimiter.
If I tried to drop and then create the procedure (or multiple statements):
Drop procedure if exists aaa;
CREATE PROCEDURE aaa()
BEGIN
END;
I receive: Error executing sql drop procedure if exists aaa;
My problem is that I need first to "drop procedure if exists" and after to create it, and I'm forced to use "delimiter" inside of script.
I don't know if it is a problem with the mysql parser, liquibase error or I made a mistake in my script.
I'm struggling with procedures (on Oracle) for some time now, and I start to believe that Liquibase was not made with procedures in mind ;)
Anyway - maybe dropProcedure and createProcedure changes would be sufficient for you?
I find executeCommand change with sqlplus to be the best solution to run scripts on Oracle...
Finally I was able run all scripts; I don't know what I made wrong when I tried first time, but now works. I had to declare the endDelimiter in sqlFile and to use this delimiter into script, also I deleted the declaration of "Delimiter //" from script
I have Oracle DB 11g Enterprise Edition and I want to create a table by reading the sql script from a file.Through java code I am reading following sql script from a file and storing it in a String sqlBlock:
CREATE SEQUENCE VerHist_SeqNum
START WITH 1
INCREMENT BY 1;
CREATE TABLE VerHist
(
SequenceNumber NUMBER(10,0) NOT NULL,
SQLFileName VARCHAR2(100) NOT NULL,
STATUS VARCHAR2(10) NOT NULL,
AppliedDate DATE NOT NULL,
DateCreated DATE
DEFAULT (SYSDATE),
DateUpdated DATE
DEFAULT (SYSDATE),
CONSTRAINT PK_VerHist PRIMARY KEY( SequenceNumber ),
CONSTRAINT UC_VerHist_SQLFileNa UNIQUE( SQLFileName )
);
CREATE OR REPLACE TRIGGER VerHist_SeqNum_TRG
BEFORE INSERT
ON VerHist
FOR EACH ROW
BEGIN
SELECT VerHist_SeqNum.NEXTVAL INTO :NEW.SequenceNumber
FROM DUAL;
END;
When I execute this query it gives
java.sql.SQLException: ORA-00911: invalid character\n
at
oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331) at
oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288) at
oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:743) at
oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:207) at
oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:946)
at
oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168)
at
oracle.jdbc.driver.OracleStatement.executeInternal(OracleStatement.java:1687)
at
oracle.jdbc.driver.OracleStatement.execute(OracleStatement.java:1653)
Following is my code to execute the sql block:
Statement stmt = conn.createStatement();
String sqlBlock = //"this contains the content of the file (it contains \n charters)";
stmt.execute(sqlBlock);
Is the newline charter invalid here, if yes, how to get this working otherwise?
Please note that when I copy paste the contents of this file and run the script through Oracle SQL Developer it runs fine.
I think the \n reference is a red-herring, and an artefact of how the error is being logged. You're trying to run two SQL statements, separated by a semi-colon, in one execute. That is not allowed. The semi-colon is a statement separator in SQL*Plus, not in SQL, and will generate an ORA-00911 even with a single statement. And execute has to be a single statement.
If you were doing DML you could wrap the statements in a PL/SQL block and execute that, but since this is DDL you can't do that unless you resort to dynamic SQL, which is going to be overly complicated and messy for what you're trying to do.
You need to put each statement in a separate file (without the trailing semi-colon on the create sequence; you still need it on the create trigger because there it is ending the trigger's PL/SQL block, not acting as a statement separator - confusing, I know), and read and execute them separately. Then each execute has a single statement, and will be much happier.
As an aside, you don't need to select your sequence value into your variable in 11g; you can now do it like this:
CREATE OR REPLACE TRIGGER VerHist_SeqNum_TRG
BEFORE INSERT
ON VerHist
FOR EACH ROW
BEGIN
:NEW.SequenceNumber := VerHist_SeqNum.NEXTVAL;
END;
When you copy paste the contains of the file to browser. Browser will treat \n as new line.
Whereas for the code /n is character only.
Try to replace \n with single space and then run it will work
sqlBlock = sqlBlock.replaceAll("\n"," ");
Remove the \n from the query. If you want the query to be formatted in sql developer, you can select the query and press Ctrl+F7
I am creating and testing a stored procedure through remote using JUnit , my user name is test and this is my code,
String sql = "create procedure test.firstProc3 AS select GETDATE()";
cursor.execute(sql);
cursor.executeQuery("firstProc3");
ANd to drop the procedure:
String dropSQL = "DROP PROCEDURE test.firstProc3";
cursor.execute(dropSQL);
Since the Drop Procedure is not working, I am unable to run the same SP for the second time ( new session) My error is:
There is already an object named 'firstProc3' in the database.
When I gave sp_help on the server side,
the table had the row with the value
firstProc3 test stored procedure
However when I give
DROP PROCEDURE test.firstProc3 in the Query analyzer, the row is getting deleted from the table.
What could be the issue in trying to do the same operation through Junit?
Are there any permissions to be set?
PS - the user test has the db_ddladmin enabled.
I managed to finally solve the problem by using the following :
declare #object_id int
select #object_id = object_id('test.firstProc5')
EXEC sp_MSdrop_object #object_id
GO
This is removing the data from the table too.And I am able to successfully run the Testcase multiple times !
Thanks for your help :)
The link I referred to is here, hope it helps someone:
http://www.mssqlcity.com/Articles/Undoc/SQL2000UndocSP.htm
Based upon the error message, I doubt it is a permissions issue. It looks like your cursor object is running the create again when you're trying to drop, or you're never really dropping the proc before running your unit test again, or the tests are running out of logical order in some fashion. Can you instantiate a new cursor object and try the drop with that, or verify the SQL being run against the database using profiler or some debug output?
Beyond that, I would be curious to see if changing your SQL commands to check for proc existence makes the error go away (not that this is the right answer):
if object_id('test.firstProc3') is null begin
-- Dynamic SQL because create must be first in a batch
exec ('create procedure test.firstProc3 as select getdate()')
end
if object_id('test.firstProc3') is not null begin
drop procedure test.firstProc3
end
Maybe something small, and maybe it is even not the case, but have you tried to use escaping in your drop statement?
something like
drop procedure [test].[firstProc3]
I can remember that i had an issue with this when i called a drop table from c# code...