I want to call this query (which works when run on SQL developer) from Java
DECLARE
TYPE my_id_tab IS
TABLE OF my_table.my_id%TYPE;
my_ids my_id_tab;
BEGIN
UPDATE my_table
SET
another_id = NULL
WHERE
another_id IS NULL
AND create_datetime BETWEEN '03-JUN-19' AND '05-JUN-19'
RETURNING my_id BULK COLLECT INTO my_ids;
COMMIT;
END;
But I believe Java is having a tough time trying to figure out that I want the collection of my_ids returned to me.
Here's what I tried so far with exception messages like java.sql.SQLException: Invalid column index
or
java.sql.SQLException: operation not allowed: Ordinal binding and Named binding cannot be combined!
final Connection connection = DataSourceUtils.getConnection(jdbcTemplate.getDataSource());
try (final CallableStatement callableStatement = connection.prepareCall(TEST_SQL))
{
callableStatement.registerOutParameter("my_ids", Types.ARRAY);
callableStatement.executeUpdate();
int[] arr = (int[]) callableStatement.getArray("my_ids").getArray();
return Arrays.stream(arr).boxed().collect(Collectors.toSet());
}
catch (final SQLException e)
{
LOG.info("threw exception, {}", e);
}
finally
{
DataSourceUtils.releaseConnection(connection, jdbcTemplate.getDataSource());
}
It's not the simplest thing, but it's pretty easy to do. You will need to create a TYPE in Oracle to define the results.
For this demo, create and populate EMP and DEPT: EMP and DEPT script
Create the TYPE, needed to define the array that will be returned:
create type t_integer_array as table of integer;
We will be running the following UPDATE, which will update only a few rows:
UPDATE emp
SET job = job -- a nonsense update
WHERE comm IS NOT NULL -- only affect some of the rows
Here is the Java:
package test;
import java.math.BigDecimal;
import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Types;
import java.util.Arrays;
public class OracleTest {
public static void main(String[] args) {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
Connection conn = DriverManager.getConnection(
"<your JDBC url>", "<your user>", "<your password>");
// Prepare the call, without defining the the output variable
// in a DECLARE section of the PL/SQL itself, you just need to
// include a "?" and then associate the correct type in the
// next step.
CallableStatement cs = conn.prepareCall(
"BEGIN\n"
+ " UPDATE emp\n"
+ " SET job = job\n"
+ " WHERE comm is not null\n"
+ " RETURNING empno BULK COLLECT INTO ?;\n"
+ "END;");
// Register the single OUT parameter as an array, using the
// type that was defined in the database, T_INTEGER_ARRAY.
cs.registerOutParameter(1, Types.ARRAY, "T_INTEGER_ARRAY");
cs.execute();
// Now get the array back, as array of BigDecimal.
// BigDecimal is used because it doesn't have precision
// problems like floating point, it will contain all digits
// that the database provided.
BigDecimal[] nums = (BigDecimal[]) (cs.getArray(1).getArray());
System.out.println(Arrays.toString(nums));
cs.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
And here's my output:
[7499, 7521, 7654, 7844]
These are the technical keys (empno) for only the rows that were affected by the update.
Related
I am having a java class with 70 fields. I also have 70 fields in a table where I need to perform CRUD operations. I have a procedure which has an object with 70 fields as object. I need to call this procedure and perform the operations. Could anyone suggest any possible solution.
First I would suggest creating a database OBJECT and a constructor for that object to represent the input parameters for your procedure.
/* set this up with your 70 values needed */
CREATE OR REPLACE TYPE Myproc_Parameters AS OBJECT(
Number_Parameter NUMBER(10)
,Varchar_Parameter1 VARCHAR2(20)
,Varchar_Parameter2 VARCHAR2(30)
,Varchar_Etc VARCHAR2(100)
,CONSTRUCTOR FUNCTION Myproc_Parameters
RETURN SELF AS RESULT);
/
CREATE OR REPLACE TYPE BODY Myproc_Parameters AS
CONSTRUCTOR FUNCTION Myproc_Parameters RETURN SELF AS RESULT AS
BEGIN
RETURN;
END;
END;
/
You will pass this object to whatever procedure you will be using for your CRUD operations.
CREATE OR REPLACE PROCEDURE Myproc_Crud (
p_My_Params IN Myproc_Parameters)
IS
BEGIN
NULL; /* CRUD logic here */
END;
/
Once you have the parameter object & procedure set up, you can call it from Java.
public class TestCallDatabaseProcedure {
public static void main(String[] args) {
try {
// set up an Oracle database connection
OracleConnection connection = getOracleConnection().unwrap(OracleConnection.class);
System.out.println("Got Connection.");
// create an object array based on the database object
// add your 70 values for the procedure here
Object[] procParameters = new Object[] {1, "param2", "param3", "param4"};
// use the object array to create a struct based on the database object
Struct structParameters = connection.createStruct("MYPROC_PARAMETERS", procParameters);
OracleCallableStatement statement = (OracleCallableStatement) connection.prepareCall(
"begin " +
" Myproc_Crud(?); " +
"end;");
// pass the struct to the callable statement executing your procedure
statement.setObject(1, structParameters);
statement.execute();
System.out.println("Statement executed.");
statement.close();
connection.close();
} catch (SQLException se) {
System.out.println("SQL exception: " + se.getMessage());
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
This is the method I use to get the Oracle database connection.
private static Connection getOracleConnection() throws Exception {
String driver = "oracle.jdbc.driver.OracleDriver";
String url = "jdbc:oracle:thin:#myhost.com:1521:mydb";
String username = "myuser";
String password = "mypasswd";
Class.forName(driver);
Connection connection = DriverManager.getConnection(url, username, password);
return connection;
}
I'm trying to print the returned value of a MySQL stored function from the JDBC code which is as follows (I am using MySQL Server 5.7.13):
package jdbc;
import java.sql.DriverManager;
import java.sql.*;
import java.util.Scanner;
public class CallableStatementsCallingFunctions {
public static void main(String... syrt)
{
try
{
try
{
Class.forName("com.mysql.jdbc.Driver");
}
catch(ClassNotFoundException e)
{
System.out.println("Error(class): "+ e);
}
try
{
Connection conn = DriverManager.getConnection("jdbc:mysql://localhost/collablestatement","root","mysql") ;
CallableStatement cs = conn.prepareCall("{call ?:=getBalance1(?)}");
String s = new Scanner(System.in).nextLine();
cs.registerOutParameter(1,Types.INTEGER);
cs.setInt(2,Integer.parseInt(s));
cs.execute();
System.out.println("Account number :" + cs.getInt(1));
conn.close();
}
catch(SQLException e)
{
System.out.println("Error(SQL) : "+e);
}
}
catch(Exception e)
{
System.out.println("Error(Fro outer try) : "+ e);
}
}
}
the stored function getBalance1(acno) is shown here
my code output is shown here
I am getting the output from the SQL command but in JDBC I am getting and SQLException saying that
parameter 1 is not an output parameter
I know that parameter 1 has been used as the placeholder of the returned value from the function in jdbc code. In prepareCall I also tried the syntax - {?:= call getBalance1(?)} , but even then getting the same Exception.
Why am I getting the exception?
I think I was getting the SQLException because I am using jdk1.8.xx in which the syntax of calling the stored function is different. The problem was solved by replacing statement
CallableStatement cs = conn.prepareCall("{call ?:=getBalance1(?)}");
in the code with
CallableStatement cs = conn.prepareCall("{? = call getBalance1(?)}");
The syntax of calling the function in the prepareCall() method as parameter is here.
getBalance1() is a MySQL FUNCTION, not a PROCEDURE, so I wouldn't expect using a JDBC CallableStatement to be applicable.
Even in your MySQL console test you are using
select getBalance1(103)
so you simply need to do the same thing in your Java code using a PreparedStatement:
PreparedStatement ps = conn.prepareStatement("select getBalance1(?)");
ps.setInt(1) = 103;
ResultSet rs = ps.executeQuery();
rs.next();
Double bal = rs.getDouble(1);
(It should be noted that since "balance" apparently refers to "money", REAL is not a good choice for the column type; details here.)
This question already has answers here:
MySQL LIMIT clause equivalent for SQL SERVER
(5 answers)
Closed 8 years ago.
I want to fetch some record(it can be 50,100 or something else that is configured by user) from database without using limit clause because our application may be work on multiple database like mysql,oracle,mssql,db2....
i did following solution
package com.test;
import java.sql.Connection;
import java.sql.Statement;
import java.sql.ResultSet;
import java.sql.DriverManager;
import java.util.Date;
public class BatchRetrieveTest extends Object {
private static final int FETCH_SIZE = 10;
public BatchRetrieveTest() {
}
public static void main(String[] args) {
BatchRetrieveTest batchRetrieveTest = new BatchRetrieveTest();
batchRetrieveTest.test();
}
void test() {
Connection conn = null;
Statement stmt2 = null;
Date start = null;
Date end = null;
int i = 0;
try {
conn = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/test",
"root", "root");
stmt2 = conn.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_READ_ONLY);
conn.setAutoCommit(false);
stmt2.setFetchSize(FETCH_SIZE);
stmt2.setPoolable(true);
start = new Date();
System.out.println(new Date() + "second execute start"
+ new Date().getTime());
ResultSet rs2 = stmt2
.executeQuery("SELECT * FROM sample_final_attendance limit 1000");
end = new Date();
System.out.println(new Date() + "*************second execute end"
+ (end.getTime() - start.getTime()));
rs2.absolute(200000);
i = 0;
while (rs2.next()) {
if (i++ > 100) {
break;
}
}
rs2.close();
stmt2.close();
end = new Date();
System.out.println(new Date() + "second read end"
+ (end.getTime() - start.getTime()));
conn.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
stmt2.close();
conn.close();
} catch (Exception e) {
}
}
}
}
Here sample_final_attendance table contains 15 columns and 3.2 millions of record
while executing this program it requires 2GB of memory and 47 seconds of execution time
here i wonder that if some table has billions of record then it fails to execute
also i used setFetchSize as suggested but problem is same
please suggest some solution
thanks in advance
Well the ASFAIK & understood, the problem is more related with the handling of data in polyglot storage. If you think, you need to resolve the same in all cases interdependent of the database type - the one common approach is to build a serving layer .
The serving layer can be a cache library or even a Map of Maps created by you. Do not attempt to query the database with large number of records at once, instead bring the data as batches, and store it as a pool of pojos. On demand of the user, you can serve the data from the serving layer.
You can make use of the memcache or hazlecast or many other cache libraries, which can be directly integrated with databases. I really don't know how complex is your situation. What I made is a suggestion. This makes up a data-grid, which can be populated with data from any databases in the background.
We have setMaxRow(int numOfRow) in Statement Object this will limit number of rows generated by the Statement Object and simply ignore the remaining.
Take the look at the doc.
I have been trying to figure out how to get SQLite working on eclipse juno. I have been following the instructions on this site http://wiki.eclipse.org/Connecting_to_SQLite. The problem is not every step is exactly as explained so I am guessing on weather I got it right or not. I feel that I have probably gotten it all correct until step 13, there is no SQL Model-JDBC Connection entry. So I have tried step 13-16 with a generic JDBC and with one that says SQLite. The SQLite one does not have a driver which is no surprise due to step 5. Any way that I have tried so far ends up failing ping with details listed below. Someone must have a better way through this process.
java.sql.SQLException: java.lang.UnsatisfiedLinkError: SQLite.Database.open(Ljava/lang/String;I)V
at SQLite.JDBCDriver.connect(JDBCDriver.java:68)
at org.eclipse.datatools.connectivity.drivers.jdbc.JDBCConnection.createConnection(JDBCConnection.java:328)
at org.eclipse.datatools.connectivity.DriverConnectionBase.internalCreateConnection(DriverConnectionBase.java:105)
at org.eclipse.datatools.connectivity.DriverConnectionBase.open(DriverConnectionBase.java:54)
at org.eclipse.datatools.connectivity.drivers.jdbc.JDBCConnection.open(JDBCConnection.java:96)
at org.eclipse.datatools.connectivity.drivers.jdbc.JDBCConnectionFactory.createConnection(JDBCConnectionFactory.java:53)
at org.eclipse.datatools.connectivity.internal.ConnectionFactoryProvider.createConnection(ConnectionFactoryProvider.java:83)
at org.eclipse.datatools.connectivity.internal.ConnectionProfile.createConnection(ConnectionProfile.java:359)
at org.eclipse.datatools.connectivity.ui.PingJob.createTestConnection(PingJob.java:76)
at org.eclipse.datatools.connectivity.ui.PingJob.run(PingJob.java:59)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Make sure you get the driver from https://bitbucket.org/xerial/sqlite-jdbc/downloads, then import the driver into your project.
Now you can test the configuration by creating a java class Sample.java
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class Sample
{
public static void main(String[] args) throws ClassNotFoundException
{
// load the sqlite-JDBC driver using the current class loader
Class.forName("org.sqlite.JDBC");
Connection connection = null;
try
{
// create a database connection
connection = DriverManager.getConnection("jdbc:sqlite:sample.db");
Statement statement = connection.createStatement();
statement.setQueryTimeout(30); // set timeout to 30 sec.
statement.executeUpdate("DROP TABLE IF EXISTS person");
statement.executeUpdate("CREATE TABLE person (id INTEGER, name STRING)");
int ids [] = {1,2,3,4,5};
String names [] = {"Peter","Pallar","William","Paul","James Bond"};
for(int i=0;i<ids.length;i++){
statement.executeUpdate("INSERT INTO person values(' "+ids[i]+"', '"+names[i]+"')");
}
//statement.executeUpdate("UPDATE person SET name='Peter' WHERE id='1'");
//statement.executeUpdate("DELETE FROM person WHERE id='1'");
ResultSet resultSet = statement.executeQuery("SELECT * from person");
while(resultSet.next())
{
// iterate & read the result set
System.out.println("name = " + resultSet.getString("name"));
System.out.println("id = " + resultSet.getInt("id"));
}
}
catch(SQLException e){ System.err.println(e.getMessage()); }
finally {
try {
if(connection != null)
connection.close();
}
catch(SQLException e) { // Use SQLException class instead.
System.err.println(e);
}
}
}
}
The code will create a database named sample.db, inserting data into, and then prints the rows.
When does the PostgreSQL JDBC driver version 9.2-1002 fetch rows from the server after executing a query? Does it fetch the rows immediately after query execution (after the client application invokes PreparedStatement.executeQuery()) or after the client application first invokes ResultSet.next() to retrieve a row from the result set? Does this depend on the value of the statement fetch size?
As the following program demonstrates, PreparedStatement.executeQuery() always retrieves rows in the result set from the server. The program also demonstrates how statement fetch size impacts row retrieval. In the case where the statement has the default fetch size of zero, executeQuery() retrieves all rows from the server and ResultSet.next() retrieves and returns the next row from memory, not from the server. (The program may even close the connection after executing the query and next() can still iterate over all rows.) In the case where fetch size is non-zero, executeQuery() retrieves the first batch of rows, the number of which equals the fetch size, and ResultSet.next() again returns the next row from memory until it consumes all rows in the current batch, at which point it retrieves the next batch of rows from the server. This pattern repeats until ResultSet.next() retrieves an empty batch from the server (one that contains zero rows).
SQL
-- Create table "test" and insert 2,000,000 integers from 1 up to 2,000,000.
WITH RECURSIVE t(n) AS
(
VALUES (1)
UNION ALL
SELECT n+1
FROM t
WHERE n < 2000000
)
SELECT n as value
INTO test
FROM t;
Java
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Date;
import java.util.Properties;
public class Start
{
public static void main( String[] args ) throws InterruptedException, SQLException
{
try
{
Class.forName( "org.postgresql.Driver" );
}
catch ( ClassNotFoundException e )
{
System.out.println( "Where is your JDBC Driver?" );
e.printStackTrace();
return;
}
System.out.println( "Registered JDBC driver" );
Connection connection = null;
try
{
final String databaseUrl = "jdbc:postgresql://localhost:5483/postgres";
final Properties properties = new Properties();
connection = DriverManager.getConnection( databaseUrl, properties );
connection.setAutoCommit(false);
Statement statement = connection.createStatement();
// Default fetch size of 0 does not create a cursor.
// Method executeQuery will retrieve all rows in result set.
statement.setFetchSize( 0 );
// Fetch size of 1 creates a cursor with batch size of 1.
// Method executeQuery will retrieve only 1 row in the result set.
//statement.setFetchSize( 1 );
System.out.println( new Date() + ": Before execute query" );
ResultSet result =
statement.executeQuery( "select * from test" );
System.out.println( new Date() + ": After execute query" );
System.out.println( new Date() + ": Sleep for 5 s" );
Thread.sleep( 5000 );
System.out.println( new Date() + ": Before process result set" );
while ( result.next() );
System.out.println( new Date() + ": After process result set" );
result.close();
statement.close();
}
catch ( SQLException e )
{
System.out.println( "Connection failed!" );
e.printStackTrace();
return;
}
finally
{
if ( connection != null )
connection.close();
}
}
}
Look at this documentation
By default the driver collects all the results for the query at once. This can be inconvenient for large data sets so the JDBC driver provides a means of basing a ResultSet on a database cursor and only fetching a small number of rows.
A small number of rows are cached on the client side of the connection and when exhausted the next block of rows is retrieved by repositioning the cursor.
Further read this
Ordinarily, libpq collects a SQL command's entire result and returns it to the application as a single PGresult. This can be unworkable for commands that return a large number of rows. For such cases, applications can use PQsendQuery and PQgetResult in single-row mode. In this mode, the result row(s) are returned to the application one at a time, as they are received from the server.
PG requires the connection to be AutoCommit = false for getting rows as cursor.
So if you use Spring jdbc and TransactionManagement you can use connection from Transaction, which has AutoCommit = false by default.
#Repository()
public class SampleRepoImpl implements SampleRepo {
private static final String SQL = "select s.id from generate_series(1,100000) as s(id)";
#Autowired
private DataSource dataSource;
#Override
#Transactional(readOnly = true)
public void findAsCursor(RecordProcessor recordProcessor) throws SQLException {
// It shouldn't close by hands, it will when the transaction finished.
Connection connection = DataSourceUtils.getConnection(dataSource);
try (PreparedStatement ps = connection.prepareStatement(SQL, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)) {
ps.setFetchSize(100);
try (ResultSet rs = ps.executeQuery();) {
while (rs.next()) {
long id = rs.getLong("id");
System.out.println("id = " + id);
}}}}}