When does the PostgreSQL JDBC driver version 9.2-1002 fetch rows from the server after executing a query? Does it fetch the rows immediately after query execution (after the client application invokes PreparedStatement.executeQuery()) or after the client application first invokes ResultSet.next() to retrieve a row from the result set? Does this depend on the value of the statement fetch size?
As the following program demonstrates, PreparedStatement.executeQuery() always retrieves rows in the result set from the server. The program also demonstrates how statement fetch size impacts row retrieval. In the case where the statement has the default fetch size of zero, executeQuery() retrieves all rows from the server and ResultSet.next() retrieves and returns the next row from memory, not from the server. (The program may even close the connection after executing the query and next() can still iterate over all rows.) In the case where fetch size is non-zero, executeQuery() retrieves the first batch of rows, the number of which equals the fetch size, and ResultSet.next() again returns the next row from memory until it consumes all rows in the current batch, at which point it retrieves the next batch of rows from the server. This pattern repeats until ResultSet.next() retrieves an empty batch from the server (one that contains zero rows).
SQL
-- Create table "test" and insert 2,000,000 integers from 1 up to 2,000,000.
WITH RECURSIVE t(n) AS
(
VALUES (1)
UNION ALL
SELECT n+1
FROM t
WHERE n < 2000000
)
SELECT n as value
INTO test
FROM t;
Java
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Date;
import java.util.Properties;
public class Start
{
public static void main( String[] args ) throws InterruptedException, SQLException
{
try
{
Class.forName( "org.postgresql.Driver" );
}
catch ( ClassNotFoundException e )
{
System.out.println( "Where is your JDBC Driver?" );
e.printStackTrace();
return;
}
System.out.println( "Registered JDBC driver" );
Connection connection = null;
try
{
final String databaseUrl = "jdbc:postgresql://localhost:5483/postgres";
final Properties properties = new Properties();
connection = DriverManager.getConnection( databaseUrl, properties );
connection.setAutoCommit(false);
Statement statement = connection.createStatement();
// Default fetch size of 0 does not create a cursor.
// Method executeQuery will retrieve all rows in result set.
statement.setFetchSize( 0 );
// Fetch size of 1 creates a cursor with batch size of 1.
// Method executeQuery will retrieve only 1 row in the result set.
//statement.setFetchSize( 1 );
System.out.println( new Date() + ": Before execute query" );
ResultSet result =
statement.executeQuery( "select * from test" );
System.out.println( new Date() + ": After execute query" );
System.out.println( new Date() + ": Sleep for 5 s" );
Thread.sleep( 5000 );
System.out.println( new Date() + ": Before process result set" );
while ( result.next() );
System.out.println( new Date() + ": After process result set" );
result.close();
statement.close();
}
catch ( SQLException e )
{
System.out.println( "Connection failed!" );
e.printStackTrace();
return;
}
finally
{
if ( connection != null )
connection.close();
}
}
}
Look at this documentation
By default the driver collects all the results for the query at once. This can be inconvenient for large data sets so the JDBC driver provides a means of basing a ResultSet on a database cursor and only fetching a small number of rows.
A small number of rows are cached on the client side of the connection and when exhausted the next block of rows is retrieved by repositioning the cursor.
Further read this
Ordinarily, libpq collects a SQL command's entire result and returns it to the application as a single PGresult. This can be unworkable for commands that return a large number of rows. For such cases, applications can use PQsendQuery and PQgetResult in single-row mode. In this mode, the result row(s) are returned to the application one at a time, as they are received from the server.
PG requires the connection to be AutoCommit = false for getting rows as cursor.
So if you use Spring jdbc and TransactionManagement you can use connection from Transaction, which has AutoCommit = false by default.
#Repository()
public class SampleRepoImpl implements SampleRepo {
private static final String SQL = "select s.id from generate_series(1,100000) as s(id)";
#Autowired
private DataSource dataSource;
#Override
#Transactional(readOnly = true)
public void findAsCursor(RecordProcessor recordProcessor) throws SQLException {
// It shouldn't close by hands, it will when the transaction finished.
Connection connection = DataSourceUtils.getConnection(dataSource);
try (PreparedStatement ps = connection.prepareStatement(SQL, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY)) {
ps.setFetchSize(100);
try (ResultSet rs = ps.executeQuery();) {
while (rs.next()) {
long id = rs.getLong("id");
System.out.println("id = " + id);
}}}}}
Related
i'm trying to check if a data is already present in my database and comparing it with user input from my java application.
But when i use equals method, i have always a false return... And i would like a "true" return.
I don't understand why it doesn't match...
Here is my code :
public boolean getTicketWithRegVehicleNumber(String vehicleRegNumber) {
Connection con = null;
boolean valueReturn = false;
try {
con = dataBaseConfig.getConnection();
PreparedStatement ps = con.prepareStatement(DBConstants.GET_VEHICLE_REG_NUMBER);
ps.setString(1, vehicleRegNumber);
ResultSet rs = ps.executeQuery();
//TODO trouver comment tester l'egalité entre la db et la saisie user
String sqlRequestResult = DBConstants.GET_VEHICLE_REG_NUMBER;
if (sqlRequestResult.equals(vehicleRegNumber)){
valueReturn = true;
}
dataBaseConfig.closeResultSet(rs);
dataBaseConfig.closePreparedStatement(ps);
} catch (Exception ex) {
logger.error("Error fetching next available slot", ex);
} finally {
dataBaseConfig.closeConnection(con);
}
return valueReturn;
}
here is my SQL request :
public static final String GET_VEHICLE_REG_NUMBER = "select t.VEHICLE_REG_NUMBER from ticket t where t.VEHICLE_REG_NUMBER = ? ";
To sum up, i have a boolean variable : returnValue, and i want that when i use equals method on my SQL selected data (a VARCHAR type that i put in a String variable) and user's input which is a String , my boolean return true.
Thanks for helping :-)
See correct Answer by Dravidian. Your code neglected to access the ResultSet you named rs. So you never retrieved any data from the database.
Here is a complete example app using the H2 Database Engine.
We hardcode the query result presented in the result set as a single row with a single column of text type with a value of Crêpe. Notice how we access the ResultSet using a loop, accessing the retrieved value with a call to getString. Then we compare strings using String#equals.
Also, notice how this example code uses try-with-resources syntax to automatically close resources such as our Connection, Statement, and ResultSet objects. Make a habit of using try-with-resources syntax to simplify your JDBC code (and similarly file i/o code, etc.).
import org.h2.jdbcx.JdbcDataSource;
import java.sql.*;
import java.time.*;
import java.util.*;
public class App5
{
public static void main ( String[] args )
{
App5 app = new App5();
app.doIt();
}
private void doIt ( )
{
JdbcDataSource dataSource = Objects.requireNonNull( new JdbcDataSource() ); // Implementation of `DataSource` bundled with H2.
dataSource.setURL( "jdbc:h2:mem:text_match;DB_CLOSE_DELAY=-1" ); // Set `DB_CLOSE_DELAY` to `-1` to keep in-memory database in existence after connection closes.
dataSource.setUser( "scott" );
dataSource.setPassword( "tiger" );
try (
Connection conn = dataSource.getConnection() ;
)
{
String sql = "SELECT 'Crêpe' ;";
try (
Statement stmt = conn.createStatement() ;
ResultSet rs = stmt.executeQuery( sql ) ;
)
{
while ( rs.next() )
{
//Retrieve by column number.
String result = rs.getString( 1 );
//Display values
System.out.println( "result: " + result );
System.out.println( "result equals crepe: " + result.equals( "crepe" ) ); // False. Missing the circumflex.
System.out.println( "result equals crêpe: " + result.equals( "crêpe" ) ); // False. Lowercase `c` does not match `C`.
System.out.println( "result equals Crêpe: " + result.equals( "Crêpe" ) ); // True.
}
}
}
catch ( SQLException e )
{
e.printStackTrace();
}
}
}
When run.
result: Crêpe
result equals crepe: false
result equals crêpe: false
result equals Crêpe: true
You're comparing your plain sql query string (DBConstants.GET_VEHICLE_REG_NUMBER) to the vehicleRegNumber parameter and no wonder they don't match.
What you need to compare is the result from your ps.executeQuery(); which is assigned to ResultSet rs.
Read the ResultSet javadoc to understand how you can extract data from it - https://docs.oracle.com/javase/8/docs/api/java/sql/ResultSet.html.
I want to call this query (which works when run on SQL developer) from Java
DECLARE
TYPE my_id_tab IS
TABLE OF my_table.my_id%TYPE;
my_ids my_id_tab;
BEGIN
UPDATE my_table
SET
another_id = NULL
WHERE
another_id IS NULL
AND create_datetime BETWEEN '03-JUN-19' AND '05-JUN-19'
RETURNING my_id BULK COLLECT INTO my_ids;
COMMIT;
END;
But I believe Java is having a tough time trying to figure out that I want the collection of my_ids returned to me.
Here's what I tried so far with exception messages like java.sql.SQLException: Invalid column index
or
java.sql.SQLException: operation not allowed: Ordinal binding and Named binding cannot be combined!
final Connection connection = DataSourceUtils.getConnection(jdbcTemplate.getDataSource());
try (final CallableStatement callableStatement = connection.prepareCall(TEST_SQL))
{
callableStatement.registerOutParameter("my_ids", Types.ARRAY);
callableStatement.executeUpdate();
int[] arr = (int[]) callableStatement.getArray("my_ids").getArray();
return Arrays.stream(arr).boxed().collect(Collectors.toSet());
}
catch (final SQLException e)
{
LOG.info("threw exception, {}", e);
}
finally
{
DataSourceUtils.releaseConnection(connection, jdbcTemplate.getDataSource());
}
It's not the simplest thing, but it's pretty easy to do. You will need to create a TYPE in Oracle to define the results.
For this demo, create and populate EMP and DEPT: EMP and DEPT script
Create the TYPE, needed to define the array that will be returned:
create type t_integer_array as table of integer;
We will be running the following UPDATE, which will update only a few rows:
UPDATE emp
SET job = job -- a nonsense update
WHERE comm IS NOT NULL -- only affect some of the rows
Here is the Java:
package test;
import java.math.BigDecimal;
import java.sql.CallableStatement;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Types;
import java.util.Arrays;
public class OracleTest {
public static void main(String[] args) {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
Connection conn = DriverManager.getConnection(
"<your JDBC url>", "<your user>", "<your password>");
// Prepare the call, without defining the the output variable
// in a DECLARE section of the PL/SQL itself, you just need to
// include a "?" and then associate the correct type in the
// next step.
CallableStatement cs = conn.prepareCall(
"BEGIN\n"
+ " UPDATE emp\n"
+ " SET job = job\n"
+ " WHERE comm is not null\n"
+ " RETURNING empno BULK COLLECT INTO ?;\n"
+ "END;");
// Register the single OUT parameter as an array, using the
// type that was defined in the database, T_INTEGER_ARRAY.
cs.registerOutParameter(1, Types.ARRAY, "T_INTEGER_ARRAY");
cs.execute();
// Now get the array back, as array of BigDecimal.
// BigDecimal is used because it doesn't have precision
// problems like floating point, it will contain all digits
// that the database provided.
BigDecimal[] nums = (BigDecimal[]) (cs.getArray(1).getArray());
System.out.println(Arrays.toString(nums));
cs.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
And here's my output:
[7499, 7521, 7654, 7844]
These are the technical keys (empno) for only the rows that were affected by the update.
I have a local SQL-Server running MariaDB and need to query Data from a database using Java and JDBC. I can connect to the Database and also write data, but a simple SELECT does not work.
I already tried to use different versions of the mysql-java-connector and checked that the SQL-Server is up to date.
Connecting to Database:
//Check wether connection already exists
if(connection != null && !connection.isClosed()){
return;
}
//Create new connection
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/statdb", "root", "");
System.out.println("Connected to Database");
//Update guilds from Database
updateGuildsInDatabase();
The method called at the end looks like this:
private void updateGuildsInDatabase() throws SQLException {
//Check for not existing connection
if(connection == null || connection.isClosed()){
init();
return;
}
ArrayList<String> localInDb = new ArrayList<>();
Statement qGStmt = connection.createStatement();
//Execute Query
ResultSet guilds = qGStmt.executeQuery("SELECT * FROM guilds;");
guilds.first();
//Adding results to List
while(guilds.next()){
localInDb.add(guilds.getString("guild_uid").toLowerCase());
}
System.out.println("Queried: " + localInDb.size());
this.guildsInDb = localInDb;
}
The table "guilds" in the database "statdb" has two columns named "guild_uid" (primary key, varchar) and "display_name" (varchar). I've added one entry with the values "guild_1" and "Test Guild".
As I have one entry in the table 'guilds' I would expect the console to look like this:
Connected to Database
Queried: 1
But the actual output looks like this:
Connected to Database
Queried: 0
The ResultSet#first() method itself moves the cursor to the first row of the result set. The ResultSet#next() method then moves the cursor forward by one, and reads whatever record is there, if a record is there.
Therefore, using your pattern will always result in skipping the first record from the result set. If the result set happens to have only a single record, then the record count will appear to be zero.
You should probably remove the call to ResultSet#first(). Use this instead:
ResultSet guilds = qGStmt.executeQuery("SELECT * FROM guilds;");
while(guilds.next()) {
localInDb.add(guilds.getString("guild_uid").toLowerCase());
}
System.out.println("Queried: " + localInDb.size());
this.guildsInDb = localInDb;
The reason why calling ResultSet#next() on a fresh result set logically works, is that by default a JDBC result set actually does not start pointing to the first record. Rather, we typically advance it to the first record with the very first call to ResultSet#next().
I don't think you need guilds.first(). Try removing it.
According to the ResultSet JavaDoc:
A ResultSet cursor is initially positioned before the first row; the first call to the method next makes the first row the current row; the second call makes the second row the current row, and so on.
I believe what is happening here is, guilds.first() set the cursor from before first row to first row. And guilds.next() in the while loop move the cursor to further one step ahead. Therefore, you are missing the first row.
The other two Answers are correct, you are skipping the first row when retrieving from the result set. No need for your line guilds.first(); as they explained, the result set automatically points at first row by default.
Example code, retrieving from ResultSet
Here is a complete example using the H2 Database Engine. This shows creating a new database (in-memory, not persisted to disk), adding a table, populating that table with a few rows, and then retrieving all those rows through a ResultSet.
The key line is while ( rs.next() ) {.
package work.basil.example;
import java.sql.*;
import java.util.*;
import org.h2.jdbcx.JdbcDataSource;
import javax.sql.DataSource;
public class GuildDemo {
public static void main ( String[] args ) {
GuildDemo app = new GuildDemo();
app.doIt();
}
private void doIt ( ) {
DataSource dataSource = null;
final String catalogName = "guild_demo_";
final String tableName = "guild_";
// Verify JDBC driver.
try {
Class.forName( "org.h2.Driver" );
JdbcDataSource ds = new JdbcDataSource(); // The `javax.sql.DataSource` interface implemented by `org.h2.jdbcx.JdbcDataSource`.
ds.setURL( "jdbc:h2:mem:" + catalogName + ";DB_CLOSE_DELAY=-1" ); // Set delay to -1 to keep in-memory database even after last connection closed.
ds.setUser( "scott" );
ds.setPassword( "tiger" );
ds.setDescription( "Dummy database for demo showing how to retrieve rows from a ResultSet." );
dataSource = ds; // Generalize from the concrete class to the interface.
} catch ( ClassNotFoundException e ) {
e.printStackTrace();
return;
}
// Connect, and create database.
try (
Connection conn = dataSource.getConnection() ;
) {
String sql = null;
// Create table.
try ( Statement stmt = conn.createStatement() ; ) {
sql = "CREATE TABLE " + "guild_" + " ( \n" +
" pkey_ UUID DEFAULT random_uuid() PRIMARY KEY , \n" +
" name_ VARCHAR NOT NULL \n" +
") ; \n";
System.out.println( "TRACE - SQL:\n" + sql );
stmt.execute( sql );
}
// Add rows.
sql = "INSERT INTO guild_ ( name_ ) \n" +
"VALUES ( ? ) " +
"; ";
List < String > names = List.of( "Alpha" , "Beta" , "Gamma" , "Delta" ); // Insert a row for each of these names.
System.out.println( "Inserting list of names: " + names );
try (
PreparedStatement ps = conn.prepareStatement( sql ) ;
) {
for ( String name : names ) {
ps.setString( 1 , name );
ps.executeUpdate();
}
}
// Retrieve rows from a `ResultSet`.
sql = "SELECT * FROM " + "guild_" + " ;";
try (
Statement stmt = conn.createStatement() ;
ResultSet rs = stmt.executeQuery( sql ) ;
) {
while ( rs.next() ) {
UUID pkey = rs.getObject( "pkey_" , UUID.class );
String name = rs.getString( "name_" );
System.out.println( "Row pkey_: " + pkey + " name_: " + name );
}
}
} catch ( SQLException e ) {
e.printStackTrace();
}
}
}
When run.
Inserting list of names: [Alpha, Beta, Gamma, Delta]
Row pkey_: 69908390-5fa6-4eee-8e12-40106db8d60d name_: Alpha
Row pkey_: 3116acb9-fcce-427f-b222-99c78c6e752a name_: Beta
Row pkey_: b3fd0930-a2e7-461a-be70-f05124fc58de name_: Gamma
Row pkey_: dddb423a-5eb2-4e5e-be16-7bb0c27c0033 name_: Delta
When we use the executeUpdate () method can not insert instructions to return ResultSet, such as a SELECT. I happen to have an update which to start a trigger, and the trigger have a command "Select * from table".
I could solve the problem by assigning this command to a variable as an example: "Select * from table " soon resultSet would be no problem for the JDBC executeUpdate.
The problem is that the system can work with triggers other integrated systems that have no domain and I can not change, so the option was to use the run command (), which as shown in the code image can I check if the return is a ResultSet ( return true). In the statement while discarding ResultSets generated in the trigger and getMoreResults returns me the number of rows affected by the Update, in the case of the frame in blue, using the statement returns the value 3, which is correct.
The problem is to use the PreparedStatement (red frame), where getUpdateCount is returning -1, but should return 3, which is the number of rows affected in the UPDATE.
Problem occurs in SQL Server 2012 database or 2008, it was not simulated with Oracle
Follows the code example with the existing trigger on the table that I'm doing update
I wonder if anyone knows why the return is different for getMoreResults.
Code Java
Trigger
public class TestarConexao {
public static void main(String[] args) throws InstantiationException, IllegalAccessException, ClassNotFoundException, SQLException {
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver").newInstance();
Connection conn = DriverManager.getConnection("jdbc:odbc:magnus;user=rh;password=rh;MARS_Connection=yes;", "rh", "rhk");
Statement stmt = conn.createStatement();
boolean is = stmt.execute("update Usu_teste set usu_codigo = 1 where usu_codigo = 1");
while (is) {
System.out.println("There is ResultSet: " + is);
is = stmt.getMoreResults();
}
System.out.println("UpdateCount with Statement: " + stmt.getUpdateCount());
PreparedStatement prepStmt = conn.prepareStatement("update Usu_teste set usu_codigo = 1 where usu_codigo = 1");
boolean isPrep = prepStmt.execute();
while (isPrep){
System.out.println("There is ResultSet: " + isPrep);
isPrep = prepStmt.getMoreResults();
}
System.out.println("UpdateCount with PreparedStatement: " +prepStmt.getUpdateCount());
}
}
I want to execute a query in Java.
I create a connection. Then I want to execute an INSERT statement, when done, the connection is closed but I want to execute some insert statement by a connection and when the loop is finished then closing connection.
What can I do ?
My sample code is :
public NewClass() throws SQLException {
try {
Class.forName("oracle.jdbc.driver.OracleDriver");
} catch (ClassNotFoundException e) {
System.out.println("Where is your Oracle JDBC Driver?");
return;
}
System.out.println("Oracle JDBC Driver Registered!");
Connection connection = null;
try {
connection = DriverManager.getConnection(
"jdbc:oracle:thin:#localhost:1521:orcl1", "test",
"oracle");
} catch (SQLException e) {
System.out.println("Connection Failed! Check output console");
return;
}
if (connection != null) {
Statement stmt = connection.createStatement();
ResultSet rs = stmt.executeQuery("SELECT * from test.special_columns");
while (rs.next()) {
this.ColName = rs.getNString("column_name");
this.script = "insert into test.alldata (colname) ( select " + ColName + " from test.alldata2 ) " ;
stmt.executeUpdate("" + script);
}
}
else {
System.out.println("Failed to make connection!");
}
}
When the select statement ("SELECT * from test.special_columns") is executed, the loop must be twice, but when (stmt.executeUpdate("" + script)) is executed and done, then closing the connection and return from the class.
Following example uses addBatch & executeBatch commands to execute multiple SQL commands simultaneously.
import java.sql.*;
public class jdbcConn {
public static void main(String[] args) throws Exception{
Class.forName("org.apache.derby.jdbc.ClientDriver");
Connection con = DriverManager.getConnection
("jdbc:derby://localhost:1527/testDb","name","pass");
Statement stmt = con.createStatement
(ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_UPDATABLE);
String insertEmp1 = "insert into emp values
(10,'jay','trainee')";
String insertEmp2 = "insert into emp values
(11,'jayes','trainee')";
String insertEmp3 = "insert into emp values
(12,'shail','trainee')";
con.setAutoCommit(false);
stmt.addBatch(insertEmp1);
stmt.addBatch(insertEmp2);
stmt.addBatch(insertEmp3);
ResultSet rs = stmt.executeQuery("select * from emp");
rs.last();
System.out.println("rows before batch execution= "
+ rs.getRow());
stmt.executeBatch();
con.commit();
System.out.println("Batch executed");
rs = stmt.executeQuery("select * from emp");
rs.last();
System.out.println("rows after batch execution= "
+ rs.getRow());
}
}
Result:
The above code sample will produce the following result.The result may vary.
rows before batch execution= 6
Batch executed
rows after batch execution= = 9
Source: Execute multiple SQL statements
In the abscence of the schema or the data contained in each table I'm going to make the following assumptions:
The table special_columns could look like this:
column_name
-----------
column_1
column_2
column_3
The table alldata2 could look like this:
column_1 | column_2 | column_3
---------------------------------
value_1_1 | value_2_1 | value_3_1
value_1_2 | value_2_2 | value_3_2
The table alldata should, after inserts have, happened look like this:
colname
---------
value_1_1
value_1_2
value_2_1
value_2_2
value_3_1
value_3_2
Given these assumptions you can copy the data like this:
try (
Connection connection = DriverManager.getConnection("jdbc:oracle:thin:#localhost:1521:orcl1", "test", "oracle")
)
{
StringBuilder columnNames = new StringBuilder();
try (
Statement select = connection.createStatement();
ResultSet specialColumns = select.executeQuery("SELECT column_name FROM special_columns");
Statement insert = connection.createStatement()
)
{
while (specialColumns.next())
{
int batchSize = 0;
insert.addBatch("INSERT INTO alldata(colname) SELECT " + specialColumns.getString(1) + " FROM alldata2");
if (batchSize >= MAX_BATCH_SIZE)
{
insert.executeBatch();
batchSize = 0;
}
}
insert.executeBatch();
}
A couple of things to note:
MAX_BATCH_SIZE should be set to a value based on your database configuration and the data being inserted.
this code is using the Java 7 try-with-resources feature to ensure the database resources are released when they're finished with.
you haven't needed to do a Class.forName since the service provider mechanism was introduced as detailed in the JavaDoc for DriverManager.
There are two problems in your code. First you use the same Statement object (stmt) to execute the select query, and the insert. In JDBC, executing a statement will close the ResultSet of the previous execute on the same object.
In your code, you loop over the ResultSet and execute an insert for each row. However executing that statement will close the ResultSet and therefor on the next iteration the call to next() will throw an SQLException as the ResultSet is closed.
The solution is to use two Statement objects: one for the select and one for the insert. This will however not always work by default, as you are working in autoCommit (this is the default), and with auto commit, the execution of any statement will commit any previous transactions (which usually also closes the ResultSet, although this may differ between databases and JDBC drivers). You either need to disable auto commit, or create the result set as holdable over commit (unless that already is the default of your JDBC driver).