We are trying to fetch data from Oracle DB using a PreparedStatement. It keeps fetching zero records while the same runs and fetches data when run from PL/SQL developer.
We found the root cause while trying to debug. While debugging the code fetched the two records properly.
We did a temporary fix by placing this piece of code.
ResultSet rs = ps.executeQuery();
while(!rs.hasNext()){
ps.executeQuery();}
This works. But this is not the best solution since it results in an unwanted DB hit.It clearly looks like a time issue. We also explicitly committed earlier transactions since they can affect the result of this query.
What could be causing this. What's the best way to solve this?
The method is quite big: I'll just post some parts here:
private static boolean loadCommission(Member member){
Connection conn = getConnection("schema1"); //obtained through connection pool
//insertion into table
conn.close();
Conn conn2 = getConnection("schema2"); //obtained through connection pool
PreparedStatement ps = conn2.prepareStatement(sql);
//this sql combines data from schema1
// and 2 with DB links
ResultSet rs = ps.executeQuery();
//business logic
conn2.close();
return true;
}
Thanks
We tried a few more things yesterday. We replaced the second connection code with direct jdbc connection like so
Connection conn = DriverManager.getConnection(URL, USER, PASS);
This too works. Now we are not sure if the delay is in getting connection from pool or in completing previous transaction like we thought earlier.
If your query selects from a materialized view, then there may be some elapsed time before it will yield results (as materialized views do not necessarily refresh instantly after a commit, depending upon how they've been created).
If this is the case, then you can resolve the problem by either selecting directly from the base table (or equivalent non-materialized views), or forcing the materialized view to refresh.
Related
i created a kafka producer on java that works fine, now im trying to store the data produced in an mySQL database but i don't know how. i tried this code but it doesn't work
try {
String MyUrl = "jdbc:mysql://130.2.2.2/pfa";
Class.forName("com.mysql.jdbc.Driver");
Connection con = DriverManager.getConnection(MyUrl, "root", "");
Statement stm = con.createStatement();
ResultSet rs = stm.executeQuery("select * from tmp");
while(rs.next()) {
producer.send(new ProducerRecord<String, String>("test",rs.getString(1)));
con.close();
}
}catch(Exception e) {System.out.println(e);}
any help is much appreciated about sending data to the database
I'm not sure I understand what you're expecting to happen from that code.
You're reading from the database, never inserting into it
You're closing the database connection after every loop iteration, meaning you'll get an exception that the database is already closed on the second loop, and therefore the resultset might terminate as well (I'm a little fuzzy on if those are lazy objects or not)
Might I suggest that you follow the Deep Dive on the JDBC connector? https://dev.to/rmoff/kafka-connect-jdbc-sink-tips-tricks-video-walkthrough-2egf
i copy pasted it literally becoz the writer said it worked
This will never be a good way to learn anything. You can read the code and type it all back out, explaining it to yourself as you go... But never blindly copy something that you assume solves your problem
I'm having a weird problem with an Grails application accessing data. Going deeper I've isolated the problem to a plain java8 small application using PreparedStatement.executeQuery vs Statement.executeQuery.
Consider the following snippet of code:
// executes in milliseconds
directSql = "select top(10) * from vdocuments where codcli = 'CCCC' and serial = 'SSSS' ORDER BY otherField DESC;";
stmt = con.createStatement();
rs = stmt.executeQuery(directSql);
// More than 10 minutes
sqlPrepared = "select top(10) * from vdocuments where codCli = ? and serial = ? ORDER BY otherField DESC;";
PreparedStatement pStatement = con.prepareStatement( sqlPrepared );
pStatement.setString(1, "CCCC");
pStatement.setString(2, "SSSS");
rsPrepared = pStatement.executeQuery();
Same query.
Data comes from a view on SqlServer (2008, I think, have no access right now) from a table with more than 15 Million records. There are indexes for all needed fields and the same query (the first one) executed from console runs also quite fast.
If I execute the slow PreparedStatement query without the ORDER clause it also runs fast.
It looks clear to me that for any cause the database it's not using indexes and make a full scan when using preparedStatement, but maybe I'm wrong so I'm open to any idea.
I thought maybe the driver (sqlserver official latest and jtds has been tested) was holding the data waiting for any kind of EOF from connection but I've checked with tcpdump on my side and no data is received.
I can't find why this is happening so any idea will be welcomed.
Thank you in advanced!
I've finally found a solution, at least in for my case. I got it here http://mehmoodbluffs.blogspot.com.es/2015/03/hibernate-queries-are-slow-sql-servers.html . Telling (driver? sqlServer?) not to send parameters as Unicode have resolved the problem.
Current connection string it's now:
String connectionUrl = "jdbc:sqlserver://server:port;databaseName=myDataBase;sendStringParametersAsUnicode=false";
And now both direct queries and preparedStatements runs at millisecond speed.
Thank you #DanGuzman for your suggestions!
I am trying to run an alter system command through JDBC which is required for me to run a query optimally.
I`m not sure if I am doing it right as I am not able to see the effect of the alter session statement. How do we persist the same session in JDBC what I mean is if I use the same connection and not close it does it mean I am using the same session?
The connection and database class are just helper classes to get a connection.
MyConnection mainDatabaseConnection = new MyConnection("jdbc:oracle:thin:#aas:111:"+tm.databaseName, "sys as sysdba", "xxx");
Database mainDatabase = new Database(mainDatabaseConnection.getConnection());
/* Fill in with data got for the main database */
//String auditQuery = mainDatabase.generateAuditQuery(tm.schemaName, tm.tableName);
String auditQuery = "select id, name, school, start, end from user where start>'11-11-11' and start<'12-12-12'";
System.out.println(auditQuery);
ResultSet rs = mainDatabase.runQuery("ALTER SESSION set optimizer_use_invisible_indexes = true");
// ResultSetMetaData md = rs.getMetaData();
// System.out.println(md.getColumnCount());
rs.close();
mainDatabase.close();
mainDatabaseConnection.close();
I am not sure if the alter session command ran successfully.
Question 2: When I run a Select query using Statement I get a resultSet. WHen I close the statement does the resultSet get closed too? So, as soon as I close the statment or connection does all the fetched data go away?
A java.sql.Connection object represents a session in Oracle. As long as you keep using the same object, you're in the same session.
With regard to closing Statements, as Mark Rotteveel commented - closing a Statement will indeed close a ResultSet that was opened by it. It would, however, be recommended to close the RestulSet once you're done with it, even (or, actually - especially) if you intend to reuse the Statement object.
In JDBC, can I use single Statement object to call executeQuery("") multiple times? Is it safe? Or should I close the statement object after each query, and create new object for executing another query.
E.G:
Connection con;
Statement s;
ResultSet rs;
ResultSet rs2;
try
{
con = getConnection();
// Initially I was creating the Statement object in an
// incorrect way. It was just intended to be a pseudocode.
// But too many answerers misinterpretted it wrongly. Sorry
// for that. I corrected the question now. Following is the
// wrong way, commented out now.
// s = con.prepareStatement();
// Following is the way which is correct and fits for my question.
s = con.createStatement();
try
{
rs = s.executeQuery(".......................");
// process the result set rs
}
finally
{
close(rs);
}
// I know what to do to rs here
// But I am asking, should I close the Statement s here? Or can I use it again for the next query?
try
{
rs2 = s.executeQuery(".......................");
// process the result set rs2
}
finally
{
close(rs2);
}
}
finally
{
close(s);
close(con);
}
Yes you can re-use a Statement(specifically a PreparedStatement) and should do so in general with JDBC. It would be inefficient & bad style if you didn't re-use your statement and immediately created another identical Statement object. As far as closing it, it would be appropriate to close it in a finally block, just as you are in this snippet.
For an example of what you're asking check out this link: jOOq Docs
I am not sure why you are asking. The API design and documentation show it is perfectly fine (and even intended) to reuse a Statement object for multiple execute, executeUpdate and executeQuery calls. If it wouldn't be allowed that would be explicitly documented in the Java doc (and likely the API would be different).
Furthermore the apidoc of Statement says:
All execution methods in the Statement interface implicitly close a statment's [sic] current ResultSet object if an open one exists.
This is an indication that you can use it multiple times.
TL;DR: Yes, you can call execute on single Statement object multiple times, as long as you realize that any previously opened ResultSet will be closed.
Your example incorrectly uses PreparedStatement, and you cannot (or: should not) be able to call any of the execute... methods accepting a String on a PreparedStatement:
SQLException - if [...] the method is called on a PreparedStatement or CallableStatement
But to answer for PreparedStatement as well: the whole purpose of a PreparedStatement is to precompile a statement with parameter placeholders and reuse it for multiple executions with different parameter values.
I can't find anything in the API docs that would state, that you shouldn't call executeQuery() on a given PreparedStatement instance more than once.
However your code does not close the PreparedStatement - a call to executeQuery() would throw a SQLException in that case - but the ResultSet that is returned by executeQuery(). A ResultSet is automatically closed, when you reexecute a PreparedStatement. Depending on your circumstances you should close it, when you don't need it anymore. I would close it, because i think it's bad style not to do so.
UPDATE Upps, I missed your comment between the two try blocks. If you close your PreparedStatement at this point, you shouldn't be able to call executeQuery() again without getting a SQLException.
A Prepared Statement tells the database to remember your query and to be prepared to accept parameterized variables to execute in that query. It's a lot like a stored procedure.
Prepared Statement accomplishes two main things:
It automatically escapes your query variables to help guard against SQL Injection.
It tells the database to remember the query and be ready to take variables.
Number 2 is important because it means the database only has to interpret your query once, and then it has the procedure ready to go. So it improves performance.
You should not close a prepared statement and/or the database connection in between execute calls. Doing so is incredibly in-efficient and it will cause more overhead than using a plain old Statement since you instruct the database each time to create a procedure and remember it. Even if the database is configured for "hot spots" and remembers your query anyways even if you close the PreparedStatement, you still incur network overhead as well as small processing time.
In short, keep the Connection and PreparedStatement open until you are done with them.
Edit: To comment on not returning a ResultSet from the execution, this is fine. executeQuery will return the ResultSet for whatever query just executed.
Firstly I am confused about your code
s = con.prepareStatement();
Is it work well?I can't find such function in JAVA API,at least one parameter is needed.Maybe you want to invoke this function
s = con.createStatement();
I just ran my code to access DB2 for twice with one single Statement instance without close it between two operation.It's work well.I think you can try it yourself too.
String sql = "";
String sql2 = "";
String driver = "com.ibm.db2.jcc.DB2Driver";
String url = "jdbc:db2://ip:port/DBNAME";
String user = "user";
String password = "password";
Class.forName(driver).newInstance();
Connection conn = DriverManager.getConnection(url, user, password);
Statement statement = conn.createStatement();
ResultSet resultSet = statement.executeQuery(sql);
int count = 0;
while (resultSet.next()) {
count++;
}
System.out.println("Result row count of query number one is: " + count);
count = 0;
resultSet = statement.executeQuery(sql2);
while (resultSet.next()) {
count++;
}
System.out.println("Result row count of query number two is: " + count);
I am deleting more than 90k rows in my table through JDBC prepared statement.
My code looks like this:
//Open JDBC Connection
//MYSQl QUERY
String fetchDataSQL="DELETE FROM MYTABLE WHERE ID=? AND X=? AND Y=?";
preparedStatement = dbConnection.prepareStatement(fetchDataSQL);
rs = preparedStatement.executeQuery();
dbConnection.setAutoCommit(false);
for (Data dt : dataList) {
preparedStatement.setLong(1, dt.getID());
preparedStatement.setLong(2, dt.getX());
preparedStatement.setLong(3, dt.getY());
preparedStatement.addBatch();
}
preparedStatement.executeBatch();
dbConnection.commit();
//cose prepared statement
//close connection
Here dataList contains more than 90k record which I want to delete.I had applied mysql indexing also to MYTABLE for id,X,Y
Unfortunately I got error
Deadlock found when trying to get lock; try restarting transaction
I have googled, I did not found working solution.
please help me to find solution or alternative if present.
Thank you
As #bodi0 points out, the MySQL Reference Manual (14.2.7.9. How to Cope with Deadlocks) has lots of advice on how to diagnose deadlocks, and how to deal with them.
In this case I can think of two possible explanations:
You are deadlocking against some other transaction being performed by a different database connection or a different database client.
You have entries in the datalist that have the same {ID, X, Y} values, so you end up adding multiple deletes for the same row or rows to the batch. Maybe this results in MySQL attempting to lock the same row twice, and deadlocking. (Just a theory ...)
But a better idea would be to diagnose the deadlock for yourself.