Code executing time is incredibly long - java

I'm trying to move data from one table to another(both table are same basically), the method that I'm using is INSERT-SELECT.
The problem that I'm facing is my java program seem like frozen there, but I can still force close it with ^C easily, so I think it might be still alive but stuck for some reason.
This is my code which stuck in this problem
public String moveData(String sql, int day) {
Connection con = null;
PreparedStatement stmt = null;
int count;
try {
con = DriverManager.getConnection(DSN, Username, Password);
stmt = con.prepareStatement(sql);
stmt.setInt(1, day);
count = stmt.executeUpdate();
return String.valueOf(count);
} catch (SQLException e) {
System.out.println("SQL exception: " + e.getMessage());
e.printStackTrace();
return "false";
} finally {
try {
stmt.close();
} catch (Exception e) {
}
try {
con.close();
} catch (Exception e) {
}
}
}
and the SQL, executing it in SQL developer is really fast with 0.231 sec for 23k row of data
insert into Request_History (customer_id, request_id, status, transaction_date, last_modified)
SELECT customer_id, request_id, status, transaction_date, current_timestamp
from Request_Log
where transaction_date <= (sysdate - NUMTODSINTERVAL(:1 ,'DAY'))
I see no problem on them, is there anything that I missed?
Update
Since there's no resolve on the program and SQL command, I'd like to change a way of thinking on the DB side.
Could anyone please tell me what kind of privileges do I need to execute my INSERT-SELECT SQL command on 11g without problem? because from what I can see that this command would only needs basic privileges such as SELECT/INSERT/UPDATE/DELETE to execute.

The time of the whole insert process depends on three factors:
How many rows are in the query result set
Is there any index, which can be used at query level (best for this would be an index on Request_Log.transaction_date
Are there any constraints, indexes which have to be maintained during the insert phase (Request_History table)
Check the exec plan of the statement, to see if there's anything - worst things are full table scans. If you allowed, you can paste here the execution plan as well.

there is probably something wrong with the way you open your connection, something with you session parameters. You might not have enough undo space or another limiting factor. Maybe you can get a dba to check on it while you execute this over java vs. SQL Developer.
alter you statement to
create table Request_History_test
as SELECT
customer_id, request_id, status, transaction_date, current_timestamp
from
Request_Log
where
transaction_date <= (sysdate - NUMTODSINTERVAL(:1 ,'DAY'))
if this goes fast in contrast to your "insert select", then contact your DBA. There might be known issues with your undo space / redo logs. I'm not a DBA, so this answer is vague, I just had similar problems once.
oh, and check your dba_hist_sqlstat to see where the time is lost.

Related

How do I check if the sql command returns null or a value?

try{
Statement stm = conn.createStatement();
String sql = "SELECT * from BOOKS WHERE ISBN_No = '" + line + "'";
ResultSet rs = stm.executeQuery(sql);
if(//values are returned) {
displayBookInfo(line);
}
else (//if it is null) {
System.out.println("No book found");
}
stm.close();
} catch (SQLException e) {
e.printStackTrace();
System.out.println("Fail to search the book" + line );
noException = false;
}
After I execute the ResultSet rs = stm.executeQuery(sql); I want to check if the query returned a value or if it was empty so that I can execute either "display book details" or "no book found message". I am just confused about how I should compare and how comparison works.
This code is a security leak. You must fix this first.
You cannot include untrusted inputs in a query like this. What if someone enters, say:
1234'; DROP TABLE books CASCADE; EXECUTE 'FORMAT C: /Y'; --
In the web form? Don't try it, you'll wipe your disk. You get the point, surely.
The right way is to use stm.prepareStatement(sql), where sql is a constant (so not something you insert user entered stuff into), using a ? where user input is needed, then calling .setString(1, line) to then tell your db driver what should go in place of the question mark.
Then, simply rs.next(), which advanced to the next row in the result (first call advances to the first row). If there are no rows left, it returns false instead. Hence, if your query returns 0 rows, the first resultSet.next() call returns false right away.
Your code also fails to close. You must use try-with-resources on everything (ResultSet, (Prepared)Statement, and most importantly the Connection), or your app will crash after a few statements.
NB: Minor nit, if all you want to know is if there's at least one result, add LIMIT 1, and just SELECT 1 FROM instead - it's less overhead that way.

JDBC exception handling: warning suppression

I am currently learning some basic java SQL coding, making a basic terminal UI for my SQL project. I have been using PostgreSQL
I am using PreparedStatement to ensure myself from SQL injections, better be safe than sorry. PreparedStatement seems to always fire warnings for some reason, which I figured is a subclass to exceptions (and should get caught in the exceptions).
In my SQL triggers and functions I have created and tested the cases where I should fire exceptions, and they are all getting and working properly catch block.
I guess using #Supresswarnings to let the compiler know I want to do suppress my warnings, but I might want to catch some warnings in the catch block, so I might be looking for a different solution.
The problem / question is:
I would like to have my prints in the try block, that is no exceptions were fired.
What could be done to have my prints after the execution of preparedStmt.executeQuery in the try block?
Is using #SuppressWarnings usually considered good coding practice when dealing with exception handling like this?
If I am incorrect about preparedStatement.executeQuery() always fires warnings, what is considered warnings in SQL language?
My code:
void registerStudent(Connection conn, String toBeInserted, String insertedTo)
throws SQLException{
ResultSet res;
String query = "INSERT INTO Registrations VALUES (?, ?)";
PreparedStatement ps = conn.prepareStatement(query);
ps.setString(1, toBeInserted);
ps.setString(2, insertedTo);
try {
ps.executeQuery();
}catch (SQLException e) {
SQLWarning warning = ps.getWarnings();
if (warning != null){
System.out.println("Yay, insert succeeded!
values: "+ toBeInserted +" were inserted into
"+insertedTo);
}else {
System.out.println("Nothing inserted, here goes the big exception SQL fired for ya'll: "
+ e.getMessage());
}
}
}
There are still some flaws. I added also the case of an AUTOINCREMENT primary key - which you can retrieve after inserting.
Flaws:
List which fields you insert, such that a later addition of a column in the database does not cause problems, and the order of columns is clear in the code.
Use try-with-resources to close statement, result set and such, even when one returns inside, or an exception is thrown.
Use executeUpdate for INSERT (and similar modifying operations) returning an update count.
You catched the SQLException, which then should be rethrown, to let the caller determine what to do on gross failure.
SQLWarnings (as does the actual SQL and the SQLException) might be interesting,
though I not often see SQLWarnings being treated. Maybe forget it for such simple case, and instead try new SQL statements first in a separate SQL tool.
So:
/**
* #return the generated primary key, the Student ID.
*/
int registerStudent(Connection conn, String toBeInserted, String insertedTo)
throws SQLException {
String query = "INSERT INTO Registrations(ToBeInserted, InsertedInto) VALUES(?, ?)";
try (PreparedStatement ps = conn.prepareStatement(query),
Statement.RETURN_GENERATED_KEYS) {
ps.setString(1, toBeInserted);
ps.setString(2, insertedTo);
int updateCount = ps.executeUpdate();
if (updateCount == 1) {
try (ResultSet rs = ps.getGeneratedKeys()) {
if (rs.next()) {
int id = rs.getInt(1);
return id;
}
}
throw new SQLException("No primary key generated");
} else {
SQLWarning warning = ps.getWarnings();
Logger.getLogger(Xyz.class.getName()).info("Not added, values: "
+ toBeInserted + " were not inserted into "+ insertedTo + ": " + warning);
throw new SQLException("Not added");
}
}
}
Thank you very much I know I really shouldn't respond to your post but it helped be a lot. I think I got confused about ps.executeQuery() and ps.executeUpdate() methods.
executeQuery() generated a warning because there was nothing being returned, although it did work to execute the update to the database with that method.
The other tips and tricks is something I will bear with me as well.

SQL PreparedStatement; Am I doing it right?

I am building a web app with a single (not pooled) full time (jdbc) connection between static classes and the database. This is expected to be a low traffic site and the static methods are synchronized. In an effort to speed up access to product information, I am trying PreparedStatements for the first time. As I test, on localhost, sure that I'm the only one running the app., it seems clear to me that my the prepared statements are slower than the unprepared statements I use earlier in the process. This may not be a fair comparison. The unprepared statements get a single result set from one table and it's done. As you can see from the code below, what I'm doing with prepared statements involves three tables and multiple queries. But since this is my first time, I would appreciate review and comment. This does actually work; i.e. all data is retrieved as expected.
The first method below (initialize()) is called once from a servlet init() method when the application is first started. The second method (getItemBatch()) retrieves information about as many product items as match a product name (Titel). My little development / test database has less than 100 (total) items and only 1-3 items matching each name; most often only 1. The server app and database are on the same machine and I'm accessing from a browser via localhost. I'm surprised by the consistent perceptible wait for this detailed product data compared to fetching a master product list (all items) mentioned above.
public static boolean initialize (Connection connArg, String dbNameArg, String dbmsArg) {
con = connArg;
dbName = dbNameArg;
dbms = dbmsArg;
sqlserver_con = connArg;
ItemBatchStatement =
"select Cartnr, Hgrupp, Prisgrupp, Moms from dbo.Centralartregister " +
"where Titel = ?";
ForArtNrStatement =
"select Artnr, Antal from dbo.Artikelregister " +
"where Cartnr = ? and Antal > 0";
ItemHgruppStatement =
"select Namn from dbo.Huvudgrupper " +
"where Hgrupp = ?";
try {
con.setAutoCommit(false);
getItemBatch = con.prepareStatement(ItemBatchStatement);
getForArtNr = con.prepareStatement(ForArtNrStatement);
getItemHgrupp = con.prepareStatement(ItemHgruppStatement);
} catch (SQLException e) {
return(false);
} finally {
try {con.setAutoCommit(true);} catch (SQLException e) {}
}
return(true);
}
-
public static synchronized String getItemBatch (String Titel) throws SQLException {
String ret_xml;
ResultSet rs = null;
ResultSet rs1 = null;
ResultSet rs2 = null;
Titel = charChange(Titel);
ret_xml = "<ItemBatch Titel='" + Titel + "'>";
try {
con.setAutoCommit(false);
getItemBatch.setString(1,Titel);
rs = getItemBatch.executeQuery();
while (rs.next()) {
getForArtNr.setInt(1,rs.getInt("Cartnr"));
rs1 = getForArtNr.executeQuery();
getItemHgrupp.setInt(1,rs.getInt("Hgrupp"));
rs2 = getItemHgrupp.executeQuery();
if (rs1.next() && rs2.next()) {
ret_xml += "<item><Hgrupp>" + rs2.getString("Namn") + "</Hgrupp><Price>" + rs.getInt("Prisgrupp") + "</Price><Moms>" + rs.getInt("Moms") + "</Moms><Cartnr>" + rs.getInt("Cartnr") + "</Cartnr><Artnr>" + rs1.getInt("Artnr") + "</Artnr></item>";
}
}
ret_xml += "</ItemBatch>";
return(ret_xml);
} catch (SQLException e) {
return("SQLException: " + e);
} finally {
try {con.setAutoCommit(true);} catch (SQLException e) {}
if (rs != null) {rs.close();}
if (rs1 != null) {rs1.close();}
if (rs2 != null) {rs2.close();}
}
}
.
UPDATE: I'm still googling and looking for ways to do better. Let me add something for your consideration.
I'm closing the prepared statements via the servlet's destroy() method; the same servlet that calls the method "initialize()" above in order to create them. The idea is to create them once (and only once) and have them available forever for all users (i.e. until the app or server is shut down). I'm going for a kind-of a pseudo-stored procedure. I would just use stored procedures straight out, but the database exists to support another application (their in-house sales system) and I'm going to use it for Internet sales read-only ... avoiding any potential conflict with their maintenance efforts or agreements etc. by not doing anything to change their database set-up. I have suggested that my app use a new account limited to read-only privileges. And come to think of it, I haven't tested whether I can use Prepared Statements in that mode; just seems like it should work.
Each result set is closed in the finally block in the method where they are created. I guess that's ok? I reuse the same RS variable name for multiple result sets (on the second two) but close only once. But wait! Do I even need that? The ResultSets are declared within the scope of the method. If resetting them without closing the old ones doesn't cause leaks, then exiting the method should do just as well on its own. It is a static method, but the ResultSets will be reset every time it's used (at the very least). So, at most, there would just be a single set of ResultSet handles available for reuse; not run-away "leakage."
I'm wondering if I can send the two select requests in the loop both at the same time, simply by turning them into one prepared statement separated by ';'. Or just found "MultipleActiveResultSets=True"; document allowing SQL Server to process multiple transaction requests on a single connection ... still investigating this. Or is there another way to create a prepared statement that would fetch ALL the data via a single submission? (Seems to me like there are too many round trips.) Finally, I might get a boost from using connection pooling, which I haven't done yet. It's low priority in my project right now, but I might have to do it before we go online.
If you create web applicaton and use some web container like tomcat, jetty etc you always can use jdbc datasource and connection pool. It is simple. Good explanation gives here
If you don't want to use connection pool I suppose it would be better use Method Scope Connections described link above

Resultset not open

I get following error on Result set
java.sql.SQLException: ResultSet not open. Verify that autocommit is OFF.
at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source)
at org.apache.derby.client.am.ResultSet.next(Unknown Source)
public ResultSet insertDb(int Id,String firstName,String lastName,String title) throws SQLException{
try {
try {
Class.forName(driver);
con = DriverManager.getConnection(connectionURL);
} catch (SQLException ex) {
Logger.getLogger(Connect.class.getName()).log(Level.SEVERE, null, ex);
} catch (ClassNotFoundException ex) {
Logger.getLogger(Connect.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println(con.getAutoCommit());
statement = con.createStatement() ;
res = statement.executeQuery("SELECT * FROM CUSTOMER") ;
con.setAutoCommit(false);
System.out.println(con.getAutoCommit());
while(res.next()){
if(res.getString("ID").equalsIgnoreCase(Integer.toString(Id))){
UNIQUE = false;
error= "Duplicate Entry Found Please Enter New Data";
throw new SQLException("Duplicate info<br>ID " + Id );
}
}
// IF value to be added IS UNIQUE
if(UNIQUE){
String qry1= "insert into CUSTOMER(ID, FIRSTNAME,LASTNAME,TITLE) values(?,?,?,?)";
stm = con.prepareStatement(qry1);
String ID=Integer.toString(Id);
stm.setString(1, ID);
stm.setString(2, firstName);
stm.setString(3, lastName);
stm.setString(4, title);
stm.executeUpdate();
}
}
catch(Exception e){
String errorMessage = "Exception caught : ";
System.out.println(errorMessage + e.toString());
}finally{
if (con != null){
con.close();
}
}
return res;
}
Try moving the setAutoCommit() and getAutoCommit() to before you create and execute the statement. Changing it after you execute the statement may be invalidating the query.
The problem is that you have closed your query before reading your resultset. Closing the query, closes the resultset, hence why you get the "ResultSet not open" error. You should close the query right at the end, in a finally block:
i.e. con.setAutoCommit(false);
will close the query and along iwth it it closes the resultset also.
Not strictly related, but your code probably doesn't do what you expect. This kind of read-modify-write code doesn't work well when there are multiple concurrent invocations.
If you imagine two invocations running though the code, it becomes clear that sometimes, depending on the execution order, BOTH invocations could reach the insert statement.
In addition, selecting from a table without using a WHERE clause is not generally useful. In this case you select '*', then iterate over all the results to see if "ID" == Id. The database is much much better at that than java is. You should add a where clause. (Note that this still won't solve the above problem)
Its also generally a bad idea to 'select *' from any table. Just pick the columns that you need. This will 'fail fast' if the schema changes and the columns that you need are no longer available, and will allow the database optimiser to do the 'right thing' about its disk accesses.
Finally, if its just a numeric ID that you are looking to assign, its normal practice to use 'autonumber' for these, rather than get the program to pick them. Different databases call them different things, so you might also know them as IDENTITY, or have to use a sequence.
In case it helps anyone down the line, I had the same error with Derby 10.5.1.1 that turned out to be a bug in the driver itself that would appear some times and not others depending on the underlying data. Upgrading the driver to a newer version (10.8.2.2) resolved the problem.

sql cleanup function. rows doesnt get really deleted

i am using java sqlite (org.sqlite.JDBC) with this i am adding a new row of data to the table, one field is supposed to take up a big amount of base64 encoded data (like pdf) the type of this field is sql-TEXT. now, if i delete the row with "DELETE FROM table WHERE id='id'" the row gets deleted as expected, my sqlite browser confirms this. but the table was befor the deletion like 4KB big, after adding the row it was 12MB and after deleting it remains 12MB big. is there a kind of cleanup i have to do?
in sqlite admin(http://sqliteadmin.orbmu2k.de/) there is a "Cleanup" button after pressing that everything is fine, which means the database shrinks to its size befor adding stuff (4KB). after asking google i realy cannot find such a sql command. it seems that only the index informations get deleted from my databasefile, not the content itself. this behavior is known from windows delete functions.
beside that, here is the java snippet i use:
public void deleteRowById(String table, int id){
try {
Connection connection = null;
Class.forName("org.sqlite.JDBC");
connection = DriverManager.getConnection("jdbc:sqlite:C:\\meinedb");
//statement = connection.createStatement();
String sql = "DELETE FROM "+table+" WHERE id='"+id+"'";
PreparedStatement pstmt = connection.prepareStatement(sql);
pstmt.executeUpdate();
pstmt.close();
connection.close();
} catch (SQLException ex) {
Logger.getLogger(FileSpinner.class.getName()).log(Level.SEVERE, null, ex);
} catch (ClassNotFoundException ex){
}
}
You can shrink a SQLite database with the VACUUM statement. Read the manual I link to for details.

Categories

Resources