Code:
// Start the query.
final ResultSet r = prepared.executeQuery();
try {
// Returning false from oneRow will stop the process.
boolean goOn = true;
while (r.next() && goOn) {
log.trace("Result: {}", toString(r));
// We did do at least one.
ran = true;
goOn = oneRow(r);
}
} finally {
try {
// Always remember to close the ResultSet.
r.close();
} catch (SQLException ex) {
log.error("Close failed", ex);
}
}
// Handle one row.
public boolean oneRow(ResultSet r) throws Exception {
String xml1 = r.getString(1);
String xml2 = r.getString(2);
if (xml1 == null && xml2 == null) {
// Probably compressed.
xml1 = decompress(r, 3);
xml2 = decompress(r, 4);
}
return false;
}
private static String decompress(ResultSet rs, int col) throws SQLException {
// Exception gets thrown here!!! ???
final InputStream compressed = rs.getBinaryStream(col);
...
As you can see this is not the obvious problem of not calling ResultSet.next(). Not only that but I have called getString in the ResultSet twice already, it is the third time that breaks it.
Note that this is an occasional issue, this code works fine most of the time.
The query looks something like:
"SELECT P1.XML XML1, "
+ "P2.XML XML2, "
+ "P1.CompressedXML CompressedXML1, "
+ "P2.CompressedXML CompressedXML2 "
+ "FROM Table1 P1 "
+ "LEFT JOIN Table2 T2 ON T2.ID = P1.ID "
+ "LEFT JOIN Table1 P2 ON P2.ID = T2.Item_Code "
+ "WHERE P1.ID = ?"
I realise this is a rather incestuous query but as I stated - this all works fine most of the time.
Added
Prompted by the answer posted by user1933888 it occurs to me that since I am using a home-grown connection-pool is it possible for a prepared statement to interfere with itself between two threads sharing different connections?
I am confident that the same connection will never be in use by two threads at the same time but the prepared statement could be shared as it should reside in the database.
As your PreparedStatement is linked to a connection, if you share the PreparedStatement between threads you would end up sharing a connection and that definitely will cause problems.
It is true that statements are generally cached in the DB to avoid having to reparse it but you'd still need one PreparedStatement object per thread to avoid confusing the DB driver as to what thread wants what.
In general, I would avoid a home brew connection pool considering that in general AppServers do the pooling for you or (if you're not in an AppServer) there are plenty of connection pool libs out there, so why reinvent the wheel.
In general:
you can shared DataSources between concurrent threads
you should NOT share Connection, PreparedStatement or ResultSet objects between concurrent threads
The connection pool will ensure that non-concurrent threads can reuse Connections and the JDBC driver will internally cache Statements
Hope that makes sense.
Related
I am currently learning some basic java SQL coding, making a basic terminal UI for my SQL project. I have been using PostgreSQL
I am using PreparedStatement to ensure myself from SQL injections, better be safe than sorry. PreparedStatement seems to always fire warnings for some reason, which I figured is a subclass to exceptions (and should get caught in the exceptions).
In my SQL triggers and functions I have created and tested the cases where I should fire exceptions, and they are all getting and working properly catch block.
I guess using #Supresswarnings to let the compiler know I want to do suppress my warnings, but I might want to catch some warnings in the catch block, so I might be looking for a different solution.
The problem / question is:
I would like to have my prints in the try block, that is no exceptions were fired.
What could be done to have my prints after the execution of preparedStmt.executeQuery in the try block?
Is using #SuppressWarnings usually considered good coding practice when dealing with exception handling like this?
If I am incorrect about preparedStatement.executeQuery() always fires warnings, what is considered warnings in SQL language?
My code:
void registerStudent(Connection conn, String toBeInserted, String insertedTo)
throws SQLException{
ResultSet res;
String query = "INSERT INTO Registrations VALUES (?, ?)";
PreparedStatement ps = conn.prepareStatement(query);
ps.setString(1, toBeInserted);
ps.setString(2, insertedTo);
try {
ps.executeQuery();
}catch (SQLException e) {
SQLWarning warning = ps.getWarnings();
if (warning != null){
System.out.println("Yay, insert succeeded!
values: "+ toBeInserted +" were inserted into
"+insertedTo);
}else {
System.out.println("Nothing inserted, here goes the big exception SQL fired for ya'll: "
+ e.getMessage());
}
}
}
There are still some flaws. I added also the case of an AUTOINCREMENT primary key - which you can retrieve after inserting.
Flaws:
List which fields you insert, such that a later addition of a column in the database does not cause problems, and the order of columns is clear in the code.
Use try-with-resources to close statement, result set and such, even when one returns inside, or an exception is thrown.
Use executeUpdate for INSERT (and similar modifying operations) returning an update count.
You catched the SQLException, which then should be rethrown, to let the caller determine what to do on gross failure.
SQLWarnings (as does the actual SQL and the SQLException) might be interesting,
though I not often see SQLWarnings being treated. Maybe forget it for such simple case, and instead try new SQL statements first in a separate SQL tool.
So:
/**
* #return the generated primary key, the Student ID.
*/
int registerStudent(Connection conn, String toBeInserted, String insertedTo)
throws SQLException {
String query = "INSERT INTO Registrations(ToBeInserted, InsertedInto) VALUES(?, ?)";
try (PreparedStatement ps = conn.prepareStatement(query),
Statement.RETURN_GENERATED_KEYS) {
ps.setString(1, toBeInserted);
ps.setString(2, insertedTo);
int updateCount = ps.executeUpdate();
if (updateCount == 1) {
try (ResultSet rs = ps.getGeneratedKeys()) {
if (rs.next()) {
int id = rs.getInt(1);
return id;
}
}
throw new SQLException("No primary key generated");
} else {
SQLWarning warning = ps.getWarnings();
Logger.getLogger(Xyz.class.getName()).info("Not added, values: "
+ toBeInserted + " were not inserted into "+ insertedTo + ": " + warning);
throw new SQLException("Not added");
}
}
}
Thank you very much I know I really shouldn't respond to your post but it helped be a lot. I think I got confused about ps.executeQuery() and ps.executeUpdate() methods.
executeQuery() generated a warning because there was nothing being returned, although it did work to execute the update to the database with that method.
The other tips and tricks is something I will bear with me as well.
I have a DAO class, that has method below. I call this one inside Transaction manager. When I ran it without "conn.commit()" line - it throws timeout exception, but when I ran it with this one - it is ok. What's the problem? As I know it is not necessary to commit if you not modify db?
#Override
public List<String> getLinks(int id) throws SQLException {
List<String> list = new ArrayList<>();
Connection conn = factory.newConnection();
Statement statement = null;
ResultSet rs = null;
try {
String expression = "select link from users.links where id=" + id + " order by id_link desc";
statement = conn.createStatement();
rs = statement.executeQuery(expression);
while (rs.next()) {
list.add(rs.getString("link"));
}
// !!!!!!!!!!!!! without next line method throw TimeoutException
conn.commit(); // <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
// !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
return list;
} catch (SQLException e) {
rollBackQuietly(conn);
e.printStackTrace();
} finally {
closeQuaitly(rs);
closeQuaitly(statement);
closeQuaitly(conn);
}
return null;
}
Seeing as the commit() call is after the line which is throwing the exception, this problem must come after repeated invocations of this method (useful information to include in your question). this leads me to believe that your Connection factory is re-using Connections, and that it is handing out "stale" Connections (Connections which have been sitting around for too long and are no longer usable). if this is all true, then you need to make your factory manage Connections better. if it is a reasonably built connection pool, it probably has some feature like "test while idle" or "test on get" which you need to enable.
I have a thread that executes and updates a database with some values. I commented out all the operations done to the data and just left the lines you can see below. While running the program with the lines you see bellow I get a memory leak.
This is what it looks like in VisualVM
Mysql Class
public ResultSet executeQuery(String Query) throws SQLException {
statement = this.connection.createStatement();
resultSet = statement.executeQuery(Query);
return resultSet;
}
public void executeUpdate(String Query) throws SQLException {
Statement tmpStatement = this.connection.createStatement();
tmpStatement.executeUpdate(Query);
tmpStatement.close();
tmpStatement=null;
}
Thread file
public void run() {
ResultSet results;
String query;
int id;
String IP;
int port;
String SearchTerm;
int sleepTime;
while (true) {
try {
query = "SELECT * FROM example WHERE a='0'";
results = this.database.executeQuery(query);
while (results.next()) {
id = results.getInt("id");
query = "UPDATE example SET a='1' WHERE id='"
+ id + "'";
SearchTerm=null;
this.database.executeUpdate(query);
}
results.close();
results = null;
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The problem has happened with many other people after researching the web,
https://forum.hibernate.org/viewtopic.php?f=1&t=987128
Is bone cp or mysql.jdbc.JDBC4Connection known for leaking?
and a few more if you google "jdbc4resultset memory leak"
The leak is yours, not MySQL's. You aren't closing the statement.
The design of your method is poor. It should close the statement, and it should return something that will survive closing of the statement, such as a CachedRowSet.
Or else it should not exist at all. It's only three lines, and it doesn't support query parameters, so it isn't really much use. I would just delete it.
You also appear to have statement as an instance member, which is rarely if ever correct. It should be local to the method. At present your code isn't even thread-safe.
You should also be closing the ResultSet in a finally block to ensure it gets closed. Ditto the Statement.
Make sure that you are explicitly closing the database connections.
I am building a web app with a single (not pooled) full time (jdbc) connection between static classes and the database. This is expected to be a low traffic site and the static methods are synchronized. In an effort to speed up access to product information, I am trying PreparedStatements for the first time. As I test, on localhost, sure that I'm the only one running the app., it seems clear to me that my the prepared statements are slower than the unprepared statements I use earlier in the process. This may not be a fair comparison. The unprepared statements get a single result set from one table and it's done. As you can see from the code below, what I'm doing with prepared statements involves three tables and multiple queries. But since this is my first time, I would appreciate review and comment. This does actually work; i.e. all data is retrieved as expected.
The first method below (initialize()) is called once from a servlet init() method when the application is first started. The second method (getItemBatch()) retrieves information about as many product items as match a product name (Titel). My little development / test database has less than 100 (total) items and only 1-3 items matching each name; most often only 1. The server app and database are on the same machine and I'm accessing from a browser via localhost. I'm surprised by the consistent perceptible wait for this detailed product data compared to fetching a master product list (all items) mentioned above.
public static boolean initialize (Connection connArg, String dbNameArg, String dbmsArg) {
con = connArg;
dbName = dbNameArg;
dbms = dbmsArg;
sqlserver_con = connArg;
ItemBatchStatement =
"select Cartnr, Hgrupp, Prisgrupp, Moms from dbo.Centralartregister " +
"where Titel = ?";
ForArtNrStatement =
"select Artnr, Antal from dbo.Artikelregister " +
"where Cartnr = ? and Antal > 0";
ItemHgruppStatement =
"select Namn from dbo.Huvudgrupper " +
"where Hgrupp = ?";
try {
con.setAutoCommit(false);
getItemBatch = con.prepareStatement(ItemBatchStatement);
getForArtNr = con.prepareStatement(ForArtNrStatement);
getItemHgrupp = con.prepareStatement(ItemHgruppStatement);
} catch (SQLException e) {
return(false);
} finally {
try {con.setAutoCommit(true);} catch (SQLException e) {}
}
return(true);
}
-
public static synchronized String getItemBatch (String Titel) throws SQLException {
String ret_xml;
ResultSet rs = null;
ResultSet rs1 = null;
ResultSet rs2 = null;
Titel = charChange(Titel);
ret_xml = "<ItemBatch Titel='" + Titel + "'>";
try {
con.setAutoCommit(false);
getItemBatch.setString(1,Titel);
rs = getItemBatch.executeQuery();
while (rs.next()) {
getForArtNr.setInt(1,rs.getInt("Cartnr"));
rs1 = getForArtNr.executeQuery();
getItemHgrupp.setInt(1,rs.getInt("Hgrupp"));
rs2 = getItemHgrupp.executeQuery();
if (rs1.next() && rs2.next()) {
ret_xml += "<item><Hgrupp>" + rs2.getString("Namn") + "</Hgrupp><Price>" + rs.getInt("Prisgrupp") + "</Price><Moms>" + rs.getInt("Moms") + "</Moms><Cartnr>" + rs.getInt("Cartnr") + "</Cartnr><Artnr>" + rs1.getInt("Artnr") + "</Artnr></item>";
}
}
ret_xml += "</ItemBatch>";
return(ret_xml);
} catch (SQLException e) {
return("SQLException: " + e);
} finally {
try {con.setAutoCommit(true);} catch (SQLException e) {}
if (rs != null) {rs.close();}
if (rs1 != null) {rs1.close();}
if (rs2 != null) {rs2.close();}
}
}
.
UPDATE: I'm still googling and looking for ways to do better. Let me add something for your consideration.
I'm closing the prepared statements via the servlet's destroy() method; the same servlet that calls the method "initialize()" above in order to create them. The idea is to create them once (and only once) and have them available forever for all users (i.e. until the app or server is shut down). I'm going for a kind-of a pseudo-stored procedure. I would just use stored procedures straight out, but the database exists to support another application (their in-house sales system) and I'm going to use it for Internet sales read-only ... avoiding any potential conflict with their maintenance efforts or agreements etc. by not doing anything to change their database set-up. I have suggested that my app use a new account limited to read-only privileges. And come to think of it, I haven't tested whether I can use Prepared Statements in that mode; just seems like it should work.
Each result set is closed in the finally block in the method where they are created. I guess that's ok? I reuse the same RS variable name for multiple result sets (on the second two) but close only once. But wait! Do I even need that? The ResultSets are declared within the scope of the method. If resetting them without closing the old ones doesn't cause leaks, then exiting the method should do just as well on its own. It is a static method, but the ResultSets will be reset every time it's used (at the very least). So, at most, there would just be a single set of ResultSet handles available for reuse; not run-away "leakage."
I'm wondering if I can send the two select requests in the loop both at the same time, simply by turning them into one prepared statement separated by ';'. Or just found "MultipleActiveResultSets=True"; document allowing SQL Server to process multiple transaction requests on a single connection ... still investigating this. Or is there another way to create a prepared statement that would fetch ALL the data via a single submission? (Seems to me like there are too many round trips.) Finally, I might get a boost from using connection pooling, which I haven't done yet. It's low priority in my project right now, but I might have to do it before we go online.
If you create web applicaton and use some web container like tomcat, jetty etc you always can use jdbc datasource and connection pool. It is simple. Good explanation gives here
If you don't want to use connection pool I suppose it would be better use Method Scope Connections described link above
I am trying to use JDBC and my query is working in some cases but not working in others.
I would really appreciate any help.
Some of my code:
public Result getSpecificTopic()
{
String query = "Select msg_body, msg_author from lawers_topic_msg";// where msg_id=2 order by msg_id desc";
try
{
con = mysql.getConnection();
//Statement stmt = con.createStatement();
PreparedStatement stmt = con.prepareStatement(query);
//stmt.setInt(1, topicId);
ResultSet rs = stmt.executeQuery(query);
int rowCount = rs.getRow();
specificTopic = ResultSupport.toResult(rs);
con.close();
stmt.close();
}
catch(Exception e)
{
}
return this.specificTopic;
}
public void setTopicId(String num)
{
this.topicId = Integer.parseInt(num);
}
public int getTopicId()
{
return this.topicId;
}
However if i change
String query = "Select msg_body, msg_author from lawers_topic_msg";
to the
String query = "Select msg_body, msg_author from lawers_topic_msg where msg_id = " + topicId;
Then the resultset retunrs nothing....
I am breaking my head here and still cannot figure out what is the problem
You still aren't closing your resources properly. That should be done in a finally block:
http://www.java-blog.com/correct-closing-jdbc-resources
As a first step, it'd be worth making sure an exception's not being thrown - at the very least log something in your catch() block.
Also be worth logging the SQL generated, and making sure that actually returns what you expect from the database when running it directly.
If you have multiple databases, it'd be worth confirming you're running against the one you think you are - I'm embarrassed to admit I've been caught out that way before.
several issues with your code, i'll keep it short:
don't encapuslate with try / catch at this layer, aspecially not since you're doing no error management. this.specificTopic looks global, so if your query fails it will return whatever was stored in this.specificTopic.
also try what BobbyShaftoe said. print in console or use your debugger. This should give you a good indication on what is wrong.
my first guess would be Integer.parseInt(num) could throw an exception. if so, the sql statement will be broken.
secondly, as Makach pointed out, there are several issues. first the catch-all
you should not use string concatenation, like
....where msg_id = " + topicId;
but rather
....where msg_id = ?"
stmt.set Int(1,topicId)
edit: it seems thats what you were trying anyways, SO sucks in some characters.