I use sqlite database and java.sql classes in servlet application to batch-insert some data into database.
There are consecutive four inserts of different kinds of data.
Each one looks like this:
PreparedStatement statement = conn
.prepareStatement("insert or ignore into nodes(name,jid,available,reachable,responsive) values(?,?,?,?,?);");
for (NodeInfo n : nodes)
{
statement.setString(1, n.name);
statement.setString(2, n.jid);
statement.setBoolean(3, n.available);
statement.setBoolean(4, n.reachable);
statement.setBoolean(5, n.responsive);
statement.addBatch();
}
conn.setAutoCommit(false);
statement.executeBatch();
conn.commit();
conn.setAutoCommit(true);
statement.close();
But sometimes I get the
java.sql.SQLException: database in auto-commit mode
I found in source code of java.sql.Connection that this exception is thrown when calling commit() while database is in autocommit mode. But I turn autocommit off before and I can't see any place for some parallel execution related issues as for now application is only turned on once.
Do you have any idea how to debug this issue? Maybe there's some other reason for this error (because I just found that exception about database not found or not well configured can be thrown when inserting null into non-null field)?.
May be an issue is with order of statements. Your database statement should be :
PreparedStatement statement1 = null;
PreparedStatement statement2 = null;
Connection connection=null;
try {
//1. Obtain connection and set `false` to autoCommit
connection.setAutoCommit(false);
//2. Prepare and execute statements
statement1=connection.prepareStatement(sql1);
statement2=connection.prepareStatement(sql2);
...
//3. Execute the statements
statement1.executeUpdate();
statement2.executeUpdate();
//4. Commit the changes
connection.commit();
}
} catch (SQLException e ) {
if (connection!=null) {
try {
connection.rollback();
} catch(SQLException excep) {}
}
}finally {
if (statement1 != null) {
statement1.close();
}
if (statement2 != null) {
statement2.close();
}
if(connection != null){
connection.setAutoCommit(true);
connection.close();
}
}
You have to prepare your Statement and create the batch after conn.setAutoCommit(false);.
When running this from a servlet, you have to make sure that the usage of the Connection is synchronized. Multiple requests could set the Connection to a different auto commit mode at nearly the same time. If you use one Connection per request, this will not be an issue. Otherwise, protect the above part with a critical section.
A tip regarding debugging which is applicable for tomcat / eclipse.
1) Enable JDPA debugging for your application server. In tomcat you can do this by adding the following lines to catalina.sh / catalina.bat:
set JPDA_ADDRESS=8000
set JPDA_TRANSPORT=dt_socket
2) Restart the application server
3) Connect with eclipse to your application server. "Debug as" --> "Remote Java Application"
4) Set a break point in above code.
5) Run the servlet.
Related
Is it possible to execute CALL system.sync_partition_metadata('dummy','dummy','FULL') using JDBC as Presto JDBC driver does not support CallableStatements?
Presto JDBC driver does not support io.prestosql.jdbc.PrestoConnection#prepareCall methods (please file an issue), but you can use Statement for this:
try (Connection connection = DriverManager.getConnection("jdbc:presto://localhost:8080/hive/default", "presto", "")) {
try (Statement statement = connection.createStatement()) {
boolean hasResultSet = statement.execute("CALL system.sync_partition_metadata('default', 'table_name', 'FULL')");
verify(!hasResultSet, "unexpected resultSet");
}
}
(BTW you can get always get more help with Presto on Trino (formerly Presto SQL) community slack)
I have a JSP/Servlet based application, the database team is complaining about the increase in open database connection.I suspect that the connection is not closed after use.
I want to make some code changes by initializing the connection string in try catch block as suggested in Java 8
I have tried closing the connection in final block which is working fine
here is the code i want to implement , Will this fix the issue?Is there any other way to close the open connections after use with little code change?
try(Connection con = DBConnectivity.openConnectionThread();
PreparedStatement ps1 = con.prepareStatement(sql1)) {
-----
------
}
You should close also PreparedStatement
try(Connection con = DBConnectivity.openConnectionThread();
PreparedStatement ps1 = con.prepareStatement(sql1))
{
-----
------
}
I have an instance of MySQL deployed on an Amazon EC2 server, along with a Java .Jar file that connects to the local database to archive metadata retrieved from an API. I have the Java file automatically executed 4 times daily via a Linux Crontab Task.
My code will always run and connect to MySQL fine initially but then it will experience a JDBC Communications link failure around a minute into its execution (the timeframe that this occurs can vary, however). This is the following error message that is displayed to the terminal:
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet successfully received from the server was 4,278 milliseconds ago.
Prior to the above error message being displayed, my program appears to freeze. The freezing of my program is evident because it will stop outputting: "A metadata point was archived". I have my program output that everytime a metadata point from an API is archived accordingly into my database's tables.
I've attempted many solutions in order to fix this problem and I'm at a complete loss as to what is causing it. I've added multiple parameters to my url string:
jdbc:mysql://localhost/MYDBNAME?serverTimezone=EST5EDT&useSSL=false&autoReconnect=true&maxReconnects=10&failOverReadOnly=false&allowPublicKeyRetrieval=true&useUnicode=true&tcpKeepAlive=true&interactiveClient=true
The autoReconnect parameter will sometimes allow my program to reconnect to the database after the communications link failure and to re-execute, but sometimes it won't help at all.
I've ensured that my program is not closing the JDBC connection until after the program's completion.
I've ensured that all of my preparedstatements and resultsets are being closed within a finally block after executing or retrieving the values from them.
I've ensured that the version of MySQL connector java driver.jar that my program is using matches the version of the installed MySQL database.
I've tried increasing MySQL's built-in timeout variables.
The following is the method that i use to initialize the connection global variable that is used exclusively by methods within this class for queries.
private void setUpConnectionToDatabase() {
try {
connection = DriverManager.getConnection(
url, username, password); // connects to server using url, username, password.
} catch (Exception e) {
System.out.println(e);
connection=null;
}
}
The following is one of my methods that queries the database, let me know if you see any errors. Other methods within the class essentially follow the same format. Notice that the connection variable itself does not get closed within the method.
public Coordinates getCoordsFromArchivedLocationData(String bioLocation) {
Coordinates coords= new Coordinates();
double lat=0;
double longit=0;
String queryStatement= "SELECT latitude, longitude FROM archived_location_data WHERE LocationName=?;";
PreparedStatement query=null;
ResultSet result=null;
try {
query = connection.prepareStatement(queryStatement);
query.setString(1, bioLocation);
result = query.executeQuery();
if(result.next()) {
lat= result.getDouble(1);
longit= result.getDouble(2);
}
coords.setLatitude(lat);
coords.setLongitude(longit);
}
catch(Exception e) {
System.out.println(e);
return coords;
}
finally {
try {
if(query != null) {
query.close();
}
if(result != null) {
result.close();
}
} catch (SQLException e) {
e.printStackTrace();
}
}
return coords;
}
I have to use a custom jdbc connection pooling library to connect to mysql db.
The problem is my that inserts are not persisted in db... To narrow down the issue, I extracted the portion of the library code inserting into db and have some strange findings:
MysqlConnectionPoolDataSource ds1 = new MysqlConnectionPoolDataSource();
ds1.setUser("usr");
ds1.setPassword("pwd");
ds1.setServerName("server");
ds1.setPort(port);
ds1.setDatabaseName("dbname");
ds1.setUseSSL(false);
ds1.setAllowPublicKeyRetrieval(true);
Connection conn = ds1.getPooledConnection("usr", "pwd").getConnection();
logger.info("connection " + conn.toString());
PreparedStatement ps = null;
ResultSet rs = null;
try {
String query = "INSERT INTO ...";
ps = conn.prepareStatement(query);
int timeout = 10;
ps.setQueryTimeout(timeout);
logger.info("timeout: " + timeout);
logger.info("Starting query execution for query: " + query);
long qeStart = System.currentTimeMillis();
ps.setString(1, "...");
ps.executeUpdate();
long qeEnd = System.currentTimeMillis();
logger.info("Query execution completed in " + (qeEnd - qeStart) + "msec.");
} catch (Exception e) {
logger.error("ERROR OCCURED", e);
System.err.println("ERROR OCCURED");
e.printStackTrace();
} finally {
closeResultSet(rs);
closeStatement(ps);
closeConnection(conn);
}
The above code does NOT work when I connect to a remote mysql db with version: 8.0.11-commercial . By not working I mean there is no error, the insert is simply lost...
When I execute the same code against my local mysql db with version : 8.0.11 hosted on windows machine, it is working...
If I change connection getting code from
Connection conn = ds1.getPooledConnection("usr", "pwd").getConnection();
to:
Connection conn = ds1.getConnection();
it also starts to work against remote mysql db with version: 8.0.11-commercial...
The autocommit mode of the underlying connection is already true...
I tried to implement a custom log4j logger with the hope of seeing some trace but that did not help either:
ds1.setLogger("com.ibtech.mysqlproblem.Log4jLogger");
My custom connection pooling library uses pooled connection so I need to get the above code working. In the client I am using mysql-connector-java-8.0.11.jar.
Any help is greatly appreciated...
It seems totally unrelated but setting paranoid property to true seemed to solve my problem...
----------------Edit : Root cause understood---------------------------------------
Problem occurs when global autocommit value is 0 in the server. Issuing change user command, clears session data and autocommit setting is lost.
A workaround solution is to set autocommit flag once again once the connection is acquired...
Thanks to Filipe for analyzing the problem..
https://forums.mysql.com/read.php?39,666702,666986#msg-666986
----------------Edit : Bug created--------------------------------------
It turned out that it is a bug in the JDBC driver. A bug has been filed...
https://bugs.mysql.com/bug.php?id=91351
Recently, my tomcat started hanging up. The requests were never replied. I figured out that it was due to connections never being returned to the connection pool.
I have used c3p0 with hibernate and the database is mysql 5.5
In order to debug the connection leaks, I added the following properties in my hibernate.cfg.xml
<property name="hibernate.c3p0.unreturnedConnectionTimeout">30</property>
<property name="hibernate.c3p0.debugUnreturnedConnectionStackTraces">true</property>
After adding them, in the logs it says :
[2013-10-12 23:40:22.487] [ INFO] BasicResourcePool.removeResource:1392 - A checked-out resource is overdue, and will be destroyed: com.mchange.v2.c3p0.impl.NewPooledConnection#1f0c0dd
[2013-10-12 23:40:22.487] [ INFO] BasicResourcePool.removeResource:1395 - Logging the stack trace by which the overdue resource was checked-out.
java.lang.Exception: DEBUG ONLY: Overdue resource check-out stack trace.
Pointing to at dao.DAOBasicInfo.getBean(DAOBasicInfo.java:69)
public static Basicinfo getBean(Integer iduser) {
Basicinfo u = null;
Session sess = NewHibernateUtil.getSessionFactory().openSession();
try {
Transaction tx = sess.beginTransaction(); //line 69
Query q = sess.createQuery("from Basicinfo where iduser=" + iduser);
u = (Basicinfo) q.uniqueResult();
if (u == null) {
u = new Basicinfo();
u.setIduser(iduser);
}
tx.commit();
} catch (Exception ex) {
ex.printStackTrace();
} finally {
sess.close();
}
return u;
}
I cross checked and Mysql says it supports transactions with InnoDB
Because of the above error, I'm having un-returned connections and then they pile up making the app unresponsive.
Please let me know what's wrong in starting a transaction and even I'm using finally and there's no exception thrown.
Some suggestions to debug it
As Steve mentioned in comments. Try to see what happens when you remove the unreturnedConnectionTimeout option.
May be your queries are taking too long. Try to log some performance stats on your code and see how much time your query is taking. May be you need to tune your query. and for short term you can also increase the unreturnedConnectionTimeout to be more than the response time on your queries.
Also try transaction timeout option in hibernate. May be set tx.setTimeout(20) and play with the timeout numbers and see if some queries timeout.
You may also want to use some profiling tool. Try VisualVM in case your Java version is supported on it. Otherwise (if on linux or mac) you may want to try Java Debugging commands on older version of java. Some of those commands are also available from JDK.
Small improvements on the code
Not sure if it will really fix your issue however you may want to add rollback for transaction in exception block. Added another try catch for tx.close to avoid another exception.
Also added a null check for session close. You may already know that one condition when finally may not completely execute - if another exception is thrown in finally block. Currently it may not be applicable in your code however in case you add more than one line in finally block make sure any exceptions are covered so next line can execute.
One more suggestion is to reduce the scope of transaction itself. Looking at the code it seems you may need the transaction only in case a uid is not found. How about limiting the transaction code inside if(u==null) block. Not sure if helps but you need not have transaction for read.
Below is my sample code
public static Basicinfo getBean(Integer iduser) {
Basicinfo u = null;
Transaction tx = null;
Session sess = NewHibernateUtil.getSessionFactory().openSession();
try {
Query q = sess.createQuery("from Basicinfo where iduser=" + iduser);
u = (Basicinfo) q.uniqueResult();
if (u == null) {
tx = sess.beginTransaction(); //line 69
u = new Basicinfo();
u.setIduser(iduser);
tx.commit();
}
} catch (Exception ex) {
ex.printStackTrace();
if(tx != null) {
try {
tx.rollback();
} catch(Exception e){e.printStackTrace;}
}
} finally {
if(sess!=null) {
sess.close();
}
}
return u;
}
One of the reasons this error comes up is when you don't make sure to
transaction.commit();
not an answer to this question, but someone who forgot to commit will also land on this page after googling the error