Develop Reference

Check db connection is active

How can I check that a connection to db is active or lost using spring data jpa?
Only the way is to make a query "SELECT 1"?

Nothing. Just execute your query. If the connection has died, either your JDBC driver will reconnect (if it supports it, and you enabled it in your connection string--most don't support it) or else you'll get an exception.
If you check the connection is up, it might fall over before you actually execute your query, so you gain absolutely nothing by checking.
That said, a lot of connection pools validate a connection by doing something like SELECT 1 before handing connections out. But this is nothing more than just executing a query, so you might just as well execute your business query.
our best chance is to just perform a simple query against one table, e.g.:
select 1 from SOME_TABLE;
https://docs.oracle.com/javase/6/docs/api/java/sql/Connection.html#isValid%28int%29

if you can use Spring Boot. Spring Boot Actuator is useful for you.
actuator will configure automatically and after it has activated ,
you can get know database status to request "health"
http://[CONTEXT_ROOT]/health
and it will return , database status like below
{"status":"UP","db":{"status":"UP","database":"PostgreSQL","hello":1}}

How to set idle Hive jdbc connection out from java code using hive jdbc

I am using hive jdbc 1.0 in my java application to create connection with hive server and execute query. I want to set the idle hive connection timeout from java code. Like say, user first creates the hive connection, and if the hive connection remains idle for next 10 minutes, then this connection object should get expired. If user uses this same connection object after 10 minutes for executing query, then hive jdbc should throw error. Can you please tell me the way to achieve this through java code.
I know there is a property hive.server2.idle.session.timeout in hive, but I don't know whether this is the right property required to be set from java code or there is some other property. I tried setting this property in jdbc connection string but it did not worked.
try {
Class.forName("org.apache.hive.jdbc.HiveDriver");
} catch (ClassNotFoundException e) {
LOG.error(ExceptionUtils.getStackTrace(e));
}
String jdbcurl = "jdbc:hive2://localhost:10000/idw?hive.server2.idle.session.timeout=1000ms";
Connection con;
con = DriverManager.getConnection(jdbcurl,"root","");
Thread.sleep(3000);
Now below I am using connection object, hive jdbc should throw error here as I used connection object after 3000 ms but I had set the idle timeout as 1000ms but hive jdbc had not thrown error
ResultSet rs = con.createStatement().executeQuery("select * from idw.emp");
Need help on this.

The hive.server2.idle.session.timeout causes a session to be terminated when it is not accessed for the specified duration. However, hive.server2.idle.session.timeout needs to be specified with hive.server2.session.check.interval set to
a positive value. basically, we need to be specify a number of session checks within the timeout interval to cause the session to be closed.
More details can be checked here
https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties#ConfigurationProperties-hive.server2.session.check.interval

You are looking at the problem the wrong way. These properties are not set in the URL, as described in the Hive AdminManual...
In the Hadoop ecosystem, server defaults are set in XML config files -- in that case /etc/hive/conf/hive-site.xml and/or hiveserver2-site.xml
Once your session is open, you can set custom values with the set <prop>=<value> statement (somewhat similar to the Oracle ALTER SESSION).
Caveat: some properties are defined as "final" in the config files and cannot be overriden. Check with your Hadoop Admin if in doubt.

Error: System Resource Exceeded

A console application executing under:
1). Multiple threads
2). Connection Pooling (as the database connections range could be 5 to 30) of type Microsoft Access using DBCP.
While executing this application at my end (not tested the database limit) it works fine. And whenever I try to introduce the same application on one of other machines it generates an error.
I'm wondering why this is happening as there is only the difference of machines here. So, it works perfectly at my end.
I don't know much about connection pooling but it seems whatever I have understood I have implemented as:
public class TestDatabases implements Runnable{
public static Map<String, Connection> correctDatabases;
#Override
public void run() {
// validating the databases using DBCP
datasource.getConnection(); // Obtaining the java.sql.Connection from DataSource
// if validated successfully °º¤ø,¸¸,ø¤º°`°º¤ø,¸,ø¤°º¤ø,¸¸,ø¤º°`°º¤ø,¸ putting them in correctDatabases
}
}
The above case is implemented using ExecutorService = Number of databases.
Finally, I'm trying to put them in a static Collection of Type
Map<String, Connection> and making use of it throughout the application. In other words: I'm trying to collect the connectionString along with the Connection in a Map.
In other parts of my application I'm simply dealing with multiple threads coming along with the Connection URL. So, to perform any database operations I'm calling the
Connection con = TestDatabases.correctDatases.get(connectUrl);
For that machine, this application works fine for around ~5 databases. And the error is always getting generated when I'm trying to fire the query using above Connection (con) as stmt.executeQuery(query);
As, I'm not able to reproduce this issue at my end, it seems something is going-on wrong with the Connection Pooling or I have not configured my application to deal with Connection Pooling correctly.
Just for your information, I'm correctly performing Connection close in finally block where my application terminates and this Application is using Quartz Scheduler as well. For Connection Pooling, a call to the following from TestDatabases class is done for setUp as:
public synchronized DataSource setUp() throws Exception {
Class.forName(RestConnectionValidator.prop.getProperty("driverClass")).newInstance();
log.debug("Class Loaded.");
connectionPool = new GenericObjectPool();
log.debug("Connection pool made.");
connectionPool.setMaxActive(100);
ConnectionFactory cf = new DriverManagerConnectionFactory(
RestConnectionValidator.prop.getProperty("connectionUrl")+new String(get().toString().trim()),
"","");
PoolableConnectionFactory pcf =
new PoolableConnectionFactory(cf, connectionPool,
null, null, false, true);
return new PoolingDataSource(connectionPool);
}
Following is the error I'm getting (at the other machine)
java.sql.SQLException: [Microsoft][ODBC Microsoft Access Driver] System resource exceeded.
Following is the Database Path:
jdbc:odbc:DRIVER= {Microsoft Access Driver (*.mdb, *.accdb)};DBQ=D:\\DataSources\\PR01.mdb
Each of those database seems to be not much heavy (its ~ 5 to 15 MB of total size).
So, I'm left with the following solutions:
1). Correction of Connection Pooling or migrate to the newer one's like c3p0 or DBPool or BoneCP.
2). Introducing batch concept - in which I will schedule my application for each group of 4 databases. It could be very expensive to deal with as any time the other schedule may also collapse.
I’m pretty sure that this is Java related error but I can’t fathom out why.

Just done the migration to BoneCP which solved my problem. I guess due to multi-threaded environment the dpcp was not providing the connection from pool rather it was trying to hit the database again and again. Maybe I could have solved the dpcp issue but migrating to BoneCP also provides advantage of performance.

ResultSet Object is closed - jtds

I am using JTDS to connect to MS-SQL 2005. I am using c3p0 as the DB Connection pool, configured with Spring.
I am randomly getting an SQLException: Invalid state, the ResultSet object is closed in a Groovy script in which I have passed a reference to the connection pool. The script is executed by a timer every so often. By random, I mean that the script works perfectly 99% of the time, but when it fails, it will do so a couple of times, then go back to working correctly again, picking up where it left off. All of the critical work is done in a transaction, pulling off of a Message Queue.
Logic below:
//passed into the groovy context
DataSource source = source;
Connection conn = source.getConnection();
...
//Have to omit proprietary DB stuff... sorry...
PreparedStatement fooStatement = conn.prepareStatement("INSERT INTO foo (x,y,z) VALUES (?,?,?) select SCOPE_IDENTITY();");
ResultSet identRes = fooStatement.executeQuery();
//This is where the execption is thrown.
identRes.next();
...
try{
log.info("Returning SQL connection.");
conn.close();
}catch(Exception ex){}
There is a separate timer thread that runs a similar groovy script, in which we have not seen this issue. That script uses similar calls to get the connection, and close it.
Originally, we thought that the second script may have been grabbing the same connection off the pool, finishing first, then closing the connection. But c3p0's documentation says that calling conn.close() should simply return it to the pool.
Has anyone else seen this, or am I missing something big here?
Thanks.

We solved this... C3P0 was configured to drop connections that were checked out longer than 30 seconds, we did this to prevent dead-lock in the database (we don't control the tuning). One of the transactions was taking horridly long to complete, and C3P0 was dropping the connection, resulting in the ResultSet Closed error. Surprisingly, however, C3P0 was not logging the incident, so we didnt see this in the application's logs.

How can I execute a stored procedure with JDBC / jTDS without using a transaction?

We run a website written in Java that uses JDBC with jTDS to access an SQL Server database.
Our database contains a complex stored procedure that typically takes 10 minutes to run. The stored procedure works fine if we execute it directly (say from SQL Server Management Studio) because it does not run in a transaction. But if we execute it using jTDS then it locks up the entire website for 10 minutes. This happens because jTDS runs it in a transaction, and so all the website requests are on hold waiting for the transaction to finish.
For example, the following locks up the website due to the transaction:
Connection connection = DriverManager.getConnection("jdbc:jtds:sqlserver://example/example");
CallableStatement callableStatement = connection.prepareCall("exec dbo.procTest");
callableStatement.execute();
Is there any way that we can run a stored procedure using JDBC / jTDS without it running in a transaction?
Please note that calling this on the jTDS connection is not valid:
connection.setTransactionIsolation(Connection.TRANSACTION_NONE);
That throws an exception stating that the Connection.TRANSACTION_NONE parameter is not supported by jTDS.
EDIT: I probably could have asked the question better: the core problem is not the transaction itself, the problem is that the transaction causes database locks to be held for 10 minutes. Either I need to get rid of the transaction, or I need locks released during the transaction.

In our case, the StoredProcedure had to be called, was throwing an exception when called from jtds:
Cannot perform a backup or restore operation within a transaction.
What I found # https://communities.bmc.com/docs/DOC-66239
is exec call need to be prepend with "SET IMPLICIT_TRANSACTIONS OFF;"
here's what spring-based snippet looked like:
try {
jdbcTemplate.getJdbcOperations.execute("SET IMPLICIT_TRANSACTIONS OFF;")
//noinspection ConvertExpressionToSAM // we have 1.6 code version
jdbcTemplate.getJdbcOperations.call(new CallableStatementCreator {
override def createCallableStatement(con: Connection): CallableStatement = {
val callableStatement = con.prepareCall(
s""" EXEC dbo.RestoreLatestCopy
| #ID = ?
|""".stripMargin)
callableStatement.setInt(1, getMappedRestoreId(fromDbName))
callableStatement
}
}, parameters ++ rsParams)
} catch {

Calling Connection#close() or Connection#commit() should submit and end the transaction. Do you close DB resources like Connection in the finally block?

I ended up using the following ugly workaround. I'm still interested if anyone can explain how I might do this properly; but the workaround will do for the moment.
The database is running on the same machine on the webserver, so I am able to use the standard SQL Server command line tools to run the stored procedure. The Java code to trigger it is:
try {
Process process = Runtime.getRuntime().exec("sqlcmd -E -d \"example\" -Q \"EXEC dbo.procTest;\"");
process.waitFor();
} catch (IOException e) {
// Handler here
} catch (InterruptedException e) {
// Handler here
}
So exactly the same stored procedure is run — the difference is that the webserver doesn't lock up because sqlcmd isn't running it in a single transaction. Yes it's ugly... but it's the only option I know of that works!