I have a Java application which is connecting to an Oracle database, but my application is losing its connection after 20 minutes. It will works after closing and logging:
My poolingDatasourceExample.java is given below:
Properties props = new Properties();
props.put("user", uname);
props.put("password", pass);
props.put("password", pass);
props.put("testWhileIdle", "true");
props.put("testOnBorrow", "true");
props.put("testOnReturn", "false");
props.put("validationQuery", "SELECT 1");
props.put("validationInterval", "30000");
props.put("timeBetweenEvictionRunsMillis", "5000");
props.put("maxActive", "100");
props.put("minIdle", "10");
props.put("maxWait", "10000");
props.put("initialSize", "10");
props.put("removeAbandonedTimeout", "60");
props.put("removeAbandoned", "true");
props.put("logAbandoned", "true");
props.put("minEvictableIdleTimeMillis", "30000");
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(connectURI, props);
Are you storing the java.sql.Connection object, and seeing it stop working after 20 minutes? That's quite normal, particularly if it's inactive in between (see, for example, this post). Inactive connections will be closed by the database and quite possibly by the network, to protect against zombie processes hogging all the resources. They're also bad at error recovery - once the Connection breaks, you need to recreate it. They're not designed for sharing either, although they are technically thread-safe.
Rather than keeping the Connection, you should keep the factory and request a new Connection when needed. Better still, use a connection pool - these typically test connections for disconnects and recreate them when needed. They also allow your application to share connections efficiently. Connection pooling is available in most app servers, but if your deployment doesn't allow that it's possible to create one of your own.
Related
I am trying to write a java code which creates a Oracle JDBC connection to a database using the following type of code:
DriverManager.registerDriver(new oracle.jdbc.OracleDriver());
con = DriverManager.getConnection(url, user, pass);
now I want to close or terminate this connection if it takes more than 200millisecs. I found that they have a method for this which is setLoginTimeout(int seconds) that only takes time in seconds is it possible to setLoginTimeout in milliseconds? If not, is there any other option to do this?
Since you are using Oracle OJDBC driver there are properties for managing timeouts in milliseconds. As per OracleConnection javadoc there are:
CONNECTION_PROPERTY_THIN_NET_CONNECT_TIMEOUT - login timeout in ms
CONNECTION_PROPERTY_THIN_READ_TIMEOUT - general read timeout in ms
You can configure them when obtaining a new connection with DriverManager.getConnection(url, props):
Properties props = new Properties();
props.setProperty("user", user);
props.setProperty("password", pass);
props.setProperty(OracleConnection.CONNECTION_PROPERTY_THIN_NET_CONNECT_TIMEOUT, "200");
props.setProperty(OracleConnection.CONNECTION_PROPERTY_THIN_READ_TIMEOUT, "200");
DriverManager.getConnection(url, props);
DriverManager.setLoginTimeout specifies a global default for all JDBC drivers in the application. It is not possible to set it in milliseconds, given DriverManager.setLoginTimeout(int seconds) is defined as
Sets the maximum time in seconds that a driver will wait while
attempting to connect to a database once the driver has been
identified.
Parameters:
seconds - the login time limit in seconds; zero means there is no limit
You will need to use a driver specific configuration property instead (if available).
I'm not sure if Oracle has this, because oracle.net.CONNECT_TIMEOUT as suggested by Karol is - as far as I know - also defined in seconds (based on TCP.CONNECT_TIMEOUT in sqlnet.ora documentation)
I have a rest service application running the Java Spring framework. The application depends on a connection to an external MySQL DB, which is connected via JDBC.
My issue is maintaining a solid connection between the rest service and the MySQL db. I have what I consider a rudimentary connection failsafe in place that looks something like:
public Connection getConnection() throws SQLException {
if(connection == null){
this.buildConnection();
}
else if(!connection.isValid(10)){ //Rebuild connection if it is no longer valid
connection.close();
this.buildConnection();
}
return connection;
}
Using this method should ensure that the connection is valid before any query is executed. My problem is that I periodically get an exception thrown when calling this method:
Could not create connection to database server. Attempted reconnect 3 times. Giving up. SQLState: 08001. ErrorCode: 0.
The things that have me super perplexed about this are:
This error only happens periodically. Many times the connection works just find.
I test this same application on my developer machine and this error never occurs.
I custom configured the MySQL DB on my own server, so I control all its config options. From this I know that this issue isn't related to the maximum number of connections allowed, or a connection timeout.
Edit - Update 1:
This service is hosted as Cloud Service on Microsoft Azure platform.
I accidentally set it up as an instance in Northern Europe, while the DB is located in North America - probably not related, but trying to paint the whole picture.
Tried the advice at this link with no success. Not using thread pools, and all ResultSets and Statements/PreparedStatements are closed after.
Edit - Update 2
After some refactoring, I was able to successfully implement a HikariCP Connection Pool as outlined by #M.Deinum below. Unfortunately, the same problem persists. Everything works great on my local machine, and all Unit Tests pass, but as soon as I push it to Azure and wait more than a few minutes between requests, I get this error, when trying to grab a connection from the pool:
springHikariCP - Connection is not available, request timed out after 38268ms. SQLState: 08S01. ErrorCode: 0.
My HikariCP configuration is as follows:
//Set up connection pool
HikariConfig config = new HikariConfig();
config.setDriverClassName("com.mysql.jdbc.Driver");
config.setJdbcUrl("jdbc:mysql://dblocation");
//Connection pool properties
Properties prop = new Properties();
prop.setProperty("user", "Username");
prop.setProperty("password", "Password");
prop.setProperty("verifyServerCertificate", "false");
prop.setProperty("useSSL","true");
prop.setProperty("requireSSL","true");
config.setDataSourceProperties(properties);
config.setMaximumPoolSize(20);
config.setConnectionTestQuery("SELECT 1");
config.setPoolName("springHikariCP");
config.setLeakDetectionThreshold(5000);
config.addDataSourceProperty("dataSource.cachePrepStmts", "true");
config.addDataSourceProperty("dataSource.prepStmtCacheSize", "250");
config.addDataSourceProperty("dataSource.prepStmtCacheSqlLimit", "2048");
config.addDataSourceProperty("dataSource.useServerPrepStmts", "true");
dataSource = new HikariDataSource(config);
Any help would be greatly appreciated.
I suggest using a proper JDBC Connection Pool like HikariCP that together with a validation query which will execute on correct intervals should give you fresh and proper connections each time.
Assuming you are using Spring and xml to configure the datasource.
<bean id="dataSource" class="com.zaxxer.hikari.HikariDataSource">
<property name="poolName" value="springHikariCP" />
<property name="dataSourceClassName" value="com.mysql.jdbc.jdbc2.optional.MysqlDataSource" />
<property name="dataSourceProperties">
<props>
<prop key="url">${jdbc.url}</prop>
<prop key="user">${jdbc.username}</prop>
<prop key="password">${jdbc.password}</prop>
</props>
</property>
</bean>
It by default validates connections on checkout. I suggest a try out.
As you are using java bases config I suggest the following
#Bean
public DataSource dataSource() {
HikariDataSource ds = new HikariDataSource();
ds.setPoolName("springHikariCP");
ds.setMaxPoolSize(20);
ds.setLeakDetectionThreshold(5000);
ds.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
ds.addDataSourceProperty("url", url);
ds.addDataSourceProperty("user", username);
ds.addDataSourceProperty("password", password);
ds.addDataSourceProperty("cachePrepStmts", true);
ds.addDataSourceProperty("prepStmtCacheSize", 250);
ds.addDataSourceProperty("prepStmtCacheSqlLimit", 2048);
ds.addDataSourceProperty("useServerPrepStmts", true);
ds.addDataSourceProperty("verifyServerCertificate", false);
ds.addDataSourceProperty("useSSL", true);
ds.addDataSourceProperty("requireSSL", true);
return ds;
}
It seems to be caused by the system variable wait_timeout of MySQL.
For MySQL 5.0, 5.1, 5.5, 5.6, the default value for wait_timeout is 28800 seconds (8 hours), and the maximum value for wait_timeout:
Linux : 31536000 seconds (365 days, one year)
Windows : 2147483 seconds (2^31 milliseconds, 24 days 20 hours 31 min 23 seconds)
The number of seconds the server waits for activity on a noninteractive connection before closing it. This timeout applies only to TCP/IP and Unix socket file connections, not to connections made using named pipes, or shared memory.
So I think you can try to use a jdbc connection to keep pinging interval some seconds, or directly using a kind of JDBC Connection Pool framework to manage jdbc connections automatically.
Hope it help. Best Regards.
assuming code you have now for '//Set up connection pool' is called only once, like bean creation, to initialize dataSource.
with that, your getConnection() would be just:
public Connection getConnection() throws SQLException {
return dataSource.getConnection();
}
and make sure wait_timeout in mysql is set to minute more than maxLifetime in hikaricp which is default 30 minutes
Hope this helps.
I have this code that uses HikariCP Connection Pool:
config.setMaximumPoolSize(100);
config.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
config.addDataSourceProperty("serverName", hostname);
config.addDataSourceProperty("port", portnumber);
config.addDataSourceProperty("databaseName", dbname);
config.addDataSourceProperty("user", username);
config.addDataSourceProperty("password", password);
config.setConnectionTimeout(30000);
config.setInitializationFailFast(false);
pooledDataSource = new HikariDataSource(config);
I monitor connections in mysql by issuing command "Show Processlist" and I see that 100 connections is created after line:
pooledDataSource = new HikariDataSource(config);
...is run. I'm sure this is not meant to happen, right? It should create connections later when I do pooledDataSource.getConnection().
What am I doing wrong? Why is it creating 100 connections immediately??
By default HikariCP runs as a fixed-sized pool. You need to set minimumIdle. That's it.
From the documentation for minimumIdle:
This property controls the minimum number of idle connections that
HikariCP tries to maintain in the pool. If the idle connections dip
below this value, HikariCP will make a best effort to add additional
connections quickly and efficiently. However, for maximum performance
and responsiveness to spike demands, we recommend not setting this
value and instead allowing HikariCP to act as a fixed size connection
pool. Default: same as maximumPoolSize
Would anyone care to elaborate how the HikariCP handles connections in the pool? How do you put a new connection in the pool, and how can you call on it / retrieve it later?
This is my current code:
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(100);
config.setDataSourceClassName("com.mysql.jdbc.jdbc2.optional.MysqlDataSource");
config.addDataSourceProperty("serverName", "localhost");
config.addDataSourceProperty("port", "8889");
config.addDataSourceProperty("databaseName", "XXX");
config.addDataSourceProperty("user", "XXX");
config.addDataSourceProperty("password", "XXX");
System.out.print("qq");
HikariDataSource ds = new HikariDataSource(config);
ds.setConnectionTimeout(800);
With a pool, you don't add a connection to the pool to retrieve it later. You do the exact inverse: you get a connection from the pool when you need one, and close the connection when you're done with it to give it back to the pool. The HikariDataSource, as its name indicates, is a DataSource. A DataSource is an object from which you get connections.
The pool handles the opening of the connection for you. It puts you in a waiting queue if no connections are available automatically, etc.
Depending on the properties of the pool, the pool can open the connections immediately or on demand, keep a given number of connections always opened, shrink the pool size after given amount of unused time, etc.
That's all very well documented: https://github.com/brettwooldridge/HikariCP#user-content-configuration-knobs-baby
Example code (Java 7 and later):
try (Connection connection = ds.getConnection()) {
// use the connection
}
Example code (Before Java 7):
Connection connection = ds.getConnection();
try {
// use the connection
}
finally {
connection.close();
}
I have created a mysqlDatasource connection using the following code:
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
If Im running my application and the connections are disconnected because mysql restarted or for some other reason, the remaining operations will fail even if the mysql server instance is running.
In that case I want to recreate a connection? Is this possible? How do I go about doing this?
In most cases when you're creating multiple connections and juggling them, it's better to use a connection pool, where the Connection objects are just that, objects, and are multiplexed to actual socket connections handled by an underlying implementation. This means that connections are created automatically when you need them and you don't need to worry about reclaiming resources and creating an appropriate number of connections.
Two prominent examples are BoneCP and C3P0.
The other option, in addition to Mr Mao's, is to define the url explicitly using setURL to allow for automatic reconnection, using the autoReconnect parameter, do note, though, that this approach is not recommended. The answer is provided here only for completeness.
Just take an help of DBCP CONNECTION and create your Connection pool like below code. You can see there is validation query which pings database at regular interval and refreshes the pool. There are other property like validateConnection you can set this property to true.
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUsername("username");
dataSource.setPassword("password");
dataSource.setUrl("jdbc:mysql://<host>:<port>/<database>");
dataSource.setMaxActive(50);
dataSource.setMaxIdle(5);
dataSource.setInitialSize(5);
dataSource.setValidationQuery("SELECT 1");
Try this.
String driver = "com.microsoft.jdbc.sqlserver.SQLServerDriver";
Class.forName(driver);
String url = "jdbc:microsoft:sqlserver://host:1433/database";
Connection conn = DriverManager.getConnection(url, "username", "password");