I work on a project where the design is to have a database per customer. I have managed to read in Tomcat documentation and establish a connection pool. Since the databases are created automatically by the application when a customer registers, I needed a way to make a dynamic pool. the following post on stack overflow gave the hint of what is required:
Using dynamic Datasource with Tomcat
and I found on the web a page showing how to do this:
Writing and using a Tomcat ObjectFactory
I still have few questions:
1- who takes care of managing the connections in the pool ?
in the datasource declaration in context.xml, we define some parameters that control various aspects like max idle time and max number of connections, etc. does tomcat handle this or do I have to take care of it in the implementation that i make when coding a class that implements the ObjectFactory interface ?
2- As I understand, when connections are closed, they return to the pool. suppose that I make a dynamic pool that has maximum 20 connections. and then 30 customers log in. so the pool returns 20 connections for the first 20 customers. when those connections are closed, they return to the pool and the next 10 customers will ask for connections to different databases than the ones already found in the pool. and the question is: will the pool automatically drop 10 connections and add new ones with the correct databases ? in if this is the case, does this really have an advantage over opening a connection upon request without a pool ? (asked with the thought of a site that will be heavily loaded. the amount of connection drop if i guessed correctly might be very high that it wouldn't justify what the pool is there to do: minimize creating connections from scratch)
3- I also wanted to ask if it is ok to implement it myself (considering I am new to this) or if i should find a library that does this (in case its not a task for the beginner in this area).
4- are there any alternatives for connection pools for managing connections to multiple databases that are added dynamically in an efficient way? (in case datasource will not help) what are the approaches used in similar situations?
Thank you in advance for your effort.
In declaring a datasource you can specify a datsource class, such as the Oracle datasource. It uses the parameters specified to manage the datasource for you. It is best to let the datasource manage itself. I have tried to write one and it is not simple.
I think the datasource will not be adequate for you, I could be wrong. A datasource requires a database url, username and password that will be fixed for the lifetime of the datasource.
Having different databases for different users will break this.
Look at the reference you gave above, the answer mentions using a resource which is different from a datasource.
Related
This question might look too broad, but I could not find a solution or any idea from existing sources in the web. There might not be exact answers for this, what I need is your suggestions on how to get this implemented.
My application has its own datasource for CRUD operations. Plus I have a new micro service to execute scheduled jobs. The idea is to maintain only a single database for both the services.
The existing service has been configured to use a connection pool for its db transactions. properties like max number of active datasources, max idle datasources at a given time etc have been configured for that service.
Now, my question is about maintaining the pool when quartz connects to the same database to query its own tables. At first place, is this possible? or is it always better to keep a separate database for quartz tables. Or Else, can I configure these two services to access same database flawlessly by doing configurations? (Remember that these two db operations happen in two separate microservices)
I am developing a tool that receives different connection parameters to test values in different databases (a plugin for Nagios in jNRPE that keeps an open connection to different databases).
Because the configuration is dynamic (there could be more databases or they can be removed) I cannot have a configuration file.
I want to know if I should have an instance of C3P0 per database or can I use the same instance and just change the URL each time I ask for a connection?
The code is at github:
https://github.com/angoca/db2-jnrpe/blob/master/src/main/java/com/github/angoca/db2_jnrpe/database/pools/c3p0/DBCP_c3p0.java
If not, how can I get multiple pool for multiple databases dynamically?
You'll need a different c3p0 DataSource for each JDBC url. A Connection pool must contain homogeneous Connections: all checked out Connections must be equivalent from a client's perspective. If Connections from multiple databases were included in the same pool, clients would have no way of specifying or knowing which db they were communicating with.
(If you are replicating, say, a read-only DB and you really want Connections from multiple sources to live in a single pool, because they are guaranteed to be equivalent from a client's point of view, you could do that by defining a custom, unpooled DataSource that round-robined or randomly chose a replicant, and then pooling the DataSource via c3p0's DataSources factory.)
It is very easy to dynamically create and configure c3p0 DataSources. See example code here.
If you capture your dynamic config as a map of c3p0 property names to values, there's also an alternative, more concise way to get a DataSource with that configuration.
I have used MySqlDataSource for in jdbc connectivity.I have used following code
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
Also i have searched there is an option of Configuring a MySQL Datasource in Apache Tomcat.
Is there any performance difference between these two? which one is best to use?
Configuring Datasource in tomcat will help you to share same data source between applications running in same tomcat. that Datasource will be managed by container (tomcat in your case).
while the Datasource created in code will be created by your application and can be used by that application only.
So if you have multiple application running on tomcat and accessing same data source, that configuring Datasource in tomcat will be good approach and have performance factor because only one data source is created and not having separate connections for each application
But if you have only single application that the first approach you have used is good one
They both use the internally the same driver, i dont think the performance is much different here, i guess if you need to access teh database only at that place and the enduser isn't supposed to use his own authentication you may use it directly from java, but if you will need the connectivity on different places it could be helpful to configure this using apache configuration, specially that if anything changes like database server, user name or whatever you don't need to get in the code to change it, this could be very important if end users have to set their own configurations.
The improvement of configuring a pool of Connections (as the one provided by tomcat) is mainly that you will actually create and close a lot less of connections.
When using a pool, when you request a Connection to a pool it will look if it has any connection already created and available for reuse and, if it has, it will provide you with it (instead of creating a new Connection, which is a heavy operation). You must still close() a Connection provided by Tomcat so Tomcat knows that it can now reuse when it is requested again.
Additionally, the advantage of the pool is that your code does not need to know the configuration data for the Connection. He just requests a Connection from a given pool and the sysadmin configures it, allowing for greater flexibility (the sysadmin does not need to know how to configure your app, just how to configure the Tomcat which is fairly more standard).
I have two options to configure my application database connection - one is using JDBC, another one is using JNDI. What will be the best option in terms of how fast those connection types work with the database.
I understand those are two different types of database connections using different principles (JDBC is a direct db connection, JNDI is a database connection pool configuration on application server side). But are there other general JDBC/JNDI pros and cons which might be more important than operating speed? If yes, what are they?
A database connection always uses JDBC. With JNDI you register a datasource in a directory service which can be looked up by its name. Thus JDBC and JNDI are completly different and not interchangeable.
I bet what you mean is choosing from
creating datasource or jdbc connection manually in your application, or
setup the datasource in the container, and application lookup the datasource through JNDI
If it is the case, always stick to 2 if possible.
The main reasons for the choice is never the performance differences. The reason for sticking to 2 is in most cases is, you need 2 to gain more advanced features from container, for example, distributed transaction.
This is what i have found about JNDI and JDBC.
JNDI: This is a technology which works like a telephone directory which is used to search the name on server and datasource remotely.
JNDI creates a connection pool. Connection pool is an environment on the server where JNDI and Database encapsulated to for Type4 connectivity.
JDBC: A Java API that enables Java programs to execute SQL statements.
This allows Java programs to interact with any SQL-compliant database.
JDBC is similar to ODBC, but is designed specifically for Java programs, whereas ODBC is language-independent.
JDBC was developed by Sun Microsystems. JNDI is faster and efficient.
Not totally clear on the question.
JNDI isn't a type of database connection. You can use JNDI to look up a DataSource, which is a factory for connections. The DataSource is part of the JDBC API though, so JNDI works with JDBC as opposed to being alternatives here.
Are you talking about using JDBC against a database for directory information, vs. using JNDI against an LDAP repo?
The real speed benefit comes from being able to reuse database connections.
Hence, you need to use an approach which provides database connection pooling, and then use the appropriate technology to get to the pool. Depending on implementation this can be either JDBC (if the driver supports it itself) or JNDI or something completely different.
If your application runs inside a web container, it is common to use JNDI to allow the pool to be configured and managed in the web container instead of inside your application.
As mentioned in previous answers, using Datasource is the same as using JDBC in terms of technology.
Nevertheless, using a Datasource is usually the preffered way because that way you have the server managing your DB connection pools.
Whether connection pooling is used does not affect application code. It does not require any code changes to the application because the application performs a lookup on a JNDI name of a previously registered data source. If the data source specifies a connection pooling implementation during JNDI registration (as described in section Creating a Data Source Using the DataDirect Connection Pool Manager), the client application benefits from faster connections through connection pooling.
The question is meaningless. Faster at what? There is nothing to compare. JDBC is a general-purpose interface to relational databases. JNDI is a general-purpose interface to naming systems. The strong probability is that the efficiency of either depends 99% on the target system being communicated with. In any case relational databases and naming systems fulfil completely different needs that are largely non-comparable. Usually JNDI is used to obtain a connection, then JDBC is used to operate with that connection.
I have a Tomcat servlet that incorporates hibernate. It works fine normally. When the servlet starts I initialize hibernate and create a session factory. I then use this session factory to generate sessions when performing various database transactions. So far so good. My problem comes after a long period of inactivity on the servlet (say when the users go home for the night and then try to log in the next morning). Suddenly, I am unable to communicate with the databse. In the logs I see
org.hibernate.exception.JDBCConectionException: Could not execute query.
If I stop and restart Tomcat, reinitializing my servlet and rebuilding my session factory, everything works fine. It is almost like the session factory itself is timing out?
Any ideas?
Thanks,
Elliott
If I stop and restart Tomcat, reinitializing my servlet and rebuilding my session factory, everything works fine. It is almost like the session factory itself is timing out?
It's not the session factory but the connections used by the session factory (e.g. MySQL is well known to timeout connections after 8 hours of inactivity by default). Either:
use a connection pool that is able to validate connections on borrow and to renew them ~or~
increase the idle timeout on the database side
OK. Suppose I use a c3P0 connection pool. How do I specify in the hibernate.cfg.xml file that I want to "validate connections on borrow" or does it do this by default?
The various options when using C3P0 are documented in Configuring Connection Testing. My advice would be to use the idleConnectionTestPeriod parameter:
The most reliable time to test
Connections is on check-out. But this
is also the most costly choice from a
client-performance perspective. Most
applications should work quite
reliably using a combination of
idleConnectionTestPeriod and
testConnectionsOnCheckIn. Both the
idle test and the check-in test are
performed asynchronously, which leads
to better performance, both perceived
and actual.
Note that for many applications, high
performance is more important than the
risk of an occasional database
exception. In its default
configuration, c3p0 does no Connection
testing at all. Setting a fairly long
idleConnectionTestPeriod, and not
testing on checkout and check-in at
all is an excellent, high-performance
approach.
To configure C3P0 with Hibernate, be sure to read the relevant instructions (and to use the appropriate properties, and the appropriate files).