I encountered a question where in we have a set of say DB connections and multiple users simultaneously request for them.
I must be able to provide them the Conn object from a pool of existing connectios, and once they are done using it, put it back into the same pool of available connections.
The approach I could think of was to have 2 sets for each used and unused connections and keep moving objects back and forth.
I am somehow not convinced it's the best way to do this.
Can anyone pls suggest a nicer approach?
I was wondering if we could mark the connection obj as in use or something so that we can do away with these two sets?
TL;DR: Use a connection pool
There are thousands of implementations to choose from:
Hikari CP
Bone CP
C3P0
Commons DBCP
and others
These pools are tested, working, optimised code. Not whatever your homebrew solution is.
The problem you encountered is just one you will encounter on the way to writing a resilient connection pool. The solution you propose is naive at best. There are many other issues, from the misuse of the pool to connection dropout that you have to handle. And not only that but the code must be threadsafe.
For general scarce resource pooling, not JDBC connections, use Commons Pool. Again, this is tested, working code.
Do not reinvent the square wheel.
Don't reinvent the wheel. There are existing solutions for this, one of them being Apache Commons pools : https://commons.apache.org/proper/commons-pool/. All you have to do is provide a factory to initialize your connections at the beginning, then Apache will provide you with a GenericObjectPool (see doc) on which you can call borrowObject before accessing a connection in the pool and then release it using returnObject.
Related
i am trying to create a connection pool for factory class ,which design pattern helps me to create connection pool .user want to specify no of connections and maximum connections externally.I have search for some time, some people proposed to use Singleton or put the initialization code inside some static block. But others said singleton is bad. So, what should be a right design pattern to use for connection pooling? Thanks.
Singleton is bad for a few reasons e.g. usually you cannot inject a Mock object. If you provide a mechanism (or do not need) to mock test it, then I do not see a problem.
Please note that there are a lot of excellent connection pools available, I would first suggest to look into existing ones.
Database Connection Pooling
Initially in my application i use apache-commons-dpcp and i faced lot of connection issues and latter after long analysis i use bonecp. I would suggest bonecp connection polling. And Strictly don't use Singleton patten. It will bring you big pain point once your application is go live.
TCP Connection Pooling
For TCP Pooling I suggest to use Netty and Take a look on Apache MINA also.
As previous commenters suggest, use libs like Apache DBCP, C3P0, BoneCP or HikariCP.
But if you want to "invent your own connection pool" and understand how it could be done for educational purpose, start with singletone and extend it to hold preconfigured number of your instances.
I created one api using jsp and hibernate. I used this option of hibernate -
<property name="hibernate.connection.pool_size">50</property>
What I want is If I am getting 1000 hit concurrently on api then it should use only 50 connection not more than that. I thought above option of hibernate will help me to achieve this. But I tested my api on 10000 hit and I checked the open connection. It was around 3000. It means pool of connection not working.
How to get this ?
From the Hibernate documentation:
Hibernate's own connection pooling algorithm is, however, quite
rudimentary. It is intended to help you get started and is not
intended for use in a production system, or even for performance
testing. You should use a third party pool for best performance and
stability. Just replace the hibernate.connection.pool_size property
with connection pool specific settings. This will turn off Hibernate's
internal pool. For example, you might like to use c3p0.
Like mentioned, c3p0 is a good option.
I'm using plain-JDBC-Database-Access in a multithreaded environment.
An exception I recently got when working with PreparedStatements (the Oracle flavour) made me aware of the fact, that they are not threadsafe.
There is, of course, always the possibility to use ThreadLocal-Variables (or synchronize access to the statement), but is there a more clever way to access a database in a multithreaded way?
Edit: To simplify the problem, I'm accessing the database read-only so parallel transactions are no concern to me.
Putting the PreparedStatement into a ThreadLocal into will not solve the problem - even the Connection must be put into ThreadLocal. But then you must make sure, that the connection is also released properly even when exceptions are thrown.
And what about transactions? How do you make sure, that one transaction does not contain stuff from independent threads?
The best way would be to adopt some patterns of EJB containers - here the infrastructure takes care of the resource and transaction management and connection pooling. But retrofitting existing code into EJB or even Spring correctly is not an easy task.
just use the ThreadLocal it's the simplest way
you could also delegate all database accesses to a single thread (through a blocking queue) which will also eliminate races. this also allows for easier batching of many statements (though only works for updates, a query requires the pending updates to be flushed first)
I have never connected to a database in java before. May I know if I should go about accessing a derby database with servlet?
I have checked this: How do I access a database from my servlet or JSP?
But I saw comments on the article saying that this is a bad way to connect. Could any one explain or show me the best way to that I should code to access my derby database?
Thank you very much.
They are all right indeed, in suggesting that. We don't don't access database directly from Servlets or JSPs, these both are meant to be web tier, isn't it?
So, what to do? Grab a JDBC tutorial. The official one is an excellent choice here. That will give you a good idea about connecting to database from Java, and grasp over JDBC API. After that you should go and read about DAO pattern, and how we employ that in real apps.
Moreover, I think you also should read about MVC pattern, because it seems to me that you are not very clear on that as well.
Once you understand all these and come up with a toy like application using all these stuff. Next step would be to have a look into Connection Pooling mechanism.
Since you are using servelt you must be using a container line Apache Tomcat. You should look to define a connection pool like this http://tomcat.apache.org/tomcat-5.5-doc/jndi-datasource-examples-howto.html. If you are using any other container then that will also have similar setup.
Other option is to create a separate DBManager kind of class which looks after initializing and returning connection. This class you can use in the servlet.
Using JDBC and having your app server's application pool is a good start. You can also use some API to make your life easier like Hibernate.
It is a "bad way", because it doesn't make use of a (JNDI-managed) connection pool to obtain connections. Although acquiring a connection costs "only" a few hundred milliseconds, this has impact in a busy multiuser environment. A connection pool will worry about opening and closing connections and release them immediately on every getConnection() call so that it effectively costs almost zero milliseconds. If you sum that up in a busy multiuser environment, then the differences are noticeable.
A connection pool is usually to be configured in flavor of a JNDI datasource which is managed by the servletcontainer in question. As you didn't mention which one you're using, I can at highest point to one of my answers which contains a Tomcat 6.0 targeted example: here.
Hope this helps.
I'm writing a Java SE (Note, not Java EE) application using Hibernate, and I need to provide a different connection to Hibernate for each thread of execution. These connections must be pooled, and each one has at the very least different authentication and, possibly, a different JDBC URL. The connections will be re-used (as can be inferred from the pooling requirement).
What parts of Hibernate/C3P0/et al do I have to override? Can this be accomplished with those tools, or do I need to write my own pooling data source?
I think the best course of action would be creating a SessionFactory for each data source, with possibly pooled connections - that's what's eqbridges suggested in his answer.
Now, Hibernate does have a ConnectionProvider hook, so I suppose you could write an implementation that would return Connections to different data sources, depending on current thread of execution and some additional parameters. Theoretically, you can then have one SessionFactory instance, which will be using different connections to different databases, supplied by your custom ConnectionProvider implementation. But, one SessionFactory holds quite a bit of data, and that data is then used by Hibernate internally, when opening a Session for a unit of work. Plus, there's a second-level cache associated with it as well.
Unfortunately, how will the factory and Sessions you open from it behave in the face of such a provider is anybody's guess. It feels like a hack to me, and I doubt it was ever considered a viable use-case for a SessionFactory. It can possibly lead to all kinds of, possibly very subtle, bugs or data corruption.
On another note, be sure to exactly measure the cost of creating multiple SessionFactories - it may not be as high as you think. Be sure to compare that with the cost of simply opening the needed JDBC connections. I don't know what kind of results you might get, but I think you should be sure about performance before you resort to more hackish solutions.
You have two questions here:
Connections are not thread safe, so each thread must have its own connection. Since you're working with Hibernate, what your application sees is actually a Session obtained from a SessionFactory. To utilize this, you call the SessionFactory#getCurrentSession() method, and configure the current session context in hibernate.cfg.xml:
<property name="current\_session\_context\_class">thread</property>
If you've properly configured thread pooling (using c3po or whatever pooling mechanism you favor) in hibernate.cfg.xml, then each thread will get a connection from that pool.
To maintain multiple data sources that the application may need to work with, then you need to configure a separate SessionFactory for each JDBC url you'd like to access. In your application you'll need to have some means of selecting with SessionFactory you'll need to choose (e.g. "client ID"), using this you can manage each of the SessionFactory instances in a Map or somesuch data structure (in a Java EE app you'd get a reference from JNDI).
To summarize (and generalize), basically a SessionFactory is essentially huge wrapper around a DataSource (and attendant connection pool). It is read-only (and hence thread safe), heavyweight and static, constructed once, and knows everything it needs to about a given DataSource.
A Session, on the other hand is essentially a lightweight wrapper around a Connection. It is not thread safe, often short-lived, and intended to be used and then thrown away.
Hope this helps!