What is the best way to implement connection pooling in hsqldb, without compromising on the speed?
Hibernate gets connections from a DataSource, uses them and closes them. You need a connection pool or it will be very inefficient, consuming a lot of resources both on your app and on the DBMS, regardless of the database server you use.
You should try out commons-dbcp from Apache-Jakarta, it's very efficient and really simple to set up. It depends on commons-pool.
You just define a BasicDataSource with DBCP and it will manage the connections from whatever JDBC driver you tell it to use. It has connection validation and lots of other stuff.
Of, if you're writing a web app, configure a connection pool on the container you will be using and use that, instead of defining your own pool.
You are comparing apples and oranges:
If you want orm compare the performance of different orm tools against the same db.
If you want connection pooling compare different connection pooling libraries against the same db.
Performing ORM incurs extra effort so it will never be as fast as direct JDBC access. That said, hibernate goes to great lengths (and very successfuly) to minimise this additional overhead. With ORM you are trading off significantly increased development productivity against a relatively small drop in performance.
Connection pooling is an orthogonal problem to orm. Most obviously, hibernate allows you to select your own connection pooling infrastructure.
Also, be aware that in practise there is often a fairly tight coupling between connection pooling and transaction mgmt. For example, a typical J2EE application will leave connection pooling to the container (via the JDBC Datasource API) and rely on declarative transactions. In this case connections and transactions are managed (approximately) together.
If you aren't in a J2EE container and you don't need orm I would simply compare C3P0, commons-pool, etc.
If you used Hibernate's own connection pooler, you could consider using c3p0? (If you're already using c3p0, I can't help further) I haven't used HSQLDB myself but I think that could be worth trying.
Related
I created one api using jsp and hibernate. I used this option of hibernate -
<property name="hibernate.connection.pool_size">50</property>
What I want is If I am getting 1000 hit concurrently on api then it should use only 50 connection not more than that. I thought above option of hibernate will help me to achieve this. But I tested my api on 10000 hit and I checked the open connection. It was around 3000. It means pool of connection not working.
How to get this ?
From the Hibernate documentation:
Hibernate's own connection pooling algorithm is, however, quite
rudimentary. It is intended to help you get started and is not
intended for use in a production system, or even for performance
testing. You should use a third party pool for best performance and
stability. Just replace the hibernate.connection.pool_size property
with connection pool specific settings. This will turn off Hibernate's
internal pool. For example, you might like to use c3p0.
Like mentioned, c3p0 is a good option.
I'm wondering if it's possible to use JPA or other persistence library with PL/Java together?
I mean using JPA persistence for manipulationg database over the special JDBC connection provided by PL/Java in stored Java procedures.
It's possible, but a really bad idea. Most ORMs are pretty memory hungry beasts, and PL/Java spawns one JVM per PostgreSQL backend (connection), so that memory-gobbling will be multiplied per-connection. Worse, many ORMs expect to be able to get numerous connections from a pool and use them freely, but when running in PL/Java with the SPI you only really have one connection since PostgreSQL backends are single-threaded and not thread-safe.
I really don't recommend it.
Is Spring SimpleJDBCTemplate production grade?
How good is it to use in the high volume (traffic) site? (Lets say 1K hits per second)
What are the choices we have to improve the performance (Connection Pooling, etc) on the Application Level?
Yes, it is production grade. Your limit at that volume isn't ran JDBC vs Spring JDBC template vs whatever. Your limit is resources if it is truly 1000 hits per second. 1000 users at the same time is a lot less hits because they they have "think time" rather than continuously hitting your app.
With any database technology, you will need to tune a lot, look at caching and be very aware of resources. And yes, you absolutely need connection pooling.
If you are talking of using SimpleJDBCTemplate over pure JDBC...go for it! It should definitely give as much performance and help you get rid of most JDBC connection make/break boiler plate code and exception handling etc.
Spring would allow you to configure connection pooling, tuning as you would do using pure JDBC.
Heres an old blog to provide you with performance comparison: http://pramatr.wordpress.com/2008/08/19/spring-jdbctemplate-the-phantom-performance-problem/
Spring has become better since.
If we have a web application which has
heavy UI (Spring MVC + JQuery with JSON)
Hibernate with JPA annotations being the domain model
extend Spring-provided DAO to code DAO layer
JBOSS being the app server with Oracle as backend
Datasource (JNDI) based connection pooling (Not an XA rather Local data source)
also has access to multiple data sources (dealing with multiple DB)
Behaviorally, lot of Data retrieval (70%) and update of data being 30%
What would be the best practices for the following to effectively consume DB connections and also see to that there is no much leakage at connection usage?
would it be better to opt for Hibernate template based DAOs?
What kind of transaction manager would be suggest-able and should we go for AOP-based transaction managementWhere
where to instantiate session and and where to close the sessions to effectively consume connections from connection pooling.
It is true that we need to handle transactions from Service layer but what happens to sessions would they be waiting for longer time (we are not using any opensessioninviewFilter)
which layer is better to handle the checked exceptions (business exceptions) and runtime exceptions.
Sorry for this being bit lengthier question, however I see that this is being a common query and I tried consolidating it. Appreciate your patience and guidance. Thanks for your help.
This sounds like a pretty typical Spring/Hibernate application, so I would recommend following current best practices, which I recently outlined in another answer. Specifically:
Do not extend Spring DAO support classes or use HibernateTemplate. Use the #Repository annotation combined with component scanning, and directly inject the SessionFactory into your DAO.
Use Spring's HibernateTransactionManager, and definitely use declarative transaction management via #Transactional as your default approach.
Let Spring manage that. It will open sessions just in time for transactions by default, but prefer the open session in view pattern enabled by Spring's OpenSessionInViewFilter.
See #3.
Always handle exceptions where they should be handled--in other words, this is a design decision. Note, however, that the Spring transaction framework by default rolls back on unchecked exceptions, but not checked, to match the behavior of the EJB spec. Make sure to set the proper rollback rules (see previous link) anywhere you use checked exceptions.
Additionally, use a connection pool, obviously. Apache Commons DBCP is a great choice. "Not much leakage in connection usage" isn't enough. You have to have zero connection leakage. Depending on Spring to manage your resources will help ensure this. As for any other performance issues, don't try to optimize prematurely. Wait until you see where your problem areas are, and then figure out the best way to solve each one individually. Since your bottlenecks will most likely be database-related, check out the performance chapter of the Hibernate reference to get an idea what you're up against. It covers the important concepts of caching and fetching strategies.
Use JPA EntityManager directly in your DAOs. Be sure not to mark it as Extended
Prefer <tx:annotation-driven /> and #Transactional - only on the service layer
The transaction manager also opens and closes sessions (if one doesn't exist already in the thread). Here it is good to know that sessions are session-per-request. Each request(=thread) has a separate session instance. But a database connection is created only if one is needed, so even if there is a transaction manager around all methods, needless connections won't be opened.
read-only transactions - use #Transactional(readOnly=true) in cases when there is only data retrieval
caching - utilize hibernate 2nd level cache to put entities in memory (instead of fetching them from the database each time)
avoid OpenSessionInView and lazy collections. This is subjective, but in my opinion all objects that leave the service layer must be initialized. For small collections (for ex. list of roles) you can have eager collections. For bigger collections use HQL queries.
I'm writing a Java SE (Note, not Java EE) application using Hibernate, and I need to provide a different connection to Hibernate for each thread of execution. These connections must be pooled, and each one has at the very least different authentication and, possibly, a different JDBC URL. The connections will be re-used (as can be inferred from the pooling requirement).
What parts of Hibernate/C3P0/et al do I have to override? Can this be accomplished with those tools, or do I need to write my own pooling data source?
I think the best course of action would be creating a SessionFactory for each data source, with possibly pooled connections - that's what's eqbridges suggested in his answer.
Now, Hibernate does have a ConnectionProvider hook, so I suppose you could write an implementation that would return Connections to different data sources, depending on current thread of execution and some additional parameters. Theoretically, you can then have one SessionFactory instance, which will be using different connections to different databases, supplied by your custom ConnectionProvider implementation. But, one SessionFactory holds quite a bit of data, and that data is then used by Hibernate internally, when opening a Session for a unit of work. Plus, there's a second-level cache associated with it as well.
Unfortunately, how will the factory and Sessions you open from it behave in the face of such a provider is anybody's guess. It feels like a hack to me, and I doubt it was ever considered a viable use-case for a SessionFactory. It can possibly lead to all kinds of, possibly very subtle, bugs or data corruption.
On another note, be sure to exactly measure the cost of creating multiple SessionFactories - it may not be as high as you think. Be sure to compare that with the cost of simply opening the needed JDBC connections. I don't know what kind of results you might get, but I think you should be sure about performance before you resort to more hackish solutions.
You have two questions here:
Connections are not thread safe, so each thread must have its own connection. Since you're working with Hibernate, what your application sees is actually a Session obtained from a SessionFactory. To utilize this, you call the SessionFactory#getCurrentSession() method, and configure the current session context in hibernate.cfg.xml:
<property name="current\_session\_context\_class">thread</property>
If you've properly configured thread pooling (using c3po or whatever pooling mechanism you favor) in hibernate.cfg.xml, then each thread will get a connection from that pool.
To maintain multiple data sources that the application may need to work with, then you need to configure a separate SessionFactory for each JDBC url you'd like to access. In your application you'll need to have some means of selecting with SessionFactory you'll need to choose (e.g. "client ID"), using this you can manage each of the SessionFactory instances in a Map or somesuch data structure (in a Java EE app you'd get a reference from JNDI).
To summarize (and generalize), basically a SessionFactory is essentially huge wrapper around a DataSource (and attendant connection pool). It is read-only (and hence thread safe), heavyweight and static, constructed once, and knows everything it needs to about a given DataSource.
A Session, on the other hand is essentially a lightweight wrapper around a Connection. It is not thread safe, often short-lived, and intended to be used and then thrown away.
Hope this helps!