A simple question: what is the more efficient way to access a db in Java/JDBC?
I'm a web developper and I'd like to write some reusable and scalable code. What is interesting for me is the use of tools like the ResultSupport: is it too expansive in terms of resource usage?
What can you suggest?
Not just JDBC specific, just general SQL stuff
If you have to rerun a query multiple times, use PreparedStatement. Use stored procedure if it is available. This is obviously not portable so YMMV.
Always close your ResultSet or Statement if you are not using it. Closing a Statement will auto close all ResultSet associated with the Statement. Still it is a good habit to close the ResultSet.
Try to restrict what can be queried eg. select * from orders where order_date between XXX and yyy. In MySQL the query may either be a full table scan or 'range' depending on how much data is returned. So deciding a how 'flexible' you want your queries to be
If you are using MySQL, use explain to optimize your query. If you are using JPA, then you don't get to see the SQL generated. (This is not strictly JDBC) You might want to enable logging on the ORM manager to display the SQL statement used. Then use explain to optimize that. You may want to use #NamedNativeQuery if the ORM generates a really convoluted query
If your JDBC driver supports batch update then use that. Batch updates is supported in PreparedStatement and ResultSet.
You can also control the commit. Good idea to turn it off if you are performing lots of updates. The call commit() yourself.
Best answer to this simple question is: "it depends".
There are many API's you can use for database access. Nearly all of them will use the JDBC API as their means to communicate with the database. So in theory, nothing can beat raw low level JDBC (just as machine code is in theory always faster than higherlevel programming languages).
But as you also like to write reusable code, I suggest you look into JPA. It's the Java standard for object persistence to relational databases. It’s performance is quite good and it’s very portable.
As JPA is just a specification, you can choose you’re own implementation: Hibernate, OpenJPA or any compliant Java EE server.
It is very important to use always a connection pool DataSource such as c3p0.
There is a project that maps java objects to mysql databases. Azirt.
Use connection pooling (either the one from your container or a standalone connection pool like c3p0 or DBCP) and something like DBUtils or Spring's JdbcTemplate.
I think the easiest and most common way is to use Spring and their JDBCTemplate.
The best approach likely depends on the stack you are using to create your web app. If you're starting afresh then Spring is a good way to go.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
We have been using JDBC for a very long time in our web applications. The main reason we used it is because we have 100% control over the code, sql and fix things by our hands. Apart from that we used triggers inside the database, and the database is developed separately by DB experts.
However many now recommend using Hibernate so we also thought about using it. But, we found the below issues.
Hibernate cannot connect with an "Existing" database. It always try to create a one of its own.
Our database might access by same application which is in different platforms (cloud, server, VPS, Personal Computer). Hibernate can make problems because of its caching in this situation.
We never like to give the "table creating work" to the java code. We create tables manually, always.
We might have to use very long and complex SQL statements. Last time we used an statement with more than 150 lines, joining more than 20 tables. We doubt whether we will face troubles in this when it comes to Hibernate.
Our SQL code is nice and standard. Hibernate generated code seems to be bit dirty for us.
We always use MySQL. Never use any other DB.
The application we create require max security, related to medical. If at least one data record is leaked, we are done.
There are lot of foreign keys, Primary Keys, Composite Keys, Unique Keys etc etc in database. In forums, some complained that Hibernate messed with those.
We decided to try hibernate because some people claims, "Are you Software Engineers? You are using already dead JDBC !!. "
Considering these, please let me know whether the above points are actually true (as I said, I got to know them via googling, discussion etc) or not. And, what are the pros and cons of Hibernate VS Java JDBC?
Answering issues listed above:
1. Hibernate cannot connect with an "Existing" database. It always try to create a one of its own.
This is wrong. Hibernate can connect to an existing database, and it doesn't always try to recreate it. You just should turn of parameter like hbm2ddl. auto.
2. Our database might access by same application which is in different platforms (cloud, server, VPS, Personal Computer). Hibernate can make problems because of its caching in this situation.
Hibernate has an adjustable cache, so this is also not a problem.
3. We never like to give the "table creating work" to the java code. We create tables manually, always.
No problem. See p.1 above. Furthemore there are several convinient libraries for indirect table creation and update (e.g. liquibase) which can be used in couple with hibernate perfectly.
4. We might have to use very long and complex SQL statements. Last time we used an statement with more than 150 lines, joining more than 20 tables. We doubt whether we will face troubles in this when it comes to Hibernate.
You can always use direct JDBC calls and invoke native SQL queries via hibernate, if it is neeeded.
5. Our SQL code is nice and standard. Hibernate generated code seems to be bit dirty for us.
Again, if you have to invoke some logic complicated SQL code instead of hibernate auto-generated - you can do it.
6. We always use MySQL. Never use any other DB.
Not a problem at all. Hibernate has special MySQL dialect support: org.hibernate.dialect.MySQLDialect.
7. The application we create require max security, related to medical. If at least one data record is leaked, we are done.
Security issues aren't related to ORM techniques. Hibernate is just logical and convinient object-oriented layer between pure database JDBC calls and programmers tools. It doesn't influence somehow on common net security.
Hibernate is a great tool and you'll find plenty of documentation, books, and blog articles about it.
I will address all your concerns:
Hibernate cannot connect with an "Existing" database. It always tries to create one of its own.
Hibernate should use a separate database schema management procedure even for integration testing. You should use an incremental versioning tool like FlywayDB to manage your schema changes.
Our database might access by same application which is in different platforms (cloud, server, VPS, Personal Computer). Hibernate can make problems because of its caching in this situation.
You don't have to use the 2nd level cache, which uses 3rd party caching implementations. All caching solutions may break transactional consistency. The first level cache guarantees session-level repeatable reads and with the optimistic locking in place you can prevent lost updates.
We never like to give the "table creating work" to the java code. We create tables manually, always.
The DB should be separated from your ORM tool. That's a best practice anyway.
We might have to use very long and complex SQL statements. Last time we used an statement with more than 150 lines, joining more than 20 tables. We doubt whether we will face troubles in this when it comes to Hibernate.
Hibernate is great for write operations and for concurrency control. You still need to use native SQL for advanced queries (window functions, CTE). But Hibernate allows you to run native queries.
Our SQL code is nice and standard. Hibernate generated code seems to be bit dirty for us.
You don't need and you shouldn't probably use the hbmdll utility anyway.
We always use MySQL. Never use any other DB.
That's even better. You can therefore use advance native queries without caring for database portability issues.
The application we create require max security, related to medical. If at least one data record is leaked, we are done.
Hibernate doesn't prevent you from securing your database or the data access code. You can still use database security measures with Hibernate too. You can even use Jasypt to enable all sorts of security-related features:
advanced password hashing
two-way encryption
There are lot of foreign keys, Primary Keys, Composite Keys, Unique Keys etc etc in database. In forums, some complained that Hibernate messed with those.
All of those are supported by Hibernate. Aside from the JPA conventions, Hibernate also offers particular mapping for any exotic mapping.
We decided to try hibernate because some people claims, "Are you Software Engineers? You are using already dead JDBC !!. "
That's not the right argument for switching from a library you already master. If you think you can benefit from using Hibernate then that's the only compelling reason for switching from JDBC.
Using plain old JDBC, does not mean you are lacking in IT industry, rather Hibernate also uses JDBC in the underlying layer.
What advantages it gives us what we should look for.
1.) Cache Mechanism.
2.) Managing sessions, transactions etc.
3.) Reduce efforts in writing queries, more utilities of hibernate like Query API, Criteria API, HQL
The questions that you have raised are more or less covered in Hibernate docs.
Also there are lot more caching strategy available ehcache, infinispan, depends on the server we are deploying, JBOSS, Weblogic, Tomcat etc. ++ environment like cloud, distributed cache etc.
Hibernate still provides you with option of turning off automatically creating schema and pointing to the one create by you.
Here are the quick answers that I know
1) You can connect to an existing database. But yeah as stated here
If you don't have a solid object model, I'd say that Hibernate is a
terrible choice.
2) As you database is been accessed from different applications so you can maintain locks. On-the-other-hand you can trun-off caching as done here.
3) You can create tables manually and connect it using .hbm.xml file.
4) You can use any type of query in hibernate like simple SQL queries criteria.
5) You can directly use SQL code in Hibernate, if you want. Other option is to use criteria.
6) Hibernate is NOT DB specific. You can go for any Database and connect it with hibernate.
7) Using locks and giving rights in database you can maintain security.
8) Agreed that foreign keys are messy in Hibernate If You Donot Handle It Well. So Use OO approach and maintain cascades well, then Hibernate will be good choice.
how to prevent my website from doing sql injection in it
I am working using Struts 2 , DB MY sql .
The best way I think is to not re-invent the wheel and use the tools already available. For a small project I would recommend to simply use prepared statements when querying your database.
http://docs.oracle.com/javase/7/docs/api/java/sql/PreparedStatement.html
You could also look into using an ORM like Hybernate. But make sure to use it as intended. Even HQL can be susceptible to injection. See: how much safe from SQL-Injection if using hibernate
The important thing is to not write your own native queries by concatenating query strings with values from untrusted sources.
I am shifting back from hibernate to plain JDBC in order to overcome the overheads incurred in using hibernate.I wanted to know how to deal with the sessions associated with hibernate.How should i convert back to Plain JDBC so that all my sessions are replaced with the JDBC connections.And please let me know if I am wrong in my thoughts that replacing a session with a connection converts back to plain JDBC as I am not well versed in these concepts and dont know if i am going in the right way.
I have used Hibernate extensively in high-performance tasks, including batch insertion of millions of records. Your problem is not with Hibernate, but with the way you are using it.
Above all, do not use Hibernate as a persistent state manager; use it as a thin layer above the raw SQL and you won't complain about performance.
Always prefer StatelessSession (it works for everything you need except save operations)`;
never use lazy fetching, use explicit joins for everythng;
never fetch whole objects, use SELECT to fetch exactly what you need;
fetch as much as possible in a single statement, avoid n+1 selects at all costs;
for large result sets, never use list, use iterate or scroll.
The list goes on, but this is what I have come up with at this moment.
As far as your direct question, it depends on the application. If it is a Spring application, then you will certainly want to use its declarative transaction management. Basically, you just put a few lines of XML config and you'll have an open DataSource in your DAO code ready to be used, with no management on your part.
If you are doing something more raw, then by all means use a connection pool library, such as the great BoneCP. You acquire connections from it and later return them to it, again with no explicit management.
Lastly, if you really want a bare-bones, unsafe and non-scalable approach, then you can create connections directly from the JDBC driver. This approach is really only for schoolwork and it is not recommended even in the smallest of production-worthy projects.
A Hibernate session is much more than a JDBC connection. It contains multiple such connections (usually managed via a JDBC Connection Pool which recycles JDBC Connection instances), a bunch of entities which are attached to, and managed by said session and other things as well (caching, etc).
Removing Hibernate and doing everything with the JDBC API-only will imply more than just replacing Hibernate Session instances with one or more JDBC connections followed by a duplication of the Hibernate code into analogous JDBC API calls. If you'd only do that, you'd simply do a lot of work for nothing, as you'd lose all of Hibernate's advantages (less verbose code, a higher level of abstraction, etc) and gain nothing of JDBC's advantages (less heap memory used, fewer method calls (yes, even with Hibernate's Javassist magic, this still counts towards performance in some cases), finer grained control of the database interactions, etc).
My advice is to first really look into the problems your app has (apparently due to Hibernate) and at least for the major ones, try to first see if you can't do something to optimize it without getting rid of Hibernate. Yes, Hibernate can become heavy and memory hungry, but more often than not, the issue with performance comes from improper use of the framework (are you sure you're fetching all the necessary associated entities in one query, or do you make Hibernate make hidden joins or pseudo joins in the background? Are you doing or you data operations on the database side, or is some of that done in Java code after a more-than-necessarily-generic Hibernate query is executed to fetch the data? etc.)
If you really need to get rid of Hibernate (maybe you need to use some very specific features of your database which are not standard SQL and which Hibernate doesn't let you access, like MySQL's ability to import big amounts of data via a custom flat-file format) then make sure that what ever it is you're replacing it with (plain JDBC, or maybe some other ORM like EclipseLink) can tackle the issue and solve it in a more performant way. Doing a small POC to test these before you start re hauling your code can save you a ton of time.
While I strongly urge you to heed the advice of Marko and Shivan, you could use hibernate to manage your connections/sessions/transactions and to execute your SQL queries without much overhead being generated.
a quick google search yielded this on executing SQL from a hibernate session.
http://www.informit.com/guides/content.aspx?g=java&seqNum=575
While I agree with both of the earlier answers, if you truly want to go down the road of executing straight SQL, I would look into this option for two reasons.
1) your sessions are already in place. If you don't have hibernate load up all of your entities I don't see how hibernate would generate that much overhead.
2)If the problem is speed, and not overhead which I have run into before, you can implement this to quickly execute native SQL in your problem areas and keep all of hibernates ORM goodies in place.
All of that being said, I would also urge you to dig into the documentation for hibernate. I have used hibernate for several high performance solutions with great success. While the nuances can be hard to grapple with in the beginning, the benefits of using hibernate (or at least something that adheres to JPA standard) far outweigh the cost of not doing so down the road scalability wise.
I want to write Java code to work with a database, no matter which database is used. My problem is that it wouldn't be Object related. There are some insertions and queries that are but most of them aren't.
Right now we are using Postgresql and pure JDBC, but we may have to make it work with Oracle.
Can Hibernate (which I've never used) solve my problem or should I go for something else ?
I have created jOOQ precisely for that. jOOQ models SQL itself as a domain-specific language in Java. It embraces using standard and vendor-specific features, such as built-in functions, window functions, stored procedures, hierarchical queries, etc. Whenever possible, a vendor-specific functionality from one database is simulated for other databases. That way, most of jOOQ-generated SQL is compatible with any of its 13 supported databases.
See also this related question here:
ORM frameworks used for insert only / query only apps
I like #unludo's JPA answer but I thought I'd add some additional details.
I would recommend changing your code to use persistance interface that you define.
public interface DataPersister {
public void saveFoo(Foo foo);
public void findFooById(int id);
...
}
Your first implementation of the interface would then be using JDBC/Postgresql. If you want to use JPA under the covers then fine. Or if you want to switch to some no-SQL database or even flat files then fine.
Once you have the separation in your own code between the usage of the data and the persistence implementation, then it is significantly easier to switch to a different persister. You can use a cheap database like H2 for testing and switch to Postgresql in production while migrating to a new database in the near future.
Hope this helps.
Problems with Hibernate is that you need to modelize your relational database like object model. Sometimes this make difficult working with existing database. So it depends your relational database.
Other framework (not JPA) is Ibatis. Try to look at this framework.
The standard for Java is JPA and it is very powerful. Hibernate is the industry standard as a JPA provider.
JPA helps you write a clean persistence layer. You may write queries which are sure not to break, because they are validated at compilation time. I like to use spring for this, it's so easy to unit test. But CDI now provides the same I believe.
It's also easy to write test classes. As a coworker once teached me, the model is the most important thing you have. You don't want it to break or you have problems.
With JPA you may also generate the schema from the entities, for any database you want to use. From experience, it's also very good.
JPA helps you put good practices at work. That's a lot of value.
Regarding #hvgotcodes answer, yes you have to be careful with the cost but you may also mix jdbc and jpa. That's what Dao's are for.
The problem with writing your own sql is you need to manually optimize it for your RDBMS. Some RDBMS support varying sql constructs.
So you need to balance that against the overhead of switching to an ORM based solution. Or, make sure you sql is 100% standard so you don't use any constructs that work in one RDBMS solution and not in another.
In your particular situation, it's probably going to be easier to just fix your sql than rework your entire persistence layer to use ORM. Sometimes the best tool is the one you know. If your current application doesn't have a concise model layer, switching to ORM is going to require a lot of work. You could of course use hibernate and just use strait sql queries, but what's the point if you are not going to model your data.
Hopefully, all your persistence concerns are in one DAO layer, with lots of integration tests, so you can quickly identify what breaks when you switch RDBMS. If you don't have integration tests that focus on persistence, then now is the time to start writing them.
We've been implementing Hibernate recently as a replacement for JDBC.
What I like is not having to constantly write SELECT, UPDATE, INSERT statements and the associated PreparedStatement and ResultSet code.
However we've been struggling with random bizarre behavior (example A) which I find hard to understand and resolve due to all the different configuration/feature options and the associated Hibernate behavior. I find some of the features like caching, lazy loading, etc, etc very cool but way more than I need - and ultimately confusing.
Is there a better middle ground for someone just looking to avoid the tediousness of JDBC but who doesn't need all the features of Hibernate?
not completely avoiding jdbc, but ... my suggestion is to use the jbdc support provided by spring framework. You still need to write your select, update and inserts, but spring nicely wraps it so you usually don't care about the result set, closing connections, cleaning up your code and such.
Have a look at this chapter from the spring documentations to see the details. You can create an entire dao layer that appears just like the hibernate dao layer, but the internal implementation is different. The RowMapper interface lets you handle the conversion from result sets to objects very nicely. Overall it provides a clean separation of concerns.
(another alternative is to use iBATIS for lightweight O/R mapping, or at least to keep your sql queries outside of Java code).
How about using Hibernate (or TopLink) as a JPA provider. IMO I find the JPA annotation-based approach to doing ORM a lot easier to understand / implement than Hibernate directly - and you can always drop down and do 'difficult' stuff with Hibernate directly.