Related
Currently I am migrating databases from Oracle to MySQL. I mainly use Java to send queries to the database using JDBC. In the process of migrating, I need to change a lot of my queries in the Java code (the queries are hard-coded) as they will not work in MySQL.
I want to be able to recode my queries in such a way that I can easily switch between the databases if problems arise; I am changing all my queries to standard SQL but there are areas where this is not possible. I am thinking of having two versions of the queries, one for Oracle and one for MySQL so I can switch between both (I will have two versions temporarily just to see if MySQL can cope with our needs). However this seems like a terrible idea - does any one have any advice on a better way they would do this?
You have a bunch of options.
Firstly, many people now use Object-Relational Mapping (ORM) tools to connect applications to SQL databases. These come in a variety of different flavours - Hibernate is popular - and allow you switch between databases at very little cost. However, they do have a fairly steep learning curve. Inexperienced developers often struggle with performance problems in ORM applications.
If you stick with "traditional" JDBC, I suggest you take the body of the SQL out of the Java code, and treat it like a resource. As Henry suggests, you could use property files, and use parameter placeholders (ideally named placeholders, using the Spring template). While this does spread the code for a given piece of functionality into two files, it makes it easy to quickly refine the SQL and test new versions.
One possibility is to store the queries in a properties file. You would have one for Oracle and one for MySql.
I must add - ORM is good advice and will work... when you are starting with a fresh application and you can design your application to work on a domain model.
In this case however there is an existing application that invokes a great number of SQL queries. ORM based queries (HQL, JPQL) translate to SQL just fine; SQL does not by definition translate to the ORM layer however, major changes will be needed to make it a more object oriented approach to the data.
The problem will still persist even when you do manage to work in an ORM layer. There is already a major difference between MySQL and Oracle in how primary key generation works for example; MySQL uses auto-numbering where Oracle uses a sequence. Likely you already have an existing datamodel that you need to reverse engineer into the ORM layer code; it isn't going to be cross-database code.
JDBC in it's documentation states that it has 4 implementations to connect to databases. I don't quite get what the 4 implementations mean, I was wondering is JDBC truly database agnostic. That is, will I need "drivers" for each type of database, like for MYSQL (jConnector)?
I am writing an app to support Oracle, MySQL and MSSQL.
Reference:
http://docs.oracle.com/javase/tutorial/jdbc/basics/gettingstarted.html
Thanks
Unfortunately, you will need drivers for all database types.
If you want to be truly database agnostic you need to use JPA:
http://en.wikipedia.org/wiki/Java_Persistence_API
With one of its implementations. One of the most popular is Hibernate:
http://en.wikipedia.org/wiki/Hibernate_%28Java%29
When using JDBC, you use an API that is database-agnostic. The interfaces of this API (Connection, Statement, etc.) are implemented by the JDBC driver of the database that you target. So if you use Oracle, you'll need an oracle driver.
Yes the JDBC API is database agnostic - you just need to provide an appropriate driver. Of course, the SQL you send won't be, unless you use JPA
JDBC in it's documentation states that it has 4 implementations to connect to databases
No it doesn't. It might state that there are (currently) 4 implementation levels. However the number of implementations depends on (at least) the number of target databases and the number of iterations of the implementation per database, both of which are considerably greater than 4.
I've explained in this answer how difficult it is to be truly vendor agnostic using JDBC alone. There are two layers to this:
JDBC is a very good abstraction of the network protocol used to connect to the database. Though you'll need a few vendor-specific quirks here and there. These quirks are fine if you're binding to a single RDBMS vendor, but if you need to support multiple products, it gets very hairy
SQL is a very good standard for a query language. Though, again, you'll need quite a few vendor-specific quirks here and there. Same as above, it's OK to support 1 RDBMS and its quirks, but very hard to support multiple.
As others have mentioned, JPA/Hibernate help you abstract over some of the differences, mostly by removing access to more "advanced" SQL features (including derived tables, unions, etc. at least as of version 5). For a more SQL centric abstraction over dialects, jOOQ is a popular option.
Disclaimer: I work for the company behind jOOQ.
I found an article from 2008 discussing how to call Java code from MySQL. There were a lot of caveats and disclaimers because the process involved working with an experimental branch of MySQL.
For a project I have in mind, it would be very useful to be be able to access Java libraries within MySQL, analogous to Oracle's Java Stored Procedures. Does this capability now exist as a standard feature of MySQL? If not, what open source RDBMSs support something similar to Oracle's Java Stored Procedures?
PostgreSQL supports pluggable procedure languages, and a project exists to extend PostgreSQL with PL/Java as the language.
I don't recommend putting too much code in the RDBMS. Tools to develop, test, and debug code in the application layer are better than tools for code in the RDBMS.
Also many developers don't understand that code inside the RDBMS should obey transaction isolation. They try to send emails from triggers and so forth. I think code with side effects should be in the application layer, so you don't create phantom effects (e.g. an email may notify of a database change, even though the change was rolled back).
If you can use HSQLDB then you can call java methods directly from SQL: http://hsqldb.org/doc/2.0/guide/sqlroutines-chapt.html#N1240C
I fully agree with Bill, but I can imagine business rules being stored (not processed) in the database. I'm thinking of drools here. The engine would be in the application, but the rules could be in the database with a management front-end.
Such a beast would be interesting for scenarios where not only the parameters change, but also the formulas can change.
It is difficult to give good advice based on the limited information that you have provided so far. However:
... the example involves a graph-based data type (chemical structures) that can't be matched to a query using built-in MySQL functions. The Java library would convert the query and contents of a text field into an in-memory object that can by matched. Keeping this logic in the DB layer would, for example, keep joins within the database, which seems like where they belong. That's the idea, at least.
I don't think I would use database-side Java in MySQL for this. Instead, I think I would consider the following options:
Use an object-relational mapping such as JDO or JPA (for example using Hibernate) to deal with the mapping between your graph-based data model and what the database provides. You don't necessarily have to use an RDBMS as the backend, but that is probably the best place to start ... unless you've already found that this is a performance issue.
Take another look at your data model and data access patterns. See if you can figure out some transformation that allows your application's main queries to be implemented as (efficient) table joins without resorting to server-side application logic.
If you do need to use server-side application logic (for performance reasons!) stick with the mechanisms supported by your RDBMS. For example, in Oracle you'd use PL/SQL and PostgreSQL you have a number of options. Be prepared to switch to a different RDBMS that better suits your application requirements.
I (personally) would avoid depending on an experimental branch of some database:
Consider what happens if the experimental branch is not merged back into the main branch. You would be stuck with your code base depending on a branch that is not supported, and is likely to stop being maintained and fizzle out.
Using a (currently) unsupported RDBMS branch will be an impediment to other folks who might want to use your software.
Now obviously, if the long term viability of your software is not a primary concern, you could choose to ignore this advice. But it probably matters to someone; e.g. your research supervisor.
I realise that this is quite an old article, but it bears updating. The ability to call java from a database trigger is is part of the "SQL Routines and Types for the Java Programming Language" (SQL/JRT) standard.
Read more about this on Wikipedia at https://en.wikipedia.org/wiki/SQL/JRT.
Amongst the compliant database engines are..
HyperSQL: http://hsqldb.org/
Oracle: https://www.oracle.com/database/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I intend to develop a small (Java) application for managing my finances. I believe I need to use an embedded database, but I have no experience regarding this issue. I tried to look at some of the available products, but I can't decide which one would be more suitable for me. H2, HSQLDB, Derby and Berkeley DB seem to be good candidates, but I still don't see how they compare to each other. I appreciate your help comparing them and helping me decide which one to use.
I intend to use Hibernate for my application (unless you would recommend using DBMS-provided API), but I also want to have the ability to edit the database easily using a SQL browsing tool (modifying schema and changing data).
Thank you.
Either
HSQLDB - Used by OpenOffice, tested and stable. It's easy to use. If you want to edit your db-data, you can just open the file and edit the insert statements.
or
H2 - Said to be faster (by the developer, who originally designed hsqldb, too)
Which one you use is up to you, depending how much performance and how much stability you need.
The developer of H2 has put up a nice performance evaluation:
http://www.h2database.com/html/performance.html
I use Apache Derby for pretty much all of my embedded database needs. You can also use Sun's Java DB that is based on Derby but the latest version of Derby is much newer. It supports a lot of options that commercial, native databases support but is much smaller and easier to embed. I've had some database tables with more than a million records with no issues.
I used to use HSQLDB and Hypersonic about 3 years ago. It has some major performance issues at the time and I switch to Derby from it because of those issues. Derby has been solid even when it was in incubator at Apache.
I needed to use Java embedded database in one of my projects and I did lot of research understanding pros and cons of each database. I wrote a blog listing pros and cons of popular embedded java databases (H2, HSQLDB, Derby, ObjectDB, Neo4j, OrientDB), you can have a look at it. I chose H2 as I thought it best suited my requirements.
Link for the blog: http://sayrohan.blogspot.in/2012/12/choosing-light-weight-java-database.html
Hope it helps!
I'd go with H2, the performance is meant to much better than Derby. Read http://www.h2database.com/html/performance.html for more info.
HSQLDB is a good candidate (the fact that it is used in OpenOffice may convinced some of you), but for such a small personnal application, why not using an object database (instead of a classic relationnal database) ?
I used DB4O in one of my projects, and I'm very satisfied with it. Being object-oriented, you don't need the whole Hibernate layer, and can directly insert/update/delete/query objects ! Moreover, you don't need to worry about the schema, you directly work with the objects and DB4O does the rest !
I agree that it may take some time to get used to this new type of database, but check the DB40 tutorial to see how easy it makes working with the DB !
EDIT: As said in the comments, DB4O handles automatically the newer versions of the classes. Moreover, a tool for browsing and updating the database outside of the application is available here : http://code.google.com/p/db4o-om/
Java DB (Sun's distribution of Apache Derby) now ships in JDK 6!
I've been wanted to do something like Jason Cohen and have been thinking this looks like the easiest way being in the JDK distro (which of last week is now a requirement for my app). Or maybe I am just lazy that way.
We use HSQLDB in production as a "no-configuration" option for our application. It allows people to trial without the hassle of setting up a real database.
However we do not support it for normal use. The reasons are several:
Slows down proportionally to the size of the data.
Difficult to access outside of our app (e.g. for custom reports).
Transactions / disk-sync is difficult to get right, so it's easy to lose data.
For at least (2) and (3), there are ways around it but it's difficult; it's much easier to e.g. install MySQL.
neo4j is:
an embedded, disk-based, fully transactional Java persistence engine that stores data structured in graphs rather than in tables
I haven't had a chance to try it yet - but it looks very promising. Note this is not an SQL database - your object graph is persisted for you - so it might not be appropriate for your existing app.
HSQLDB may cause problems for large applications, its not quite that stable.
The best I've heard (not first hand experience however) is berkleyDB. But unless you opensource it, it will cost you an arm and a leg to use due to licensing...see this http://www.oracle.com/technology/software/products/berkeley-db/htdocs/licensing.html for details.
ps. berkleyDB is not a relational database in case you didnt know.
Good comparison tool can be found here: http://www.jpab.org/All/All/All.html
Notice also the Head to Head DBMS/JPA Comparisons
Most things have been said already, but I can just add that I've used HSQL, Derby and Berkely DB in a few of my pet projects and they all worked just fine. So I don't think it really matters much to be honest. One thing worth mentioning is that HSQL saves itself as a text file with SQL statements which is quite good. Makes it really easy for when you are developing to do tests and setup data quickly. Can also do quick edits if needed. Guess you could easily transfer all that to any database if you ever need to change as well :)
I am a big fan of DB4O for both .Net and Java.
Performance has become much better since the early releases. The licensing model isnt too bad, either. I particularly like the options available for querying your objects. Query by example is very powerful and easy to get used to.
What criteria will you use to evaluate these ? If you don't know yet, then you don't need to decide right now. Try to make your application as database-implementation-agnostic as you can - providing the appropriate wrappers, data access objects etc., and make this decision when you have all the facts to hand and you have to decide.
If you're using relational databases and SQL then the above shouldn't be too hard (using JDBC etc). Make sure you have plenty of surrounding tests so that when you want to switch between databases, you can determine that your application's functionality remains the same.
I ran into the same issue some time ago. I didn't know which database to go for, so my first solution used Derby (or HSQLDB?), and I was later able to switch to HSQLDB (or Derby ? Can't remember which solution worked) once I'd determined where I had issues (relating to performance) and which solution would really work for me.
I have used Derby and i really hate it's data type conversion functions, especially date/time functions. (Number Type)<--> Varchar conversion it's a pain.
So that if you plan use data type conversions in your DB statements consider the use of othe embedded DB, i learn it too late.
Latest Derby Version data type conversions
I personally favor HSQLDB, but mostly because it was the first I tried.
H2 is said to be faster and provides a nicer GUI frontend (which is generic and works with any JDBC driver, by the way).
At least HSQLDB, H2 and Derby provide server modes which is great for development, because you can access the DB with your application and some tool at the same time (which embedded mode usually doesn't allow).
I guess I'm a little late (a lot late;-)) to this post, but I'd like to add Perst, an open source, object-oriented embedded database for Java &.NET. for your consideration. Perst is an open source / dual license embedded database for Java. The distribution is compatible with Google's Android platform, and also includes Perst Lite for Java ME. We've even built an Android benchmark and produced a whitepaper on the subject...you can take a look here: http://www.mcobject.com/index.cfm?fuseaction=download&pageid=581§ionid=133
All the best,
Chris
If I am correct H2 is from the same guys who wrote HSQLDB. Its a lot better if you trust the benchmarks on their site. Also, there is some notion that sun community jumped too quickly into Derby.
I realize you mentioned SQL browsing, but everything else in your question makes me want to suggest you also consider DB4O, which is a great, simple object DB.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I intend to develop a small (Java) application for managing my finances. I believe I need to use an embedded database, but I have no experience regarding this issue. I tried to look at some of the available products, but I can't decide which one would be more suitable for me. H2, HSQLDB, Derby and Berkeley DB seem to be good candidates, but I still don't see how they compare to each other. I appreciate your help comparing them and helping me decide which one to use.
I intend to use Hibernate for my application (unless you would recommend using DBMS-provided API), but I also want to have the ability to edit the database easily using a SQL browsing tool (modifying schema and changing data).
Thank you.
Either
HSQLDB - Used by OpenOffice, tested and stable. It's easy to use. If you want to edit your db-data, you can just open the file and edit the insert statements.
or
H2 - Said to be faster (by the developer, who originally designed hsqldb, too)
Which one you use is up to you, depending how much performance and how much stability you need.
The developer of H2 has put up a nice performance evaluation:
http://www.h2database.com/html/performance.html
I use Apache Derby for pretty much all of my embedded database needs. You can also use Sun's Java DB that is based on Derby but the latest version of Derby is much newer. It supports a lot of options that commercial, native databases support but is much smaller and easier to embed. I've had some database tables with more than a million records with no issues.
I used to use HSQLDB and Hypersonic about 3 years ago. It has some major performance issues at the time and I switch to Derby from it because of those issues. Derby has been solid even when it was in incubator at Apache.
I needed to use Java embedded database in one of my projects and I did lot of research understanding pros and cons of each database. I wrote a blog listing pros and cons of popular embedded java databases (H2, HSQLDB, Derby, ObjectDB, Neo4j, OrientDB), you can have a look at it. I chose H2 as I thought it best suited my requirements.
Link for the blog: http://sayrohan.blogspot.in/2012/12/choosing-light-weight-java-database.html
Hope it helps!
I'd go with H2, the performance is meant to much better than Derby. Read http://www.h2database.com/html/performance.html for more info.
HSQLDB is a good candidate (the fact that it is used in OpenOffice may convinced some of you), but for such a small personnal application, why not using an object database (instead of a classic relationnal database) ?
I used DB4O in one of my projects, and I'm very satisfied with it. Being object-oriented, you don't need the whole Hibernate layer, and can directly insert/update/delete/query objects ! Moreover, you don't need to worry about the schema, you directly work with the objects and DB4O does the rest !
I agree that it may take some time to get used to this new type of database, but check the DB40 tutorial to see how easy it makes working with the DB !
EDIT: As said in the comments, DB4O handles automatically the newer versions of the classes. Moreover, a tool for browsing and updating the database outside of the application is available here : http://code.google.com/p/db4o-om/
Java DB (Sun's distribution of Apache Derby) now ships in JDK 6!
I've been wanted to do something like Jason Cohen and have been thinking this looks like the easiest way being in the JDK distro (which of last week is now a requirement for my app). Or maybe I am just lazy that way.
We use HSQLDB in production as a "no-configuration" option for our application. It allows people to trial without the hassle of setting up a real database.
However we do not support it for normal use. The reasons are several:
Slows down proportionally to the size of the data.
Difficult to access outside of our app (e.g. for custom reports).
Transactions / disk-sync is difficult to get right, so it's easy to lose data.
For at least (2) and (3), there are ways around it but it's difficult; it's much easier to e.g. install MySQL.
neo4j is:
an embedded, disk-based, fully transactional Java persistence engine that stores data structured in graphs rather than in tables
I haven't had a chance to try it yet - but it looks very promising. Note this is not an SQL database - your object graph is persisted for you - so it might not be appropriate for your existing app.
HSQLDB may cause problems for large applications, its not quite that stable.
The best I've heard (not first hand experience however) is berkleyDB. But unless you opensource it, it will cost you an arm and a leg to use due to licensing...see this http://www.oracle.com/technology/software/products/berkeley-db/htdocs/licensing.html for details.
ps. berkleyDB is not a relational database in case you didnt know.
Good comparison tool can be found here: http://www.jpab.org/All/All/All.html
Notice also the Head to Head DBMS/JPA Comparisons
Most things have been said already, but I can just add that I've used HSQL, Derby and Berkely DB in a few of my pet projects and they all worked just fine. So I don't think it really matters much to be honest. One thing worth mentioning is that HSQL saves itself as a text file with SQL statements which is quite good. Makes it really easy for when you are developing to do tests and setup data quickly. Can also do quick edits if needed. Guess you could easily transfer all that to any database if you ever need to change as well :)
I am a big fan of DB4O for both .Net and Java.
Performance has become much better since the early releases. The licensing model isnt too bad, either. I particularly like the options available for querying your objects. Query by example is very powerful and easy to get used to.
What criteria will you use to evaluate these ? If you don't know yet, then you don't need to decide right now. Try to make your application as database-implementation-agnostic as you can - providing the appropriate wrappers, data access objects etc., and make this decision when you have all the facts to hand and you have to decide.
If you're using relational databases and SQL then the above shouldn't be too hard (using JDBC etc). Make sure you have plenty of surrounding tests so that when you want to switch between databases, you can determine that your application's functionality remains the same.
I ran into the same issue some time ago. I didn't know which database to go for, so my first solution used Derby (or HSQLDB?), and I was later able to switch to HSQLDB (or Derby ? Can't remember which solution worked) once I'd determined where I had issues (relating to performance) and which solution would really work for me.
I have used Derby and i really hate it's data type conversion functions, especially date/time functions. (Number Type)<--> Varchar conversion it's a pain.
So that if you plan use data type conversions in your DB statements consider the use of othe embedded DB, i learn it too late.
Latest Derby Version data type conversions
I personally favor HSQLDB, but mostly because it was the first I tried.
H2 is said to be faster and provides a nicer GUI frontend (which is generic and works with any JDBC driver, by the way).
At least HSQLDB, H2 and Derby provide server modes which is great for development, because you can access the DB with your application and some tool at the same time (which embedded mode usually doesn't allow).
I guess I'm a little late (a lot late;-)) to this post, but I'd like to add Perst, an open source, object-oriented embedded database for Java &.NET. for your consideration. Perst is an open source / dual license embedded database for Java. The distribution is compatible with Google's Android platform, and also includes Perst Lite for Java ME. We've even built an Android benchmark and produced a whitepaper on the subject...you can take a look here: http://www.mcobject.com/index.cfm?fuseaction=download&pageid=581§ionid=133
All the best,
Chris
If I am correct H2 is from the same guys who wrote HSQLDB. Its a lot better if you trust the benchmarks on their site. Also, there is some notion that sun community jumped too quickly into Derby.
I realize you mentioned SQL browsing, but everything else in your question makes me want to suggest you also consider DB4O, which is a great, simple object DB.