may i know in 'nosql' there is limitation just like bigtable where we should 'denormalized' our table/entity ?
any api wrapper that allow we to write code once and can be used for google app engine bigtable and nosql ? (something like hiberanate)
Yes, for example in MongoDB you don't have joins since it is non-relational, so it does change how we store and browse the data.
As MongoDB is non-relational (no
joins), references ("foreign keys")
between documents are generally
resolved client-side by additional
queries to the server. Two conventions
are common for references in MongoDB:
first simple manual references, and
second, the DBRef standard, which many
drivers support explicitly.
It seems that the consensus is to denormalize and duplicate to accelerate the reads to avoid the cost of joining distributed data all toghether, with the join and merge logic done on the application level.
As to whether it is an absolute requirement to denormalize the database, I am not sure (other SO members can probably enlighten us). But I think the database should be modeled with these "limitations" in the mind along with a good study of how the data is going to be queried. This should give the least impedance to the process.
See Also:
Bigtable database design theory
GAE - How to live with no joins?
Any API wrapper that allow we to write
code once and can be used for google
app engine BigTable and nosql ?
(something like Hibernate)
JDO is datastore-agnostic, so it might just provide what you want to some extent.
Seems there are lots of recent projects to use JDO and JPA with "NoSQL" products.
See:
Datanucleus-Cassandra
Datanucleus-Cassandra-Plugin
Any API wrapper that allow we to write code once and can be used for google app engine BigTable and nosql ? (something like Hibernate)
While abstraction libraries definitely help portability, you have to take into consideration the particular platform you're running on. If you're going to go with Google App Engine, you have to be aware of the incurred startup costs inherent with additional abstraction libraries.
You should weigh the pros and cons of using something like JDO or JPA. Also take a look at the Objectify library that offers a more native interface that has the downside of being coupled to the App Engine Datastore.
Related
Currently I am migrating databases from Oracle to MySQL. I mainly use Java to send queries to the database using JDBC. In the process of migrating, I need to change a lot of my queries in the Java code (the queries are hard-coded) as they will not work in MySQL.
I want to be able to recode my queries in such a way that I can easily switch between the databases if problems arise; I am changing all my queries to standard SQL but there are areas where this is not possible. I am thinking of having two versions of the queries, one for Oracle and one for MySQL so I can switch between both (I will have two versions temporarily just to see if MySQL can cope with our needs). However this seems like a terrible idea - does any one have any advice on a better way they would do this?
You have a bunch of options.
Firstly, many people now use Object-Relational Mapping (ORM) tools to connect applications to SQL databases. These come in a variety of different flavours - Hibernate is popular - and allow you switch between databases at very little cost. However, they do have a fairly steep learning curve. Inexperienced developers often struggle with performance problems in ORM applications.
If you stick with "traditional" JDBC, I suggest you take the body of the SQL out of the Java code, and treat it like a resource. As Henry suggests, you could use property files, and use parameter placeholders (ideally named placeholders, using the Spring template). While this does spread the code for a given piece of functionality into two files, it makes it easy to quickly refine the SQL and test new versions.
One possibility is to store the queries in a properties file. You would have one for Oracle and one for MySql.
I must add - ORM is good advice and will work... when you are starting with a fresh application and you can design your application to work on a domain model.
In this case however there is an existing application that invokes a great number of SQL queries. ORM based queries (HQL, JPQL) translate to SQL just fine; SQL does not by definition translate to the ORM layer however, major changes will be needed to make it a more object oriented approach to the data.
The problem will still persist even when you do manage to work in an ORM layer. There is already a major difference between MySQL and Oracle in how primary key generation works for example; MySQL uses auto-numbering where Oracle uses a sequence. Likely you already have an existing datamodel that you need to reverse engineer into the ORM layer code; it isn't going to be cross-database code.
I want to implement a hotel booking system with Play framework 2.0 (Java). The app will do hotel booking with no banking transactions (credit card will be provided only as identification method to prevent fraud), user will select the desirable room and date range and the app will make the booking and update the rooms availability.
I am considering mongo over mySQL for performance reasons and also because my models will have translatable fields in a few languages which will have a lot of joins if it is in mySQL.
For Availability check I don't quite figure it out, if it is simpler in mySQL rather than mongo.
Is MongoDB suitable for that kind of web app, or I am in the wrong path?
Two simple statements :
Stick with what you know. If you know plain SQL, go for that.
Premature optimization is the root of all evil. If even before starting your application, you are already thinking of optimization, there is something terribly wrong with your way of working.
You can use mongoDB for this kind of application. I personally have some security concerns when it comes to sensitive data (i.e. credit card infirmation) as deleting the information does not automatically mean that it is unrecoverable.
From wikipedia on MongoDB:
E-commerce. Several sites are using MongoDB as the core of their ecommerce infrastructure (often in combination with an RDBMS for the final order processing and accounting).
There are some very intresting articles on pros an cons on stackoverflow Pros and cons of MongoDB?
If you want to learn a new NoSQL technology by all means go for it, but if you want to play it safe stick to the mysql solution. These applications (booking) often work well with BI solutions/data mining and this fact alone would make a noSql approach a no-go for me.
I've experience with Toplink to translate objects to database and vica versa. But this was all part of a JSP site and now I did some EJB stuff with it to. Now is my question: is it good to work with stuff like Toplink in a Java Desktop application or is it more common to use native sql stuff from Java?
Maybe some experience of prof. developpers might be good. I need to develop a seriously application for a client. I'm doing it in Java and I'm gonna store the data in a database.
Thanks
ORM is nice if your data model is well structured, not overly complex and, most of all, if you have control over it.
Legacy databases or poorly modelled ones are harder to be represented with ORM, and doing so would be strongly discouraged, as your application would add further complexities over those implied by the model itself.
If you are comfortable with some ORM tool such as Hibernate and your database is fairly well done, go for it. They sure save you a lot of boilerplate code and have some nice query optimization code under the hood. Otherwise, you may want to use JDBC directly or some other framework to simplify JDBC use but still using plain SQL. For such situations I recommend MyBatis.
TopLink (and EclipseLink/JPA) work just as well in a desktop application as in a server side application. In fact TopLink has been around since the 90s with client-server Smalltalk apps before the server side was popular.
It's dependent on your use cases
ORM technologies can nicely abstract away database specifics and allow you to concentrate of the domain model. However, there are circumstances where using an ORM layer is not appropriate (extremely large data sets can cause performance issues for example, database schemas that are difficult to map to objects is another).
I would recommend using a JPA compliant technology such as Hibernate. That way you're using the ORM that implements a Java standard and you can more or less swap in and out implementations.
For everything else then JDBC is a flexible friend
depends on database volume too. For databases with huge data try using hibernate. It might be of great help rather than writing JDBC code
Working on a app built on GAE in java. Having trouble figuring out whether i want to use DatastoreServices and the Entity class, or make class's for my objects, and using the PersistenceManager.
Can someone explain the differences of these two services?
If you don't have a fixed reason to use either of those, take a look at Objectify. It's a much better and more usable library for storage.
The Datastore services from the SDK are low level services that let you talk directly with Bigtable RPC. You'll be writing code that translates into Bigtable's data formats and API, and calling Bigtable's RPC methods.
The JDO specification and its persistence manager are a Java standard that deals with creating data classes and their storage and retrieval. It's one level of abstraction higher than the direct datastore services. Google has implemented the JDO spec to run on GAE, so you can use it if you want to.
Objectify is an alternative to the JDO that isn't an enterprise Java standard, but is a lot more easy and fun to use. It follows the Python API quite closely, which makes it quite nice and concise.
My vote for DatastoreService because PersistenceManager (JDO) consumes more resources ( = more money) and it is slower.
http://gaejava.appspot.com/ — here you can compare them. Try to run this test a couple times.
I found an article from 2008 discussing how to call Java code from MySQL. There were a lot of caveats and disclaimers because the process involved working with an experimental branch of MySQL.
For a project I have in mind, it would be very useful to be be able to access Java libraries within MySQL, analogous to Oracle's Java Stored Procedures. Does this capability now exist as a standard feature of MySQL? If not, what open source RDBMSs support something similar to Oracle's Java Stored Procedures?
PostgreSQL supports pluggable procedure languages, and a project exists to extend PostgreSQL with PL/Java as the language.
I don't recommend putting too much code in the RDBMS. Tools to develop, test, and debug code in the application layer are better than tools for code in the RDBMS.
Also many developers don't understand that code inside the RDBMS should obey transaction isolation. They try to send emails from triggers and so forth. I think code with side effects should be in the application layer, so you don't create phantom effects (e.g. an email may notify of a database change, even though the change was rolled back).
If you can use HSQLDB then you can call java methods directly from SQL: http://hsqldb.org/doc/2.0/guide/sqlroutines-chapt.html#N1240C
I fully agree with Bill, but I can imagine business rules being stored (not processed) in the database. I'm thinking of drools here. The engine would be in the application, but the rules could be in the database with a management front-end.
Such a beast would be interesting for scenarios where not only the parameters change, but also the formulas can change.
It is difficult to give good advice based on the limited information that you have provided so far. However:
... the example involves a graph-based data type (chemical structures) that can't be matched to a query using built-in MySQL functions. The Java library would convert the query and contents of a text field into an in-memory object that can by matched. Keeping this logic in the DB layer would, for example, keep joins within the database, which seems like where they belong. That's the idea, at least.
I don't think I would use database-side Java in MySQL for this. Instead, I think I would consider the following options:
Use an object-relational mapping such as JDO or JPA (for example using Hibernate) to deal with the mapping between your graph-based data model and what the database provides. You don't necessarily have to use an RDBMS as the backend, but that is probably the best place to start ... unless you've already found that this is a performance issue.
Take another look at your data model and data access patterns. See if you can figure out some transformation that allows your application's main queries to be implemented as (efficient) table joins without resorting to server-side application logic.
If you do need to use server-side application logic (for performance reasons!) stick with the mechanisms supported by your RDBMS. For example, in Oracle you'd use PL/SQL and PostgreSQL you have a number of options. Be prepared to switch to a different RDBMS that better suits your application requirements.
I (personally) would avoid depending on an experimental branch of some database:
Consider what happens if the experimental branch is not merged back into the main branch. You would be stuck with your code base depending on a branch that is not supported, and is likely to stop being maintained and fizzle out.
Using a (currently) unsupported RDBMS branch will be an impediment to other folks who might want to use your software.
Now obviously, if the long term viability of your software is not a primary concern, you could choose to ignore this advice. But it probably matters to someone; e.g. your research supervisor.
I realise that this is quite an old article, but it bears updating. The ability to call java from a database trigger is is part of the "SQL Routines and Types for the Java Programming Language" (SQL/JRT) standard.
Read more about this on Wikipedia at https://en.wikipedia.org/wiki/SQL/JRT.
Amongst the compliant database engines are..
HyperSQL: http://hsqldb.org/
Oracle: https://www.oracle.com/database/