Develop Reference

Java Persistence: Design with it in mind from the beginning or add in later?

I am doing things now like writing classes and their unit tests -- business logic. Without question, I will need to have something like JPA to allow me to store these classes and initialize the application from a database. I also know that I will need to do a lot of operations within a transaction.
My question is, does it make sense to implement the business logic first and then worry about persistence later or am I asking for trouble in this way -- perhaps instead I should be incorporating persistence in my design from the start: it might be very hard to add it in later? Or is there an approach where business logic can be totally ignorant of persistence? The reason I am guessing not is that the persistent classes need annotations.
Anyway, I could have been more succinct -- maybe the title says it all.
Cheers.

Isolating the implementation from a particular technology is a best practice. In general, you should be better off developing an application without preparing to use JPA.
For this, you can use a separate domain model for your business logic. The domain objects should be mapped to/from a persisteable representation at the boundaries of your business logic layer.
Domain driven design, clean architecture, hexagonal architecture (and probably some others) are different but closely related approaches that emphasize the separation of business domain from frameworks.
The primary benefit is a clean separation of concerns. You can reach a very good testability for your code with very fast tests that do not rely on the DB. You also can switch persistence technologies (going with in-mem DBs or flat files if you should so desire) with much less pain.
The downside is that you will have to define a boundary mapping between your domain classes and persistent classes.
Having said that, sometimes fully embracing a framework can have it's own benefits that have to be weighed against clean design. When creating a simple, one-off webapp, it can make sense to use JPA entities all the way - even using attached JPA entites for display in the UI - this is called 'transaction view'.
The expected benefit is simplicity - sometimes there is no use in introducing a 'business logic' layer if there is no logic to speak of.

Using DAO pattern

I am trying to use the DAO pattern in my multiple web app projects. I have three different web applications and they share two different databases. Each databases have number of tables.
Now I am wondering how I can make my program modular by using best practice. I am thinking of making:
DAO project which have two factory class for each database, DAO interfaces for each tables and DTO for each tables.
Then in each web app project I am planning to write implementation code for DAO interface and necessary utility class for getting and closing the connections.
Is this approach good? The doubt/problem i am having is with this design if I am going to ship any one of the project I have to ship DAO project also but that will contain unnecessary info about other databases.
Or will it be good to attach all necessary DAO in web app itself? If so then I have to write same DAO ode for each web app.
Hope anyone can provide me the clear path for this DB connection using DAO pattern.

In general, you're headed in the right direction by separating your concerns.
You mention the multiple web apps rely on the two databases. Does each web app rely on both databases? If so, I'd consider creating a single DAO project to encapsulate all the data access logic.
If it's more a mix and match (web app a uses db a, web app b uses db b, web app c uses a and b), I'd consider having two DAO projects, one per database, unless there's a lot of combined logic - that is, when an app uses both databases, it's doing joins between them [yes, I have had projects that do this].
I'd also recommend looking at an Object/Relational Mapping (ORM) framework such as Hibernate and/or a Dependency Injection framework such as Spring, which can help simplify the process of separating the various projects and then using them together.
You're clearly planning a pretty ambitious project, so taking advantage of existing frameworks to minimize recreating the wheel will let you focus on your specific problem domain.

Use JPA to access DB. If not possible then use JdbcTemplate (Spring)
EntityManager (JPA) is a kind of a DAO
DAO only where it makes sense (e.g. complex, reusable logic using an EntityManager)
Use pooled connections/ DataSources
DTO are usually only needed if your objects need to leave the JVM (e.g. remote EJB services, web services,...)
use EJBs for container-managed transactions
consider the Gateway pattern (a stateful session bean and an extended persistence context, see "Real World Java EE Patterns – Rethinking Best Practices" by Adam Bien) and just return the attached entity.

How does Spring fit into my application architecture?

I'm currently rebuilding an existing PHP application using Java. I therefore have an existing frontend GUI and an existing database schema that I'm working with.
Here is the technology stack I'm working towards:
jQuery, client-side
Wicket, front-end
Spring, ???
Hibernate, ORM
MySQL, database
Before reading about Spring in both Wicket In Action and the Hibernate documentation, I envisioned wiring the two together through my own business logic. I have experience with JBoss Seam, but am told that Spring is hardly comparable (although the documentation suggests otherwise, IMO). Short of adding a book about Spring to my reading list (I haven't found an appropriate one with good reviews yet), I'm at a loss.
What benefit(s) will Spring provide in this technology stack?
Subjective & optional follow up question: what reference material (book, website, etc) will get me started with the portion of Spring 3 I may utilize?

First, you can make your web application without Spring. But Spring will greatly facilitate things. Spring framework is a lightweight, non-invasive. Spring is like a kind of conductor. Among other things Spring helps you in:
To keep your objects loosely coupled. This will make your application more flexible and open to future changes
Powerful support for transactions through the AOP (Aspect Oriented Programming).
Object-relational mapping (ORM) integration module. Spring doesn’t attempt to implement its own ORM solution, but does provide hooks into several popular ORM frameworks, including Hibernate, Java Persistence API, Java Data Objects, and iBATIS SQL Maps. Spring’s Transaction management supports each of these ORM frameworks as well as JDBC.
The Spring MVC framework. Even though Spring integrates with several popular MVC frameworks, it also comes with its own very capable MVC framework that promotes Spring’s loosely coupled techniques in the web layer of an application.
A good book about Spring: Pro Spring

Spring, as noted in this review is non-invasive. It just wires your application components. And provides useful classes that make using other frameworks easier (JMS, JPA, etc). Spring doesn't force you to use its classes or interfaces anywhere.
What it handles is the creation of your components (objects), so that you can refer to a class' dependencies, without instantiating them. I.e. you say what your class needs, not how it is obtaining it. This makes the application very flexible.
That's in short - for more, read the linked article. It's not about the latest version, but that doesn't matter.

In addition to dependency injection, Spring offers features like declarative transaction management, simple integration with ORM, aspect-oriented programming support and many other nice things.
For documentation see Spring reference: http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html

#Dolph, in the simplest term, think of Spring as your application framework at the highest degree. This "framework" provides several "component buckets" where you can easily plug in different types of implementations. For example, for ORM, you may choose to use Hibernate over JPA or TopLink, for front end, you may choose Wicket over Struts or SpringMVC, and so forth.
The whole beauty of this (besides all the goodies stated in other posts) is it allows you easily swap out any of these implementations easily in the future. So, you can essentially rip out Hibernate one day and replace with TopLink, and it will never cause ripple effect to other components.
Another beauty of using Spring is your code becomes less clutter and has loose dependencies with other classes because you spend less time creating new objects yourself, Spring handles that for you. That said, you will quickly realize how easy for you to test your code because your API to be tested becomes very atomic. This is one primary reason why folks get discouraged in writing testcases, because they quickly realize that in order for them to test one API, they have to construct whole lot of things just to test that. Because of that, the whole process is brittle, imagine if you change that API, you need to reconstruct everything before testing it again.
Pro Spring book is good, mentioned by #JLBarros. I like Spring in Action very much. It is very very easy to read when I first got started with Spring. This is probably one reference book that I read from skin to skin.

Do aspects substitute repositories?

I started experimenting with Spring Roo just recently. It does a very nice job helping one build a domain model with integrated persistence rather quickly. As it adds persistence functionality in aspects, I started think about the following question:
Roo adds finders (load an instance of a class from the database which meets variable criteria) in an aspect to the actual class/entity. In DDD this is IMHO the responsibility of repositories. Repositories are explicit classes which show up in the design. Of course as an aspect the repository functionality is hidden in an entity and is pretty much invisible.
So here is the question: Is an aspect a real substitute for a explicit repository class? Are there any downsides to the Roo AOP approach?

Adding finders to your domain classes feels more natural from a user's point of view but it mingles your layers. Grails uses the same approach by adding static finder*() save(), ... methods.
Apart from the aestetics it might have practical drawbacks when not used in web application setting:
Your domain classes are now tied to your database. If you transfer these objects to rich clients via RMI or HttpInvoker the client cannot and often may not use the find* methods because there is no session / database connection available on the client.
I generally prefer allowing domain classes to reference service layer interfaces to prevent an anemic domain model (http://martinfowler.com/bliki/AnemicDomainModel.html). This has its own set of drawbacks but at least provides a clear boundary. On the client the concrete implementation behind a service interface can then just proxy all method calls to the server (or just use a synamic proxy with spring remoting or sth similar).
So to answer your question: It might be a substitute but you should be aware of the possible negative consequences which make your domain classes (i.e. your core business logic) less portable between systems.

This depends on how complicated your applications persistence layer is and how much control you have over it. If your application is simple enough to be implemented via JPA, then it all could be handled via Roo aspects. However if you are mapping legacy tables or need advanced DB stuff, then you may find yourself in a situation where Spring-JDBC is the only way out and in these cases a repository/dao model may still be useful.
I consider it logical inconsistent (and a break of layer responsibility) to be mixing two persistence models and so as most of my applications requires such advanced DB constructs I stick strictly with a repository model.

I think adding repository methods to domain objects is bad design. The right place would be static methods in the domain class. But domain objects and their management are two different things that should be separated. I would prefer domain objects and repositories.
I guess the motivation was to achieve something Rails/Grails like with Java.

How does ORM work under the covers? Also what is the best way to have persistent objects in Java?

How does ORM work? Are objects serialized into BLOBs?
In Java, is JDO still the way to go for this? What else is available? Seems like there was a lot of talk of EJB, direct object serialization, and JDO.

To answer your first question, here is an extract from Hibernate in Action, that says that there are various ways to implement ORM:
Pure relational
The whole application, including the
user interface, is designed around the
relational model and SQL-based
relational operations. This approach,
despite its deficiencies for large
systems, can be an excellent solution
for simple applications where a low
level of code reuse is tolerable.
Direct SQL can be fine-tuned in every
aspect, but the drawbacks, such as
lack of portability and
maintainability, are significant,
especially in the long run.
Applications in this category often
make heavy use of stored procedures,
shifting some of the work out of the
business layer and into the database.
Light object mapping
Entities are represented as classes
that are mapped manually to the
relational tables. Hand-coded SQL/JDBC
is hidden from the business logic
using well-known design patterns.
This approach is extremely widespread
and is successful for applications
with a small number of entities, or
applications with generic,
metadata-driven data models. Stored
procedures might have a place in this
kind of application.
Medium object mapping
The application is designed around an
object model. SQL is generated at
build time using a code generation
tool, or at runtime by framework code.
Associations between objects are
supported by the persistence
mechanism, and queries may be
specified using an object-oriented
expression language. Objects are
cached by the persistence layer. A
great many ORM products and homegrown
persistence layers support at least
this level of functionality. It’s well
suited to medium-sized applications
with some complex transactions,
particularly when portability between
different database products is
important. These applications usually
don’t use stored procedures.
Full object mapping
Full object mapping supports
sophisticated object modeling:
composition, inheritance,
polymorphism, and “persistence by
reachability.” The persistence layer
implements transparent persistence;
persistent classes do not inherit any
special base class or have to
implement a special interface.
Efficient fetching strategies (lazy
and eager fetching) and caching
strategies are implemented
transparently to the application. This
level of functionality can hardly be
achieved by a homegrown persistence
layer—it’s equivalent to months or
years of development time. A number
of commercial and open source Java ORM
tools have achieved this level of
quality. This level meets the
definition of ORM we’re using in this
book. Let’s look at the problems we
expect to be solved by a tool that
achieves full object mapping.

ORM = Object Relational Mapping, attributes of the objects are mapped to columns in the realational database. That mapping is arbitrary, so that could be done to blobs, in practise what is most useful tends to natural mappings - Strings to Varchars, int to integers etc.
JPA is the place to look for a standard for ORM. JPA replaces the EJB CMP approach, which was found to be cumbersome. JPA allows you to express the mapping as Java annotations and also allows the mappings to be specified in configutration files, when supporting multip[le databases the latter can be useful.
JPA has a query language so that you can construct queries against object attributes.
JPA is supported by the major App Server vendors and also by products such as Hibernate.
I found JPA pretty nice to work with, more so than EJB CMP.
I would recommend still using EJB Session Beans facades for transaction mamangement and security - the annotation-based approach makes EJB 3 way easier to use than EJB 2, minimal coding overhead.

JDO is actually standard ORM too, and provides a more complete a specification than JPA (1 + 2). JPQL is more focussed on RDBMS concepts and hence mimics SQL. JDOQL follows Java syntax so is more object based. Depends if your app is ever considered to go away from RDBMS. If so then JPA is not the way to go. If it is solely for RDBMS then JPA is definitely a consideration.
Whether objects are serialized into BLOBs depends on your configuration. You can do that for complex object types if you wish, but then they won't be queryable. If you instead persist them in a native form then you can also query them, leading to more efficient apps.
--Andy (DataNucleus - JDO and JPA persistence)