I want to have two servers running, both with a small local database accessed via JPA. The 2nd server is a hot standby ready to take over.
How do I keep the databases in sync, and how do I handle caching etc at the JPA layer?
I am curious to know if anyone has tried this, and what technologies they used?
For caching you can use Ehcache. It comes with Hibernate by default if you are using that JPA Persistence Provider. You will need to specify which entities are cacheable using annotations and you can also configure the caching policy for specific entities through XML.
For database replication I wouldn't go for a JPA/JDBC based solution apart from basic IP failover which could be supported by your driver depending on what the database is. Replication is something the database should be responsible for not the application. Most of the popular databases (Postgresql, Mysql, Ms-SQL, Oracle, etc.) nowadays support some kind of replication or clustering, so depends on what you're using.
Related
I have a SpringBoot application running on a multitenant architecture.
I've two databases Admin and Client (both are MySQL) and both these databases have a User table
Client can add users to the User Table but I need them to get synchronized in the User table of Admin database.
Is there a way I can achieve this?
I've read about flyway migrations but I think it works more on database schema changes and not values.
Please ignore my mistakes as this my first question, any help would be appreciated.
This looks like a solution to your problem:
SymmetricDS is software that replicates relational database tables between multiple databases. It can also be used to replicate files and directories between multiple hosts. It uses a light-weight, web-based protocol to send and receive data, which makes it easy to work with firewalls. Replication is done in the background asynchronously, allowing data changes in offline mode. It supports most commercial and open source database platforms.
How does it work?
Triggers are installed in the database to guarantee that data changes are captured. This means that applications continue to use the database as usual without any special driver software. The triggers are written to be as small and efficient as possible. Routing and syncing of data is done outside of the database in the SymmetricDS process.
SymmetricDS supports many databases and can replicate across different databases, including Oracle, MySQL, MariaDB, PostgreSQL, MS SQL and many more.
https://www.symmetricds.org/docs/faq
You need to create some event from the flow where client adds user to the User Table.
If this "client" flow is in same java service then you can make use of Spring's Asynchronous Event Handling or have a method(which does the data copy) marked with #Async. This ensures the data copy happens in separate thread.
If the "client" flow is in different java service, then any publisher-subscriber model can be used (some opensource frameworks available are kafka, rabbitmq etc).
Now to connect to two datasources at the same time, Spring's RoutingDataSource will come handy in this scenario as it works on "lookup key" to choose the datasource. Or else you can hardcode two datasource beans in your config (since it is fixed in your case).
I have a web application which receives requests to save orders in a database. I want to write to 2 different databases - one Cassandra instance and one PostgreSQL instance. I am using plain Java and JDBC (with apache DBUtis) with a lightweight web application library at the front.
What I am unsure about is how to implement transactionality across the two databases, i.e. if a write to one of the databases fails, then rollback the other write and put an error message in the error log.
Are there any mechanisms in Java to implement this? I know of such a thing as two phase commit, is that what I would be looking for here? Are there any alternatives?
Both Cassandra & PostgreSQL support linearizability and compare-and-set (CAS), so you can implement transactions on the client side.
If you want Serializable Isolation level then you should take a look on the Percolator's transactions. The Percolator's transactions are quite known in the industry and have been used in the Amazon's DynamoDB transaction library, in the CockroachDB database and in the Google's Pecolator system itself. A step-by-step visualization of the Percolator's transactions may help you to understand it.
If you expect contention and can deal with Read Committed isolation level then RAMP transactions by Peter Bailis may suit you. I also created a step-by-step RAMP visualization.
The third approach is to use compensating transactions also known as the saga pattern. It was described in the late 80s in the Sagas paper but became more actual with the raise of distributed systems. Please see the Applying the Saga Pattern talk for inspiration.
We are making an application which is machine dependent.
Now we deploy our application on several client machines but problem is that every client have database from different vendors. Currently we are facing conflicts between SQL Server and Oracle.
Our Application built on Oracle database and now we want to access SQL Server. Is there any way to do it as I am little amateur with databases and I dont want to change the queries and configuration settings for different databases.
What you need is something that provides a layer that provides database independence. There are a variety of ways of doing this.
You could use a ORM (Object Relational Mapping) technology such as JPA (with Hibernate being the prime example). A typical JPA implementation has backends for a range of different databases.
You could use an existing product that supports database independence by (roughly speaking) mapping SQL statements between different SQL dialects.
Some people suggest using ODBC.
You could implement a DAO API with different DAO implementation classes for each backend database. If you stick to SQL-92 conformant DDL and DML as much as possible, there is a good chance that there will be a lot of commonality between the DAO implementations. (JDBC provides database independence at the API level, provided you don't use vendor specific extensions. I recall having problems with Oracle's JDBC drivers doing things in non-standard ways ... but they may have gotten their act together now.)
... I dont want to change the queries and configuration settings for different databases.
If you use an ORM and its query language, you won't have to. If you implement your database stuff using SQL and JDBC, it is largely a matter of sticking to the SQL standard and the standard JDBC API methods respectively.
Related question:
DB Independent Java Programming - Suggestions?
Recommended approach is to use ORM tools like Hibernate. if that is not possible then use StoredProcedures ( make sure that it uses only normal PL SQL and no database specific features) for database operations
Create Linked Server and use openquery if necessary.
http://technet.microsoft.com/en-us/library/ms188279.aspx
I am working on a Java application that requires horizontal partitioning of data in different PostgreSQL servers. I would like to use a JPA framework and Spring for transaction management.
The most popular frameworks for sharding data with JPA seem to be Hibernate Shards, which appears to be no longer in development, and OpenJPA Slice, which does not support virtual shards (one of my requirements). Are there any other options that I'm missing, or a way to get around the OpenJPA limitation?
Thanks in advance for your input!
You can have a look at Sharding-JDBC, it is a JDBC driver for shard databases and tables, can adapter for any ORM frameworks.
Have you looked into EclipseLink? It supports JPA2 and horizontal sharding.
The documentation says it supports Sharding over multiple database instance
I'm developing a Java EE application to run on Glassfish:
Database (javaDB, MS SQL, MySQL or Oracle)
EJB layer with JPA (Toplink essentials - from Glassfish) for
database access
JSF/Icefaces based web UI accessing the EJB layer
The application will have a lot of concurrent web client, so I want to
run it on different physical servers and use a load-balancer. My
problem is now how to keep the applications synchronized. I intend to
set up multiple servers, each running Glassfish with my EAR app
installed. Whenever on one of the servers data is added to or removed
from the database (via JPA, no direct SQL queries), this change should
be reflected in the JPA layer on the other servers.
I've been looking
around for solutions to this, but couldn't find anything I really like
(the full Toplink from Oracle claims to have a solution, but don't
know). Doing a refresh before every access to a JPA entity could work,
but is far from efficient.
Are there any patterns, libraries, ... that could help here?
Whenever on one of the servers data is added to or removed from the database (via JPA, no direct SQL queries), this change should be reflected in the JPA layer on the other servers.
I don't understand what state is maintained and need to be updated at the "JPA layer". Entity managers are isolated and the way to deal with concurrency in JPA is to use locking (optimistic or pessimistic, the former scaling better). Maybe you could clarify what you need exactly and what you saw.
See also
Java Persistence/Locking