This question might look too broad, but I could not find a solution or any idea from existing sources in the web. There might not be exact answers for this, what I need is your suggestions on how to get this implemented.
My application has its own datasource for CRUD operations. Plus I have a new micro service to execute scheduled jobs. The idea is to maintain only a single database for both the services.
The existing service has been configured to use a connection pool for its db transactions. properties like max number of active datasources, max idle datasources at a given time etc have been configured for that service.
Now, my question is about maintaining the pool when quartz connects to the same database to query its own tables. At first place, is this possible? or is it always better to keep a separate database for quartz tables. Or Else, can I configure these two services to access same database flawlessly by doing configurations? (Remember that these two db operations happen in two separate microservices)
Related
I'm trying to understand the use of Java XA Datasource.
But I can't still figure when to use it, and when not to use it.
I read that XA Datasource used when we use two databases.
But I'm not sure what is the meaning of two database.
For example:
I had two layer of classes (Service and DAO)
A method in service layer annotated as a transaction, invoke two methods in DAO.
Each method in DAO open new connection to database and close it in the end of the method.
If I use one instance of database, and each method in DAO write to different table, do I have to use XA Datasource? since transaction occured in service layer but only in one instance database
Systems such as databases, but also for example queueing systems that you use through JMS have the concept of transactions. A transaction is treated as a unit of work; at the end of doing the work, such as inserting, updating or deleting records in the database, you commit the transaction and then the database definitively does the work; or you rollback the transaction and then everything done in the transaction is cancelled.
In some cases, your software has to perform operations over multiple different systems. For example, you might need to insert data into multiple databases, or insert something in the database and put a message on a queue.
If you want to do such combinations of operations as if they are in one transaction, then you need a distributed transaction system - a system that can combine the transactions of the different systems into one. That way you can write your code as if it's running inside a single transaction; the distributed transaction system automatically commits or rolls back the transactions in the underlying systems.
To make it more concrete: Suppose that you insert a record in a database, and put a message on a queue, and you want to do this inside one transaction. When something goes wrong with putting the message on the queue, you also want the database transaction to be rolled back, so that you don't have a record in the database but not a corresponding message on the queue. Instead of manually keeping track of the transactions of the database and the queue system (including handling all combinations of possible errors), you can use a distributed transaction system.
XA is a standard for working with distributed transactions. You can work with XA transactions in Java through the Java Transaction API (JTA). Java EE servers have support for this built-in. If you're not using a Java EE server, then you can use a separate library that implements JTA such as Narayana or Atomikos.
Each method in DAO open new connection to database and close it in the end of the method.
Normally this isn't how you should write DAOs. Opening a database connection is a relatively slow operation; if you open a new database connection for every method that you call in a DAO, your program is most like going to run slowly. You should at least use a connection pool, that manages a number of database connections and allows you to reuse already open connections.
If I use one instance of database, and each method in DAO write to different table, do I have to use XA Datasource? since transaction occured in service layer but only in one instance database
If your DAO methods each open their own connection, then they will run in separate transactions. Whether this is a problem or not depends on what your application needs to do. An XA datasource is not the solution to make them run in one transaction. Instead, you should both let them use the same connection and transaction.
XA transactions are really only useful if you are using multiple database systems or other systems, and you want to be able to perform transactions that span across these systems.
My project connects to a database using hibernate, getting connections from a connection pool on JBoss. I want to replace some of the reads/writes to tables with publish/consume from queues. I built a working example that uses OracleAQ, however, I am connecting to the DB using:
AQjmsFactory.getQueueConnectionFactory followed by createQueueConnection,
then using createQueueSession to get a (JMS) QueueSession on which I can call createProducer and createConsumer.
So I know how to do what I want using a jms.QueueSession. But using hibernate, I get a hibernate.session, which doesn't have those methods.
I don't want to open a new connection every time I perform an action on a queue - which is what I am doing now in my working example. Is there a way to perform queue operations from a hibernate.session? Only with SQL queries?
I think you're confusing a JMS (message queue) session with a Hibernate (database) session. The Hibernate framework doesn't have any overlap with JMS, so it can't be used to do both things.
You'll need 2 different sessions for this to work:
A Hibernate Session (org.hibernate.Session) for DB work
A JMS Session (javax.jms.Session) to to JMS/queue work
Depending on your use case, you may also want an XA transaction manager to do a proper two-phase commit across both sessions and maintain transactional integrity.
I was also looking for some "sane" way how to use JMS connection to manipulate database data. There is not any. Dean is right, you have to use two different connections to the same data and have distributed XA transaction between them.
This solution opens a world of various problems never seen before. In real life distributed transactions can really be non-trivial. Surprisingly in some situations Oracle can detect that two connections are pointing into the same database and then two-phase commit can be bypassed - even when using XA.
I am developing a tool that receives different connection parameters to test values in different databases (a plugin for Nagios in jNRPE that keeps an open connection to different databases).
Because the configuration is dynamic (there could be more databases or they can be removed) I cannot have a configuration file.
I want to know if I should have an instance of C3P0 per database or can I use the same instance and just change the URL each time I ask for a connection?
The code is at github:
https://github.com/angoca/db2-jnrpe/blob/master/src/main/java/com/github/angoca/db2_jnrpe/database/pools/c3p0/DBCP_c3p0.java
If not, how can I get multiple pool for multiple databases dynamically?
You'll need a different c3p0 DataSource for each JDBC url. A Connection pool must contain homogeneous Connections: all checked out Connections must be equivalent from a client's perspective. If Connections from multiple databases were included in the same pool, clients would have no way of specifying or knowing which db they were communicating with.
(If you are replicating, say, a read-only DB and you really want Connections from multiple sources to live in a single pool, because they are guaranteed to be equivalent from a client's point of view, you could do that by defining a custom, unpooled DataSource that round-robined or randomly chose a replicant, and then pooling the DataSource via c3p0's DataSources factory.)
It is very easy to dynamically create and configure c3p0 DataSources. See example code here.
If you capture your dynamic config as a map of c3p0 property names to values, there's also an alternative, more concise way to get a DataSource with that configuration.
I have used MySqlDataSource for in jdbc connectivity.I have used following code
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
Also i have searched there is an option of Configuring a MySQL Datasource in Apache Tomcat.
Is there any performance difference between these two? which one is best to use?
Configuring Datasource in tomcat will help you to share same data source between applications running in same tomcat. that Datasource will be managed by container (tomcat in your case).
while the Datasource created in code will be created by your application and can be used by that application only.
So if you have multiple application running on tomcat and accessing same data source, that configuring Datasource in tomcat will be good approach and have performance factor because only one data source is created and not having separate connections for each application
But if you have only single application that the first approach you have used is good one
They both use the internally the same driver, i dont think the performance is much different here, i guess if you need to access teh database only at that place and the enduser isn't supposed to use his own authentication you may use it directly from java, but if you will need the connectivity on different places it could be helpful to configure this using apache configuration, specially that if anything changes like database server, user name or whatever you don't need to get in the code to change it, this could be very important if end users have to set their own configurations.
The improvement of configuring a pool of Connections (as the one provided by tomcat) is mainly that you will actually create and close a lot less of connections.
When using a pool, when you request a Connection to a pool it will look if it has any connection already created and available for reuse and, if it has, it will provide you with it (instead of creating a new Connection, which is a heavy operation). You must still close() a Connection provided by Tomcat so Tomcat knows that it can now reuse when it is requested again.
Additionally, the advantage of the pool is that your code does not need to know the configuration data for the Connection. He just requests a Connection from a given pool and the sysadmin configures it, allowing for greater flexibility (the sysadmin does not need to know how to configure your app, just how to configure the Tomcat which is fairly more standard).
I work on a project where the design is to have a database per customer. I have managed to read in Tomcat documentation and establish a connection pool. Since the databases are created automatically by the application when a customer registers, I needed a way to make a dynamic pool. the following post on stack overflow gave the hint of what is required:
Using dynamic Datasource with Tomcat
and I found on the web a page showing how to do this:
Writing and using a Tomcat ObjectFactory
I still have few questions:
1- who takes care of managing the connections in the pool ?
in the datasource declaration in context.xml, we define some parameters that control various aspects like max idle time and max number of connections, etc. does tomcat handle this or do I have to take care of it in the implementation that i make when coding a class that implements the ObjectFactory interface ?
2- As I understand, when connections are closed, they return to the pool. suppose that I make a dynamic pool that has maximum 20 connections. and then 30 customers log in. so the pool returns 20 connections for the first 20 customers. when those connections are closed, they return to the pool and the next 10 customers will ask for connections to different databases than the ones already found in the pool. and the question is: will the pool automatically drop 10 connections and add new ones with the correct databases ? in if this is the case, does this really have an advantage over opening a connection upon request without a pool ? (asked with the thought of a site that will be heavily loaded. the amount of connection drop if i guessed correctly might be very high that it wouldn't justify what the pool is there to do: minimize creating connections from scratch)
3- I also wanted to ask if it is ok to implement it myself (considering I am new to this) or if i should find a library that does this (in case its not a task for the beginner in this area).
4- are there any alternatives for connection pools for managing connections to multiple databases that are added dynamically in an efficient way? (in case datasource will not help) what are the approaches used in similar situations?
Thank you in advance for your effort.
In declaring a datasource you can specify a datsource class, such as the Oracle datasource. It uses the parameters specified to manage the datasource for you. It is best to let the datasource manage itself. I have tried to write one and it is not simple.
I think the datasource will not be adequate for you, I could be wrong. A datasource requires a database url, username and password that will be fixed for the lifetime of the datasource.
Having different databases for different users will break this.
Look at the reference you gave above, the answer mentions using a resource which is different from a datasource.