Should I define datasources in application or in app server? - java

I've developed applications (running on Jboss server) with two different teams. One team had datasource configuration inside of the application WAR file and another one had it inside of the application server's standalone.xml. And I'm not sure which approach is better.
So, here are some advantages that I've found in defining datasource inside of the server's standalone.xml.
Defining datasource in server's standalone.xml is more secure than in war file. If the database connection credentials stored in the server's standalone.xml, which is almost never modified, it is safer than having the password inside of the war file, which is often transferred from developer's machines to the server and database configuration is spread by all developers computers.
By having datasource in standalone.xml, we can make war file smaller, as JDBC driver could be installed as a module and can be removed from war file. In addition, loading JDBC as module is more efficient as from classpath.
We can put datasource inside of the standalone.xml if we don't want the application development team be aware of database connection settings.
By having datasource configurations in application WAR file, I see the following advantages, which are:
Development team doesn't have permission to change Jboss configuration files in environment where Jboss is running. So DB connection only can be defined in application.
It is useful in development state, when developers often need to switch between different database connections. For example, the developer can specify connection when building a WAR file.
So I'd like to know if there are another advantages in both approaches. And on your opinion which approach is better?

In addition to the points noted in the question, another advantage in having the datasources outside the application is that it allows for using the same war file in different regions. This would allow the team to have different databases for different environments, like Test, Perf and Prod while using the same war file.
You can do a deployment once and then the war file which has been tested by your QA team can be promoted to the production region. This would make sure that no untested code goes into higher regions while avoiding the need for SCM forks and code freezes.

I favour having the datasource exposed by the application server with a caveat..
You need your development team to at least know their way around with your application server or to have at least access to the jboss console to review the configuration or change it..
Reason being that for example they need to monitor the connection usage of the datasource connection pool.. Since you're speaking about jboss, I don't know if the "live" bean for the datasource with jboss AS exposes the same information natively of for example oracle ucp (ucp .getStatistics is a godSend for more than one reason..).
Consider that EVEN if you internalize the datasource in xml, using the concept of profiles you can make some field of the xml being "filled" with some specific value in one or another property file based on the profile the application is loaded with..
e.g with spring you can surely do
<beans profile="local">
<bean id="dataSource" class="oracle.ucp.jdbc.PoolDataSourceFactory" factory-method="getPoolDataSource">
<property name="URL" value="jdbc:oracle:thin:#db00-ccea.labucs.int:1521:CCEA"/>
<property name="user" value="myuser_DCI"/>
<property name="password" value="mypassword_DCI"/>
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleConnectionPoolDataSource"/>
<property name="connectionPoolName" value="${name.connection.pool}"/>
<property name="minPoolSize" value="5"/>
<property name="maxPoolSize" value="1000"/>
<property name="maxIdleTime" value="3000"/>
<property name="maxStatements" value="3000"/>
<property name="validateConnectionOnBorrow" value="true" />
<property name="inactiveConnectionTimeout" value="3000" />
<property name="connectionWaitTimeout" value="3000"/>
<property name="abandonedConnectionTimeout" value="3000"/>
<qualifier value ="dataSourceDCI" />
</bean>
<orcl:pooling-datasource id="dataAltSource"
url="jdbc:oracle:thin:#db00-ccea.labucs.int:1521:CCEA" username="some_OWN" password="some_OWN"/>
<util:properties id="flyway.prop" location="classpath:config_local.properties"/>
</beans>
meaning on local profile load the properties from the config_loca.properties file inside the classpath
and have as well
<beans profile="qa">
<bean id="dataSource" class="oracle.ucp.jdbc.PoolDataSourceFactory" factory-method="getPoolDataSource">
<property name="URL" value="jdbc:oracle:thin:#db00-ccea.labucs.int:1521:CCEA"/>
<property name="user" value="myuserQA_DCI"/>
<property name="password" value="myuserQA_DCI"/>
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleConnectionPoolDataSource"/>
<property name="connectionPoolName" value="${name.connection.pool}"/>
<property name="minPoolSize" value="5"/>
<property name="maxPoolSize" value="1000"/>
<property name="maxIdleTime" value="3000"/>
<property name="maxStatements" value="3000"/>
<property name="validateConnectionOnBorrow" value="true" />
<property name="inactiveConnectionTimeout" value="3000" />
<property name="connectionWaitTimeout" value="3000"/>
<property name="abandonedConnectionTimeout" value="3000"/>
<qualifier value ="dataSourceDCI" />
</bean>
<bean id="dataAltSource" class="oracle.ucp.jdbc.PoolDataSourceFactory" factory-method="getPoolDataSource">
<property name="URL" value="jdbc:oracle:thin:#db00-ccea.labucs.int:1521:CCEA"/>
<property name="user" value="myuserQA_OWN"/>
<property name="password" value="myuserQA_OWN"/>
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleConnectionPoolDataSource"/>
<property name="connectionPoolName" value="${name.connection.pool}"/>
<property name="minPoolSize" value="5"/>
<property name="maxPoolSize" value="1000"/>
<property name="maxIdleTime" value="3000"/>
<property name="maxStatements" value="3000"/>
<property name="validateConnectionOnBorrow" value="true" />
<property name="inactiveConnectionTimeout" value="3000" />
<property name="connectionWaitTimeout" value="3000"/>
<property name="abandonedConnectionTimeout" value="3000"/>
</bean>
<util:properties id="flyway.prop" location="file:///${prefix.iam.dir}/${filecert.iam.path}/external.properties"/>
</beans>
thus in your QA environment or other non-dev environment you instead refer to the external xml file and not to the one integrated in the war..
You can even include username and password to be "filled" via the internal//external properties file to enhance the security if you're concerned.

In order to properly verify your application works, you should try in a staging server before sending it to production.
In that scenario, the war file you install into production, should be the same that you tested, so you shouldn't need to change anything for te application to work in a different environment with different database connections.
So, the database configuration shouldn't be in the war file but in the application server. Additionally, you make the system administrator's live easier because they don't need to manipulate (uncompress and change) your war to install it in a server.
In the very early development live of the application, it can be useful to add the database (and any other development configuration) to reduce the time a developer can put his/her hands on the project and starts programming without having to configure the application in a development application server.

For me, the number one benefit of having all data-source configuration out of the war file is ease of deployment.
If I read your question correctly, you can't ever deploy the exact same build to multiple environments if you include any configuration in the build. A direct implication is you can never deploy you DEV build to QA and more importantly, you can't deploy your QA build to PROD or UAT. This might cause headaches if your process is audited.
If I mis-understood your question, please let me know, otherwise, I hope this helps.

Related

PostgreSQL - User lacks priviledge or object not found

I have an application that is using Hibernate and PostgreSQL, I compile it with Maven to a WAR and deploy it to a Tomcat 8.0, and sometimes I have the problem of
this error :
"User lacks priviledge or object not found: TABLE_NAME".
Well obviously the answers found to this problem are always "Check the priviledges to that user, and check if the table/database exists".
This absolutely is not the problem because this happens sporadically, i.e. My application is working fine, and when I deploy a new version of it, sometimes after the deploy it throws out this error.
I always deploy my application with Jenkins, and the deploys to Tomcat are successful, the application is working, but if I invoke an action that consumes a database service, it goes haywire with this error. This clears up after 2 to 3 clean deploys on it (i.e. I stop Tomcat, clean the directory, start it and run deploy with Jenkins again with the same version of code).
Still it is very odd that after a clean deploy it does not solve the problem. Even after restarting the whole server (not Tomcat, the Windows Server), it still happens.
The closest explanation I read about possible causes to this, is that sometimes it can create a ghost connection to the database and when it tries to query something it can't because it is not really connected.
The database is on the same server as the Tomcat, so the application points to localhost to connect to the database. I don't think this is related because it also happens in a Quality Rnvironment where the database is in a different server.
My configuration file regarding database is this:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<alias name="appDataSource" alias="jpaDataSource" />
<alias name="jpaTransactionManager" alias="appTransactionManager" />
<bean id="appDataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.postgresql.Driver" />
<property name="url" value="${postgre.url}" />
<!-- jdbc:postgresql://127.0.0.1:5432/DEV_DATABASE -->
<property name="username" value="${postgre.username}" />
<property name="password" value="${postgre.password}" />
</bean>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="jpaDataSource" />
<property name="jpaVendorAdapter" ref="jpaVendorAdapter" />
<property name="packagesToScan" value="${persistence.jpa.packagesToScan}" />
<property name="jpaProperties">
<props>
<prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop>
<prop key="hibernate.jdbc.use_get_generated_keys">true</prop>
</props>
</property>
<property name="sharedCacheMode" value="ENABLE_SELECTIVE" />
<property name="validationMode" value="NONE" />
</bean>
</beans>
Anyone got ideias of what this could be?
It's been a few months and I've been able to find out what was happening.
Apparently I had a configuration of an embedded-database and sometimes (though rarely) the application would connect to this embedded-database and not the one that I had configured to connect to Postgres Database.
The tag was:
<jdbc:embedded-database id="appDataSource" />
Because this was an empty mockup database, it would fire up the error of "User lacks priviledge or object not found" because the table did not exist in this empty database.
After removing this, the problem was solved, it now always connects to the correct database.

Second level caching in hibernate while migrating from JBoss to TomEE

I need to migrate my application built in GWT from Jboss to tomEE. The application has second level caching enabled in hibernate using Jboss cache. Is it possible to use Jboss cache in tomEE or do I have to find an alternative? If possible, can anyone please help me with the configuration of hibernate.cfg.xml? Below is the configuration
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.driver_class">com.sybase.jdbc3.jdc.Sybdriver</property>
<property name="hibernate.connection.url">xyz...</property>
<property name="hibernate.connection.username">xyz..</property>
<property name="hibernate.connection.password">xyz..</property>
<property name="transaction.factory_class">org.hibernate.transaction.JTATransactionFactory</property>
<property name="transaction.manager_lookup_class">org.hibernate.transaction.JBossTransactionManagerLookup</property>
<property name="dialect">org.hibernate.dialect.SybaseASE15Dialect</property>
<property name="generate_statistics">false</property>
<property name="jdbc.use_scrollable_resultset">false</property>
<property name="cache.provider_class">org.hibernate.cache.jbc.JBossCacheRegionFactory</property>
<property name="cache.use_second_level_cache">true</property>
<property name="cache.use_minimal_puts">true</property>
<property name="cache.use_structured_entries">true</property>
<property name="cache.use_query_cache">true</property>
<property name="cache.region.factory_class">org.hibernate.cache.jbc2.JndiMultiplexedJBossCacheRegionFactory</property>
<property name="cache.region.jbc2.cachefactory">java:CacheManager</property>
<property name="cache.region.jbc2.cfg.entity">mvcc-entity</property>
<property name="cache.region.jbc2.cfg.collection">mvcc-entity</property>
<property name="cache.region.jbc2.cfg.query">local-query</property>
<mapping resource="book.hbm.xml" />
</session-factory>
</hibernate-configuration>
We cannot achieve the second level caching using JBoss cache in TomEE. We have implemented a similar caching mechanism using Ehcache which is a little slow but works fine.
<property name="cache.region.jbc2.cfg.entity">mvcc-entity</property>
<property name="cache.region.jbc2.cfg.collection">mvcc-entity</property>
<property name="cache.region.jbc2.cfg.query">local-query</property>
We see that entity, collection, and query caching are enabled by using JBoss cache implementation specific to JBoss server. In the case of TomEE, we cannot have the same configuration. You would have to implement a different caching technique using different caching technologies available and match it with the usage of second-level cache(like say read-only, read-write, transactional etc.). In a similar setup, I had used "Ehcache" and that had solved the problem.

What is the proper way to create a datasource in a spring web application?

I have two ways to create datasource:
Inside spring context
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/mkyongjava" />
<property name="username" value="root" />
<property name="password" value="password" />
Tomcat JNDI
<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jdbc/UsersDB"/>
What are the benefits and problems of creating the datasource using spring or using tomcat jndi ?
There is nothing like good practice among this. The good one is the one which is good for use case. They both have pros and cons.
When u have use cases where u are trying to deploy the apps on a server environment where multiple apps share a common datasource or datasource components are managed by the Server administration then probably the best way to use is through a JNDI context. The server has the DataSource kept ready so that the application can directly get that object, bind to itself and use the datasource.
When u have use case where application runs inside a embedded server container or cloud containers where the server also is embedded along the build packs creating datasource with in the application and using them would be a good solution. Also in non web application environments creating datasource would be good.
To summarize, use cases where datasource is non managed by the application better to go for a JNDI context.
Try to figure out what is the deployment model of your application and figure out what is good for u. Also try to use a datasourcepool implementations to improve performance like DBCP, C3P0 or Hikarcp.
The first approach, using the class DriverManagerDataSource, is useful for test or standalone environments outside of j2ee container.
The second approach, using the jndi data soure, is recommended to be used in the j2ee container.
The below reference clearly states these approaches.
http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/jdbc/datasource/DriverManagerDataSource.html
I am using it as below. Works for me since project is configured at the time of architecture design. And its live in production.
<!-- Local Apache Commons DBCP DataSource that refers to a combined database -->
<!-- (see dataAccessContext-jta.xml for an alternative) -->
<!-- The placeholders are resolved from jdbc.properties through -->
<!-- the PropertyPlaceholderConfigurer in applicationContext.xml -->
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="validationQuery" value="${jdbc.validationquery}"/>
<property name="testOnBorrow" value="${jdbc.testonborrow}"/>
<property name="driverClassName" value="${jdbc.driverClassName}"/>
<property name="url" value="${jdbc.url}"/>
<property name="username" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
</bean>
<!-- Transaction manager for a single JDBC DataSource -->
<!-- (see dataAccessContext-jta.xml for an alternative) -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
you can also refer to this. Best contribution of Genious to java developers.
https://www.mkyong.com/spring-boot/spring-boot-jdbc-oracle-database-commons-dbcp2-example/
The best practice is to not to expose the database configuration in any of the code that you have written. It is always best to configure inside the server console and expose it as a jndi. Inside spring configuration file, get the database instance as:
<jee:jndi-lookup id="dataSource" jndi-name="jdbc"/>
and configure the database transaction manager as:
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
The key note here is do not expose the database configuration.

Using Ignite for Hibernate 2nd level cache with S3 discovery, changes are not progagated

We have deployed a cluster to AWS configured to use Ignite as a hibernate 2nd level cache, and we're using S3 discovery. On startup we can see the nodes do connect to each other correctly.
However, when a hibernate entity is updated on one server, the change is not propagated through the cluster so other servers continue to have the old value.
Here is a snippet of our ignite config..
```
<!-- Basic configuration for transactional cache. -->
<bean id="transactional-cache" class="org.apache.ignite.configuration.CacheConfiguration" abstract="true">
<property name="cacheMode" value="REPLICATED"/>
<property name="atomicityMode" value="TRANSACTIONAL"/>
<property name="writeSynchronizationMode" value="FULL_ASYNC"/>
<property name="backups" value="0"/>
</bean>
<bean id="ignite.cfg" class="org.apache.ignite.configuration.IgniteConfiguration" >
<property name="discoverySpi" ref="discoverySpi"/>
<property name="gridName" value="hibernate-grid"/>
<property name="cacheConfiguration">
<list>
<bean parent="transactional-cache">
<property name="name" value="hibernate.io.milton.vfs.db.Auditable"/>
</bean>
```
The exact same application code, when configured with hazelcast 2nd level cache, does work.
I'm sure there's just something wrong with our config. Can anyone suggest a pointer?
Most likely you hit a known issue with binary format which is used as the default one in Ignite. See the warning callout here for details: https://apacheignite-mix.readme.io/docs/hibernate-l2-cache#l2-cache-configuration
Switching to OptimizedMarshaller should fix the issue.

Using JNDI for for my datasource and location of properties files

I want to use jndi in my spring mvc application for 2 things:
datasource settings
store the location of my properties file so I can do a lookup in my code
I actually have 2 applications, 1 is a spring mvc and the other is a spring so I'm planning on using the same jndi settings for both.
Also, the 2 applications run on different contains (tomcat and jetty), I've never used jndi before so I hope I can define this in a single location and have both applications point to the same jndi file (assuming its a file).
So as for #1, my datasource settings in my spring context file are currently:
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost/mydb_development"/>
<property name="username" value="devuser1"/>
<property name="password" value="123"/>
<property name="maxActive" value="100"/>
<property name="maxIdle" value="30"/>
<property name="maxWait" value="1000"/>
<property name="defaultAutoCommit" value="true"/>
<property name="removeAbandoned" value="true"/>
<property name="removeAbandonedTimeout" value="60"/>
<property name="testOnBorrow" value="true"/>
<property name="validationQuery" value="SELECT 1"/>
</bean>
Can someone tell me how I can extract this out to use JNDI, confused, is it a separate file that has to be in my class loader, or do I hard code the path in my spring context file?
And as for #2, in my code I am currently loading a properties file that is in my class path like:
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
InputStream is = classLoader.getResourceAsStream("test.properties");
Now the test.properties file is in my classpath so it works out of the box, but what I want to know/understand is if I could somehow use jndi to lookup the location of the properties file (if that makes sense?) and not have this file in my classpath potentially.
I did google around and I know in spring I can pull in jndi using:
<jee:jndi-lookup id="dataSource" jndi-name="jdbc/mysqlDataSource" />
But what I don't understand is, where is the actual file that has the settings? Can it be in a location other than my web.xml file? (my other application again is using spring, but it is a daemon so it doesn't have a web.xml file).
The goal for me is to be able to change the values of the jndi file (like username, passwords, file paths) and NOT have these embedded into a jar. I want to be able to manually edit these files on the production server or dev servers and just restart the container.
I'm kinda rusty with spring but I remember using a PropertyPlaceHolderConfigurer, something like:
<bean id="propertyPlaceholderConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="ignoreResourceNotFound">
<value>true</value>
</property>
<property name="ignoreUnresolvablePlaceholders">
<value>true</value>
</property>
<property name="locations">
<list>
<value>file:path to your file
</value>
<value>file:path to another file if needed
</value>
</list>
</property>
</bean>
Then you can use the values on the .properties defined on the list directly. i.e. if you have a property like
db.driver=com.mysql.jdbc.Driver
you can use it like
bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="driverClassName" value="${db.driver}"/>
when you want to define a value.
If you want further control over the file, the path to the file itself can be registered on your server and you can look it up through JNDI, changing
file:path to your file
into
file:${propertiesFilePath}
I'm pretty sure you can look up those configuration properties by code too by a normal JNDI lookup.

Categories

Resources