Develop Reference

Wait for database connection in Spring Boot without an exception

I want to create a microservice with Spring Boot. For persistence i use a mariadb database. To wait for the database which is running in a docker container, i implemented the following code like shown here:
#Bean
public DatabaseStartupValidator databaseStartupValidator(DataSource dataSource) {
var dsv = new DatabaseStartupValidator();
dsv.setDataSource(dataSource);
dsv.setTimeout(60);
dsv.setInterval(7);
dsv.setValidationQuery(DatabaseDriver.MYSQL.getValidationQuery());
return dsv;
}
The code is working very well, my application is now waiting for the database connection. But i get an exception at startup of the application:
java.sql.SQLNonTransientConnectionException: Could not connect to Host ....
...
...
...
In the next line i get an information, that it will wait for the database:
021-04-07 21:29:40.816 INFO 16569 --- [ main] o.s.j.support.DatabaseStartupValidator : Database has not started up yet - retrying in 7 seconds (timeout in 57.65 seconds)
After that the application is starting as expected. So i think everything is working fine, but what i have to do to suppress the Exception? In the linked article it should work without an exception. Do i have to implement the "dependsOnPostProcessor" function? Which dependency i have to use? Sorry, possible a dumb question, i am new to spring boot.

to get rid of that exception you can state the below directive in your application.properties file:
logging.level.com.zaxxer.hikari=OFF
Keep in mind that if the application will not be able to get in contact with the db your spring crashes after a while due to that exception. In addition the above directive prevent you to see any logging activity related to Hikari.
In summary you hide the appearance of the exception until it is possible before the application dies due to timeout.
hoping I clarified a bit the case

Yes indeed you need to add the "depends-on" for the beans that rely on the data source. Note the following part of the documentation:
To be referenced via "depends-on" from beans that depend on database startup, like a Hibernate SessionFactory or custom data access objects that access a DataSource directly.
If I understand it well, this means that beans such as an EntityManagerFactory which rely on the database will now have to go through the DatabaseStartupValidator bean and wait for the DB startup. I don't know what caused your exception, but usually there is an EntityManagerFactory involved, so try adding the DependsOn on this object at least.
This is how the linked article is doing it:
#Bean
public static BeanFactoryPostProcessor dependsOnPostProcessor() {
return bf -> {
// Let beans that need the database depend on the DatabaseStartupValidator
// like the JPA EntityManagerFactory or Flyway
String[] flyway = bf.getBeanNamesForType(Flyway.class);
Stream.of(flyway)
.map(bf::getBeanDefinition)
.forEach(it -> it.setDependsOn("databaseStartupValidator"));
String[] jpa = bf.getBeanNamesForType(EntityManagerFactory.class);
Stream.of(jpa)
.map(bf::getBeanDefinition)
.forEach(it -> it.setDependsOn("databaseStartupValidator"));
};
}
You may not necessarily have Flyway configured, but the main thing to note is the dependency itself is referenced by the bean name databaseStartupValidator which is the name of the method that creates the bean.

Testing Spring Application with JUnit - Keep data after testclass

I am working on a bigger Spring Boot application that has a lot of tests (more like integration tests). My job now is to speed the process of the tests up. I have found, that testData, we need to test the application are set up multiple times in one testrun, if i run multiple test classes. We use something like this to setup the Data in the classes (edit: the Repository and testDataBuilder are #Autowired):
#BeforeEach
public void setup() {
if (Repository.findByShortId("someId") == null) {
testDataBuilder.createTestData();
}
}
Within the testclass, this works fine. But if my testrun gets to the next class, it seems to drop the data (the data are normally stored in a database, I think within the test the data gets stored in an in - memory database, not sure about that.
I tried multiple things to make this work but nothing did work out in the end:
building data in an abstract class that every test extends
working with #commit on some tests
using a testSuite and trying to create data before all tests like this:
#RunWith(Suite.class)
#SuiteClasses({testClass1.class ...})
#SpringBootTest
#ActiveProfiles({ "unit-test" })
public class testSuite {
#ClassRule
public static setupTestData setup = new setupTestData();
}
this didn't work, because spring does not run at the time, the #ClassRule is run.
What is the best way to setup the testData so not every testClass has to set them up ?

Why doesnt your Test #Configuration map the proper database beans by having H2 inmemory database as a dependency?
Spring/Springboot caches context automatically without you having to do anything special.
If your #Repository uses the H2 memory, then it will be cached across all Tests cases.
H2 can also be configured to write to a file (Instead of memory),if you want it instead of memory.
By default, closing the last connection to a database closes the database. For an in-memory database, this means the content is lost.
To keep the database open, add ;DB_CLOSE_DELAY=-1 to the database URL.
To keep the content of an in-memory database as long as the virtual
machine is alive, use jdbc:h2:mem:test;DB_CLOSE_DELAY=-1.
jdbc:h2:~/test;DB_CLOSE_DELAY=-1; <-- Writes to a file, and disable closing as long as the VM is alive
http://www.h2database.com/html/features.html#database_only_if_exists

#BeforeAll JUnit/spring-boot-test alternative that runs when application context starts

I'm writing a #Repository/#Service integration test that leverages an embedded database. In my test class, I would like to preload my database with some data.
I'm currently using #BeforeEach to load in my sample data, however, this code is run upon each test in my class.
Is there any way that I can load in my test data after Spring application context has loaded, but before any test has been run?
My current approach:
#BeforeEach
public void before() {
repository.save(...); // -> prepopulates repository with sample data
}
#Test
public void testService() {
service.get(...); // -> gathers existing record
}
#Test
public void deleteById() {
service.delete(...); // -> deletes existing record
}
However... with this, I am required to flush out the records after every test. Otherwise any unique constraints can easily be violated.
Rather than using #BeforeEach which is required to run before every test... is it possible to load this in in a #BeforeAll kind of fashion that happens after the spring application context has been loaded?

Is there any way that I can load in my test data after Spring application context has loaded
Basically yes, I think you can do that:
The idea is to load the SQL data when the application context is started or in the process of being started.
For example, spring boot integration with Flyway works this way (the bean of Flyway is created and loaded). So, in theory, you could merely use Flyway with test migrations that will contain all the relevant SQL scripts of test data generation.
How can you do this technically?
Here is one way:
Create a special bean (just like the way it works with Flyway) that would depend on your repository and in post construct save the data:
#Component
public class SqlGenerationBean {
#Autowired
private MyRepository repo;
#PostConstruct
public void init() {
repo.save();
}
}
Another way of doing is to create a listener that will be called upon the application context started and again will call the same repo.save().
In both cases the bean/listener code should not be accessible from production (it's only for tests): so put it somewhere under src/test/java for example
Now once the application context is started you can use a neat trick:
Mark your tests with #Transactional annotation. Spring will wrap the code in an artificial transaction that will be rolled back automatically (even if the test succeeds) so that all the data that you'll modify during the test will be rolled back and basically before each test, you'll have the same state (that is identical to the state of the database when/after the application context starts). Of course, if you use DDL in the test, some databases can't make it a part of transaction but it depends on the database really.
Another interesting point here is that the application context can be cached even between the test cases (created only once), so keep this in mind.

In this case I would just create a constructor for the test class. It will be triggered before everything.
#BeforeEach runs before each tests but after all initialisations .
you can also just use Mockito and mock the result without need to clean and overcomplicate

Just add following snippet to your code. This is just like you can do to detect that Spring application is really started.
#Configuration
public class AppConfig implements ApplicationListener<ApplicationReadyEvent> {
/**
* This is to indicate in the logs when the application has actually started and everything is loaded.
*/
#Override
public void onApplicationEvent(ApplicationReadyEvent event) {
ApplicationContext context = event.getApplicationContext();
Environment env = context.getEnvironment();
// do what you want on application start
}
}
P.S. For database manipulation in test #Sql is the best candidate as was mentioned in comment.

How to make App server to start even if database is down?

I am using spring & hibernate. my application has 3 modules. Each module has a specific database. So, Application deals with 3 databases. On server start up, if any one of the databases is down, then server is not started. My requirement is even if one of the databases is down, server should start as other module's databases are up, user can work on other two modules. Please suggest me how can i achieve this?
I am using spring 3.x and hibernate 3.x. Also i am using c3p0 connection pooling.
App server is Tomcat.
Thanks!

I would use the #Configuration annotation to make an object who's job it is to construct the beans and deal with the DB down scenario. When constructing the beans, test if the DB connections are up, if not, return a Dummy Version of your bean. This will get injected into the relevant objects. The job of this dummy bean is to really just throw an unavailable exception when called. If your app can deal with these unavailable exceptions for certain functions and show that to the user while continuing to function when the other datasources are used, you should be fine.
#Configuration
public class DataAccessConfiguration {
#Bean
public DataSource dataSource() {
try {
//create data source to your database
....
return realDataSource;
} catch (Exception) {
//create dummy data source
....
return dummyDataSource;
}
}
}

This was originally a comment:
Have you tried it? You wouldn't know whether a database is down until you connect to it, so unless c3p0 prevalidates all its connections, you wouldn't know that a particular database is down until you try to use it. By that time your application will have already started.

How to populate Java (web) application with initial data using Spring/JPA/Hibernate

I want to setup my database with initial data programmatically. I want to populate my database for development runs, not for testing runs (it's easy). The product is built on top of Spring and JPA/Hibernate.
Developer checks out the project
Developer runs command/script to setup database with initial data
Developer starts application (server) and begins developing/testing
then:
Developer runs command/script to flush the database and set it up with new initial data because database structures or the initial data bundle were changed
What I want is to setup my environment by required parts in order to call my DAOs and insert new objects into database. I do not want to create initial data sets in raw SQL, XML, take dumps of database or whatever. I want to programmatically create objects and persist them in database as I would in normal application logic.
One way to accomplish this would be to start up my application normally and run a special servlet that does the initialization. But is that really the way to go? I would love to execute the initial data setup as Maven task and I don't know how to do that if I take the servlet approach.
There is somewhat similar question. I took a quick glance at the suggested DBUnit and Unitils. But they seem to be heavily focused in setting up testing environments, which is not what I want here. DBUnit does initial data population, but only using xml/csv fixtures, which is not what I'm after here. Then, Maven has SQL plugin, but I don't want to handle raw SQL. Maven also has Hibernate plugin, but it seems to help only in Hibernate configuration and table schema creation (not in populating db with data).
How to do this?
Partial solution 2010-03-19
Suggested alternatives are:
Using unit tests to populate the database #2423663
Using ServletContextListener to gain control on web context startup #2424943 and #2423874
Using Spring ApplicationListener and Spring's Standard and Custom Events #2423874
I implemented this using Spring's ApplicationListener:
Class:
public class ApplicationContextListener implements ApplicationListener {
public void onApplicationEvent(ApplicationEvent event) {
if (event instanceof ContextRefreshedEvent) {
...check if database is already populated, if not, populate it...
}
}
}
applicationContext.xml:
<bean id="applicationContextListener" class="my.namespaces.ApplicationContextListener" />
For some reason I couldn't get ContextStartedEvent launched, so I chose ContextRefreshedEvent which is launched in startup as well (haven't bumped into other situations, yet).
How do I flush the database? Currently, I simply remove HSQLDB artifacts and a new schema gets generated on startup by Hibernate. As the DB is then also empty.

You can write a unit test to populate the database, using JPA and plain Java. This test would be called by Maven as part of the standard build lifecycle.
As a result, you would get an fully initialized database, using Maven, JPA and Java as requested.

The usual way to do this is to use a SQL script. Then you run a specific bash file that populate the db using your .sql
If you want to be able to programmatically set your DB during the WebApp StartUp you can use a Web Context Listener. During the initialization of your webContext you can use a Servlet Context Listener to get access to your DAO (Service Layer.. whatever) create your entities and persist them as you use to do in your java code
p.s. as a reference Servlet Life Cycle
If you use Spring you should have a look at the Standard and Custom Events section of the Reference. That's a better way to implement a 'Spring Listener' that is aware of Spring's Context (in the case you need to retrieve your Services form it)

You could create JPA entities in a pure Java class and persist them. This class could be invoked by a servlet but also have a main method and be invoked on the command line, by maven (with the Exec Maven Plugin) or even wrapped as a Maven plugin.
But you're final workflow is not clear (do you want the init to be part of the application startup or done during the build?) and requires some clarification.

I would us a Singleton bean for that:
import javax.annotation.PostConstruct;
import javax.ejb.Startup;
import javax.ejb.Singleton;
#Singleton
#Startup
public class InitData {
#PostConstruct
public void load() {
// Load your data here.
}
}

Depend on your db. It is better to have script to set up db

In the aforementioned ServletContextListener or in a common startup place put all the forthcoming code
Define your data in an agreeable format - XML, JSON or even java serialization.
Check whether the initial data exists (or a flag indicating a successful initial import)
If it exists, skip. If it does not exist, get a new DAO (using WebApplicationContextUtils.getRequiredWebApplicationContext().getBean(..)) , iterate all predefined objects and persist them via the EntityManager in the database.

I'm having the same problem. I've tried using an init-method on the bean, but that runs on the raw bean w/o AOP and thus cannot use #Transactional. The same seems to go for #PostConstruct and the other bean lifecycle mechanism.
Given that, I switched to ApplicationListener w/ ContextRefreshedEvent; however, in this case, #PersistenceContext is failing to get an entity manager
javax.persistence.PersistenceException: org.hibernate.SessionException: Session is closed!
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:630)
at org.hibernate.ejb.QueryImpl.getSingleResult(QueryImpl.java:108)
Using spring 2.0.8, jpa1, hibernate 3.0.5.
I'm tempted to create a non-spring managed entitymanagerfactory and do everything directly but fear that would interfere w/ the rest of the Spring managed entity and transaction manager.

I'm not sure if you can get away from using some SQL. This would depend if your develoeprs are staring with an empty database with no schema defined or if the tables are there but they are empty.
If you starting with empty tables then you could use a Java approach to generating the data. I'm not that familiar with Maven but I assume you can create some task that would use your DAO classes to generate the data. You could probably even write it using a JVM based scripting language like Groovy that would be able to use your DAO classes directly. You would have a similar task that would clear the data from the tables. Then your developers would just run these tasks on the command line or through their IDE as a manual step after checkout.
If you have a fresh database instance that I think you will need to execute some SQL just to create the schema. You could technically do that with executing SQL calls with hibernate but that really doesn't seem worth it.

Found this ServletContextListener example by mkyong. Quoting the article:
You want to initialize the database connection pool before the web
application is start, is there a “main()” method in whole web
application?
This sounds to me like the right place where to have code to insert initial DB data.
I tested this approach to insert some initial data for my webapp; it works.

I found some interesting code in this repository: https://github.com/resilient-data-systems/fast-stateless-api-authentication
This works pretty neat in my project:
#Component
#DependsOn({ "dataSource" })
public class SampleDataPopulator {
private final static Logger log = LoggerFactory.getLogger(SampleDataPopulator.class);
#Inject
MyRepository myRepository
#PostConstruct
public void populateSampleData() {
MyItem item = new ResourceDatabasePopulator();
myRepository.save(item);
log.info("Populated DB with sample data");
}
}

You can put a file called data.sql in src/main/resources, it will be read and executed automatically on startup. See this tutorial.
The other answers did not work for me.