Develop Reference

Spring boot test failing to get data populated at start of test

I have a sql file which looks like this:
--Testdata
INSERT into cms_patch.configuration_main(document_key, config)
values('TEST/PhoenixTest', '{"b" : "c"}');
I am trying to run a test and want to seed this data to the table before that test, but the data is getting rolled back before the test happens.
Test class:
#Sql({"/sql/cms_dao_data.sql"})
public class CmsDaoImplTest extends AbstractPhoenixDao {
#Autowired
private CmsDao cmsDao;
#BeforeMethod(alwaysRun = true)
public void beforeMethod() {
this.cmsDao = new CmsDaoImpl();
}
#Test
#Sql("/sql/cms_dao_data.sql")
public void containsDocumentKey() {
Boolean result = cmsDao.containsDocumentKey("TEST/PhoenixTest");
assertTrue(result);
}
}
abstract class:
#Transactional
#TestPropertySource({"classpath:application-test.properties"})
#ContextConfiguration(classes = {RepositoryContext.class}, loader = AnnotationConfigContextLoader.class)
public abstract class AbstractPhoenixDao extends AbstractTransactionalTestNGSpringContextTests {
public AbstractPhoenixDao() {
}
}
Can someone please help me to know why the data is getting rolled back before the test happens.
the data is getting created because if I keep 2 values in insert with same primary key the sql run at start gives an error.
we are using postgres.

Junit 5 with test application.properties

I am learning Junit 5 and test cases.
I am using spring boot version '2.2.6.RELEASE and JUnit 5,
in my application, I have a method that processes based on the boolean flag from property file.
\src\main\resources\application.properties
#data base connection properties
spring.app.datasource.url=jdbc:mysql://localhost:3306/student_db
spring.app.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.app.datasource.username=root
spring.datasource.password=root
spring.app.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL8Dialect
#additional properties
spring.property.name=shrikant
spring.property.enable=false
database connection properties are used to create the database connection
Datasource.java
#Value("${spring.app.datasource.url}")
private String url;
#Value("${spring.app.datasource.driver-class-name}")
private String className;
#Value("${spring.app.datasource.username}")
private String userName;
#Value("${spring.datasource.password}")
private String password;
#Value("${spring.app.jpa.properties.hibernate.dialect}")
private String dialect;
controller class
#RestController
public class Controller {
#Value("${spring.property.name}")
private String name;
#Value("${spring.property.enable}")
private boolean status;
public void validateObject(String surName) {
if (status) { # if this flag is true then only process
System.out.println("name= " + name);
System.out.println("surName= " + surName);
}
}
ControllerTest.java
#SpringBootTest
class ControllerTest {
#Autowired
private Controller controller;
#Test
void show() {
controller.validateObject("sharma");
}
by default the flag is false, so every time test case runs it never processes the object.
so I tried to create aplication.properties in the test folder
\src\test\resources\application.properties
spring.property.name=vishal
spring.property.enable=true
but now it's giving me an error that
Could not resolve placeholder 'spring.app.datasource.url'
but I don't want to provide DB connection URL, I am not connecting to the database while testing.
Q1 - how to change the value of properties file for test case only.
Q2 - is it mandatory to provide all the keys of \src\main\resources\application.properties is \src\test\resources\application.properties?
I am new in test case, so little explained answers would be welcomed.
Update:-
I found that
#SpringBootTest
#TestPropertySource(properties = {"spring.property.name=vishal", " spring.property.status=true"})
class ControllerTest {
will solve the issue temporarily, by providing keys along with values, but I have a lot of keys, which cannot be provided in such a way.

If you use #SpringBootTest then your test will load the whole Spring context. This means it will create all your beans and try to wire them together. If you inject property values to your beans, you have to specify them all for such tests as otherwise, you won't be able to boot the application.
What might help you in such a situation is to use test annotations like #WebMvcTest or #DataJpaTest to focus on testing just slices of your application. Using #WebMvcTest you'll get an application context just containing controllers and everything related to your web layer. Other beans (like service classes) can be mocked with #MockedBean.
Next, for testing business logic in service classes try not to use #SpringBootTest and rather rely on plain JUnit 5 and Mockito to verify your code.
You still might want to have some integration tests that use #SpringBootTest to make sure everything is working together. In such case, you can hard code any static property inside application.properties in src/test/resources/ or using the annotation like you already did: #TestPropertySource(properties = {"spring.property.name=vishal", " spring.property.status=true"}).
When it comes to providing a database, you can either configure an embedded database (which I would try to avoid) or use the same database as in production. Testcontainers helps you a lot when it comes to providing external infrastructure for your tests like a database.
An example setup with Spring Boot >= 2.2.6 and JUnit might look like the following:
#Testcontainers
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class ApplicationIT {
#Container
public static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer()
.withPassword("password")
.withUsername("username");
#DynamicPropertySource
static void postgresqlProperties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", postgreSQLContainer::getJdbcUrl);
registry.add("spring.datasource.password", postgreSQLContainer::getPassword);
registry.add("spring.datasource.username", postgreSQLContainer::getUsername);
}
#Test
public void contextLoads() {
}
}

For Junit5, annotate your test class with path to application.properties in your src/main/resources:
#ExtendWith(SpringExtension.class)
#TestPropertySource("classpath:application.properties")
public class MyTest {
#Value("${property.name}")
private int myProperty;
tests...
}
property gets loaded

Database alters and Spring Boot JPA

I want to run alters on the embedded derby database after a desktop application has been updated. To do this I attempted to create a JPS repository with my alter queries similar to this:
#Repository
public interface DatabaseUpdates {
#Modifying
#Query("alter table my_table add column my_column clob", nativeQuery = true)
void alterMyTableAddMyColumn();
}
And use it in the updater class like this
#Component
public class DatabaseUpdater {
#Autowired
private DatabaseUpdates databaseUpdates;
private void alterMyTable() {
// some logic that checks it the update needs to happen is here
databaseUpdates.alterMyTableAddMyColumn();
}
}
This throws NoSuchBeanDefinitionException: No qualifying bean of type DatabaseUpdates exceptions.
I also tried having DatabaseUpdates extend org.springframework.data.repository.Repository but then had problems with what to use as the domain type.
I am using spring-boot-starter-data-jpa 1.2.3.RELEASE.
Edit: This is to support a legacy app and I am looking for straight forward ways to support database updates. Database migrations tools are great but not in scope for this fix. I've been wishing this project had been using one for more than a month now and will be integrating one into the next version soon.

That sounds like an awful idea. There are frameworks, which were specifically developed for your usecase and are supported by Spring Boot, like Flyway and Liquibase. You are then able to specify your changes either in SQL or Java code (for Flyway) or in an XML, SQL, YAML or JSON file (for Liquibase). Those frameworks will check which migrations are necessary and they also keep track when a migration was executed.

I wouldn't put this in Spring JPA. If you mention it is temporary, maybe try some JDBC directly and run the queries on the database. Try to log the changes you make in a table (something similar to what liquibase is doing), so you don't run the queries multiple times.
Maybe try to run this before Spring builds your DataSource in your DatabaseConfiguration class.
#Configuration
#EnableJpaRepositories("")
#EnableTransactionManagement
public class DatabaseConfiguration implements EnvironmentAware {
#Bean(destroyMethod = "close")
public DataSource dataSource(DataSourceProperties dataSourceProperties, ApplicationProperties applicationProperties)
throws SQLException {
LOG.debug("Configuring Datasource");
//some code
alterSchema();
return new HikariDataSource(config);
}
private void alterSchema() {
// JDBC connection and queries to alter the schema
}
}

While not a best practice, you can temporarily use native JDBC, acquire connection from your existing DataSource. If you are only adding a column, make sure to check if the object(column) already exist.

Your repository can execute a transaction and build a native query with an EntityManager like the following code
import org.springframework.stereotype.Repository;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.transaction.Transactional;
#Repository
public class DatabaseUpdates{
#PersistenceContext
private EntityManager entityManager;
//ALTER TABLE `my_table` ADD COLUMN `my_column` VARCHAR(100) AFTER `after_column`;
#Transactional
public void alterMyTableAddMyColumn(String tableName, String columnName,
String columnType, String afterColumnName) {
String query = "ALTER TABLE `" + tableName + "` ADD COLUMN `" + columnName + "` " +
columnType + " AFTER `" + afterColumnName + "`";
entityManager.createNativeQuery(query).executeUpdate();
}
}
Your updater class calls your repository like this
import org.springframework.stereotype.Component;
import org.springframework.beans.factory.annotation.Autowired;
#Component
public class DatabaseUpdater {
#Autowired
private DatabaseUpdates databaseUpdates;
private void alterMyTable() {
// some logic that checks it the update needs to happen is here
String tableName = "my_table";
String columnName = "my_column";
String columnType = "VARCHAR(100)";
String afterColumnName = "after_column";
databaseUpdates.alterMyTableAddMyColumn(tableName, columnName,
columnType, afterColumnName);
}
}

How to write a unit test for a method, which creates database table on the fly?

tl;dr;. I have a method for creating new database table on the fly, and I want to write a unit test for it. Unfortunately, test runner does not perform rollback after tests in a proper way, and the table still remains in the DB after tests finished. What should I do?
Long story:
I am not very familiar neither with Java Persistence nor with Java Spring, so, if you found current solution ugly (as for me, it is rather ugly), please, tell me how to improve it - I will very appreciate your opinion.
I have a SysDatastoreService with the following implementation of addStaticDatastore method.
#Service
public class SysDatastoreServiceImpl implements SysDatastoreService {
#Autowired
private SysDatastoreRepository datastoreRepository;
#Autowired
private DataSource dataSource;
#Override
#Transactional
public Optional<SysDatastore> addStaticDatastore(String name, String tableName, String ident, Long ord) {
String createTableSql = PostgresTableSqlBuilder.createTableInPublicSchemaWithBigintPkAndFkId(
tableName,
SysObject.TABLE_NAME,
Optional.of(SysObject.ID_COLUMN_NAME)).buildSql();
Optional<SysDatastore> sysDatastore = Optional.empty();
try(
Connection connection = dataSource.getConnection();
Statement statement = connection.createStatement()
) {
connection.setAutoCommit(false);
Savepoint beforeTableCreation = connection.setSavepoint();
try {
statement.execute(createTableSql);
sysDatastore = Optional.ofNullable(
datastoreRepository.save(new SysDatastore(name, tableName, DatastoreType.STATIC, ident, ord)));
} catch(SQLException e) {
e.printStackTrace();
}
if(!sysDatastore.isPresent()) {
connection.rollback(beforeTableCreation);
} else {
connection.commit();
}
} catch(SQLException e1) {
e1.printStackTrace();
}
return sysDatastore;
}
}
So, as you can see, I receive new connection from DataSource and try to create new table. In success, I will create a new entry in the SysDataStoreRepository, and, if this fails, I will perform a rollback for the table creation.
There are some disadvantages of current approach, one of them is that table creation and entry insertion operates on separate connections (am I right?).
But I have some problem while writing a unit test. This is what I tried:
#Transactional(propagation = Propagation.REQUIRED)
#RunWith(SpringJUnit4ClassRunner.class)
#TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = true)
#ContextConfiguration(locations = "file:src/main/webapp/WEB-INF/rest-servlet.xml")
public class SysDatastoreServiceTest {
#Autowired
private SysDatastoreService sysDatastoreService;
#Autowired
private DataSource dataSource;
#Test
public void testAddStaticDatastore() throws Exception {
Optional<SysDatastore> sysDatastore =
sysDatastoreService.addStaticDatastore("New static datastore", "new_datastore_table",
"NEW_DATASTORE_TABLE", 42L);
assertTrue(sysDatastore.isPresent());
assertEquals("New static datastore", sysDatastore.get().getName());
assertEquals("NEW_DATASTORE_TABLE", sysDatastore.get().getIdent());
assertEquals("new_datastore_table", sysDatastore.get().getTableName());
assertEquals(DatastoreType.STATIC, sysDatastore.get().getDynType());
assertEquals(42L, sysDatastore.get().getOrd().longValue());
assertTrue(dataSource.getConnection()
.getMetaData()
.getTables(null, null, sysDatastore.get().getTableName(), null)
.next());
}
This test seems pretty easy: I just compare all the fields, and then checks database for a new table.
However, this test fails when I run it twice or more times. Looking at the database I noticed, that the table new_datastore_table still remained in the schema. I guess, it was not rollbacked properly because of hand-written transaction and raw sql execution, but I am not sure.
Question: How should I write a test case for this method in a proper way? And, in case if the current approach is fundamentally wrong, how it should be changed?
Side notes: I use PostgreSQL database, and it cannot be replaced with non-relational database.

First a CREATE TABLE is a DDL statement, not a DML one. That means that a rollback will not delete the table. If you want to clean your database, you must explicitely remove it in a #After or #AfterClass method.
But do you really need to do the tests on the PostgreSQL database? Spring has great support for in memory databases and the default embedded database is HSQL which has a pretty good support for postgresql syntax. Provided you have no complex statement, it could be enough and avoids cluttering the main database for (potentially destructive) unit tests.
You could create the database in a #BeforeClass method. Here is an oversimplified example:
private static DriverManagerDataSource dataSource;
#BeforeClass
public static void setupClass() throws Exception {
ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
populator.addScript(new ClassPathResource("path/to/package/defaults.sql"));
dataSource = new DriverManagerDataSource();
dataSource.setUrl("jdbc:hsqldb:mem:pgtest;sql.syntax_pgs=true");
dataSource.setUsername("SA");
Connection con = dataSource.getConnection();
assertNotNull(con);
populator.populate(con);
con.close();
}

JUnit + Derby + Spring: drop in-memory db after every test

In my unit tests I autowired some DataSources, which use URLs like
jdbc:derby:memory:mydb;create=true
to create an in-memory DBs.
To drop an in-memory Derby db you have to connect with:
jdbc:derby:memory:mydb;drop=true
I would like this to happen after every test and start with a fresh db. How can I do this using Spring?

How to shutdown Derby in-memory database Properly
gave me a hint to a solution:
mydb.drop.url = jdbc:derby:memory:mydb;drop=true
...
<bean id="mydbDropUrl" class="java.lang.String">
<constructor-arg value="${mydb.drop.url}" />
</bean>
...
#Resource
private String mydbDropUrl;
#After
public void tearDown() {
try {
DriverManager.getConnection(mydbDropUrl);
} catch (SQLException e) {
// ignore
}
}
A downside is the use of the String constructor which accepts a String (an immutable String object around an immutable String object). I read that there is a #Value annotation in Spring 3, which might help here, but I'm using Spring 2.5.
Please let me know if you have a nicer solution.

There is a database-agnostic way to do this if you are using Spring together with Hibernate.
Make sure the application context will be created / destroyed before / after every test method:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration({"classpath*:application-context-test.xml"})
#TestExecutionListeners({DirtiesContextTestExecutionListener.class,
DependencyInjectionTestExecutionListener.class})
#DirtiesContext(classMode = ClassMode.AFTER_EACH_TEST_METHOD)
public abstract class AbstractTest {
}
Instruct Hibernate to auto create the schema on startup and to drop the schema on shutdown:
hibernate.hbm2ddl.auto = create-drop
Now before every test
the application context is created and the required spring beans are injected (spring)
the database structures are created (hibernate)
the import.sql is executed if present (hibernate)
and after every test
the application context is destroyed (spring)
the database schema is dropped (hibernate).
If you are using transactions, you may want to add the TransactionalTestExecutionListener.

After spring test 3, you can use annotations to inject configurations:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration("/spring-test.xml")
public class MyTest {
}

Just do something like:
public class DatabaseTest implements ApplicationContextAware {
private ApplicationContext context;
private DataSource source;
public void setApplicationContext(ApplicationContext applicationContext) {
this.context = applicationContext;
}
#Before
public void before() {
source = (DataSource) dataSource.getBean("dataSource", DataSource.class);
}
#After
public void after() {
source = null;
}
}
Make your bean have a scope of prototype (scope="prototype"). This will get a new instance of the data source before every test.

If you use the spring-test.jar library, you can do something like this:
public class MyDataSourceSpringTest extends
AbstractTransactionalDataSourceSpringContextTests {
#Override
protected String[] getConfigLocations() {
return new String[]{"classpath:test-context.xml"};
}
#Override
protected void onSetUpInTransaction() throws Exception {
super.deleteFromTables(new String[]{"myTable"});
super.executeSqlScript("file:db/load_data.sql", true);
}
}
And an updated version based on latest comment, that drops db and recreates tables before every test:
public class MyDataSourceSpringTest extends
AbstractTransactionalDataSourceSpringContextTests {
#Override
protected String[] getConfigLocations() {
return new String[]{"classpath:test-context.xml"};
}
#Override
protected void onSetUpInTransaction() throws Exception {
super.executeSqlScript("file:db/recreate_tables.sql", true);
}
}

This is what we do at the start of every test.
Drop all Previous Objects.
Create all tables mentioned in the create_table.sql
Insert values onto the created tables based on what you want to test.
#Before
public void initialInMemoryDatabase() throws IOException, FileNotFoundException {
inMemoryDerbyDatabase.dropAllObjects();
inMemoryDerbyDatabase.executeSqlFile("/create_table_policy_version_manager.sql");
inMemoryDerbyDatabase.executeSqlFile("/insert_table_policy_version_manager.sql");
}
Works like a charm!