Parallelization of unit and integration tests in springboot application - java

I am trying to parallelize tests in my spring boot application. For this, I have made the changes in the maven surefire plugin to include the parallel argument with classes as value and thread count argument with 5 as value. But when I run maven clean install, mvn clean install I get PessimisticLockingFailure or Pessimistic Exception.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<parallel>classes</parallel>
<threadCount>5</threadCount>
</configuration>
</plugin>
For my test cases, I have configured H2 database in the application properties file present under the test folder. After reading on google I found out that I need to include MVCC=TRUE;DEFAULT_LOCK_TIMEOUT=10000 in the database url. So, therefore the application properties file has below configuration:
spring:
datasource:
type: com.zaxxer.hikari.HikariDataSource
url: jdbc:h2:TestDB;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE;MVCC=TRUE;DEFAULT_LOCK_TIMEOUT=10000
jpa:
database-platform: io.github.jhipster.domain.util.FixedH2Dialect
database: H2
open-in-view: false
show-sql: true
hibernate:
ddl-auto: none
naming:
physical-strategy: org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
implicit-strategy: org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
properties:
hibernate.id.new_generator_mappings: true
hibernate.connection.provider_disables_autocommit: true
hibernate.cache.use_second_level_cache: false
hibernate.cache.use_query_cache: false
hibernate.generate_statistics: false
hibernate.hbm2ddl.auto: validate
hibernate.jdbc.time_zone: UTC
Now I understand, that this Pessimistic lock is being done because it should be in order to follow ACID properties. Also what I understood that in the case of concurrency H2 uses pessimistic locking instead of optimistic locking, which makes sense as we do not want in corrupt data. But then how do I parallelize my tests. I also see that while executing these tests maven uses different threads, as the thread name is printed, but then I get this Pessimistic Exception.
Caused by: org.hibernate.PessimisticLockException: could not execute statement
at com.x.y.z.repository.a.ProductionDataRepositoryTest.setUp(ProductionDataRepositoryTest.java:45)
Caused by: org.h2.jdbc.JdbcSQLException:
Timeout trying to lock table ; SQL statement:
insert into my_table
One thought that came to my mind is that maybe, we should run individual h2 instances per thread, that if there are 4 threads then 4 h2 instances, but then wouldn't it be expensive, and if not then how to do that?
Or is there some other approach in achieving parallelization of tests in spring boot, java and maven.
Kindly suggest.

Related

Spring boot micro-service should come up irrespective of Db status

What I am looking for is some suggestions on this behavior: Spring boot app(Considered as a microservice) should come up, irrespective of Db status.
Why I am doing this?
Based on my understanding of the microservice all services should be independent of each other.
I am using Spring boot with JPA (org.springframework.boot' version '2.5.7'). I am able to achieve this using the below configuration
spring:
datasource:
driverClassName: org.postgresql.Driver
url: jdbc:postgresql://localhost:5433/xxx?createDatabaseIfNotExist=true&characterEncoding=utf8&enabledTLSProtocols=TLSv1.2&useSSL=false
username: xx
password: xx
continueOnError: true
initialize: false
initialSize: 0
timeBetweenEvictionRunsMillis: 5000
minEvictableIdleTimeMillis: 5000
minIdle: 0
jpa:
show-sql: true
hibernate:
naming_strategy: org.hibernate.cfg.DefaultNamingStrategy
properties:
hibernate:
dialect: org.hibernate.dialect.PostgreSQLDialect
hbm2ddl:
auto: none
temp:
use_jdbc_metadata_defaults: false
But now the issue is I have to make hbm2ddl.auto none. Due to this, I am losing update schema functionalists which is one of the very essential functionality.
Requirement:
case 1. Service should be up and running irrespective of DB status
case 2. Jpa/Hibernate should update the database schema by comparing the existing schema with the entity mappings and generate the appropriate schema migration scripts (hbm2ddl.auto: update)
Can we achieve both? If yes how? or Do I have to compromise with one?
If I am going with only "case 1" do I have to rely on running schema updates manually or is there any other way?
thanks in advance
As mentioned in the comment you can exclude org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfigurationif you wish for spring boot to ignore the status of the data source (link on how to do that). In order to still have the schema auto-generated/controlled outside of your actual db you could use something like liquibase, which is used a lot in production environments.

spring.datasource.data picked up, but script never executed

I am trying to set up in-memory H2 tables for a test class in my Spring boot application.
My config looks something like:
spring:
jpa:
show-sql: true
generate-ddl: true
properties:
hibernate:
dialect: org.hibernate.dialect.H2Dialect
hibernate:
ddl-auto: create-drop
datasource:
# not sure which one to use so added both just in case
initialization-mode: always
initialize: true
platform: h2
# casting a wide net here, but no cookie - completely ignored
data: data-h2.sql,classpath*:data-h2.sql, classpath:data-h2.sql
url: jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
driver-class-name: org.h2.Driver
username: sa
password:
As you can see, I'm trying to load a data-h2.sql script upon db initialization.
Unfortunately, the property is ignored no matter the value.
I am certain the configuration file is being picked up properly (e.g. among others, I desperately added a #Value("${spring.datasource.data}" -annotated property in my test class and the value was indeed populated correctly).
As an alternative, I could annotate the test class with #Sql("classpath:data-h2.sql") which did run the script - however it did so for every test, while I wanted the script to be run once before any test execution.
I also tried removing that and using a blank schema.sql and moving the population to data.sql (as suggested here), but Spring would complain about the empty schema file - which is useless to me, because my schema is auto-generated and I certainly don't want to re-create it (NB: probably a conflict with a hibernate property if memory serves).
I've browsed some of the answers here, but the only one I could use, is not working.
The only solution I can see is to keep the #Sql annotation, but try and clear the tables after every test with another #Sql annotation launching another script on #After.
This seems insane to me - there must be a better solution.
Am I missing something more esoteric than it already is in my configuration?
Simply put your file in the /main/resources directory (what you already did)
Spring Boot 2:
The correct property is:
spring.datasource.initialization-mode=always
Read more about this topic in the Spring Boot Documentation: https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-initialize-a-database-using-spring-jdbc
Spring Boot 1
You only have to place data-h2.sql in the classpath
Read more about this topic in the Spring Boot Documentation:
https://docs.spring.io/spring-boot/docs/1.5.8.RELEASE/reference/htmlsingle/#howto-initialize-a-database-using-spring-jdbc

How to load initial data in H2 when using flyway on spring boot application?

I'm creating a spring boot application using Flyway to migration and want to use a memory database for development profile, but the problem is that data is lost every time I restart application. So I need to insert some data when my application start in development profile. I tried to put a file called data.sql on src/main/resource to spring load it when application starts but it doesnt work (It didnt run the script). I tried to put INIT=runscript from 'classpath:data.sql' in the h2 url but it tries to run it before Flyway migration execution so the tables doesnt exist yet. Can anyone give me an other way to do it?
My application.yml:
spring:
datasource:
url: jdbc:h2:mem:testdb;IFEXISTS=FALSE
username: sa
password:
driver-class-name: org.h2.Driver
jpa:
show-sql: true
hibernate:
ddl-auto: none
properties:
hibernate:
dialect: org.hibernate.dialect.H2Dialect
flyway:
enabled: true
Per documentation , a profile-specific customised flyway spring.flyway.locations can be configured. The profile-specific scripts runs when that profile is active . So a dev profile configured will work on this requirement.
The initialisation script can be placed as part of migration folder which will run and populate the db.
An example can be found here

How to cause SpringBoot/Hibernate to create a schema for unit testing?

Environment
SpringBoot 2 with H2 as test dependency.
Production works
Jar is deployed to cloud. A DB2 service is configured with driver and connection details, and automatically bound to the java application. There's no configuration in the jar itself. There is the application.properties file but it's empty. This part works fine and I'm hoping that a solution exists which will not require me to create property files and profiles.
Local Unit Test crashes on 'schema xxx not found'
#Entity(table="Employee", schema="acme")
class Employee {
...
}
#RunWith(SpringRunner.class)
#DataJpaTest
public class EmployeeTest {
...
}
No data source configuration exists.
SpringBoot sees H2 dependency and selects Hibernate by default.
Hibernate sees the Entity definition and attempts to drop the table first.
The drop uses the schema name drop table acme.employee if exists. No schema has been created so process fails with JdbcSQLSyntaxErrorException: Schema "acme" not found.
I tried #TestPropertySource(properties ="jdbc:h2:testb;INIT=CREATE SCHEMA IF NOT EXISTS acme;") with no luck.
I've found issues like this on the web and potential solutions. But they go very far into Hibernate and/or Spring configuration files. I would really want to avoid this. It's only local unit test that fails so I'm hoping to find a solution that is contained within the test.
If you need a different behaviour for your tests, that's basically a different profile. Although you prefer not to define properties files, the solution below doesn't go too deep into configuration and allows you to be very explicit in your tests.
Create application-test.properties (or yml if you prefer) with:
spring.datasource.url = jdbc:h2:mem:testb;init=CREATE SCHEMA IF NOT EXISTS acme;
Select Spring profile in your test classes with the annotation #ActiveProfiles:
#SpringBootTest
#ActiveProfiles("test")
class MyTest {
...
}
Give it a go. H2 should create the schema just before Hibernate tries to create the tables.
The absolute simplest means of doing this is to use h2 (or hsql as you are), and set ddl-auto to create-drop.
spring:
datasource:
driver-class-name: org.h2.Driver
password:
url: jdbc:h2:acme
username: sa
jpa:
database-platform: org.hibernate.dialect.H2Dialect
hibernate:
ddl-auto: create-drop

MYSQL schema keeps getting dropped

I have a spring-boot application running using docker on a Ubuntu server, with MYSQL DB running on the server itself not in a docker image. The schema I am using for the spring-boot app keeps getting deleted randomly and I always have to restore it using backed up dump. I cannot determine the reason behind this weird issue. I tried googling this issue and I could not find anything about this issue.
What could possibly cause a DB to be dropped?!?
It is random like once a week! may be more than that. I cannot even track that.
My application.yml has this:
spring:
datasource:
url: jdbc:mysql://localhost/schemaname?useSSL=false
username: username
password: password
tomcat:
validation-query: SELECT 1
test-on-borrow: true
jpa:
show-sql: true
hibernate:
ddl-auto: update
properties:
hibernate:
dialect: org.hibernate.dialect.MySQL5Dialect
You shouldn't use ddl-auto for a production app.
Quote from Java Persistence with Hibernate.
WARNING: We've seen Hibernate users trying to use SchemaUpdate to
update the schema of a production database automatically. This can
quickly end in disaster and won't be allowed by your DBA.
This option is intended to be used only for development.
If you don't want to manually run your SQL migrations, you should consider tools like flyway or liquibase.

Categories

Resources