When we write an application, we also write migrations for database schema and data, we use liquibase for that.
For example I am deploying an application locally as developer on my machine, application uses elasticsearch, but before adding some data I need to execute some query/mappings.
So, I want to add that query/mappings to some files, so, when another developer will deploy application on his machine, that query will be executed automatically.
What tools/library are using for migrations ElasticSearch schema?
I created a long time ago a project I'm still maintaining which creates an index with all settings and mappings if it does not exist at startup.
elasticsearch-beyonder is available at https://github.com/dadoonet/elasticsearch-beyonder.
For now, it works only with a TransportClient but I'm planning to add support for the Java REST Client as well.
Related
I try to use the MongoDB in the spring-boot project.
I tried a different of non-usable tutorial and stopped on the official documentation
I created cluster and now I ready to use MongoDB in my project.
Documentation says:
As I understand the pwd - this is the password, so my application.properties looks like:
spring.data.mongodb.uri=mongodb+srv://root:pass#88.155.XX.XXX.mongodb.net/mygrocerylist
spring.data.mongodb.database=mygrocerylist
this is all what I added to the application.properties - there are no other information in the tutorial.
After I started project, I receive error, that repeats every several seconds (however project still launched - I receive error even during system logging):
com.mongodb.MongoConfigurationException: No SRV records available for
host 88.155.21.126 at
com.mongodb.internal.dns.DefaultDnsResolver.resolveHostFromSrvRecords(DefaultDnsResolver.java:64)
~[mongodb-driver-core-4.4.2.jar:na] at
com.mongodb.internal.connection.DefaultDnsSrvRecordMonitor$DnsSrvRecordMonitorRunnable.run(DefaultDnsSrvRecordMonitor.java:78)
~[mongodb-driver-core-4.4.2.jar:na] at
java.base/java.lang.Thread.run(Thread.java:829) ~[na:an]
please, can anyone to explain how to exactly add MongoDB to the spring project?
I prefer to use such solution, since I need to complete the task and launch my project in the another PC. In any case - I tried to use mongoDb in the docker and this way also not work for me, and I not sure the other side use MongoDB locally.
I'm making an application for a school project, but I'm running into the issue that when I try to run the unit tests that it tries to connect to the database while starting up the application, which isn't required for the tests (because it will be mocked), and is not available in the CI/CD pipeline.
jdbc connection error
I'm building my project in Java Maven Springboot and would like to know how I can prevent it from trying to connect to the database when running my test.
here is a link to my repository: https://gitlab.com/kwetter_jack/Kwetter_posts/-/tree/ci_cd_setup
Your test classes have #SpringBootTest annotation which will start a Spring application context - as your application uses a database the tests will also try to setup and use a database connection.
The simplest solution is to remove the annotation so the tests no longer try to connect to a database. You'll probably need to mock some more dependencies as a result as Spring is no longer creating these for you. You could also have a look at https://www.baeldung.com/spring-boot-testing for some other ideas how you could alter your tests.
Alternatively if you do want / need the application context to run you can add a application.yaml for the tests that defines and uses a in memory DB so the tests have something to connect to - see https://www.baeldung.com/spring-boot-h2-database for details how to do this.
Just change value under spring.datasource to H2 database to prevent
The application connect the real database.
Test application.yml
FYI, You no need to copy all config from original application.yml, just only some config that you need to override.
while I was investigating the spring boot H2 in-memory database (as suggested by Chris Olive and Paranaaan) I also came across the option of using a test container. after looking into this I saw that this enables the project to create a temp docker container with a MySQL image that I can use during the testing of my project, considering I was planning on using docker anyway for the integration testing of my microservices project I attempted this and it worked as I had hoped it would.
if anyone is interested in the test container solution that I used, the information can be found here:
https://www.testcontainers.org/modules/databases/mysql/
I'm trying to make Spring Boot application with Flyway (and Hikari pool) to start the server even when the DB is not available at that time.
I need to support cases when:
1. DB is not available when applicaition starts (it should run Flyway after DB starts, it can be up to 30 mins).
2. DB goes offline during the application lifetime and then goes back up.
I got a problem with the first case, Flyway always tries to do migrations even when DB is not available and application stops.
I tried adding spring.datasource.continue-on-error: true but Flyway ignores that, and I couldn't find any flyway configuration that would allow such operation.
Is it possible or should I wrap Flyway and do it myself?
Spring boot 2.1.4
A couple of points to consider
What is the desired behavior of the application when the DB is really not available when the instance of java application? Ok, so flyway won't start, but how the application will be able to handle requests that will have to reach the database?
Flyway itself relies on DataSource bean, maybe on hibernate if you use it, and these are much more complicated infrastructures than flyway itself?
Maybe if the database is not available the application won't need to start at all?
Instead it worth to rely on orchestrators (like kubernetes, ECS or whatever that will recognize that the application didn't start and will try to retrigger the start again, again, and again till the database will be ready)?
This is my recommendation in general.
Now, assuming find answers to all these questions and still, want to proceed with this path:
Spring Boot by itself works like this when it comes to flyway integration:
If the relevant classes (Flyway class) exist on classpath and spring.flyway.enabled=true then the bean of flyway starts and spring boot does its magic.
Technically the relevant auto configuration can be found in class org.springframework.boot.autoconfigure.flyway.FlywayAutoConfiguration (org.springframework.boot:spring-boot-autoconfigure module)
I think the way to go is to disable flyway, and given that fact that beans like DataSource are available (somehow) - create a Flyway Bean by yourself and trigger the migration in some kind of loop in the background that will exit only if the migration actually succeeds (or already applied)
I am working on a Spring MVC project with running Tomcat server and using Eclipse STS IDE.
I need to test my project's functionality with different databases (SQL, oracle and postgreSQL database).
To test same functionality in all databases, every time I am changing applicationProperies.properties file to change database configuration and restarting the sever. Is there any way where I can test my application parallely with all databases?
I am developing a web app using Spring MVC and Hibernate that I hope to package up as a WAR file and distribute for users to deploy where needed.
The application will require the database connection details before deploying, can anyone advise on the best practice of how to do this? currently my database.properties file is packaged as part of the WAR file, is there an easier way than asking the user to manually edit this file inside the WAR file or in the source and then expect them to build the war file themselves?
I would also like an easy way for users to create a single admin user on first deploy easily - is it best to just provide a sql script to create that or is there a way to include the functionality in the web app so on first deploy it creates the user?
Thanks
Since web applications are not focused to "mass production", I can't tell if something like that is possible.
You can make an SQL script containing all your CREATE statements limited by "IF NOT EXISTS" to avoid deleting existing data. Hibernate can execute a sql script at the moment of creating the SessionFactory, if Hibernate finds a file named "import.sql" located in the classpath root, Hibernate will execute it. Also you can always edit the database.properties of an application already deployed without re-packaging the WAR.
Try Liquibase, http://www.liquibase.org/ . It does not allow your users to setup database connection details but it allows you to manage the database schema (create and update the schema) and default data.