I currently have one database connected and it is working. I would like to connect another (and eventually 2 more) databases. How do I do so? There should be a solution using only annotations and properties files.
I also read this question, How to use 2 or more databases with spring?, but I dont know how it works too well/ if it will apply. I'm not using a controller class and I dont know what that does. I'm also not sure how the config class they mention in the answer actually connects to the specific DO.
This is my application.properties file: (marked out username and password but its there in my file)
hibernate.dialect=org.hibernate.dialect.SQLServer2012Dialect
hibernate.show_sql=true
hibernate.format_sql=true
hibernate.default_schema=dbo
hibernate.packagesToScan=src.repositories.LMClientRepository.java
spring.jpa.generate-ddl=true
spring.jpa.hibernate.naming-strategy=org.hibernate.cfg.DefaultNamingStrategy
spring.datasource.username=***
spring.datasource.password=***
spring.datasource.url=jdbc:sqlserver://schqvsqlaod:1433;database=dbMOBClientTemp;integratedSecurity=false;
spring.datasource.testOnBorrow=true
spring.datasource.validationQuery=SELECT 1
spring.jpa.database=dbMOBClientTemp
spring.jpa.show-sql=true
spring.jpa.hibernate.dialect=org.hibernate.dialect.SQLServer2012Dialect
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
This is my application file:
package testApplication;
import java.util.ArrayList;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.orm.jpa.EntityScan;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import fileRetrieval.InputFileParse;
import lmDataObjects.LMClientDO;
import lmDataObjects.LoadMethodDO;
import repositories.LMClientRepository;
import repositories.LoadMethodRepository;
#SpringBootApplication
#EnableJpaRepositories(basePackageClasses = LoadMethodRepository.class)
#EntityScan(basePackageClasses = LoadMethodDO.class)
#EnableCaching
public class Application {
private static final Logger log = LoggerFactory.getLogger(Application.class);
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Bean
public CommandLineRunner demo(LoadMethodRepository lm_repo, LMClientRepository lmc_repo) {
return (args) -> {
List<LMClientDO> lmlist = InputFileParse.getMultiGroupfile();
List<String> uniqueMediaIds = new ArrayList(InputFileParse.getUniqueMediaIds());
for (int i = 0; i < InputFileParse.getUniqueMediaIds().size(); i ++){
lm_repo.save(new LoadMethodDO(uniqueMediaIds.get(i)));
}
for (int i = 0; i < lmlist.size(); i++){
lmc_repo.save(new LMClientDO(lmlist.get(i).getClientId(), lmlist.get(i).getMediaId()));
}
//Here is where I would like to do stuff with data from the other database that I have not connected yet
};
}
}
I also made a new properties file called application-MTS.properties and I put data for the new database in there. Still unsure of what to do with it.
spring.datasource.username=***
spring.datasource.password=***
spring.datasource.url=jdbc:sqlserver://SCHQVSQLCON2\VSPD:1433;database=dbMTS;integratedSecurity=false;
Spring Data has understood this is a common use case that people may want so they created a example project on how to do this. I would review the spring-data-examples for multiple-datasources.
The important aspect is to look at the OrderConfig and CustomerConfig classes as they define the two data sources.
For Hibernate you can have mutiple hbm.xml files and get that in a class and use it.
Something like this:
if ("yourCondition".equals(definedCondition)) {
sf = new Configuration().configure("example.cfg.xml").buildSessionFactory();
} else {
sf = new Configuration().configure("exampleTwo.cfg.xml").buildSessionFactory();
}
You can have separate database and login info in the xml files.
If you want two separate connections, you can store them in separate methods and call them to create multiple sessions.
Related
I am trying to write integration test for neo4j using spring boot. I am using test container. Can anyone help me how to load database dump file to testcontainer?
here's one way to do this (using current Neo4j 5.3). I have a dump file called neo4j.dump created from a local instance and I use withCopyFileToContainer to copy it to the container before it starts.
As I use the community edition in this example, there is no online backup/restore, but only dump/load. So therefor I have to change the startup command. The load needs to happen before Neo4j starts. I can't stop Neo4j in the container because it would stop the container.
Therefor I create a small shell script that executes the desired other command and than delegates to the original entry point.
This script is transferred with file mode 0100555, corresponding to r-xr-xr-x so that it is executable.
Finally, the container is started with the above script as command.
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.images.builder.Transferable;
import org.testcontainers.utility.MountableFile;
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class LoadDumpTest {
Neo4jContainer<?> neo4j;
Driver driver;
#BeforeAll
void initNeo4j() {
neo4j = new Neo4jContainer<>("neo4j:5.3.0")
.withCopyFileToContainer(MountableFile.forClasspathResource("neo4j.dump"),
"/var/lib/neo4j/data/dumps/neo4j.dump")
.withCopyToContainer(Transferable.of("""
#!/bin/bash -eu
/var/lib/neo4j/bin/neo4j-admin database load neo4j
/startup/docker-entrypoint.sh neo4j
""", 0100555), "/startup/load-dump-and-start.sh")
.withCommand("/startup/load-dump-and-start.sh")
.withLogConsumer(f -> System.out.print(f.getUtf8String()));
neo4j.start();
driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", neo4j.getAdminPassword()));
}
#Test
void dataShouldHaveBeenLoaded() {
try (var session = driver.session()) {
var numNodes = session.run("MATCH (n) RETURN count(n)").single().get(0).asLong();
Assertions.assertTrue(numNodes > 0);
}
}
#AfterAll
void stopNeo4j() {
neo4j.stop();
}
}
Edit:
If you are on a recent enterprise edition, Christophe suggested the following solution on Twitter, which I do personally think it's superior, as it is less hacky.
https://github.com/ikwattro/neo4j-5-testcontainers-restore-backup/blob/main/src/test/java/dev/ikwattro/Neo4j5RestoreBackupExampleTest.java
It makes use of seed URIs for databases. Copying or binding the resource in his example works the same as in mine.
If you're using Neo4j 5, I suggest you make backups instead of dumps. Using backups you can take advantage of the seedUri option when creating a database, meaning you can create a database from an URI pointing to a backup on disk ( or in the neo4j container ). https://neo4j.com/docs/operations-manual/current/clustering/databases/#cluster-seed-uri
Here is an example using Testcontainers
package dev.ikwattro;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.neo4j.driver.AuthTokens;
import org.neo4j.driver.Driver;
import org.neo4j.driver.GraphDatabase;
import org.neo4j.driver.Session;
import org.neo4j.driver.SessionConfig;
import org.testcontainers.containers.BindMode;
import org.testcontainers.containers.Neo4jContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import static org.assertj.core.api.Assertions.assertThat;
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#Testcontainers(disabledWithoutDocker = true)
public class Neo4j5RestoreBackupExampleTest {
#Container
private Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5.3.0-enterprise")
.withAdminPassword("password")
.withEnv("NEO4J_dbms_memory_heap_max__size", "256M")
.withEnv("NEO4J_dbms_databases_seed__from__uri__providers", "URLConnectionSeedProvider")
.withClasspathResourceMapping("backups", "/backups", BindMode.READ_ONLY)
.withEnv("NEO4J_ACCEPT_LICENSE_AGREEMENT", "yes");
#BeforeAll
void beforeAll() {
neo4j.start();
createDbFromBackup();
}
#Test
void testCreatingDbFromBackup() {
try (Driver driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", "password"))) {
try (Session session = driver.session(SessionConfig.forDatabase("worldcup22"))) {
var result = session.run("MATCH (n) RETURN count(n) AS c").single().get("c").asLong();
assertThat(result).isPositive();
}
}
}
private void createDbFromBackup() {
try (Driver driver = GraphDatabase.driver(neo4j.getBoltUrl(), AuthTokens.basic("neo4j", "password"))) {
try (Session session = driver.session(SessionConfig.forDatabase("system"))) {
session.run("""
CREATE DATABASE worldcup22 OPTIONS { existingData: "use", seedUri: "file:///backups/world-cup-2022-neo4j.backup"}
""");
}
}
}
}
You can find a working maven project here https://github.com/ikwattro/neo4j-5-testcontainers-restore-backup
I am trying to configure the Confluent - ConsumerTimestampsInterceptor to support the Confluent KAFKA Replication and have configured Java spring boot application as mentioned below
application.properties
#consumer timestamp interceptor
interceptor.classes=io.confluent.connect.replicator.offsets.ConsumerTimestampsInterceptor
timestamp.producer.security.protocol=PLAINTEXT
timestamp.producer.sasl.mechanism=NONE
#timestamp.producer.ssl.endpoint.identification.algorithm=
#timestamp.producer.sasl.jaas.config=
src.consumer.group.id=lkc-302y2
Consumer.java
package com.example.demo.service;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Service;
import com.example.demo.model.ProductKafka;
import java.io.IOException;
import java.util.ArrayDeque;
import java.util.LinkedList;
import java.util.Queue;
import java.util.logging.Logger;
#Service
public class Consumer {
private static final Logger LOGGER = Logger.getLogger(Consumer.class.getName());
public static ArrayDeque<ProductKafka> consumeQueue = new ArrayDeque<>();
#KafkaListener(autoStartup = "false", topics = "#{'${spring.kafka.topics}'.split('\\\\ ')}", groupId = "#{'${spring.kafka.groupId}'}")
public void consume(ProductKafka productKafka) throws IOException {
consumeQueue.offer(productKafka);
LOGGER.info(String.format("#### -> Logger Consumed message -> %s", productKafka.toString()));
System.out.printf("#### -> Consumed message -> %s", productKafka.toString());
}
}
and below is the project structure
enter image description here
however it is not working.
Our KAFKA administrator mentioned that though I have the Interceptor added KAFKA consumer is not configured to use the interceptor.
I don't know how to make the consumer use the Interceptor
I am trying to migrate application from spring-boot 1.5.9.RELEASE to 2.0.0.CR1.
In the spring-boot 2 org.springframework.boot.autoconfigure.session.StoreType.HASH_MAP was removed. In my application it was used in application.yml
spring:
session:
store-type: hash_map
What should I do to preserve previous behavior with new version of spring-boot and why this store-type was removed?
you can use the following:
#EnableSpringHttpSession
#Configuration
public class SpringHttpSessionConfig {
#Bean
public MapSessionRepository sessionRepository() {
return new MapSessionRepository(new ConcurrentHashMap<>());
}
}
see equivalent question on SpringSession git repository
Below config worked for me. This configuration was used for testing and lower environment. for production we used redis instead of none or hash_map.
Add below code in application-local.yml file.
spring:
session:
store-type: none
server:
servlet:
max-http-header-size: 20000
session:
cookie:
max-age: 300 # absolute session timeout in seconds
timeout: 300
Create config class.
import java.util.concurrent.ConcurrentHashMap;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.session.MapSessionRepository;
import org.springframework.session.SessionRepository;
import org.springframework.session.config.annotation.web.http.EnableSpringHttpSession;
import org.springframework.session.web.http.HeaderHttpSessionStrategy;
import org.springframework.session.web.http.HttpSessionStrategy;
#EnableSpringHttpSession
#Configuration
#ConditionalOnProperty(prefix = "spring.session", value = "store-type", havingValue = "none")
public class HttpSessionConfig {
#Value("${server.servlet.session.timeout:300}")
private Integer maxInactiveIntervalInSeconds;
#Bean
public SessionRepository sessionRepository() {
SessionRepository sessionRepository = new MapSessionRepository(new ConcurrentHashMap<>());
((MapSessionRepository) sessionRepository)
.setDefaultMaxInactiveInterval(maxInactiveIntervalInSeconds);
return sessionRepository;
}
}
I want to run unit tests on a database other than the default one. Here is my application.conf:
application.secret="[cut]"
application.langs="en"
db.default.driver=com.mysql.jdbc.Driver
db.default.url="jdbc:mysql://localhost:3306/city_game?characterEncoding=UTF-8"
db.default.user=root
db.default.password=""
db.test.driver=com.mysql.jdbc.Driver
db.test.url="jdbc:mysql://localhost:3306/play_test?characterEncoding=UTF-8"
db.test.user=root
db.test.password=""
ebean.default="models.*"
ebean.test="models.*"
logger.root=ERROR
logger.play=INFO
logger.application=DEBUG
BaseModelTest.java:
package models;
import com.avaje.ebean.Ebean;
import com.avaje.ebean.EbeanServer;
import com.avaje.ebean.config.ServerConfig;
import com.avaje.ebeaninternal.server.ddl.DdlGenerator;
import com.avaje.ebean.config.dbplatform.MySqlPlatform;
import com.avaje.ebeaninternal.api.SpiEbeanServer;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import play.test.FakeApplication;
import play.test.Helpers;
import java.io.IOException;
public class BaseModelTest
{
public static FakeApplication app;
public static DdlGenerator ddl;
#BeforeClass
public static void startApp() throws IOException
{
app = Helpers.fakeApplication();
Helpers.start(app);
String serverName = "test";
EbeanServer server = Ebean.getServer(serverName);
ServerConfig config = new ServerConfig();
ddl = new DdlGenerator();
ddl.setup((SpiEbeanServer) server, new MySqlPlatform(), config);
}
#AfterClass
public static void stopApp()
{
Helpers.stop(app);
}
#Before
public void dropCreateDb() throws IOException
{
// Drop
ddl.runScript(false, ddl.generateDropDdl());
// Create
ddl.runScript(false, ddl.generateCreateDdl());
}
}
However, I get results saved in the default database, and the test one has its tables created but empty. What I expect is to have the results written to the test db and default one untouched.
I somehow ended with different approach.
I still created separate real test database instance (because of stored procedures), but instead I used the Play1-like approach.
I have separates configuration sides beneath my main configuration (e.g. test configuration, prod specific stuff, stage specific stuff etc ..)
I load it via Global.scala as shown below (please note the exaple provided below works in in Play for java developers version as well)
object Global extends GlobalSettings {
override def onLoadConfig(config: Configuration, path: File, cl: ClassLoader, mode: Mode.Mode): Configuration = {
val modeFile: String = s"application.${mode.toString.toLowerCase}.conf"
Logger.error(s"Loading {${path.toURI}conf/application.conf}")
Logger.error(s"Appending mode specific configuration {${path.toURI}conf/$modeFile}")
val modeConfig = config ++ Configuration(ConfigFactory.load(modeFile))
super.onLoadConfig(modeConfig, path, cl, mode)
}
}
And the application.test.conf config file is as follows:
# test database
db.default.logStatements=false
db.default.jndiName=DefaultDS
db.default.url="jdbc:postgresql://127.0.0.1:5432/db-test"
db.default.user=user
db.default.password="password!##$"
db.default.driver=org.postgresql.Driver
This way I get following benefits:
I still write my tests the usual way
Play evolutions gets tested on CI / jenkins as well
I have to write my tests in the way I could safely retun them on the existing db instance w/ minimal assumptions about data and userbase. That way I'm 90% certain I will be able to run those against staging / prod environment with much less friction. (Controversial point)
I think you should separate your code
as these
#BeforeClass
public static void startApp() throws IOException {
app = Helpers.fakeApplication();
Helpers.start(app);
}
#Before
public void dropCreateDb() throws IOException {
String serverName = "test";
EbeanServer server = Ebean.getServer(serverName);
ServerConfig config = new ServerConfig();
DdlGenerator ddl = new DdlGenerator((SpiEbeanServer) server, new MySqlPlatform(), config);
// Drop
ddl.runScript(false, ddl.generateDropDdl());
// Create
ddl.runScript(false, ddl.generateCreateDdl());
}
I know that there is a configuration file called web.xml
What I want to achieve is have another configuration file that has application specific configuration and it has to be read when the web server is started. I also want a Class to be able to read this configuration. Is there a way I can configure this is web.xml file itself or is there another way
You can use the Apache Commons Configuration. Have a look at the user guide. Since you want it to be done on startup here is a sample ServletContextListener:
package test;
import java.io.File;
import java.net.MalformedURLException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.apache.commons.configuration.Configuration;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.XMLConfiguration;
public class ConfigurationListener implements ServletContextListener {
#Override
public void contextInitialized(ServletContextEvent sce) {
ServletContext context = sce.getServletContext();
File configFile;
try {
configFile = new File(context.getResource("/WEB-INF/configuration.xml").getPath());
Configuration config = new XMLConfiguration(configFile);
context.setAttribute("configuration", config);
} catch (ConfigurationException | MalformedURLException ex) {
Logger.getLogger(ConfigurationListener.class.getName()).log(Level.SEVERE, null, ex);
}
}
#Override
public void contextDestroyed(ServletContextEvent sce) {}
}
Now get your configuration anywhere in your web application like this:
Configuration config = (Configuration) request.getServletContext().getAttribute("configuration");
I would create a class to hold the configuration though rather than adding it as an attribute to the ServletContext. The class would simply provide access to the configuration through a static method.