Develop Reference

Export from Oracle and Import to Azure SQL reusing Entities Spring Boot

I'm trying to fetch data from an Oracle DB and import it to Azure Sql. The databases have the same structure and therefore I was thinking if I could use the same entities and repositories with different datasources for it.
#Entity
#Table(name = "Street", indexes = {#Index(name = "street_id_index", columnList = "streetid")})
public class Street {
#Id
private String streetid;
public String streetcode;
public String streetname;
public String streetnameaddd;
#ManyToOne(cascade = CascadeType.ALL)
#JoinColumn(name = "municipalitycode", referencedColumnName = "municipalitycode")
public Municipality municipalitycode;
public String is_deleted;
}
public interface Street_Repository extends JpaRepository<Street,String> {
List<Street> findAll();
}
#Configuration
public class AzureConfig {
#Bean("AzureDataSource")
#ConfigurationProperties(prefix = "spring.datasource-azure-sql")
public DataSource dataSourceAzureSql(){
return DataSourceBuilder.create().build();
}
}
#Configuration
public class OracleConfig {
#Bean
#ConfigurationProperties(prefix = "spring.datasource-oracle")
public DataSource dataSourceOracle(){
return DataSourceBuilder.create().build();
}
}
spring:
datasource-azure-sql:
jdbc-url: jdbc:sqlserver://xxxxxxx.database.windows.net:1433;database=xxxxxxxxxxx-xxx;user=xxxxx#xxxxx;password=xxxxxxxxx;encrypt=true;trustServerCertificate=false;hostNameInCertificate=xxxxxxxx;loginTimeout=30
datasource-oracle:
jdbc-url: jdbc:oracle:thin:#xxxxx.xx.xxx:xxx:ssid
#Repository
public class Stree_Repo implements Street_Repository {
private JdbcTemplate jdbcTemplate;
#Autowired
#Qualifier("AzureDataSource")
public void setDataSource(DataSource dataSource){
this.jdbcTemplate = new JdbcTemplate(dataSource);
}
#Autowired
#Qualifier("OracleDataSource")
public void setDataSource2(DataSource dataSource){
this.jdbcTemplate = new JdbcTemplate(dataSource);
}
#Override
public List<Street> findAll() {
return null;
}
... rest override methods...
}
So the classe Street_Repo implements the interface and because this Entity is part of both Oracle and Azure I was wondering if there was a possible of not repeating the Entity and all the other classes associated.
Thank you in advance!

Is there a way to map mutliple databases with same table names in a dynamic way?

First time I encounter this problem. The situation is:
I have more than 100 SQL databases, each one correspond to a different company and each one have the same three tables (same table names, same column names, same column data type).
Is there some way to map all these databases dynamically?
With dynamically I mean to have one class to which I can refer and make any CRUD operation.

After some research I could see what I wanted to do:
Basically I needed to change my data source in run time, for that I used a spring framework interface called AbstracRoutingDataSource.
Example:
Implementing AbstractRoutingDataSource:
public class MultiRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return DBContextHolder.getCurrentDb();
}
}
DatabaseContextHolder:
public class DBContextHolder {
private static final ThreadLocal<DBTypeEnum> contextHolder = new ThreadLocal<>();
public static void setCurrentDb(DBTypeEnum dbType) {
contextHolder.set(dbType);
}
public static DBTypeEnum getCurrentDb() {
return contextHolder.get();
}
public static void clear() {
contextHolder.remove();
}
}
Database type enum:
public enum DBTypeEnum{
DATASOURCE1("DATASOURCE1"),
DATASOURCE2("DATASOURCE2");
DBTypeEnum(final String dbTypeEnum){
this.dbTypeEnum = dbTypeEnum;
}
private String dbTypeEnum;
public String dbTypeEnum(){
return dbTypeEnum;
}
}
Persistence configuration:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
basePackages = "base.packages.path",
entityManagerFactoryRef = "multiEntityManager",
transactionManagerRef = "multiTransactionManager"
)
public class PersistenceConfiguration {
private final String PACKAGE_SCAN = "base.package.path";
#Bean(name = "dataSource1")
#ConfigurationProperties("spring.datasource1")
public DataSource dataSource1() {
return DataSourceBuilder.create().build();
}
#Bean(name = "dataSource2")
#ConfigurationProperties("spring.datasource2")
public DataSource dataSource2() {
return DataSourceBuilder.create().build();
}
#Bean(name = "multiRoutingDataSource")
public DataSource multiRoutingDataSource() {
Map<Object, Object> targetDataSources = new HashMap<>();
targetDataSources.put(DBTypeEnum.DATASOURCE1, dataSource1());
targetDataSources.put(DBTypeEnum.DATASOURCE2, dataSource2());
MultiRoutingDataSource multiRoutingDataSource = new MultiRoutingDataSource();
multiRoutingDataSource.setDefaultTargetDataSource(dataSource1());
multiRoutingDataSource.setTargetDataSources(targetDataSources);
return multiRoutingDataSource;
}
#Bean(name = "multiEntityManager")
public LocalContainerEntityManagerFactoryBean multiEntityManager() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(multiRoutingDataSource());
em.setPackagesToScan(PACKAGE_SCAN);
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(hibernateProperties());
return em;
}
#Bean(name = "multiTransactionManager")
public PlatformTransactionManager multiTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
multiEntityManager().getObject());
return transactionManager;
}
#Bean(name = "dbSessionFactory")
public LocalSessionFactoryBean dbSessionFactory() {
LocalSessionFactoryBean sessionFactoryBean = new LocalSessionFactoryBean();
sessionFactoryBean.setDataSource(multiRoutingDataSource());
sessionFactoryBean.setPackagesToScan(PACKAGE_SCAN);
sessionFactoryBean.setHibernateProperties(hibernateProperties());
return sessionFactoryBean;
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put("hibernate.show_sql", true);
properties.put("hibernate.format_sql", true);
return properties;
}
}
Then you need to have all of your database information in a .properties file:
spring.datasource1.jdbcUrl=jdbc:sql:sql-url:3306/datasource1
spring.datasource1.username=username
spring.datasource1.password=password
spring.datasource1.driver-class-name= Driver
spring.datasource2.jdbcUrl=jdbc:sql:sql-url:3306/datasource2
spring.datasource2.username=username
spring.datasource2.password=password
spring.datasource2.driver-class-name= Driver
Then you need to map your entity:
#Entity
#Table(name = "table_name")
#Getter
#Setter
public class MyEntity implements Serializable {
#Id
#Column(name = "ID", columnDefinition = "varchar(17)")
private String id;
//more fields...
}
I used spring CrudRepositories interface for this entity
public interface IMyEntityRepository extends CrudRepository<MyEntity, String> {
}
Finally my controller is prepared to change datasource depending on a JSON field in my request.
JSON:
{
"dataSource":"DATASOURCE1"
//more fields ...
}
RESTController:
#PutMapping("/url/{id}")
public ResponseEntity<?> editMyEntity(#RequestBody RequestObject request #PathVariable String id){
DBContextHolder.setCurrentDb(DBTypeEnum.valueOf(request.getDataSource);
iMyEntitiRepository.getMyEntity(id);
//...
}

What is wrong of this code? (Multiple Datasource Oracle)

Currently I have to connect two Oracle Datasources in my Spring Boot proyect. I think that I did well the configuration because Spring validated my connection but when I call a service this always response me "Empty". I think is because the TransactionManager don't change.
This is my project structure:
- project
⊢---- config
⊢---- controllers
∟---- models
⊢---- dao
| ⊢---- db1
| ∟---- db2
⊢---- entity
| ⊢---- db1
| ∟---- db2
⊢---- services
⊢---- db1
∟---- db2
This are my entities classes:
package project.models.entity.db1;
//// IMPORTS
#Entity(name = "User")
#Table(name = "T_USER")
public class User implements Serializable {
private static final long serialVersionUID = 1L;
#Id
private Long id;
private String name;
private String surname;
private String username;
private String password;
////// CONSTRUCT
////// GETTERS & SETTERS
////// HASCODE, EQUALS AND TOSTRING
}
package project.models.entity.db2;
//// IMPORTS
#Entity(name = "Shareholder")
#Table(name = "Shareholder")
public class Shareholder implements Serializable {
private static final long serialVersionUID = 1L;
#Id
private Integer id;
private Integer numberTitles;
////// CONSTRUCT
////// GETTERS & SETTERS
////// HASCODE, EQUALS AND TOSTRING
}
This are my repositories classes:
package project.models.dao.db1;
//// IMPORTS
#PersistenceContext(name = "db1")
#Repository
public interface IUserDao extends JpaRepository<User, Long> {
}
package project.models.dao.db2;
//// IMPORTS
#PersistenceContext(name = "db2")
#Repository
public interface IShareholderDao extends JpaRepository<Shareholder, Integer> {
}
This are my services classes:
package project.models.services.db1.impl;
//// IMPORTS
#Service
public class UserServiceImpl implements IUserService {
#Autowired
private IUserDao userDao;
#Override
#Transactional("transactionManager")
public List<User> findAll() {
return userDao.findAll();
}
#Override
#Transactional("transactionManager")
public User findById(Long id) {
return userDao.findById(id).orElse(null);
}
package project.models.services.db2.impl;
//// IMPORTS
#Service
public class ShareholderServiceImpl implements IShareholderService {
#Autowired
private IShareholderDao shareholderDao;
#Override
#Transactional("db2TransactionManager")
public List<Shareholder> findAll() {
return shareholderDao.findAll();
}
#Override
#Transactional("db2TransactionManager")
public Shareholder findById(Integer id) {
return shareholderDao.findById(id).orElse(null);
}
}
And this are my config classes:
package project.config;
//// IMPORTS
#Configuration
#ConfigurationProperties(prefix = "oracle1")
#EnableJpaRepositories(entityManagerFactoryRef = "entityManagerFactory", transactionManagerRef = "transactionManager", basePackages = {
"project.models.dao.db1" })
#EnableTransactionManagement
#Validated
public class Db1Config {
#NotNull
private String username;
#NotNull
private String password;
#NotNull
private String url;
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory() throws SQLException {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource());
em.setPackagesToScan(new String[] { "project.models.entity.db1" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(hibernateProperties());
em.setPersistenceUnitName("db1");
return em;
}
#Bean(name = "dataSource")
DataSource dataSource() throws SQLException {
OracleDataSource dataSource = new OracleDataSource();
dataSource.setUser(username);
dataSource.setPassword(password);
dataSource.setURL(url);
dataSource.setImplicitCachingEnabled(true);
dataSource.setFastConnectionFailoverEnabled(true);
return dataSource;
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager() throws SQLException {
return new JpaTransactionManager(entityManagerFactory().getObject());
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put("hibernate.dialect", "org.hibernate.dialect.Oracle10gDialect");
properties.put("hibernate.show_sql", "true");
properties.put("hibernate.hbm2ddl.auto", "validate");
return properties;
}
}
package project.config;
//// IMPORTS
#Configuration
#ConfigurationProperties(prefix = "oracle2")
#EnableJpaRepositories(entityManagerFactoryRef = "db2EntityManagerFactory", transactionManagerRef = "db2TransactionManager", basePackages = {
"project.models.dao.db2" })
#EnableTransactionManagement
#Validated
public class Db2Config {
#NotNull
private String username;
#NotNull
private String password;
#NotNull
private String url;
#Bean(name = "db2EntityManagerFactory")
public LocalContainerEntityManagerFactoryBean db2EntityManagerFactory() throws SQLException {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(db2DataSource());
em.setPackagesToScan(new String[] { "project.models.entity.db2" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
em.setJpaProperties(hibernateProperties());
em.setPersistenceUnitName("db2");
return em;
}
#Bean(name = "db2DataSource")
DataSource db2DataSource() throws SQLException {
OracleDataSource dataSource = new OracleDataSource();
dataSource.setUser(username);
dataSource.setPassword(password);
dataSource.setURL(url);
dataSource.setImplicitCachingEnabled(true);
dataSource.setFastConnectionFailoverEnabled(true);
return dataSource;
}
#Bean(name = "db2TransactionManager")
public PlatformTransactionManager db2TransactionManager() throws SQLException {
return new JpaTransactionManager(db2EntityManagerFactory().getObject());
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put("hibernate.dialect", "org.hibernate.dialect.Oracle10gDialect");
properties.put("hibernate.show_sql", "true");
properties.put("hibernate.hbm2ddl.auto", "validate");
return properties;
}
}
And my main application:
package project;
//// IMPORTS
#SpringBootApplication
#EnableAutoConfiguration(exclude = { DataSourceAutoConfiguration.class, HibernateJpaAutoConfiguration.class,
DataSourceTransactionManagerAutoConfiguration.class })
#EnableCaching
public class ProjectApplication {
#Autowired
private IUserService userService;
#Autowired
private IShareholderService shareholderService;
public static void main(String[] args) {
SpringApplication.run(ProjectApplication.class, args);
}
#PostConstruct
void init() {
List<User> user = userService.findAll();
List<Shareholder> shareholder = shareholderService.findAll();
System.out.println();
System.out.println(user);
System.out.println(shareholder);
System.out.println();
}
}
And always response me:
[User [id=39, name=Jack, surname=Sparrow, ...], [...], ...]
[]
What is the error? Because I don't see it.
Thank you very much

In Db2Config class, package being scanned is different than Db1Config.
Db1Config:
em.setPackagesToScan(new String[] { "project.models.entity.db1" });
Db2Config
em.setPackagesToScan(new String[] { "project.models.dao.db2" });

No transactional EntityManager available when Persisting an object

I have got No transactional EntityManager available exception when I tried to persist an object.
I have a Spring project with the following configuration class:
#Configuration
#ComponentScan
#EnableTransactionManagement
#EnableJpaRepositories
public class SpringConfiguration {
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean emf(DataSource dataSource) {
LocalContainerEntityManagerFactoryBean emf =new LocalContainerEntityManagerFactoryBean();
emf.setDataSource(dataSource);
Properties jpaProperties = new Properties();
jpaProperties.put("hibernate.hbm2ddl.auto", "create-drop");
jpaProperties.put("hibernate.show_sql", "true");
emf.setJpaProperties(jpaProperties);
emf.setPackagesToScan(
new String[] {"ds.core.entity"});
emf.setJpaVendorAdapter(
new HibernateJpaVendorAdapter());
return emf;
}
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(EntityManagerFactory emf,DataSource dataSource) {
JpaTransactionManager tm =
new JpaTransactionManager();
tm.setEntityManagerFactory(emf);
tm.setDataSource(dataSource);
return tm;
}
And an entity class:
#Entity
public class Type {
#Id #GeneratedValue
private Long id;
private String name;
}
and Jpa implementation class :
#Repository
public class JpaTypeRepo implements TypeRepo {
#PersistenceContext
private EntityManager em;
#Override
public Type insertRecord(Type data) {
em.persist(data);
return data;
}
}
And finally a Service Class with Init():
#Transactional
#Service
public class InitDbService {
#Autowired
private TypeRepo typeRepo;
#PostConstruct
public void init() {
Type type=new Type();
type.setName("newName");
typeRepo.insertRecord(type);
}}

just add #Transactional annotation to the class :JpaTypeRepo
as following:
#Transactional
#Repository
public class JpaTypeRepo implements TypeRepo {
#PersistenceContext
private EntityManager em;
#Override
public Type insertRecord(Type data) {
em.persist(data);
return data;
}
}

Spring Hibernate: Entity seems not persisted, no failures either

My model is
import javax.annotation.Nonnull;
import javax.persistence.Entity;
import org.joda.time.DateTime;
#Entity
public class Network extends AbstractEntity {
private long networkId;
private String name;
private boolean active;
private DateTime createdAt;
private String createdBy;
private DateTime updatedAt;
private String updatedBy;
...
}
AbstractEntity is
#MappedSuperclass
public class AbstractEntity {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
private Long id;
public Long getId() {
return id;
}
#Override
public boolean equals(final Object obj) {
if (this == obj) {
return true;
}
if (obj == null || getClass() != obj.getClass()) {
return false;
}
final AbstractEntity abstractEntity = (AbstractEntity) obj;
return id.equals(abstractEntity.id);
}
#Override
public int hashCode() {
return id == null ? 0 : id.hashCode();
}
}
I have Repository where I want to add entity. I do
#Repository
public class NetworkRepositoryImpl implements NetworkRepository {
#PersistenceContext
private EntityManager entityManager;
#Override
#Nonnull
public List<Network> findAll() {
//noinspection JpaQlInspection
final Query query = entityManager.createQuery("SELECT e FROM Network e");
return (List<Network>)query.getResultList();
}
#Nonnull
#Override
#Transactional
public Network save(#Nonnull final Network network) {
if (network.getId() == null) {
entityManager.persist(network);
return network;
} else {
return entityManager.merge(network);
}
}
}
and I Test it as
#Test
public void testAllNetworks() {
final Network network = new Network(1, "US", true, DateTime.now());
final Network networkInDb = networkRepository.save(network);
assertNotNull(networkInDb.getId());
final List<Network> networks = networkRepository.findAll();
assertTrue(networks.isEmpty());
}
my ApplicationConfig looks like
#Configuration
#ComponentScan
#EnableJpaRepositories
public class ApplicationConfig {
#Bean
public DataSource dataSource() {
final EmbeddedDatabaseBuilder embeddedDatabaseBuilder = new EmbeddedDatabaseBuilder();
embeddedDatabaseBuilder.setType(EmbeddedDatabaseType.H2);
return embeddedDatabaseBuilder.build();
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
final HibernateJpaVendorAdapter jpaVendorAdapter = new HibernateJpaVendorAdapter();
jpaVendorAdapter.setDatabase(Database.H2);
jpaVendorAdapter.setGenerateDdl(true);
final LocalContainerEntityManagerFactoryBean localContainerEntityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
localContainerEntityManagerFactoryBean.setJpaVendorAdapter(jpaVendorAdapter);
localContainerEntityManagerFactoryBean.setPackagesToScan("com.org.comma.persistence");
localContainerEntityManagerFactoryBean.setDataSource(dataSource());
localContainerEntityManagerFactoryBean.setPersistenceUnitName("test-comma-pu");
return localContainerEntityManagerFactoryBean;
}
#Bean
public PlatformTransactionManager transactionManager() {
final JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory().getObject());
return transactionManager;
}
}
My test fails on line
assertNotNull(networkInDb.getId());
meaning the entity is not persisted
What I am doing wrong here?

In my case, I was using wrong annotation.
I was using
import javax.transaction.Transactional;
instead, what I needed was
import org.springframework.transaction.annotation.Transactional;
Atleast I see it try persisting and failing somewhere else

If your test fails because Id value is null, then it is expected behavior.
When hibernate persist an entity, it will not fetch the auto-generated key value automatically. You should either find the entity again or refresh it manually.
In similar situation, I usually refresh the entity like below.
entityManager.persist(network);
entityManager.flush();
entityManager.refresh(network);
P.S: You should flush before refreshing to make it reliable.