I'm using MongoDB and PostgreSQL in my application. The need of using MongoDB is we might have any number of new fields that would get inserted for which we'll store data in MongoDB.
We are storing our fixed field values in PostgreSQL and custom field values in MongoDB.
E.g.
**Employee Table (RDBMS):**
id Name Salary
1 Krish 40000
**Employee Collection (MongoDB):**
{
<some autogenerated id of mongodb>
instanceId: 1 (The id of SQL: MANUALLY ASSIGNED),
employeeCode: A001
}
We get the records from SQL, and from their ids, we fetch related records from MongoDB. Then map the result to get the values of new fields and send on UI.
Now I'm searching for some optimized solution to get the MongoDB results in PostgreSQL POJO / Model so I don't have to fetch the data manually from MongoDB by passing ids of SQL and then mapping them again.
Is there any way through which I can connect MongoDB with PostgreSQL through columns (Here Id of RDBMS and instanceId of MongoDB) so that with one fetch, I can get related Mongo result too. Any kind of return type is acceptable but I need all of them at one call.
I'm using Hibernate and Spring in my application.
Using Spring Data might be the best solution for your use case, since it supports both:
JPA
MongoDB
You can still get all data in one request but that doesn't mean you have to use a single DB call. You can have one service call which spans to twp database calls. Because the PostgreSQL row is probably the primary entity, I advise you to share the PostgreSQL primary key with MongoDB too.
There's no need to have separate IDs. This way you can simply fetch the SQL and the Mongo document by the same ID. Sharing the same ID can give you the advantage of processing those requests concurrently and merging the result prior to returning from the service call. So the service method duration will not take the sum of the two Repositories calls, being the max of these to calls.
Astonishingly, yes, you potentially can. There's a foreign data wrapper named mongo_fdw that allows PostgreSQL to query MongoDB. I haven't used it and have no opinion as to its performance, utility or quality.
I would be very surprised if you could effectively use this via Hibernate, unless you can convince Hibernate that the FDW mapped "tables" are just views. You might have more luck with EclipseLink and their "NoSQL" support if you want to do it at the Java level.
Separately, this sounds like a monstrosity of a design. There are many sane ways to do what you want within a decent RDBMS, without going for a hybrid database platform. There's a time and a place for hybrid, but I really doubt your situation justifies the complexity.
Just use PostgreSQL's json / jsonb support to support dynamic mappings. Or use traditional options like storing json as text fields, storing XML, or even EAV mapping. Don't build a rube goldberg machine.
Related
I am using Spring boot to create a web app. I need to store and query a list of patients however the table Patients has 30+ Columns so it is too big to use a java model. I was wondering if there is a way where I can query ( using sql script ) my Patient table and then display the list on html
I'm not sure what you mean by "big model".
I'd recommend using JdbcTemplate f you mean "I don't want all that JPA/Hibernate machinery". It's the lightest, simplest way to interact with relational databases using SQL.
You don't need a complex object model. Map each row into a Map of column name/value pairs and return a List of them for each query. Simple.
I'm not expert but I think you need a model but you can simplified this model. You can add properties as you will use. Incoming SQL query response assign to your simple model. Also I think it's good that you want to simplify the model. It will be useful.
In .NET C#, we used Odata to filter, page, sort the database results from SQL database. Odata in .NET would actually go into the database, and query WHERE, ORDER By Filters to database, instead of extracting all the database results, and applying filtering on the api memory.
I am curious if GraphQL, queries the database internally or applies filtering on the API memory set.
Resource:
https://graphql.org/
GraphQL is mainly a specification that defines a query language , a type system , a way/framework such that you have to follow it to implement for querying or mutating the data based on this query language and the type system. (i.e. implement various resolvers in the GraphQL term).
It does not define anythings related to where the data should be stored. So it does not define anything related to the SQL and SQL database.
I am curious if GraphQL, queries the database internally or applies
filtering on the API memory set.
So it depends on how you implement it. To have a good performance, of course you have to convert the query type that you defined to an efficient SQL with an efficient where and limit clause , and send to DB to query the result set internally.
I'm trying to change the default Database when the program starts. First, I start with default Database (main_database) and after select some tables, I want to change to another table (second_database).
I use this code but doesn't work:
String qlQuery = "USE second_database;";
Query query = entityManager.createNativeQuery(qlQuery);
query.getResultList();
Server server = serverService.findById(1);
but it seems like getResultList is only for Selects.
How can I solve that problem?
I'm using Spring Boot and JPA.
Thanks!
SOLUTION:
Tenancy (to Spring Boot and JPA):
https://javadeveloperzone.com/spring-boot/spring-boot-jpa-multi-tenancy-example/
I can think of two options, but it all depends on your usecases:
1. Creating live views in "default_database"
You are working with 31 child databases, but do you need access to all their data? If this cannot be answered (e.g. you expect as a future requirement to need access to arbitrary table from one of these 31 databases) live views is a no go.
If it is determined that all the data that you will ever need for your application is - from DB 1 - table A and B; from DB2 - table C; from DB3 - table D and E; and so on - it might be a good approach to create views.
You should also take into account table data size and operations to be performed (e.g. read only or also writes?)
2. Configuring Hibernate for multitenancy:
You can configure hibernate to execute queries in different databases.
You need to take care of the following things:
the multitenancy strategy - for MySql use MultiTenancyStrategy.DATABASE
the MultiTenancyConnectionProvider implementation which you can pass via hibernate.multi_tenant_connection_provider property
the CurrentTenantIdentifierResolver implementation which you can pass via hibernate.tenant_identifier_resolver property
You can follow the official doc for more details and code samples as well as this excellent hands-on article by the master himself.
I am working on an JavaEE application, and there are almost 1000+ tables in the database, now I have to query the records by the parametes from the client.
Generally I will create one Entity for each table, and create the Dao,Service to do the query.
However I meet two problems:
1 Number of the tables
As I said, 1000+ table with almost 40+ columns for each, it would a nightmare to create the entity one by one.
2 Scheme update
Even I can create the Entity by program, the schema of the data may change sometime which is out of my control.
And in my application, only read operations are related to these kinds of data,no update,delete,create required.
So I wonder if the following solution is possible:
1 Use Map instead of POJOs
Do not create POJOs at all, use the native Map to wrap the columns and values.
2 Row mapping
When querying using Hibernate or Spring JdbcTemplate or something else, use a mapper to map each row to an entry in the map.
If yes, I would use the ResultMetaData to detect the column name,type,value:
ResultMetaData rmd=rs.getMetaData();
for(int i=0;i<rmd.getColumnCount();i++){
Type t=rmd.getType(i)
if(t==....){
...
}else if(t=...){
...
}
}
Looks like part of JPA's job, any library can used here?
If not, any other alternatives?
Recently I am working on a bilingual project and for some reference tables I need to sort the data. But because it is bilingual, the data is coming from different languages (in my case English and French) and I like to sort them all together, for example, Île comes before Inlet.
Ordinary Order By will put Île at the end of the list. I finally came up with using nativeQuery and sort the data using database engine's function (in oracle is about using NLS_SORT)
But I am tight with database engine and version, so for example if I change my database to postgres then the application will break. I was looking for native JPA solution (if exists) or any other solutions.
To archive this, without use native query JPA definition, I just can see two ways:
Create a DB view which includes escaped/translated columns based on DB functions. So, the DB differences will be on the create view sentence. You can define a OneToOne relation property to original entity.
Create extra column which stores the escaped values and sort by it. The application can perform the escape/translate before store data in DB using JPA Entity Listeners or in the persist/merge methods.
Good luck!