I have 3 MySQL tables and i need to generate their corresponding entities in java (eclipse)
is there an easy way ?
im working on a spring boot application
While the majority of people will recommend jpa(spring data) this is codeFirst approach, for DB first approach I have a great framework recommendation:
If you have some time on your hand I will go and try Jooq
Jooq website
Jooq has a maven dependency for creating all necessary objects from your db:
Jooq auto code generation from db
codeGen configuration in maven
Jooq can connect to your db, and even connect to your liquibase schema upgrades so that the object are always on the same stage with the db
I don't have any connection to Jooq but Lukas Eder (#lukaseder) is an awsome guy, he always answer questions in stack Overflow, so you are in good hands, also if you are using reactor Jooq is fully compatible(also compatible with JavaRX)
Related
I would like to find a reliable way to test my Maria DB schema with Jooq.
This is what I have now:
let Jooq with Gradle to extract an XML schema from the real DB (i.e. Maria instance) via org.jooq.codegen.XMLGenerator, this step will generate a maria_information_schema.xml;
use this schema to generate all the Java Classes.
To test all the classes I will write I have develop a technique:
fire a H2 embedded in ram database;
build a java InitDatabase.java class which manually set a DB schema as similar as possible with the MariaDB one;
preform all the test on the in ram DB.
This procedure works perfectly as far as I don't change something on the real DB and I forgot to do the same on the InitDatabase.java class.
My question is: is there a way to use the XML schema or the generated java classes to create an H2 Database with the same schema as the MariaDb one without writing manually all the create table statement?
Thanks
This is an open ended question with no obvious, "correct" answer. jOOQ's official take here is that you may want to re-consider using alternative RDBMS for testing what you could be doing with your target RDBMS directly, specifically using testcontainers.
You could combine this approach with jOOQ's code generation for a more streamlined development process.
In short, jOOQ's recommendation is to use:
Flyway or Liquibase for database change management
Testcontainers for code generation
Testcontainers for integration testing
Other, equivalent products are obviously also possible.
My goal is to enable schema and data migration for an existing application.
This kind of question seems to have been asked many times, however with different requirements and circumstances as mine, I think.
Since I am inexperienced in this domain, allow me to lay out the architecture of the app and my assumptions first.
Architecture
The app is a multi-user, enterprise desktop application with a backend server that can persist to any major DB (MySql, Postgresql, SQL Server, Oracle DB, etc). It is assumed the DB is on-premise and maintained by our clients.
The tech stack used is a fairly common Hibernate+Spring+RMI/JMS-Combo.
Currently, migrations are done by the server in the following way:
On server start it checks for the latest expected schema version
If larger than the current version, start migration to next version until current==latest:
Create new database
Load (whole) latest schema (SQL script with a lot of CREATE TABLE ...)
Migrate data (in Java classes using 2 JDBC-Connections to old and new schema)
Load (all) latest constraints (SQL script with a lot of ALTER TABLE ...)
This migration is slow and forward-only. But it is simple. The problem is, that until now the schema scripts and the queries in the data migrations have been using MySQL-syntax and features.
Note that by migrate data I mean: the backend server copies the data from the old schema to the new one, transforming it if necessary.
Also, the migration process starts automatically on-premise of our clients. Meaning, we only have control over the JDBC connection, but no direct access to the database nor knowledge about the specific database being used (MySQL, SQL Server,...).
The goal is to either replace or augment this migration scheme with a database independent one.
Assumptions and research
StackOverflow 1 2 3 4 5 6 7: Answers state to use Hibernate's inbuilt feature. However, the docs state that this is not production ready. Also, AFAICT, all answers are concerned with schema migration only.
Liquibase: Uses a custom DSL (in XML/JSON/YAML/etc) to allow for database independent schema migration only.
DBUnit: Uses custom XML-DSL to capture snapshots of states of databases. Can not recreate a snapshot of schema version 1 to version 2.
flyway: In principle same as Liquibase. But is not database independent because SQL-Scripts are used for migrations.
JOOQ: A database independent Query-DSL in Java on top of JDBC. Comparable to Criteria API but without the drawbacks of JPA. Should in principle allow for database independent data migration, however, does not help with schema migration.
JPA-Query languages like HQL, JPQL, Criteria API are not sufficient because
One cannot reference tables not mapped by the entity manager. E.g. join tables, metadata and audit tables.
A copy of all versions of the Entity classes needs to be kept around for the mapping.
Question
I realize, that as this question stands now, it will be dismissed as opinion-based.
However, I am not necessarily looking for specific solutions to this problem ( I doubt there exists a clear solution for such a complex problem space ) but rather to validate my assumptions.
Namely, is it true, that
Liquibase and Flyway are mainly concerned with schema migration and data migration is left as an exercise for the reader?
in order for Flyway to support multiple, different databases, one needs to duplicate the migrations scripts per database?
by and large, the problem of database independent data migration remains unresolved in enterprise Java?
Even if I was to combine Liquibase/Flyway with JOOQ, I do not see how to perform a data migration, because Liquibase/Flyway migrate databases in place. The old database gets destroyed and with it the opportunity to transform the old data to the new schema.
Thanks for your attention!
Let's break it down a little bit. You're right in that this is largely opinion based, but here's what I've noticed in my experiences.
Liquibase and Flyway are mainly concerned with schema migration and data migration is left as an exercise for the reader?
You can do data migration with liquibase and flyway. It's something I've done pretty often. Take the example where I want to split a User table into User and Address tables. I'd write a migration script, which is basically just a sql file, to create the new Address table and the copy all the relevant data into it.
in order for Flyway to support multiple, different databases, one needs to duplicate the migrations scripts per database?
Possibly, flyway and liquibase are better thought of as database versioning tools. If my app needs version 10 of the database, these tools would help me get to that point. Again, the migration scripts are just basic .sql files. If you're using some mysql specific functions then those will just go in the migration script and they wouldn't work on a sql server
by and large, the problem of database independent data migration remains unresolved in enterprise Java?
Eh, I'm not sure about this one. I agree its a problem, but in practice it's not a huge one. For the past 8+ years, I've only written ansi sql. It should be portable everywhere. So in theory, we can lift those applications on to a different database. JPA and the various implementations help with those differences. Depending on how your project was built, say an application that has all of its business logic in implementation specific sql functions, then it's going to be a headache. If you're using the database for CRUD, and I'd argue that's all you should be using it for, then it's not a huge deal.
So all that said, I think you might have the wrong idea about flyway and liquibase. Like i said earlier, they aren't really 'migration tools' so much as they are database versioning tools. With a list of specific sql migration scripts that are ordered, i can guarantee the state of my database at any version. I'm not sure these are tools that I'd use to 'migrate' a legacy SQL Server based application into a PostGres based application.
I currently use jOOQ to generate Java code from my database and Flyway to manage my binary (Java) migrations as well as SQL migrations.
However, I run into problems when I modify existing tables. For example, if I were to drop a column in one migration and a past binary migration was dependent on that column, the migration will have a syntax error because the field wouldn't exist in jOOQ anymore.
I know I could just comment out the body of the migration but that kind of defeats the whole purpose of Flyway or any sort of database version manager if I can't rerun my migrations or makes it very tedious (run 1 migration, uncomment, run next, generate jOOQ, etc)
Is there a better way to approach this problem?
I'd argue this is a workflow problem.
You are effectively upgrading an API with each migration, expecting legacy consumers of that API to continue to work would be nothing short of miraculous.
jOOQ is a great tool but using it in this context (to assist migrations) is certainly going to lead to trouble.
My suggestion would be to rethink you schema evolution strategy; using raw SQL, which comes naturally to Flyway, and leave jOOQ for exclusively assisting your application instead.
I am new in the hibernate world. My project task is to represent xml result from the SQL database. To do so, first step, I am trying to generate POJOs and mapping xml from my db by using hibernate tools auto generating feature. When, i am trying to generate a simple database (2/3 tables) its working fine.
But while i trying to convert my real database, which has 28 tables within different table relationships. Then i am facing the below problem. Not able to connect the database tables. Hibernate tools configuration showing this terrible message.(Foreign key name (fk_p_einheit) mapped to different tables! previous: org.hibernate.mapping.Table(public.einheit_quelle) current:org.hibernate.mapping.Table(public.spalten)
I checked google to get the solution but not get any proper solution related to eclipse IDE. Just got one BUG report from NetBeans site(https://netbeans.org/bugzilla/show_bug.cgi?id=205863).
My setup configuration is:eclipse-jee-luna-SR1-win32-x86_64, postgres (sql 9.1-901.jdbc4) and hibernate core (4.3.5.Final) with maven project and hibernate tools.
Any one please help me.
You need an object model that maps all of those 28 tables and relationships. Do you have one? If not, why are you using Hibernate?
You can ask Hibernate to generate an object model for you from the schema. Try it.
Why are you using all these unknown technologies (Hibernate, Eclipse, etc.) to do something simple?
If you're thinking in terms of tables and columns, and not objects, I'd say you'd be better off writing JDBC and straight SQL. ORM tools aren't for you.
I'm looking for a general solution for upgrading database schema with ORM tools, like JPOX or Hibernate. How do you do it in your projects?
The first solution that comes to my mind is to create my own mechanism for upgrading databases, with SQL scripts doing all the work. But in this case I'll have to remember about creating new scripts every time the object mappings are updated. And I'll still have to deal with low-level SQL queries, instead of just defining mappings and allowing the ORM tools to do all the job...
So the question is how to do it properly. Maybe some tools allow for simplifying this task (for example, I heard that Rails have such mechanism built-in), if so please help me decide which ORM tool to choose for my next Java project.
LiquiBase is an interesting open source library for handling database refactorings (upgrades). I have not used it, but will definitely give it a try on my next project where I need to upgrade a db schema.
I don't see why ORM generated schemas are any different to other DB schemas - the problem is the same. Assuming your ORM will spit out a generation script, you can use an external tool to do the diff
I've not tried it but google came back with SQLCompare as one option - I'm sure there are others.
We hand code SQL update scripts and we tear down the schema and rebuild it applying the update scripts as part of our continuous build process. If any hibernate mappings do not match the schema, the build will fail.
You can check this feature comparison of some database schema upgrade tools.
A comparison of the number of questions in SOW of some of those tools:
mybatis (1049 questions tagged)
Liquibase (663 questions tagged)
Flyway (400 questions tagged)
DBDeploy (24 questions tagged).
DbMaintain can also help here.
I think your best bet is to use an ORM-tool that includes database migration like SubSonic:
http://subsonicproject.com/2-1-pakala/subsonic-using-migrations/
We ended up making update scripts each time we changed the database. So there's a script from version 10 to 11, from 11 to 12, etc.. Then we can run any consecutive set of scripts to skip from some existing version to the new version. We stored the existing version in the database so we could detect this upon startup.
Yes this involved database-specific code! One of the main problems with Hibernate!
When working with Hibernate, I use an installer class that runs from the command-line and has options for creating database schema, inserting base data, and dynamically updating the database schema using SchemaUpdate. I find it to be extremely useful. It also gives me a place to put one-off scripts that will be run when a new version is launched to, for example, populate a new field in an existing DB table.