I have spring boot application which is has spring ws and backend as Stored procedure, I am trying to write integration tests for this and looking for inmemmory database which supports stored procedure?
i tried with h2 database but it looks for java functions to work on stored procedure. i sthere any direct mechanism where I can put my stored procedure with minimal effort?
so, there is no ready made solution. i needed this for integration testing but solution provide by h2 database required me to rewrite the stored procedure in java classes. http://www.h2database.com/html/features.html#user_defined_functions
Considering the pain to write integration test and amount of effort, I decided to go against it. Hope it may help to some else.
Related
As the application gets complicated, one thing that change a lot is the queries, especially if they are complex queries. Wouldn't it be easier to maintain the queries in the db rather then the resources location inside the package, so that it can be enhanced easily without a code change. What are the drawbacks of this?
You can use stores procedures, to save your queries in the database. Than your Java code can just call the procedure from the database instead of building a complex query.
See wikipedia for a more detailed explanation about stored procedures:
https://en.wikipedia.org/wiki/Stored_procedure
You can find details about the implementation and usage in the documentation of your database system (MySql, MariaDb, Oracle...)
When you decide to move logic to the database, you should use a version control system for databases like liquibase: https://www.liquibase.org/get-started/quickstart
You can write the changes to you database code in xml, json or even yaml and check that in in your version control system (svn, git...). This way you have a history of the changes and can roll back to a previous version of your procedure, if something goes wrong.
You also asked, why some people use stored procedures and others keep their queries in the code.
Stored procedures can encapsulate the query and provide an interface to the data. They can be faster than queries. That is good.
But there are also problems
you distribute the buisiness logic of your application to the database and the programm code. It can realy be troublesome, if the logic is spread through all technical layers of your applicaton.
it is not so simple anymore to switch from a Oracle database to a MariaDb, if you use specific features of the database system. You have to migrate or rewrite the procedures.
you have to integrate liquibase or another system into you build pipeline, to keep track of you database changes.
So it depends on the project and it's size, if either of the solutions is better.
I am testing this api that creates databases/tables in postgres. For automated testing, i was thinking along the lines of, having a setup method that creates a database with tables setup and populated with required data (1000 entries/rows).
I was thinking of an elegant way of doing this? Any thoughts apart from writing code that loops over 1000 times and writing data stored in a csv to postgres table?
Honestly CSV, XML, or any other structured format seems fine to me. Is there a reason you don't want to do that? Using the pg_dump command to export data from an existing DB and using pg_restore could be a good option too.
Your other idea about writing code to generate the data isn't bad either. The benefit of writing code is that your test isn't coupled to a data file.
Also, I would take a look at the H2 database because it has PostgreSQL compatibility mode, and you can actually embed it in your unit/integration tests instead of relying on a PostgreSQL server to be set up and configured in your tests. We've used H2 to test our PostgreSQL app, and it's worked well. The downside is you can't be 100% sure that just because your test passes against H2 that it will against PostgreSQL.
If you really prefer to use postgresql (instead of H2) for testing you could use liquibase. It's a database schema management tool that supports (among others) bulk loading data from csv (http://www.liquibase.org/documentation/changes/load_data.html).
It also offers spring integration if your application use it.
This question is extracted from a comment I posted here:
What's the best strategy for unit-testing database-driven applications?
So I have a huge database schema for a legacy application (with quite an old code base) that has many tables, synonyms, triggers, and dblinks. We and we have (finally) started to test some part of the application.
Our tests are already using mocks, but in order to test the queries that we are using we have decided to use an in-memory db with short-lived test dataset.
But the setup of the in-memory database requires a specific SQL script for the db schema setup. The script is not the real DDL we have in production because we can not import it directly.
To make things harder, the database contains functions and procedures that needs to be implemented in Java (we use the h2 db, and that is the way to declare procedures).
I'm afraid that our test won't break the day the real db will change and we will spot the problem only at runtime, potentially in production.
I know that our tests are quite at the border between integration and unit. However with the current architecture it is quite hard to insulate the test from the db. And we want to have proper tests for the db queries (no ORM inside).
What would be solution to have a DDL as close as possible of the real one and without the need to manually maintain it ?
If your environments are Dockerized I would highly suggest checking out Testcontainers (https://www.testcontainers.org/modules/databases/). We have used it to replace in-memory databases in our tests with database instances created from production DDL scripts.
Additionally, you can use tmpfs mounting to get performance levels similar to in-memory databases. This is nicely explained in following post from Vlad Mihalcea: https://vladmihalcea.com/how-to-run-integration-tests-at-warp-speed-with-docker-and-tmpfs/.
This combination works great for our purposes (especially when combined with Hibernate auto-ddl option) and I recommend that you check it out.
Since I'm not really proficient with databases, some details may be irrlevant, but I'll include everything:
As part of a project in my University, we're creating a website that uses JSP, servlets and uses a MySQL server as backend.
I'm in charge of setting up the tables on the DB, and creating the Java classes to interact with it. However, we can only connect to the MySQL server from inside the University, while we all (7 people) work mostly at home.
I'm creating an interface QueryHandler which has a method that takes a string (representing a query) and returns ResultSet. My question is this: How do I create a class that implements this interface which will simulate a database and allow others to use different DBHandlers and not know the difference and allow me to test different queries without connecting to the actual MySQL database?
EDIT: I'm not so sure on the differences between SQL databases, but obviously all the queries I run on MySQL should run on the mock.
Why not just install your own MySQL database for testing? It runs on Windows, Mac and Linux, and it's not too resource heavy. I have it installed on my laptop for local testing.
Your API appears to be flawed. You should not be returning ResultSets to clients. By doing so, you are forever forcing your clients to rely on a relational database backend. Your data access layer needs to hide all of the details of how your data is actually structured and stored.
Instead of returning a ResultSet, consider returning a List or allowing the client to supply a Stream that your data access component can write to.
This will make unit tests trivial for the clients of the API and will allow you to swap storage mechanisms at will.
Try derby. It's a free server you can use to test against, if you don't mind having to change drivers when you go back to SqlServer. You might be limited in the kind of queries you can run though. I'm not sure if SqlServer has any special syntax outside of standard SQL.
How about using a HSQLDB for offline tests? It wont behave exactly like a MySQL DB but is a fast in memory SQL DB that should satisfy most of your needs.
The best way in my experience is multiple database instances and or schemas. Normally you'd have one for each user to do their development against/sanity checking the running application, one for an automated build for running unit tests and ideally one for each user to run their unit tests against. And of course instances/schemas for demos, integration testing. Apart from the practial side, being able to do this ensures deploying/upgrading the app/database will be pretty near faultless too.
Assuming you have a DAO layer, the only code that needs access to a real database at the unit test level is the DAO implementation, the business layer should be using a mock DAO implementation.
I am a newbie in Hibernate.
I am working on a cloud service data access layer.
Currently we are using Hibernate for OR mapping and as data access layer using Hibernate annotations. But lately i have been asked to implement Hibernate/Data Access layer in such a way that my stored procedures be in HQL and we can change our DB at a short notice and port our entire code.
The closest i can think in this regard is by using Named queries , where stored procedures are at DB side and my hibernate is resolving the stored procedure calls using named queries.
The reason for all that is the notion that since stored procedures are precompiled therefore they give good performance and security optimization for a large cloud service implementation.
currently i am using java , hibernate and Mysql.
Can anybody examine my assumptions and validate or give/suggest some better alternatives.
Performance and security are top priority.
I think the approach you outlined is great.
That is exactly what I would do if I were in your position. (I'm on Hibernate backed by MySql also, and have considered doing this if needed for performance reasons.)
Since parsing and optimizing statements is fast with the most DBMSes, I prefer not to use stored procedures if my application is the 'owner' of the catalogue(s).
With stored procedures, migration and maintainance can become more difficult which outweights the little tiny performance profits.
Cases, where I see the benefits of stored procedures:
I'm not the owner of the database. Access to data is provided by database developers / maintainers (like you find in Datawarehouses often). So stored procedures are an interface to the data.
Statements are complex and the runtime is unpredictable or should have no affects on my application (like triggering long running transactions or batches).
Hope, that'll help you with your decision.