Develop Reference

Clickhouse jdbc schema [Airbyte integration]

I'm really looking for and advice on how to set up data streaming between Oracle db (just for example) and Clickhouse database using Airbyte ETL tool.
I've created the source and destination connectors and successfuly tested the connections but now I'm facing the issue in syncing the data. While the source is sending the data correctly, the destination (Clickhouse) JDBC driver requires the schema to accept those tables.
So, I don't know how to retrieve this schema and where it should be stored. Maybe there're some means to create this schema, because when I inspect the log it displays such thing:
So, basically, the error is connected with Exception in thread "main" java.lang.IllegalStateException: jdbc destinations must specify a schema.
Please I will be really grateful for providing any means for resolving that issue!

I think use should try version 0.1.1 of ClickHouse destination. It fixes your issue https://github.com/airbytehq/airbyte/pull/8982

How to create a database if it does not exist with springboot/JPA and SQL Server?

I am surprised I haven't found an SO question that answers this. I am trying to connect a springboot/JPA application to an SQL Server on my local machine. I have the application setup so that it can connect to a database if it it exists, but if I change the JDBC URL to create the database if it doesn't exist then it fails. Here is what the properties look like when it fails.
spring.jpa.hibernate.ddl-auto=update
spring.datasource.url=jdbc:sqlserver://localhost:1433;databaseName=testing;createDatabaseIfNotExist=true;
spring.datasource.username=hello
spring.datasource.password=Hello1234
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.jpa.database-platform=org.hibernate.dialect.SQLServer2008Dialect
spring.jpa.show-sql=true
Here is a snippet of the error I receive when starting the app:
com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user
'hello'. ClientConnectionId:971a3369-258b-4713-bddc-cda559b9fe94 at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:262)
~[mssql-jdbc-8.4.0.jre11.jar:na] at
com.microsoft.sqlserver.jdbc.TDSTokenHandle
If anybody has any thoughts as to how I can change this so the database is created if it does not exist I would be very thankful. Thanks in advnace.

I don't think a database can be created using JPA.
It has to be created manually or in some other ways, but JPA won't do that for you.
And it would be a bad practice as well to create the database using the application itself, and the use of same credentials.

Yes, definitely you can auto-create a database with JPA for that
spring.datasource.url=jdbc:sqlserver://localhost:1433;databaseName=testing;
createDatabaseIfNotExist=true;
line should be changed to:
spring.datasource.url=jdbc:sqlserver://localhost:1433
/testing?createDatabaseIfNotExist=true

In practice your application should never create your database so its not really a problem most of the time(Outside small databases like sqlite3). Different databases would handle this situation differently as well.
In your case I do not see this as a valid jdbc parameter in the documentation.
I would recommend creating the database in advance with a privileged user separate from your application user.

Make schema created by Spring JDBC the default schema

I'm using Spring Boot and JDBC for my database connection. I placed schema.sql at the classpath to initialize a schema and tables.
Because the schema doesn't exist yet while connecting to the datasource, I have to configure the datasource in application.properties like so:
spring.datasource.url=jdbc:mysql://localhost:3306/
schema.sql:
CREATE DATABASE IF NOT EXISTS <schema_name>
USE <schema.name>;
CREATE TABLE...
So I select the schema after creating it. This obviously doesn't persist for too long.
How do I configure this properly? Is there a way to select a default schema after the create script or maybe change the datasource url?

With JDBC you need to use Connection.setCatalog to switch between databases. You should not use USE <databasename> as the JDBC driver itself needs to be aware of which database it is operating on.

Based on your code from the schema.sql
USE <schema.name>;
This will not work for your java environment. The schema.sql will be executed and finished, which will not cater your requirement to set the default schema.
The General approach will be to use JDBC URL as;
jdbc:mysql://localhost:3306/DB_NAME
This will set the default db as DB_NAME.
Assumptions: I am assuming that you want to connect to single node DB without any loadbalancer or failover mechanism to be used. URL may change based on these features to be configured.
If you are not specifying the DB_NAME in the URL, it means their is no default schema.
You have 2 options to access the DB in that case.
1) Always use the Connection.setCatalog() method to specify the desired database in JDBC applications, rather than the USE database statement.
2) fully specify table names using the database name (that is, SELECT dbname.tablename.colname FROM dbname.tablename...) in your SQL. Opening a connection without specifying the database to use is generally only useful when building tools that work with multiple databases, such as GUI database managers.
For more details refer to the below mysql portal for reference.
https://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html

Fallback Database with Hibernate

do someone of you know a way to have 2 Databases running parallel?
We are using Hibernate 4 and as a main database Postgres 9.3 - this db is hosted on another machine then the application - if the database is down we still have to save some stuff.
So first intention was to write it into a csv, but I'm not a friend of writing stuff into an unordered file. So I want just to use a fallback Database (thinking of H2 Database). Does someone has experience with such a construct?
We are also using Spring 4 - I would just set up another datasource + sessionfactory + transactionmanager - and add the name at the #Transactional method to use the right manager. any other ideas?
Thank you!!

You can extend the Spring AbstractRoutingDataSource and configure two actual data sources:
a primary PostgreSQL data source
a secondary H2 data source
The application logic will see only one data source, which is the router which will decide which data source is going to switch to on demand.
When the primary data source is down you need to instruct the router to pick the fall-back H2 one.

Get JDBC connection working in play framework (java)

Was wondering, what steps am I missing to get a jdbc embeded h2 database working in my play application? Following these docs.
So far editted Application.conf file to contain this:
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:databases/test"
db.default.user=test
db.default.password="testtest"
Next I created a libs directory and added the jar file
h2-1.3.174.jar
Is this necessary or does the provided driver handle all types of h2 databases (embeded and server - I know it handles in memory)?
Now in the controler how can I access the database? Do I have to start/shutdown the database?
I know I can get connections from the getConnection() method in play.db. But everytime I execute a statement through this connection I get an exception saying no data is available. If I then check - looks like directory
databases/test
was not created so no database files exist.
What am I missing?

H2 works out of the box. Just create a new project in the terminal.
Otherwise, to your listing:
I think you should change db.default.url="jdbc:h2:databases/test" to db.default.url="jdbc:h2:mem:play"
don't need to create lib directories. It's all handeled by the build in dependency mgmt sbt
Just use the model objects and call save/update. No need to call start/shutdown
you are in a framework, it's all there ready for you...
I think you should start reading the documentation from the beginning to the end and examine the example applications. It's all there what you are looking for.

In addition to myborobudur's answer I'll only mention, that you don't need to use memory database, as you can for an instance use file storage (Embedded) or even run H2 as a server and then connect to it with TCP in Server Mode... Everything is clearly described in H2 documentation.