After opening a database connection, I want to check if the database is newly minted or not. I am using H2 which automatically creates a DB if one doesn't exist.
I tried this check:
db.Public.PUBLIC.getTables().isEmpty()
but that returns a static list of tables (without querying the schema in the database).
I could write raw SQL to get the table list, but that would be specific to the database engine. Is there a generic alternative in jOOQ?
You cannot use:
db.Public.PUBLIC.getTables().isEmpty()
Because generated meta information is not connected to the database. Instead, you may want to take a look at DSLContext.meta(). In your case, you'd simply write:
DSL.using(configuration).meta().getTables().isEmpty();
If you run this test very often, that's of course not a very performant way of checking if there are any tables, as it will fetch all tables into memory just to run an isEmpty() check. I suggest issuing an actual query instead:
int numberOfTables =
DSL.using(configuration)
.select(count())
.from("information_schema.tables")
.where("table_schema = 'PUBLIC'")
.fetchOne(0, int.class);
A future jOOQ version (after 3.11) will be able to offer you actual object existence predicates that can be used in SQL or elsewhere:
https://github.com/jOOQ/jOOQ/issues/8038
Related
I am trying to log a “change summary” from each INSERT/UPDATE MySQL/SQL Server query that executes in a Java program. For example, let’s say I have the following query:
Connection con = ...
PreparedStatement ps = con.prepareStatement(“INSERT INTO cars (color, brand) VALUES (?, ?)”);
ps.setString(1, “red”);
ps.setString(2, “toyota”);
ps.executeUpdate();
I want to build a “change set“ from this query so I know that one row was inserted into the cars table with the values color=red and brand=toyota.
Ideally, I would like MySQL/SQL Server to tell me this information as that would be the most accurate. I want to avoid using a Java SQL parser because I may have queries with “IF EXISTS BEGIN ELSE END”, in which case I would want to know what was the final query that was inserted/updated.
I only want to track INSERT/UPDATE queries. Is this possible?
What ORM do you use? If you don't use one, now could be the time to start - you give the impression that you have all these prepared statement scattered throughout the code, which is something that needs improving anyway.
Using something like Hibernate means you can just activate its logging and keep the query/parameter data. It might also make you focus your data later a bit more (if it's a bit haphazardly structured right now).
If you're not willing to switch to using an ORM consider creating your own class, perhaps called LoggingPreparedStatement, that is identical to normal PreparedStatement (subclass or wrapper of PreparedStatement such that it uses all the same method names etc so it's a drop in replacement) and logs whatever you want. Use find/replace across the code base to switch to using it.
As an alternative to doing it on the client side, you can get the database to do It. For SQL server it has change tracking, don't know what there is for MySQL but it'll be something proprietary. For something consistent, most DB have triggers that have some mechanism of identifying old and new data and you can stash this in a history table(s) to see what was changed and when. Triggers that keep history have a regularity to their code that means they can be programmatically generated from a list of the table columns and datatypes, so you can query the db for the column names (most db have some virtual tables that tell you info about the real tables) etc and generate your triggers in code and (re)apply them whenever schema changes. The advantage of using triggers is that they really easily identify the data that was changed. The disadvantage is that this is all they can see so if you want your trigger to know more you have to add that info to the table or the session so the trigger can access it - stuff like who ran the query, what the query was. If you're not willing to add useless columns to a table (and indeed, why should you) you can rename all your tables and provide a set of views that select from the new names and are named the old names. These new views can expose extra columns that your client side can update and the views themselves can have INSTEAD OF triggers that update the real tables. Doesn't help for selections though because deleting data doesn't need any data from the client, so the whole thing is a mess. If you were going that wholesale on your DB you'd just switch to using stored procedures for your data modifications and embark on a massive job to change your client side calls. An alternative that is also well leveraged for SQL Server is the CONTEXT_INFO variable, a 128byte variable block of binary data that lives for the life of your connection/session or it's newer upgrade SESSION_CONTEXT, a 256kb set of key value pairs. If you're building something at the client side that logs the user, query and parameter data and you're also building a trigger that logs the data change you could use these variables, programmatically set at the start of each data modification statement, to give your trigger something more involved than "what is the current time" to identify which triggered dataset relates to which query logged. Generating a guid in the client and passing it to the db in some globally readable way that means the database trigger can see it and log it in the history table , tying the client side log of the statement and parameters to the server side set of logged row changes
I need to find RDBMS table metadata for an application. I am able to find Column Metadata, table existence using java.sql.DatabaseMetaData and row count by executing query on java.sql.Statement.
How need to check
boolean canRead()
boolean canWrite()
on table.
I found isReadOnly() method in DatabaseMetaData class. But it
Retrieves whether this database is in read-only mode.
Is there any way to check these features generically for RDBMS databases at table level?
You are looking for getTablePrivileges()
Retrieves a description of the access rights for each table available in a catalog. Note that a table privilege applies to one or more columns in the table. It would be wrong to assume that this privilege applies to all columns (this may be true for some systems but is not true for all.)
The column PRIVILEGE from the ResultSet will contain the information you are looking for. If I remember correctly you will get one row in the result for each privilege that was granted.
As the JavaDoc for getTablePrivileges() suggests, you might need to also check getColumnPrivileges() - but that depends on your use case and the DBMS you expect to support
When I run a query in BigQuery Web UI, the results are displayed in a table where both name and type of each field are known (even when a field is a result of COUNT(), AVG(), ... operation, type of field is known, of course).
The results can be then directly exported as a table/json/csv.
My question is, when I retrieve query results in my java project, e.g. with a query:
String query = "SELECT nationality, COUNT(DISTINCT personID) AS population
FROM Dataset.Table
GROUP BY nationality";
PCollection<TableRow> result = p.apply(BigQueryIO.Read.fromQuery(query));
... is it possible to obtain the schema of TableRow in result PCollection, without explicitly defining it?
I think it must be possible, since it's possible with the same query when using BigQuery Web UI.
But I can't figure out how to do it ...
TableSchema schema = // function of PCollection<TableRow> result ?
result.apply(BigQueryIO.Write
.named("Write Results Table")
.to(getTableReference(tableName))
.withSchema(schema));
That way query results could be always automatically exported/saved into a new table (only the table name then needs to be explicitly provided).
Any ideas? Any help would be appreciated :)
Unfortunately, Dataflow SDK doesn't expose a schema returned by BigQuery via Dataflow's BigQueryIO API. There's no "good" workaround within the Dataflow API alone.
Defining a schema manually is one workaround.
Alternatively, you could make a separate query to BigQuery directly via jobs: query at pipeline construction time, whose result can then be passed to BigQueryIO.Write transform. This may incur additional cost, but that can probably be mitigated by altering the query slightly to reduce the amount of data processed. Correctness of the output is not relevant, since you'd be storing the schema only.
Conceptually - you should write the function which will iterate thru all cells of given TableRow and for each - get name and type and while iterating you will create respective TableSchema.
For simple schemas, I would expect, it should be relatively easy.
For schemas with records, repeated, etc. this could be more complex
I'm trying to create a table in mysql through java. I'm using putty for this by the way. Here is a bit of the code I have so far but it doesn't work.
rs=s.executeQuery("CREATE TABLE test(id CHAR(2),name VARCHAR(3),PRIMARY KEY(id)); ");
while(rs.next())
{
System.out.println(rs.getString(1));
}
catch (SQLException ex)
{
System.out.println("SQLException:"+ ex.getMessage());
}
executeQuery() is for quires (usually SELECT) that return a ResultSet.
With DML (and DDL) queries you need to use executeUpdate() method.
For more information and examples use Setting Up Tables tutorial.
See this post here: Data Manipulation Statements
You should be using executeUpdate() if you wish to actually modify the database.
Your query is ok! But you don't get a result set! the CREATE TABLE won't give an rows or columns.
You have been tricked be the documentation:
Returns:
a ResultSet object that contains the data produced by the given query; never null
however
Throws:
SQLException - if a database access error occurs,... the given SQL statement produces anything other than a single ResultSet object, ...
In my opinion a call of "execute" would be the proper way.
I don't think its ever a good idea to generate your database schema via Java. Use the utility tool that comes with your database to create your schema. This way, you or anyone (such as a DBA) can create your tables, views, indexes, constraints, grant permissions, etc without having to know Java. You can even have your database utility generate an SQL script that you can run to re-generate the schema from scratch. Last point: I believe you will be better off calling your primary key test_id and making it type numberic, long, or int. this way, when you refer to it as a foreign key in another table, you will immediately know it refers back to the test table.
I am using Hiberante to connect to postgres database. I am trying to insert a record into the database. I have the values for the record in a string array which I got from a csv file. This is my dao code
StringBuffer query=new StringBuffer("insert into t_wonlist values(");
for(int i=0;i<67;i++){
query.append(values[i]+",");
}
query.deleteCharAt(query.lastIndexOf(","));
query.append(");");
sessionfactory.getCurrentSession().createSQLQuery(query.toString()).executeUpdate();
System.out.println("Query executed");
sessionfactory.getCurrentSession().flush();
I am using StringBuffer, so that I can append the values into the query using a for loop.
but when I execute the query I am getting the following exception
org.postgresql.util.PSQLException: No value specified for parameter 1.
I am sure that the number of parameters is correct. Can someone help me. Thanks
You're approaching this in a bizarre and backwards manner.
The immediate problem is probably failure to escape/quote a ? in one of the input strings, so PgJDBC thinks it's a query parameter. That doesn't mean you should fix it by escaping/quoting question marks, it's a sign you're taking entirely the wrong approach.
Please read this page on SQL injection and this site.
You're using the Hibernate ORM, so you'd usually be using the JPA interface or the direct Hibernate interface to create new domain objects and persisting them. The typical approach is to new an object, then use the EntityManager.persist method (if using JPA) or the Session.save method (if using Hibernate directly) to persist the entities.
If you want to use direct JDBC instead you should be creating a JDBC PreparedStatement, setting its parameters, and then applying it. See this tutorial. Since you're loading CSV you'd usually do this in a JDBC batch, though this doesn't actually gain you much in PostgreSQL.
Better yet, since you're importing CSV you can probably just use PostgreSQL's built-in COPY command via PgJDBC's CopyManager to stream the changes efficiently into the target table.