The latest Java JDBC drivers for postgres claim to support UUIDs natively; working against Postgres 9.2 (mac).
Indeed, when a PreparedStatement is used, I can step through the driver code, and even walk
through the specialised 'setUuid' function in AbstractJdbc3gStatement.java. By all indications, it should 'just work'.
However, it does not work. The database flings back an error, which I receive thus:
Caused by: org.postgresql.util.PSQLException: ERROR: operator does not exist: uuid = bytea
Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Position: 139
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) ~[postgresql-9.2-1002.jdbc4.jar:na]
Yes, indeed, setUuid in the JDBC driver does send that as a bytea :
private void setUuid(int parameterIndex, UUID uuid) throws SQLException {
if (connection.binaryTransferSend(Oid.UUID)) {
byte[] val = new byte[16];
ByteConverter.int8(val, 0, uuid.getMostSignificantBits());
ByteConverter.int8(val, 8, uuid.getLeastSignificantBits());
bindBytes(parameterIndex, val, Oid.UUID);
} else {
bindLiteral(parameterIndex, uuid.toString(), Oid.UUID);
}
}
What gives?
Is there some magic rune required in the actual database to bless this conversion ?
tl;dr
myPreparedStatement.setObject(
… ,
java.util.UUID.randomUUID()
)
Details
(a) Show us your code.
PreparedStatement::setObject does work when passing a java.util.UUID. You likely have some other issue in your code.
(b) See my blog post UUID Values From JDBC to Postgres for a bit of discussion and example code.
// Generate or obtain data to store in database.
java.util.UUID uuid = java.util.UUID.randomUUID(); // Generate a random UUID.
String foodName = "Croissant";
// JDBC Prepared Statement.
PreparedStatement preparedStatement = conn.prepareStatement( "INSERT INTO food_ (pkey_, food_name_ ) VALUES (?,?)" );
int nthPlaceholder = 1; // 1-based counting (not an index).
preparedStatement.setObject( nthPlaceholder++, uuid );
preparedStatement.setString( nthPlaceholder++, foodName );
// Execute SQL.
if ( !( preparedStatement.executeUpdate() == 1 ) ) {
// If the SQL reports other than one row inserted…
this.logger.error( "Failed to insert row into database." );
}
(c) I'm not sure what you mean by
The latest Java JDBC drivers for postgres claim to support UUIDs natively
Which driver? There are at least two open-source JDBC drivers for Postgres, the current/legacy one and a new rewrite "next generation" one. And there are other commercial drivers as well.
"natively"? Can you link to the documentation you read? The SQL spec has no data type for UUID (unfortunately ☹), therefore the JDBC spec has no data type for UUID. As a workaround, the JDBC driver for Postgres uses the setObject and getObject methods on PreparedStatement move the UUID across the chasm between Java ↔ SQL ↔ Postgres. See the example code above.
As the PreparedStatement JDBC doc says:
If arbitrary parameter type conversions are required, the method setObject should be used with a target SQL type.
Perhaps by "natively", you confused Postgres' native support for UUID as a data type with JDBC having a UUID data type. Postgres does indeed support UUID as a data type, which means the value is stored as 128-bits rather than multiple times that if it were stored as as ASCII or Unicode hex string. And being native also means Postgres knows how to build an index on a column of that type.
The point of my blog post mentioned above was that I was pleasantly surprised by how simple it is to bridge that chasm between Java ↔ SQL ↔ Postgres. In my first uneducated attempts, I was working too hard.
Another note about Postgres supporting UUID… Postgres knows how to store, index, and retrieve existing UUID values. To generate UUID values, you must enable the Postgres extension (plugin) uuid-ossp. This extension wraps a library provided by The OSSP Project for generating a variety of kinds of UUID values. See my blog for instructions.
By the way…
If I knew how to petition the JDBC expert group or JSR team to make JDBC aware of UUID, I certainly would. They are doing just that for the new date-time types being defined in JSR 310: Date and Time API.
Similarly, if I knew how to petition the SQL standards committee to add a data type of UUID, I would. But apparently that committee is more secretive than the Soviet Politburo and slower than a glacier.
I used the following approach to add UUID and other objects to postgres:
PGobject toInsertUUID = new PGobject();
toInsertUUID.setType("uuid");
toInsertUUID.setValue(uuid.toString());
PreparedStmt stmt = conn.prepareStatement(query);
stmt.setObject(placeHolder,toInsertUUID);
stmt.execute();
This way you will be stopping yourself from doing type casting. This piece of code worked perfectly for me for any time for example even json.
This worked for me using the org.postgresql.postgresql 42.2.5
myPreparedStatement.setObject(4, UUID.randomUUID(),java.sql.Types.OTHER)
Without java.sql.Types.OTHER I got an error
try
.setParameter("uuid", uuid, PostgresUUIDType.INSTANCE);
Related
We are using Oracle version 12.1.0.2.0 with ojdbc7-1.0.jar and JDK 8.
We use Hikari as connection pool if it's relevant and it's using oracle.jdbc.driver.OracleDriver
This is the recommended driver to use in oracle
Oracle Database version JDBC specification compliance
12.1 or 12cR1 ojdbc7.jar with JDK 7 and JDK 8
In Spring application,we are using PreparedStatement, and we want to support national character as characters.
We use setNString which works for new/updated queries
Sets the designated paramter to the given String object. The driver converts this to a SQL NCHAR or NVARCHAR or LONGNVARCHAR value (depending on the argument's size relative to the driver's limits on NVARCHAR values) when it sends it to the database.
Can we add support (platform/config level) for existing code using setString without code change?
try (PreparedStatement ps = conn.prepareStatement(SQL_STATEMENT)) {
ps.setString(1, THAI_TEXT);
}
Currently setString doesn't handle national characters (save as ?) although columns defined as NVARCHAR2(50)
Must we replace all setString with setNString?
or is there a flag/property/upgrade/fix that can add the national support using setString?
You might get away with oracle.jdbc.defaultNChar=true, see 19.2 in the JDBC Guide (however I find Oracle and the JDBC API highly confusing in this regard, it should work with setString)
I’m struggling to insert a JSON object into my postgres v9.4 DB. I have defined the column called "evtjson" as type json (not jsonb).
I am trying to use a prepared statement in Java (jdk1.8) to insert a Json object (built using JEE javax.json libraries) into the column, but I keep running into SQLException errors.
I create the JSON object using:
JsonObject mbrLogRec = Json.createObjectBuilder().build();
…
mbrLogRec = Json.createObjectBuilder()
.add("New MbrID", newId)
.build();
Then I pass this object as a parameter to another method to write it to the DB using a prepared statement. (along with several other fields) As:
pStmt.setObject(11, dtlRec);
Using this method, I receive the following error:
org.postgresql.util.PSQLException: No hstore extension installed.
at org.postgresql.jdbc.PgPreparedStatement.setMap(PgPreparedStatement.java:553)
at org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:1036)
I have also tried:
pStmt.setString(11, dtlRec.toString());
pStmt.setObject(11, dtlRec.toString());
Which produce a different error:
Event JSON: {"New MbrID":29}
SQLException: ERROR: column "evtjson" is of type json but expression is of type character varying
Hint: You will need to rewrite or cast the expression.
But, at least this tells me that the DB is recognizing the column as type JSON.
I did try installing the hstore extension, but it then told me that it was not an hstore object.
OracleDocs shows a number of various methods to set the parameter value in the preparedStatement, but I'd rather not try them all if someone knows the answer. (http://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html) These also reference an additional parameter, SQLType, but I can't find any reference to these.
Should I try setAsciiStream? CharacterStream? CLOB?
This behaviour is quite annoying since JSON strings are accepted without problems when used as literal strings in SQL commands.
There is a already an issue for this in the postgres driver Github repository (even if the problem seems the be the serverside processing).
Besides using a cast (see answer of
#a_horse_with_no_name) in the sql string, the issue author offers two additional solutions:
Use a parameter stringtype=unspecified in the JDBC connection URL/options.
This tells PostgreSQL that all text or varchar parameters are actually
of unknown type, letting it infer their types more freely.
Wrap the parameter in a org.postgresql.util.PGobject:
PGobject jsonObject = new PGobject();
jsonObject.setType("json");
jsonObject.setValue(yourJsonString);
pstmt.setObject(11, jsonObject);
You can do it like this and you just need the json string:
Change the query to:
String query = "INSERT INTO table (json_field) VALUES (to_json(?::json))"
And set the parameter as a String.
pStmt.setString(1, json);
You have two options:
Use statement.setString(jsonStr) and then handle the conversion in the sql statement:
PreparedStatement statement = con.prepareStatement(
"insert into table (jsonColumn) values (?::json)");
statement.setString(1, jsonStr);
Another option is to use PGobject to create a custom value wrapper.
PGobject jsonObject = new PGobject();
PreparedStatement statement = con.prepareStatement(
"insert into table (jsonColumn) values (?)");
jsonObject.setType("json");
jsonObject.setValue(jsonStr);
statement.setObject(1, jsonObject);
I personally prefer the latter as the query is cleaner
Passing the JSON as a String is the right approach, but as the error message tells you, you need to cast the parameter in the INSERT statement to a JSON value:
insert into the_table
(.., evtjson, ..)
values
(.., cast(? as json), ..)
Then you can use pStmt.setString(11, dtlRec.toString()) to pass the value
Most answers here defines ways of inserting into postgres json field with jdbc in a non-standard way, ie. it is db implementation specific. If you need to insert a java string into a postgres json field with pure jdbc and pure sql use:
preparedStatement.setObject(1, "{}", java.sql.Types.OTHER)
This will make the postgres jdbc driver (tested with org.postgresql:postgresql:42.2.19) convert the java string to the json type. It will also validate the string as being a valid json representation, something that various answers using implicit string casts does not do - resulting in the possibility of corrupt persisted json data.
As others have mentioned, your SQL string needs to explicitly cast the bind value to the PostgreSQL json or jsonb type:
insert into t (id, j) values (?, ?::json)
Now you can bind the string value. Alternatively, you can use a library that can do it, for example jOOQ (works out of the box) or Hibernate (using a third party UserType registration). The benefits of this is that you don't have to think about this every time you bind such a variable (or read it). A jOOQ example:
ctx.insertInto(T)
.columns(T.ID, T.J)
.values(1, JSON.valueOf("[1, 2, 3]"))
.execute();
Behind the scenes, the same cast as above is always generated, whenever you work with this JSON (or JSONB) data type.
(Disclaimer: I work for the company behind jOOQ)
if using spring boot: adding the following line to application.properties helped:
spring.datasource.hikari.data-source-properties.stringtype=unspecified
as Wero wrote:
This tells PostgreSQL that all text or varchar parameters are actually
of unknown type
Instead of passing json object pass its string value and cast it to json in the query.
Example:
JSONObject someJsonObject=..........
String yourJsonString = someJsonObject.toString();
String query = "INSERT INTO table (json_field) VALUES (to_json(yourJsonString::json))";
this worked for me.
I am using Appscan source edition for Java Secure Coding. It is reporting an SQL injection in my application. The issue is that we are generating the query dynamically in code so I cannot use a prepared statement. Instead I have to us e Esapi.encoder().encodeForSql(new OracleCodec(), query). AppScan does not consider this to mitigate the SQL injection issue.
final String s = "SELECT name FROM users WHERE id = " +
Esapi.encoder().encodeForSql(new OracleCodec(), userId);
statement = connection.prepareStatement(s);
This code additionally does not work for ESAPI.encoder()
How can I resolve this issue?
what you should do is
final String s = "SELECT name FROM users WHERE id = ?"
statement = connection.prepareStatement(s);
statement.setString(1, userId);
The documentation for encodeForSQL recommends a PreparedStatement which you can still use which dynamically generated queries:
Encode input for use in a SQL query, according to the selected codec
(appropriate codecs include the MySQLCodec and OracleCodec). This
method is not recommended. The use of the PreparedStatement interface
is the preferred approach. However, if for some reason this is
impossible, then this method is provided as a weaker alternative. The
best approach is to make sure any single-quotes are double-quoted.
Another possible approach is to use the {escape} syntax described in
the JDBC specification in section 1.5.6. However, this syntax does not
work with all drivers, and requires modification of all queries.
Let's check what the encoder is doing to see why your code have an injection vulnerability. The encoder calls encodeCharacter in the oracle codec which simply replaces single quotes with two single quotes:
public String encodeCharacter( char[] immune, Character c ) {
if ( c.charValue() == '\'' )
return "\'\'";
return ""+c;
}
This only makes sense if the value is inside single quotes, which string values would be. If id is actually an integer and you wanted to concatenate it with the query, then you would convert it to an int type first instead of using this encoder.
I am working with some legacy code that performs database operations in a generic way, so that the User/developer can work with a different database by changing only the JDBC driver.
I have a problem with PostgreSQL JDBC driver. My test case:
//ddl
CREATE TABLE test
(
id numeric,
name text,
)
//java code
String sqlCmd = "INSERT INTO test values (?, ?)";
PreparedStatement ps = connection.prepareStatement( sqlCmd );
ps.setString( 1, "1" );
ps.setString( 1, "name1" );
ps.executeUpdate();
With Postgres, the result of this case is an exception with message: "can't cast string to int..."
Is it inappropriate to use PreparedStatement.setString() to set values that database expects to be numeric?
Should I expect the JDBC driver to automatically convert java types to database types?
This test passes with other databases, including H2 and MySQL. Does the failure with PostgreSQL reflect a bug in the JDBC driver? Is it possible to make this case work without changing code?
The documentation for java.sql.PreparedStatement has this to say:
Note: The setter methods (setShort, setString, and so on) for setting IN parameter values must specify types that are compatible with the defined SQL type of the input parameter. For instance, if the IN parameter has SQL type INTEGER, then the method setInt should be used.
Whether a particular database or JDBC driver allows you to be sloppy about that is its own affair, but you are not justified in expecting that all drivers will allow such slop, even if certain ones do.
while migrating from oracle database to postgresql, I found that may help you to use setString with numeric types and date types as well.
Just you have to use the connection parameter stringtype=specified as mentioned in the documentation.
https://jdbc.postgresql.org/documentation/head/connect.html
You are using setString() method to insert integers and postgres could not do that, Use
ps.setInt(1, INTEGER_VALUE);
I'm using official JDBC driver for PostgreSQL, but I'm stuck with the following issues:
No support for PostgreSQL-ish data structures such as UUIDs.
Common JDBC weirdness, such as:
No function to escape values for consuming by PostgreSQL.
Limited support for executing heterogeneous statements in batch.
No rewriting of multiple insert statements into single insert statement when inserting many rows in one table.
So, the question — is there any PostgreSQL database driver which can leverage full power of PostgreSQL without much boilerplate? I'm also use Scala language for development, so if driver is designed specifically for Scala it would be so much awesome awesome.
Some of this seems to be (unless I'm not understanding) user error in using JDBC. JDBC is a pretty ugly API, so never ask if you can do it elegantly, just ask if you can do it at all.
Escaping and inserting multiple rows should be handled, as #ColinD and #a_horse pointed out, with Prepared statements and batch operations. Under the hood, I would expect a good JDBC implementation to do the things you want (I am not familiar with PostgreSQL's implementation).
Regarding UUIDs, here is a solution:
All that PostgreSQL can do is convert string literals to uuid.
You can make use of this by using the data type
org.postgresql.util.PGobject, which is a general class used to
represent data types unknown to JDBC.
You can define a helper class:
public class UUID extends org.postgresql.util.PGobject {
public static final long serialVersionUID = 668353936136517917L;
public UUID(String s) throws java.sql.SQLException {
super();
this.setType("uuid");
this.setValue(s);
}
}
Then the following piece of code will succeed:
java.sql.PreparedStatement stmt =
conn.prepareStatement("UPDATE t SET uid = ? WHERE id = 1");
stmt.setObject(1, new UUID("a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11"));
stmt.executeUpdate();
The driver supports batched statements to speed up bulk inserts.
And using batched statements is a lot more portable than using proprietary INSERT syntax (and as far as I can tell, there is no big different between a multi-row insert and batched inserts)
Check out PreparedStatement.addBatch()
The reason why UUID is not supported is probably that UUID is not part of the Postgres core, just a contrib module.
Edit
Regarding the execute heterogeneous statements
The Postgres driver does support different types of statements in the a batch.
The following works fine:
Connection con = DriverManager.getConnection("jdbc:postgresql://localhost/postgres", "foo", "bar");
con.setAutoCommit(false);
Statement stmt = con.createStatement();
stmt.addBatch("create table foo (id integer, data varchar(100))");
stmt.addBatch("insert into foo values (1, 'one')");
stmt.addBatch("insert into foo values (2, 'two')");
stmt.addBatch("update foo set data = 'one_other' where id = 1");
stmt.executeBatch();
con.commit();
Although you do lose the automatic escaping that PreparedStatement gives you.
I realise this doesn't answer your entire question, but hopefully it will be useful all the same.
I'm using Java 6 and Postgres 8.4. The driver I'm using is in my Maven POM file as:
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>8.4-702.jdbc4</version>
</dependency>
I'm using PreparedStatement.getObject() and PreparedStatement.setObject() with Java's java.util.UUID class to retrieve and store UUIDs.
For example:
pstm.setObject(1, guid); //where pstm is a PreparedStatement and guid is a UUID
and:
//where rs is a ResultSet
UUID myGuid = (UUID) rs.getObject("my_uuid_column_name");
Works fine.
With newer drivers, the following is alsow supported
UUID myGuid = rs.getObject("my_uuid_column_name", UUID.class);
No support for PostgreSQL-ish data structures such as UUIDs.
On the contrary, the current JDBC driver (9.2-1002 JDBC 4) for Postgres 9.x does indeed support UUID via the setObject and getObject commands. You cannot get any more direct or simpler than that (in any database, Postgres or any other) because JDBC does not recognize UUID as a data type.
As far as I can tell, there is no need to create a helper class as suggest in another answer by Yishai.
No need to do any casting or go through strings.
See my blog post for more discussion and code example.
Code example excerpt:
java.util.UUID uuid = java.util.UUID.randomUUID();
…
preparedStatement.setObject( nthPlaceholder++, uuid ); // Pass UUID to database.
Take a look at O/R Broker, which is a Scala JDBC-based library for relational database access.