I am working with some legacy code that performs database operations in a generic way, so that the User/developer can work with a different database by changing only the JDBC driver.
I have a problem with PostgreSQL JDBC driver. My test case:
//ddl
CREATE TABLE test
(
id numeric,
name text,
)
//java code
String sqlCmd = "INSERT INTO test values (?, ?)";
PreparedStatement ps = connection.prepareStatement( sqlCmd );
ps.setString( 1, "1" );
ps.setString( 1, "name1" );
ps.executeUpdate();
With Postgres, the result of this case is an exception with message: "can't cast string to int..."
Is it inappropriate to use PreparedStatement.setString() to set values that database expects to be numeric?
Should I expect the JDBC driver to automatically convert java types to database types?
This test passes with other databases, including H2 and MySQL. Does the failure with PostgreSQL reflect a bug in the JDBC driver? Is it possible to make this case work without changing code?
The documentation for java.sql.PreparedStatement has this to say:
Note: The setter methods (setShort, setString, and so on) for setting IN parameter values must specify types that are compatible with the defined SQL type of the input parameter. For instance, if the IN parameter has SQL type INTEGER, then the method setInt should be used.
Whether a particular database or JDBC driver allows you to be sloppy about that is its own affair, but you are not justified in expecting that all drivers will allow such slop, even if certain ones do.
while migrating from oracle database to postgresql, I found that may help you to use setString with numeric types and date types as well.
Just you have to use the connection parameter stringtype=specified as mentioned in the documentation.
https://jdbc.postgresql.org/documentation/head/connect.html
You are using setString() method to insert integers and postgres could not do that, Use
ps.setInt(1, INTEGER_VALUE);
Related
I’m struggling to insert a JSON object into my postgres v9.4 DB. I have defined the column called "evtjson" as type json (not jsonb).
I am trying to use a prepared statement in Java (jdk1.8) to insert a Json object (built using JEE javax.json libraries) into the column, but I keep running into SQLException errors.
I create the JSON object using:
JsonObject mbrLogRec = Json.createObjectBuilder().build();
…
mbrLogRec = Json.createObjectBuilder()
.add("New MbrID", newId)
.build();
Then I pass this object as a parameter to another method to write it to the DB using a prepared statement. (along with several other fields) As:
pStmt.setObject(11, dtlRec);
Using this method, I receive the following error:
org.postgresql.util.PSQLException: No hstore extension installed.
at org.postgresql.jdbc.PgPreparedStatement.setMap(PgPreparedStatement.java:553)
at org.postgresql.jdbc.PgPreparedStatement.setObject(PgPreparedStatement.java:1036)
I have also tried:
pStmt.setString(11, dtlRec.toString());
pStmt.setObject(11, dtlRec.toString());
Which produce a different error:
Event JSON: {"New MbrID":29}
SQLException: ERROR: column "evtjson" is of type json but expression is of type character varying
Hint: You will need to rewrite or cast the expression.
But, at least this tells me that the DB is recognizing the column as type JSON.
I did try installing the hstore extension, but it then told me that it was not an hstore object.
OracleDocs shows a number of various methods to set the parameter value in the preparedStatement, but I'd rather not try them all if someone knows the answer. (http://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html) These also reference an additional parameter, SQLType, but I can't find any reference to these.
Should I try setAsciiStream? CharacterStream? CLOB?
This behaviour is quite annoying since JSON strings are accepted without problems when used as literal strings in SQL commands.
There is a already an issue for this in the postgres driver Github repository (even if the problem seems the be the serverside processing).
Besides using a cast (see answer of
#a_horse_with_no_name) in the sql string, the issue author offers two additional solutions:
Use a parameter stringtype=unspecified in the JDBC connection URL/options.
This tells PostgreSQL that all text or varchar parameters are actually
of unknown type, letting it infer their types more freely.
Wrap the parameter in a org.postgresql.util.PGobject:
PGobject jsonObject = new PGobject();
jsonObject.setType("json");
jsonObject.setValue(yourJsonString);
pstmt.setObject(11, jsonObject);
You can do it like this and you just need the json string:
Change the query to:
String query = "INSERT INTO table (json_field) VALUES (to_json(?::json))"
And set the parameter as a String.
pStmt.setString(1, json);
You have two options:
Use statement.setString(jsonStr) and then handle the conversion in the sql statement:
PreparedStatement statement = con.prepareStatement(
"insert into table (jsonColumn) values (?::json)");
statement.setString(1, jsonStr);
Another option is to use PGobject to create a custom value wrapper.
PGobject jsonObject = new PGobject();
PreparedStatement statement = con.prepareStatement(
"insert into table (jsonColumn) values (?)");
jsonObject.setType("json");
jsonObject.setValue(jsonStr);
statement.setObject(1, jsonObject);
I personally prefer the latter as the query is cleaner
Passing the JSON as a String is the right approach, but as the error message tells you, you need to cast the parameter in the INSERT statement to a JSON value:
insert into the_table
(.., evtjson, ..)
values
(.., cast(? as json), ..)
Then you can use pStmt.setString(11, dtlRec.toString()) to pass the value
Most answers here defines ways of inserting into postgres json field with jdbc in a non-standard way, ie. it is db implementation specific. If you need to insert a java string into a postgres json field with pure jdbc and pure sql use:
preparedStatement.setObject(1, "{}", java.sql.Types.OTHER)
This will make the postgres jdbc driver (tested with org.postgresql:postgresql:42.2.19) convert the java string to the json type. It will also validate the string as being a valid json representation, something that various answers using implicit string casts does not do - resulting in the possibility of corrupt persisted json data.
As others have mentioned, your SQL string needs to explicitly cast the bind value to the PostgreSQL json or jsonb type:
insert into t (id, j) values (?, ?::json)
Now you can bind the string value. Alternatively, you can use a library that can do it, for example jOOQ (works out of the box) or Hibernate (using a third party UserType registration). The benefits of this is that you don't have to think about this every time you bind such a variable (or read it). A jOOQ example:
ctx.insertInto(T)
.columns(T.ID, T.J)
.values(1, JSON.valueOf("[1, 2, 3]"))
.execute();
Behind the scenes, the same cast as above is always generated, whenever you work with this JSON (or JSONB) data type.
(Disclaimer: I work for the company behind jOOQ)
if using spring boot: adding the following line to application.properties helped:
spring.datasource.hikari.data-source-properties.stringtype=unspecified
as Wero wrote:
This tells PostgreSQL that all text or varchar parameters are actually
of unknown type
Instead of passing json object pass its string value and cast it to json in the query.
Example:
JSONObject someJsonObject=..........
String yourJsonString = someJsonObject.toString();
String query = "INSERT INTO table (json_field) VALUES (to_json(yourJsonString::json))";
this worked for me.
I want to insert "0001-01-01" as a value into a date field by using Java PreparedStatement.
But it throws exception when I tried this:
String sql = "insert into mytable values(?)"
ps = conn.prepareStatement(sql);
ps.setDate(1, java.sql.Date.valueOf("0001-01-01"));
ps.executeUpdate(); // throws exceptions here.
The error is :
The supplied value is not a valid instance of data type datetime. Check the source data for invalid values. An example of an invalid value is data of numeric type with scale greater than precision.
If I don't use PreparedStatement, I can insert "0001-01-01". However,
prepare statement seems not allow me to insert this value.
It will work if I inserted "1969-01-01" instead of "0001-01-01".
Any ideas?
Updates:
Here are more info that might be needed.
we use sql server 2012.
we have to use "0001-01-01" because these values were already there. I am changing some very very old codes to use prepare statement. So I have to insert the same values in the same functionality.
Updates 2:
We are using "date" datatype, not "datetime" datatype.
Based on this https://msdn.microsoft.com/en-us/library/bb630352.aspx, "0001-01-01" is not out of range for "date" field.
In addition, I am able to insert "0001-01-01" to the date field without using prepare statement. i.e.
String sql = "insert into mytable values('0001-01-01')"
java.sql.Statement statement = conn.createStatement();
statement.executeUpdate(sql);
So it is not sql server's problem or db field's problem.
Try using the different suitable JDBC driver.
When I run the sql statement below through psql it works fine, but when I try to run the same query by building it with a preparedstatement, it fails.
INSERT INTO Hvbp
(provider_number, weighted_clinical_process,
weighted_patience_experience, total_performance_score,
coordinates, latitude, longitude, address, city, state, zip)
VALUES
('010092', 43.909090909091, 13.5, 57.409090909091,
'POINT(33.206201 -87.525480)', 33.206200613000476,
-87.52548020899968, '809 UNIVERSITY BOULEVARD EAST', 'TUSCALOOSA', 'AL', '');
The error I keep getting is
org.postgresql.util.PSQLException: ERROR: column "coordinates" is of type geography but expression is of type character varying
Hint: You will need to rewrite or cast the expression.
Position: 203
The coordinates column is of type GEOGRAPHY(POINT)
I know this is an old problem, but I just spend most of the day debugging the same basic problem and finally found a fix
What you're trying to do, is provide the POINT using WKT and have the server automatically convert that into a Geometry.
And as you've found that works if you include the WKT inside the body of the SQL, but fails if you use a parameter on a prepared statement.
There are 3 options for fixing it:
Use st_GeographyFromText in your SQL like so:
INSERT INTO Hvbp(coordinates) VALUES( st_GeographyFromText(?) )
And then set your parameters as WKT with setString.
Depending on the frameworks involved, that might not be possible for you.
Use setObject on the preparedStatement instead of setString. For example:
ps.setObject(1, "POINT(33.206201 -87.525480)", java.sql.Types.OTHER )
Change your JDBC driver settings to send strings as unspecified type, and then the server will do type conversions for you. To do that you want to change your JDBC URL to something like
"jdbc:postgresql:my_db?stringtype=unspecified"
#Tim - thank you for your help with a similar problem - I had to write ST_GeometryFromText into my database and the JDBC Driver threw a similar exception as #Hanks got.
For further reference and clarification for others - this is my result using Java with JDBC:
INSERT INTO streets.points ( point_id, the_geom )
VALUES( ?, ST_GeomFromText( ? , 25832) );
And the inserted Geometry-String looked like that:
POINT(33.206201 -87.525480)
The latest Java JDBC drivers for postgres claim to support UUIDs natively; working against Postgres 9.2 (mac).
Indeed, when a PreparedStatement is used, I can step through the driver code, and even walk
through the specialised 'setUuid' function in AbstractJdbc3gStatement.java. By all indications, it should 'just work'.
However, it does not work. The database flings back an error, which I receive thus:
Caused by: org.postgresql.util.PSQLException: ERROR: operator does not exist: uuid = bytea
Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Position: 139
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2157) ~[postgresql-9.2-1002.jdbc4.jar:na]
Yes, indeed, setUuid in the JDBC driver does send that as a bytea :
private void setUuid(int parameterIndex, UUID uuid) throws SQLException {
if (connection.binaryTransferSend(Oid.UUID)) {
byte[] val = new byte[16];
ByteConverter.int8(val, 0, uuid.getMostSignificantBits());
ByteConverter.int8(val, 8, uuid.getLeastSignificantBits());
bindBytes(parameterIndex, val, Oid.UUID);
} else {
bindLiteral(parameterIndex, uuid.toString(), Oid.UUID);
}
}
What gives?
Is there some magic rune required in the actual database to bless this conversion ?
tl;dr
myPreparedStatement.setObject(
… ,
java.util.UUID.randomUUID()
)
Details
(a) Show us your code.
PreparedStatement::setObject does work when passing a java.util.UUID. You likely have some other issue in your code.
(b) See my blog post UUID Values From JDBC to Postgres for a bit of discussion and example code.
// Generate or obtain data to store in database.
java.util.UUID uuid = java.util.UUID.randomUUID(); // Generate a random UUID.
String foodName = "Croissant";
// JDBC Prepared Statement.
PreparedStatement preparedStatement = conn.prepareStatement( "INSERT INTO food_ (pkey_, food_name_ ) VALUES (?,?)" );
int nthPlaceholder = 1; // 1-based counting (not an index).
preparedStatement.setObject( nthPlaceholder++, uuid );
preparedStatement.setString( nthPlaceholder++, foodName );
// Execute SQL.
if ( !( preparedStatement.executeUpdate() == 1 ) ) {
// If the SQL reports other than one row inserted…
this.logger.error( "Failed to insert row into database." );
}
(c) I'm not sure what you mean by
The latest Java JDBC drivers for postgres claim to support UUIDs natively
Which driver? There are at least two open-source JDBC drivers for Postgres, the current/legacy one and a new rewrite "next generation" one. And there are other commercial drivers as well.
"natively"? Can you link to the documentation you read? The SQL spec has no data type for UUID (unfortunately ☹), therefore the JDBC spec has no data type for UUID. As a workaround, the JDBC driver for Postgres uses the setObject and getObject methods on PreparedStatement move the UUID across the chasm between Java ↔ SQL ↔ Postgres. See the example code above.
As the PreparedStatement JDBC doc says:
If arbitrary parameter type conversions are required, the method setObject should be used with a target SQL type.
Perhaps by "natively", you confused Postgres' native support for UUID as a data type with JDBC having a UUID data type. Postgres does indeed support UUID as a data type, which means the value is stored as 128-bits rather than multiple times that if it were stored as as ASCII or Unicode hex string. And being native also means Postgres knows how to build an index on a column of that type.
The point of my blog post mentioned above was that I was pleasantly surprised by how simple it is to bridge that chasm between Java ↔ SQL ↔ Postgres. In my first uneducated attempts, I was working too hard.
Another note about Postgres supporting UUID… Postgres knows how to store, index, and retrieve existing UUID values. To generate UUID values, you must enable the Postgres extension (plugin) uuid-ossp. This extension wraps a library provided by The OSSP Project for generating a variety of kinds of UUID values. See my blog for instructions.
By the way…
If I knew how to petition the JDBC expert group or JSR team to make JDBC aware of UUID, I certainly would. They are doing just that for the new date-time types being defined in JSR 310: Date and Time API.
Similarly, if I knew how to petition the SQL standards committee to add a data type of UUID, I would. But apparently that committee is more secretive than the Soviet Politburo and slower than a glacier.
I used the following approach to add UUID and other objects to postgres:
PGobject toInsertUUID = new PGobject();
toInsertUUID.setType("uuid");
toInsertUUID.setValue(uuid.toString());
PreparedStmt stmt = conn.prepareStatement(query);
stmt.setObject(placeHolder,toInsertUUID);
stmt.execute();
This way you will be stopping yourself from doing type casting. This piece of code worked perfectly for me for any time for example even json.
This worked for me using the org.postgresql.postgresql 42.2.5
myPreparedStatement.setObject(4, UUID.randomUUID(),java.sql.Types.OTHER)
Without java.sql.Types.OTHER I got an error
try
.setParameter("uuid", uuid, PostgresUUIDType.INSTANCE);
I'm using official JDBC driver for PostgreSQL, but I'm stuck with the following issues:
No support for PostgreSQL-ish data structures such as UUIDs.
Common JDBC weirdness, such as:
No function to escape values for consuming by PostgreSQL.
Limited support for executing heterogeneous statements in batch.
No rewriting of multiple insert statements into single insert statement when inserting many rows in one table.
So, the question — is there any PostgreSQL database driver which can leverage full power of PostgreSQL without much boilerplate? I'm also use Scala language for development, so if driver is designed specifically for Scala it would be so much awesome awesome.
Some of this seems to be (unless I'm not understanding) user error in using JDBC. JDBC is a pretty ugly API, so never ask if you can do it elegantly, just ask if you can do it at all.
Escaping and inserting multiple rows should be handled, as #ColinD and #a_horse pointed out, with Prepared statements and batch operations. Under the hood, I would expect a good JDBC implementation to do the things you want (I am not familiar with PostgreSQL's implementation).
Regarding UUIDs, here is a solution:
All that PostgreSQL can do is convert string literals to uuid.
You can make use of this by using the data type
org.postgresql.util.PGobject, which is a general class used to
represent data types unknown to JDBC.
You can define a helper class:
public class UUID extends org.postgresql.util.PGobject {
public static final long serialVersionUID = 668353936136517917L;
public UUID(String s) throws java.sql.SQLException {
super();
this.setType("uuid");
this.setValue(s);
}
}
Then the following piece of code will succeed:
java.sql.PreparedStatement stmt =
conn.prepareStatement("UPDATE t SET uid = ? WHERE id = 1");
stmt.setObject(1, new UUID("a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11"));
stmt.executeUpdate();
The driver supports batched statements to speed up bulk inserts.
And using batched statements is a lot more portable than using proprietary INSERT syntax (and as far as I can tell, there is no big different between a multi-row insert and batched inserts)
Check out PreparedStatement.addBatch()
The reason why UUID is not supported is probably that UUID is not part of the Postgres core, just a contrib module.
Edit
Regarding the execute heterogeneous statements
The Postgres driver does support different types of statements in the a batch.
The following works fine:
Connection con = DriverManager.getConnection("jdbc:postgresql://localhost/postgres", "foo", "bar");
con.setAutoCommit(false);
Statement stmt = con.createStatement();
stmt.addBatch("create table foo (id integer, data varchar(100))");
stmt.addBatch("insert into foo values (1, 'one')");
stmt.addBatch("insert into foo values (2, 'two')");
stmt.addBatch("update foo set data = 'one_other' where id = 1");
stmt.executeBatch();
con.commit();
Although you do lose the automatic escaping that PreparedStatement gives you.
I realise this doesn't answer your entire question, but hopefully it will be useful all the same.
I'm using Java 6 and Postgres 8.4. The driver I'm using is in my Maven POM file as:
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>8.4-702.jdbc4</version>
</dependency>
I'm using PreparedStatement.getObject() and PreparedStatement.setObject() with Java's java.util.UUID class to retrieve and store UUIDs.
For example:
pstm.setObject(1, guid); //where pstm is a PreparedStatement and guid is a UUID
and:
//where rs is a ResultSet
UUID myGuid = (UUID) rs.getObject("my_uuid_column_name");
Works fine.
With newer drivers, the following is alsow supported
UUID myGuid = rs.getObject("my_uuid_column_name", UUID.class);
No support for PostgreSQL-ish data structures such as UUIDs.
On the contrary, the current JDBC driver (9.2-1002 JDBC 4) for Postgres 9.x does indeed support UUID via the setObject and getObject commands. You cannot get any more direct or simpler than that (in any database, Postgres or any other) because JDBC does not recognize UUID as a data type.
As far as I can tell, there is no need to create a helper class as suggest in another answer by Yishai.
No need to do any casting or go through strings.
See my blog post for more discussion and code example.
Code example excerpt:
java.util.UUID uuid = java.util.UUID.randomUUID();
…
preparedStatement.setObject( nthPlaceholder++, uuid ); // Pass UUID to database.
Take a look at O/R Broker, which is a Scala JDBC-based library for relational database access.