Develop Reference

Using Prepared Statement to get results from Oracle Database where column size is char[2] [duplicate]

I have a tricky issue with the Oracle JDBC driver's handling of CHAR data types. Let's take this simple table:
create table x (c char(4));
insert into x (c) values ('a'); -- inserts 'a '
So when I insert something into CHAR(4), the string is always filled with whitespace. This is also done when I execute queries like this:
select * from x where c = 'a'; -- selects 1 record
select * from x where c = 'a '; -- selects 1 record
select * from x where c = 'a '; -- selects 1 record
Here, the constant 'a' is filled with whitespace as well. That's why the record is always returned. This holds true when these queries are executed using a JDBC PreparedStatement as well. Now the tricky thing is when I want to use a bind variable:
PreparedStatement stmt =
conn.prepareStatement("select * from x where c = ?");
stmt.setString(1, "a"); // This won't return any records
stmt.setString(1, "a "); // This will return a record
stmt.executeQuery();
This is a workaround:
PreparedStatement stmt =
conn.prepareStatement("select * from x where trim(c) = trim(?)");
stmt.setString(1, "a"); // This will return a record
stmt.setString(1, "a "); // This will return a record
stmt.executeQuery();
EDIT: Now these are the constraints:
The above workaround is not desireable as it modifies both the contents of c and ?, AND it makes using indexes on c quite hard.
Moving the column from CHAR to VARCHAR (which it should be, of course) is not possible
EDIT: The reasons for these constraints is because I ask this question from the point of view of the developer of jOOQ, a database abstraction library. So my requirements are to provide a very generic solution that doesn't break anything in jOOQ's client code. That is why I'm not really a big fan of the workaround. And that's why I don't have access to that CHAR column's declaration. But still, I want to be able to handle this case.
What would you do instead? What's a good practice for handling CHAR data types when I want to ignore trailing whitespace?

If you want
stmt.setString(1, "a"); // This won't return any records
to return a record, try
conn.prepareStatement("select * from x where c = cast(? as char(4))")

I don't see any reason to use CHAR datatype even if it is char(1) in Oracle. Can you change the datatype instead?

Gary's solution works well. Here's an alternative.
If you are using an Oracle JDBC driver, the call to prepareStatement() will actually return an OraclePreparedStatement, which has a setFixedCHAR() method that automatically pads your inputs with whitespace.
String sql = "select * from x where c = ?";
OraclePreparedStatement stmt = (OraclePreparedStatement) conn.prepareStatement(sql);
stmt.setFixedCHAR(1, "a");
...
Obviously, the cast is only safe if you are using the Oracle driver.
The only reason I would suggest that you use this over Gary's answer is that you can change your column sizes without having to modify your JDBC code. The driver pads the correct number of spaces without the developer needing to know/manage the column size.

I have nice fix for this. You have to add one property while getting connection from database.
NLS_LANG=american_america.AL32UTF8
or in Java connection you can use below code:
java.util.Properties info = new java.util.Properties();
info.put ("user", user);
info.put ("password",password);
info.put("fixedString","TRUE");
info.put("NLS_LANG","american_america.AL32UTF8");
info.put("SetBigStringTryClob","TRUE");
String url="jdbc:oracle:thin:#"+serverName;
log.debug("url="+url);
log.debug("info="+info);
Class.forName("oracle.jdbc.OracleDriver");
conn = DriverManager.getConnection(url,info);

the other way is modify your sql as
select * from x where NVL(TRIM(c),' ') = NVL(TRIM('a'),' ')

Simply add RTRIM() to the column name(which is defimed) in the update query.

Is there an function to retrieve the binary XML from oracle result set

I am trying to extract the data from XMLTYPE COLUMN "ATTRIBUTE_XML2" STORE AS SECUREFILE BINARY XML from an Oracle 12C database.
I am using this select query in my code:
select xmlserialize(document a.xmlrecord as clob) as xmlrecord from tablename
ResultSet rset = stmt.executeQuery();
OracleResultSet orset = (OracleResultSet) rset;
while (orset.next()) {
oracle.sql.CLOB xmlrecord = (oracle.sql.CLOB) orset.getClob(1);
Reader reader = new BufferedReader(xmlrecord.getCharacterStream());
}
Here "orset.getClob" is taking more memory in oracle DB and we are getting out of process memory in the oracle database. Currently we have the XML type storage as CLOB and business is interested to change it to BINARY XML.
Is there any option for retrieving the binary XML from the oracle result set?
Please note that i have tried "orset.getClob" which results in memory error, since it is changing the binary XML to clob.
Also tried with " XMLType xml = (XMLType) orset.getObject(1);" this is working fine, but it is taking 27 minutes for fetching 1 million XML records.
Whereas the same 1 million completed in 5 minutes if the table type storage is CLOB instead of BINARY XML.
Is there any other option for retrieving the BINARY XML ?

The Oracle documentation for Using JDBC to Access XML Documents in Oracle XML DB states that:
You can select XMLType data using JDBC in any of these ways:
Use SQL/XML function XMLSerialize in SQL, and obtain the result as an oracle.sql.CLOB, java.lang.String or oracle.sql.BLOB in Java. The Java snippet in Example 13-2 illustrates this.
Call method getObject() in the PreparedStatement to obtain the whole XMLType instance. The return value of this method is of type oracle.xdb.XMLType. Then you can use Java functions on class XMLType to access the data. Example 13-3 shows how to do this.
So you should be able to use XMLSERIALIZE( DOCUMENT your_binary_xml_column AS BLOB ) in SQL and then use OracleResultSet#getBLOB(int) to get the binary data.
Paraphrasing Oracle's Example 13-2 to cast to a BLOB instead of a CLOB:
DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver());
Connection conn = DriverManager.getConnection("jdbc:oracle:oci8:#", "QUINE", "CURRY");
OraclePreparedStatement stmt = (OraclePreparedStatement) conn.prepareStatement(
"SELECT XMLSerialize(DOCUMENT e.poDoc AS BLOB) poDoc FROM po_xml_tab e");
ResultSet rset = stmt.executeQuery();
OracleResultSet orset = (OracleResultSet) rset;
while(orset.next())
{
// the first argument is a BLOB
oracle.sql.BLOB clb = orset.getBLOB(1);
// now use the BLOB inside the program
}

Output parameter from SQL Server stored procedure truncated at 4000 characters

I have a problem with the output parameter of a stored procedure when it contains more than 4000 characters. The response seems to be truncated by the JDBC driver? How can I get the full result?
The stored procedure answers with the complete response (> 4000 characters) but I can not open it from Java. I have tried both jTDS and Microsoft's JDBC driver 6.0. Here is my code:
CallableStatement pstmt = con.prepareCall("{call sp_horus_get_consultorios_stv(?)}");
pstmt.registerOutParameter(1, -1);
pstmt.setString(1, "");
pstmt.execute();
String sp_horus_get_consultorios_stv = pstmt.getString(1);
This works with stored procedures in sybase.

I was able to recreate your issue using Microsoft JDBC Driver 6.x. I found that I could avoid the problem by commenting out the setString call:
try (CallableStatement pstmt = conn.prepareCall("{call usp_horus_get_consultorios_stv(?)}")) {
pstmt.registerOutParameter(1, Types.LONGNVARCHAR);
//pstmt.setString(1, ""); // disabled
pstmt.execute();
String sp_horus_get_consultorios_stv = pstmt.getString(1);
System.out.println(sp_horus_get_consultorios_stv.length()); // > 4000 characters
}
Unfortunately, that fix did not solve the problem under jTDS 1.3.1. It appears that jTDS still suffers from the limitation described here. So, for jTDS it appears that we have to do something like this:
String sql =
"DECLARE #out NVARCHAR(MAX);" +
"EXEC usp_horus_get_consultorios_stv #out OUTPUT;" +
"SELECT #out;";
try (
Statement st = conn.createStatement();
ResultSet rs = st.executeQuery(sql)) {
rs.next();
String sp_horus_get_consultorios_stv = rs.getString(1);
System.out.println(sp_horus_get_consultorios_stv.length()); // > 4000 characters
}

From what I understand, your output parameter is of type NVARCHAR(8000), which is the maximum explicit number allowed, and outputs a 4000 character Unicode string lenght (2 bytes per char).
However, and lucky you, there another possibility : NVARCHAR(MAX), that basically allows an infinite string lenght (well, not infinite, but almost :
What is the maximum number of characters that nvarchar(MAX) will hold?
You should change your output paramater type to NVARCHAR(MAX).
Happy coding ;)

How to convert DB2 binary data to UTF-8 at query level

I am connected to IBM DB2 database with java but data is stored as binary format in database so when I fetch any value it comes as binary or hexdecimal format. How can I convert this in binary data in utf-8 at query level.
Sample code to fetch data -
String sql = "SELECT poMast.ORDNO from AMFLIBL.POMAST AS poMast ";
Class.forName("com.ddtek.jdbc.db2.DB2Driver");
String url = "jdbc:datadirect:db2://hostname:port;DatabaseName=dbName;";
Connection con = DriverManager.getConnection(url, "username","password");
PreparedStatement preparedStatement = con.prepareStatement(sql);
ResultSet rs = preparedStatement.executeQuery();
System.out.println("ResultSet : \n");
System.out.println(" VNDNO");
while (rs.next())
{
System.out.println(rs.getString("ORDNO"));
}

You probably need to use the CAST expression:
SELECT CAST(poMast.ORDNO as VARCHAR(50)) from AMFLIBL.POMAST AS poMast
Adjust the VARCHAR length to your needs. The string is in the database codepage (often UTF-8 these days) and converted to the client/application codepage when fetched.

you can "cast" the result from your select to utf8 like below.
String sql = "SELECT poMast.ORDNO, CAST(poMast.ORDNO AS VARCHAR(255) CCSID UNICODE) FROM AMFLIBL.POMAST AS poMast ";
src: cast db2

In my case, somehow bad UTF-8 data had gotten into varchars in a 1208/UTF-8 DB. Prior to conversion, when querying such data via the JDBC driver, the DB returned -4220 via the JDBC driver. This is fixable at the JDBC driver level by adding this property:
java -Ddb2.jcc.charsetDecoderEncoder=3 MyApp
see:
https://www.ibm.com/support/pages/sqlexception-message-caught-javaiocharconversionexception-and-errorcode-4220
The Db2 LUW Command Line Processor fixed it long ago as an APAR, so this error is only seen via the JDBC driver when the above property is not set.
But, if you want to fix the data in the db, this works:
update <table_name> set <bad_data_col> = cast(cast( <bad_data_col> as vargraphic) as varchar);
1st db2 treats (casts) the bad data as a binary where "anything goes" and then converts (casts) it back to valid UTF-8. After the casts, the JDBC driver shows the same result with or without the special property set and returns no errors.

Can't store UTF-8 Content in MySQL Using Java PreparedStatement

For some strange reason I can't seem to add UTF-8 data to my MySQL database. When I enter a non-latin character, it's stored as ?????. Everything else is stored fine. So for example, "this is an example®™" is stored fine, but "和英辞典" is stored as "????".
The connection url is fine:
private DataSource getDB() throws PropertyVetoException {
ComboPooledDataSource db = new ComboPooledDataSource();
db.setDriverClass("com.mysql.jdbc.Driver");
db.setJdbcUrl("jdbc:mysql://domain.com:3306/db?useUnicode=true&characterEncoding=UTF-8");
db.setUser("...");
db.setPassword("...");
return db;
}
I'm using PreparedStatement as you would expect, I even tried entering "set names utf8" as someone suggested.
Connection conn = null;
PreparedStatement stmt = null;
ResultSet rs = null;
try {
conn = db.getConnection();
stmt = conn.prepareStatement("set names utf8");
stmt.execute();
stmt = conn.prepareStatement("set character set utf8");
stmt.execute();
... set title...
stmt = conn.prepareStatement("INSERT INTO Table (title) VALUES (?)");
stmt.setString(1,title);
stmt.execute();
} catch (final SQLException e) {
...
The table itself seems to be fine.
Default Character Set: utf8
Default Collation: utf8_general_ci
...
Field title:
Type text
Character Set: utf8
Collation: utf8_unicode_ci
I tested it by entering in Unicode ("和英辞典" specifically) through a GUI editor and then selecting from the table -- and it was returned just fine. So this seems to be an issue with JDBC.
What am I missing?

On your JDBC connection string, you just need set the charset encoding like this:
jdbc:mysql://localhost:3306/dbname?characterEncoding=utf8

There is 2 points in the mysql server to check in order to correctly set the UTF-8 charset.
Database Level
This is obtained by creating it :
CREATE DATABASE 'db' CHARACTER SET 'utf8';
Table Level
All of the tables need to be in UTF-8 also (which seems to be the case for you)
CREATE TABLE `Table1` (
[...]
) DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci;
The important part being DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci
Finally, if your code weren't handling utf8 correctly, you could have forced your JVM to use utf8 encoding by changing the settings by on startup :
java -Dfile.encoding=UTF-8 [...]
or changing the environment variable
"**JAVA_TOOLS_OPTIONS**" to -Dfile.encoding="UTF-8"
or programmatically by using :
System.setProperty("file.encoding" , "UTF-8");
(this last one may not have the desire effect since the JVM caches value of default character encoding on startup)
Hope that helped.

Use stmt.setNString(...) instead of stmt.setString(...).
Also don't forget to check column collation in database side.

If you log in to your mysql database and run show variables like 'character%';
this might provide some insight.
Since you're getting a one-to-one ratio of multi-byte characters to question marks then it's likely that the connection is doing a character set conversion and replacing the Chinese characters with the replacement character for the single-byte set.

Also check locale -a on ubuntu default Ubuntu works with en_us locale and doesn't have other locale installed.
must specify characterEncoding=utf8 while connecting through JDBC.

add at the end of your DB connection url - (nothing else needed)
ex.
spring.datasource.url = jdbc:mysql://localhost:3306/dbname?characterEncoding=utf8