Develop Reference

UCanAccess in Java returning wrong order with ORDER BY clause in column with special characters

Using the Microsoft Access Database (2007-2013) with a .mdb file, I created a simple test_table table with only one text column "name" and inserted the following test values:
óbito,
fanatico,
orbita,
fanático,
fanta,
órbita,
fantástico,
obito,
obituario,
orbitando
When I execute the query SELECT * FROM test_table ORDER BY name using MS Access query design, the following ordered result is returned:
fanatico,
fanático,
fanta,
fantástico,
obito,
óbito,
obituario,
orbita,
órbita,
orbitando
This order is totally correct and expected.
Now, I need to retrieve and use these values in my Java software. In order to do this, I am using the UCanAccess JDBC driver on version 5.0.0 to connect to the database. The connection itself is successfully being opened, but, when I execute the same query above, it returns the following:
fanatico
fanta
fantástico
fanático
obito
obituario
orbita
orbitando
óbito
órbita
And this is NOT the correct order (for instance, óbito should come immediately after obito). The desired order should consider accented words as if they were the same as the equivalent unnacented word.
It doesn't matter if óbito comes before or after obito, but they must be together.
I tried using COLLATE, tried changing the charset, etc, but nothing worked. Has anyone gone through something similar and could you help me solve this issue? Thanks in advance.

The driver is sorting by their binary representation and/or the individual ASCII characters. Both provide the sort order you provided at the bottom. This is entirely a problem created by the driver, and "fixes" are going to be limited.
There is a workaround posted in the JDBC driver changelog, under the 2.0.9.3 Release notes: WORKAROUND suggested: if you want the same behaviour of Access: select * from table2 order by orderJet( COLUMN1).
If that doesn't work, then you either need to
a) subvert the driver's sorting by creating/maintaining a SORTORDER column in the original database that holds the same word with all accented characters stripped, or b) find a way to change the sort after it arrives from the driver. Neither of these are preferable, so I hope the workaround provided by the developer is sufficient.

Java by default do not perform locale-sensitive String comparison.
In your example I tried following ran program as below for natural sorting
List<String> strings = Arrays.asList(new String[]{"óbito",
"fanatico",
"orbita",
"fanático",
"fanta",
"órbita",
"fantástico",
"obito",
"obituario",
"orbitando"});
Collections.sort(strings);
System.out.println("Output = " + strings);
the output is
Output = [fanatico, fanta, fantástico, fanático, obito, obituario, orbita, orbitando, óbito, órbita]
Now jut by replacing sort by below line
Collections.sort(strings, Collator.getInstance(Locale.US));
I am getting output which you are expecting
Output = [fanatico, fanático, fanta, fantástico, obito, óbito, obituario, orbita, órbita, orbitando]
Giving you example above to understand the difference when you use string comparison using locale-sensitive. There are ways you can handle this from your code or from db configuration.
You can check here for example

Did you try enforcing a CharSet?
Have a look here CharSet for MS Access '97 DB using UCanAccess
class DatabaseOpener : JackcessOpenerInterface {
override fun open(fl: File, pwd: String?): Database {
return DatabaseBuilder.open(fl).apply {
this.charset = charset("Cp1252")
}
}
}
// URL
"jdbc:ucanaccess://<path-to-mdb-file>;memory=false;jackcessOpener=${DatabaseOpener::class.qualifiedName!!}"
When using plain JDBC connection you could try adding a connection parameter:
private static java.sql.ResultSet executeDataTable(String sql) throws Exception {
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
String conStr = "jdbc:ucanaccess://" + dataDir + "ABC.mdb";
Properties props = new java.util.Properties();
props.put("charSet", "Cp1252");
java.sql.Connection con = java.sql.DriverManager.getConnection(conStr, props);
java.sql.Statement stmt = con.createStatement();
return stmt.executeQuery(sql);
}
You need to check what your charset might be. So potentially replace 'Cp1252'.

How do I define a java defined stored proc to return multiple result sets?

I'm trying to define a java (written in Kotlin) method that returns multiple result sets when called as a stored procedure. Code is below. the Hsqldb website;s features page indicates that this should be possible, what am I missing? I currently get an array out of bounds error on index 1:
val createProcedure = """
CREATE PROCEDURE GET_CACHED(dir VARCHAR(100), hashCode INT)
MODIFIES SQL DATA
LANGUAGE JAVA
DYNAMIC RESULT SETS 9
EXTERNAL NAME 'CLASSPATH:integration.FileCache.getResultSets'
"""
#JvmStatic
#Throws(SQLException::class)
public fun getResultSets(conn: Connection, dir: String, hashCode: Int, result: Array<ResultSet?>) {
val file = getFile(dir, hashCode, "sql")
//A list of cached sql statements
val sqlList = BufferedReader(InputStreamReader(file.inputStream())).readLines()
val stmt = conn.createStatement()
for(i in sqlList.indices) {
result[i] = stmt.executeQuery(sqlList[i])
}
}
Given that I can set a breakpoint and reach the inside of the function, I don't think I need to add any more of my code, but if that is wrong let me know.
I'm in a quest to find an in-memory database that can handle multiple result sets for testing purposes (We're modernizing an application, setting up tests first, containerized testing is currently out of reach). I've tried H2, sqlite, and now hsqldb, if there's a better solution I'm open to it.

HSQLDB supports multiple result test, but the Guide states :HyperSQL support this method of returning single or multiple result sets from SQL/PSM procedures only via the JDBC CallableStatement interface. Note the reference to SQL/PSM, rather than SQL/JRT. Currently there is no Java mechanism to return multiple result sets from a Java language PROCEDURE.
For test purposes, you could use a text template consisting of an SQL/PSM CREATE PROCEDURE statement written in SQL, with placeholders for the actual SQL statements that you want to execute. Process the template with your test SQL statements from your file and execute the resulting CREATE PROCEDURE statement.

Introducing a named parameter breaks jOOQ query

To query a PostgreSQL 10.11 database, I am using jOOQ 3.12.4, which comes bundled with Spring Boot 2.2.
Let's assume I have built a query using jOOQ like this:
final String[] ids = ...;
final var query = dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.in(ids));
final Map<String, List<MyTable>> changeDomains = query.fetch().intoGroups(MY_TABLE.ID, MyTable.class);
This code runs fine and produces the expected results. But when I refactor my query and introduce a named parameter (to reuse the query in multiple parts of my code), like this:
final String[] ids = ...;
final var query = dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.in(param("ids")));
final Map<String, List<MyTable>> changeDomains = query.bind("ids", ids).fetch().intoGroups(MY_TABLE.ID, MyTable.class);
I suddenly start to get the following error:
org.springframework.jdbc.BadSqlGrammarException: jOOQ; bad SQL grammar ...; nested exception is org.postgresql.util.PSQLException: ERROR: operator does not exist: text = character varying[]
Hinweis: No operator matches the given name and argument type(s). You might need to add explicit type casts.
Edit: I get the same error when I use
MY_TABLE.ID.in(param("ids", String[].class))
instead.
How can I solve or work around this problem?

A better solution to your code reuse approach
But when I refactor my query and introduce a named parameter (to reuse the query in multiple parts of my code)
While you could use jOOQ this way (be careful, when mutating and reusing jOOQ queries in a non-threadsafe way!), it is generally recommended to use jOOQ in a more functional way, see e.g.:
https://blog.jooq.org/2017/01/16/a-functional-programming-approach-to-dynamic-sql-with-jooq/
https://www.jooq.org/doc/latest/manual/sql-building/dynamic-sql/
You don't gain much by re-using a jOOQ query, specifically, there's hardly any performance gain.
So, instead of this:
final var query = dslContext.selectFrom(MY_TABLE)
.where(MY_TABLE.ID.in(param("ids")));
final Map<String, List<MyTable>> changeDomains = query
.bind("ids", ids).fetch().intoGroups(MY_TABLE.ID, MyTable.class);
Write this:
public ResultQuery<MyTableRecord> query(String[] ids) {
return dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.in(ids));
}
// And then:
final Map<String, List<MyTable>> changeDomains = query(ids)
.fetch().intoGroups(MY_TABLE.ID, MyTable.class);
The actual problem you ran into:
jOOQ, JDBC, and SQL don't support single bind value IN lists. While it seems useful to write this:
SELECT * FROM t WHERE c IN (:bind_value)
And passing an array or list as a single bind value, this is not supported in SQL. Some APIs might pretend that this is supported (but behind the scenes replace the single bind value by multiple ?, ?, ..., ?
PostgreSQL supports the = ANY (:bind_value) operator with arrays
SELECT * FROM t WHERE c = ANY (:bind_value)
You could use it in jOOQ using
dslContext.selectFrom(MY_TABLE).where(MY_TABLE.ID.eq(any(ids)));
That way, you could call the bind() method to replace the array prior to execution. However, I still recommend you write functions returning queries dynamically.

JPA CreateNativeQuery returns wrong date

We recently updated a project from using Hibernate 4 to Hibernate 5.2, and with that came the need to update all of our Criteria to use JPA. For the most part things are in working order, but I have one query that is no longer behaving. One of the fields on the table we are querying is of type DATE in the database.
When I query directly on the table I get back the date- say it is "2017-04-20." However, when I run the same query on our development server, using JPA's createNativeQuery, I get back the date "2017-04-19"
I don't think this is an issue with the query as I run the exact same query both through a mysql terminal and through java and get different results. The query that I run is the one that is logged in my below example. I think it may be a timezone issue as I don't have this problem on my local environment, just on my dev server, but it also wasn't a problem until we updated to the new versions of Hibernate.
public List<ResponseDTO> getDashboardData(String date, Integer page, Integer pageSize, AbstractDashboard dashboard) {
List<ResponseDTO> processed = new ArrayList<ResponseDTO>();
String query = getDashboardQuery(date, dashboard);
logger.info("Dashboard Query: " + query);
List<Object[]> raw = createNativeQuery(query).getResultList();
return raw.stream().map(r->new ResponseDTO(r)).collect(Collectors.toList());
}
And the constructor of my DTO object:
public ResponseDTO(Object[] r) {
this.date = ((Date) r[0]).toLocalDate();
System.out.println(this.date.toString());//This date does not match what is in the db.
this.type = (String) r[1];
this.label = (String) r[2];
this.value = (Double) r[3];
}
Edit:
I think it's actually an issue with the java.sql.Date type, because I tried printing that out on my dev server and it also returns "2017-04-19" instead of the 20th. I don't get why this doesn't match the results when I run the query in a mysql console, it seems like they should be the same to me.

Try another JDBC driver
Ok, I got it working. For what it's worth, this seems like complete madness to me. The "Aha!" moment came while reading this answer to a different question.
Since it is the mysql driver that dictates how the date is parsed in the system. I rolled back my mysql driver as that was one of the packages that I updated. Suddenly everything behaved as expected.

Converting cassandra blob type to string

I have an old column family which has a column named "value" which was defined as a blob data type. This column usually holds two numbers separated with an underscore, like "421_2".
When im using the python datastax driver and execute the query, the results return with that field parsed as a string:
In [21]: session.execute(q)
Out[21]:
[Row(column1=4776015, value='145_0'),
Row(column1=4891778, value='114_0'),
Row(column1=4891780, value='195_0'),
Row(column1=4893662, value='105_0'),
Row(column1=4893664, value='115_0'),
Row(column1=4898493, value='168_0'),
Row(column1=4945162, value='148_0'),
Row(column1=4945163, value='131_0'),
Row(column1=4945168, value='125_0'),
Row(column1=4945169, value='211_0'),
Row(column1=4998426, value='463_0')]
When I use the java driver I get a com.datastax.driver.core.Row object back. When I try to read the value field by, for example, row.getString("value") I get the expected InvalidTypeException: Column value is of type blob. Seems like the only way to read the field is via row.getBytes("value") and then I get back an java.nio.HeapByteBuffer object.
Problem is, I cant seem to convert this object to string in an easy fashion. Googling yielded two answers from 2012 that suggest the following:
String string_value = new String(result.getBytes("value"), "UTF-8");
But such a String constructor doesn't seems to exist anymore.
So, my questions are:
How do I convert HeapByteBuffer into string?
How come the python driver converted the blob easily and the java one did not?
Side Note:
I could debug the python driver, but currently that seems too much work for something that should be trivial. (and the fact that no one asked about it suggests Im missing something simple here..)

Another easier way is to change the CQL statement.
select column1, blobastext(value) from YourTable where key = xxx
The second column would be type of String.

You can also get direct access to the Java driver's serializers. This way you don't have to deal with low-level details, and it also works for other types.
Driver 2.0.x:
String s = (String)DataType.text().deserialize(byteBuffer);
Driver 2.1.x:
ProtocolVersion protocolVersion = cluster.getConfiguration().getProtocolOptions().getProtocolVersion();
String s = (String)DataType.text().deserialize(byteBuffer, protocolVersion);
Driver 2.2.x:
ProtocolVersion protocolVersion = cluster.getConfiguration().getProtocolOptions().getProtocolVersion();
String s = TypeCodec.VarcharCodec.instance.deserialize(byteBuffer, protocolVersion);

For version 3.1.4 of the datastax java driver the following will convert a blob to a string:
ProtocolVersion proto = cluster.getConfiguration().getProtocolOptions().getProtocolVersion();
String deserialize = TypeCodec.varchar().deserialize(row.getBytes(i), proto);

1.) Converting from byte buffer in Java is discussed in this answer.
2.) Assuming you're using Python 2, it's coming back as a string in Python because str is the binary type.