Develop Reference

UCanAccess in Java returning wrong order with ORDER BY clause in column with special characters

Using the Microsoft Access Database (2007-2013) with a .mdb file, I created a simple test_table table with only one text column "name" and inserted the following test values:
óbito,
fanatico,
orbita,
fanático,
fanta,
órbita,
fantástico,
obito,
obituario,
orbitando
When I execute the query SELECT * FROM test_table ORDER BY name using MS Access query design, the following ordered result is returned:
fanatico,
fanático,
fanta,
fantástico,
obito,
óbito,
obituario,
orbita,
órbita,
orbitando
This order is totally correct and expected.
Now, I need to retrieve and use these values in my Java software. In order to do this, I am using the UCanAccess JDBC driver on version 5.0.0 to connect to the database. The connection itself is successfully being opened, but, when I execute the same query above, it returns the following:
fanatico
fanta
fantástico
fanático
obito
obituario
orbita
orbitando
óbito
órbita
And this is NOT the correct order (for instance, óbito should come immediately after obito). The desired order should consider accented words as if they were the same as the equivalent unnacented word.
It doesn't matter if óbito comes before or after obito, but they must be together.
I tried using COLLATE, tried changing the charset, etc, but nothing worked. Has anyone gone through something similar and could you help me solve this issue? Thanks in advance.

The driver is sorting by their binary representation and/or the individual ASCII characters. Both provide the sort order you provided at the bottom. This is entirely a problem created by the driver, and "fixes" are going to be limited.
There is a workaround posted in the JDBC driver changelog, under the 2.0.9.3 Release notes: WORKAROUND suggested: if you want the same behaviour of Access: select * from table2 order by orderJet( COLUMN1).
If that doesn't work, then you either need to
a) subvert the driver's sorting by creating/maintaining a SORTORDER column in the original database that holds the same word with all accented characters stripped, or b) find a way to change the sort after it arrives from the driver. Neither of these are preferable, so I hope the workaround provided by the developer is sufficient.

Java by default do not perform locale-sensitive String comparison.
In your example I tried following ran program as below for natural sorting
List<String> strings = Arrays.asList(new String[]{"óbito",
"fanatico",
"orbita",
"fanático",
"fanta",
"órbita",
"fantástico",
"obito",
"obituario",
"orbitando"});
Collections.sort(strings);
System.out.println("Output = " + strings);
the output is
Output = [fanatico, fanta, fantástico, fanático, obito, obituario, orbita, orbitando, óbito, órbita]
Now jut by replacing sort by below line
Collections.sort(strings, Collator.getInstance(Locale.US));
I am getting output which you are expecting
Output = [fanatico, fanático, fanta, fantástico, obito, óbito, obituario, orbita, órbita, orbitando]
Giving you example above to understand the difference when you use string comparison using locale-sensitive. There are ways you can handle this from your code or from db configuration.
You can check here for example

Did you try enforcing a CharSet?
Have a look here CharSet for MS Access '97 DB using UCanAccess
class DatabaseOpener : JackcessOpenerInterface {
override fun open(fl: File, pwd: String?): Database {
return DatabaseBuilder.open(fl).apply {
this.charset = charset("Cp1252")
}
}
}
// URL
"jdbc:ucanaccess://<path-to-mdb-file>;memory=false;jackcessOpener=${DatabaseOpener::class.qualifiedName!!}"
When using plain JDBC connection you could try adding a connection parameter:
private static java.sql.ResultSet executeDataTable(String sql) throws Exception {
Class.forName("net.ucanaccess.jdbc.UcanaccessDriver");
String conStr = "jdbc:ucanaccess://" + dataDir + "ABC.mdb";
Properties props = new java.util.Properties();
props.put("charSet", "Cp1252");
java.sql.Connection con = java.sql.DriverManager.getConnection(conStr, props);
java.sql.Statement stmt = con.createStatement();
return stmt.executeQuery(sql);
}
You need to check what your charset might be. So potentially replace 'Cp1252'.

HSQLDB "precision or scale out of range" error when UCanAccess loads database

I'm writing tools in java that connects to different access databases. I have a particular scenario that throws the following exception -5592:UCAExc:::4.0.4 precision or scale out of range
The exception is listed under HSQLDB exceptions:
5592=42592 precision or scale out of range (so I know who generates it)
and according to this answer, this other answer, and the definitions in here, both precision and scale refer to characteristics of a double.
The exception occurs only when I use ucanaccess to connect to an old mdb (pre-access 2003) trhough a frontend database with linked tables to the mdb. However, ucanaccess can connect to the old mdb directly without any issue.
If I use the frontend in Access to modify the database it works fine, so I assume MS worked around the problem at least within Access.
My guess is that my ucanaccess connection is attempting to treat the database as a more modern one, being fooled by the front-end facade. But why this exception?
Minimal Complete Verifiable Example:
Here is a minimal example to replicate the issue containing the offending mdb, is a gzip tarball which includes the following jar, the databases involved and a helpful readme file.
public static void main(String[] args) {
String query= "SELECT nombre FROM encemp where cveemp=1";
try {
Connection frontEndToAccdb = DriverManager.getConnection("jdbc:ucanaccess://FrontEndPointingToAccdb.accdb");
PreparedStatement statement = frontEndToAccdb.prepareStatement(query);
ResultSet resultSet = statement.executeQuery();
if (resultSet.next()) System.out.println("Querying Accdb BackEnd through front end OK");
Connection directConnectionToMdb = DriverManager.getConnection("jdbc:ucanaccess://X:/BackendOld.mdb");
statement = directConnectionToMdb.prepareStatement(query);
resultSet = statement.executeQuery();
if (resultSet.next()) System.out.println("Querying mdb BackEnd directly OK");
//This is the one that will generate the exception
Connection frontEndToMdb = DriverManager.getConnection("jdbc:ucanaccess://FrontEndPointingToMdb.accdb");
statement = frontEndToMdb.prepareStatement(query);
resultSet = statement.executeQuery();
if (resultSet.next()) System.out.println("Querying mdb BackEnd through front end OK");
} catch (SQLException ex) {
System.out.printf("%s:%s\n", ex.getErrorCode(),ex.getMessage());
}
I've been struggling navigating the DatabaseMetaData and still have no clue why the offending mdb is generating the exception.

I took the liberty of updating your title. Hopefully the new wording conveys what you mean.
Please do the following:
a) Query the column definitions in question directly. For example:
// GetMetaData():
try (ResultSet rsMD = connChem.getMetaData().getTables(null, null, null, null)) {
while (rsMD.next()) {
String tblName = rsMD.getString("TABLE_NAME");
System.out.println(tblName);
}
}
... or ...
-- SQL "select" from MSysObjects
SELECT Name
FROM MSysObjects
WHERE Left([Name],1)<>"~"
AND Left([Name],4)<>"MSys"
AND Type In (1,4,6)
ORDER BY Name;
b) Compare the DB column definitions (above) with the corresponding Hibernate class members (for example, in your favorite IDE).
c) Compare the results for an "old" .mdb (which "works") vs. a "new" .accdb.
Please update your post with the results.

As it turns out, this particular issue has nothing to do with DECIMAL numbers, or numeric columns at all. It is a bug in UCanAccess caused by a bug in Jackcess where the width of an Access_97 TEXT field is mis-reported as half of its actual width.
The database in question has a table with a column defined as TEXT(1), so Jackcess getLengthInUnits reports zero (because of integer division). UCanAccess tries to create a table in the HSQLDB backing database as
CREATE CACHED TABLE DETCAD(SERIE VARCHAR(0), ...
and the VARCHAR(0) is triggering the "precision or scale out of range" exception.
jackcess-2.2.2.jar fixes the issue. You can use it to replace the jackcess-2.1.11.jar file that ships with UCanAccess 4.0.4.

Precision is the maximum total number of digits in a decimal. Scale is the number of digits to the right of the decimal point (ref). So if you had decimal(20,10) then it would be a decimal with 10 digits to the left, and 10 digits to the right of the decimal for a total of 20 digits. i.e. 1234567890.1234567890. The maximum precision is 38, default is 18.
Examples
decimal(38,0): '12345678912345678912345678912345678900'
decimal(38,1): '1234567891234567891234567891234567890.0'
decimal(38,2): '123456789123456789123456789123456789.00'
decimal(38,3): '12345678912345678912345678912345678.900'
decimal(10,8): '12.12345678'

Java ResultSet giving returning wrong values for some columns

I have an applicaton that executes a query using NamedParameterJdbcTemplate. The resultSet is then parsed row by row using ResultSet.next().
Now in some cases during multi threading scenarios, this goes wrong. The result set is returning wrong values. When I execute the same query in SQLDeveloper, I am seeing the correct values. Not sure what could be the problem behind this.
while (rs.next()) {
count++;
long dbKy = rs.getLong("DBKY");
pAttrs = map.get(dbKy );
if (pAttrs== null) {
pAttrs= new HashMap<String, String>();
map.put(dbKy , pAttrs);
}
log.info( "PrintingResultSet!!::"+rs.getLong("DBKY")
+"::"+rs.getString(ATTR_NAME)
+"::"+rs.getString(ATTR_VAL)
+"::"+rs.getString(Constants.VAL));
pAttrs.put(rs.getString(ATTR_NAME),rs.getString(ATTR_VAL));
}
EDIT: This code is in the repo layer of SpringBoot application. Multithreading is, this issue happens when multiple requests are sent simultaneously. I have printed Thread id in my logs and it confirms that this happens only in multi threaded scenarios.
The value that is being returned actually is the value of some other row.

What values (wrong values) do you see when you are trying to display the resultset. If you see some unknown texts or symbols then probably it could be "encoding" issue. I suggest you to please refer on how to encode values like some special characters/symbols on your service layer since no doubt you will be able to see the data in the database by using the query but if that data contains some special characters/symbols then there is a need of encoding "UTF-8".
Thanks!

MySQL Query Parsing using java

I need to parse a query which a user enters, say in a text box, and then what I need, is that I want to encrypt all the values in query leaving the query keywords. To convert it into an equivalent query that can be performed on an encrypted database.
Such as,
select name from employee where salary = 10000
I need an equivalent query as,
select name_enc from employee_enc where salary_enc = 10000_enc
where name_enc,employee_enc, salary_enc and 10000_enc are the encrypted values of name, employee, salary and 10000. I need to do this in java and the the database I'm using is MySQL Server where the table Employee is already encrypted.
Please provide any necessary help. Thanks in advance.

You may want to consider using code from Alibaba's Druid project. Although designed as a sophisticated connection pooling library, this project supports a very advanced parser and AST for ANSI SQL and non-ANSI dialects such as MySQL, Oracle, SQL Server, etc. The project is open source and bears the very liberal Apache License Version 2.0.
The main entry points into this part of the library is SQLUtils.java. You can use values returned from SQLUtils.parseStatements to access a typed model of the statements:
List<SQLStatement> statements = SQLUtils.parseStatements(sql, JdbcConstants.MYSQL);
for (SQLStatement statement : statements) {
if (statement instanceof SQLSelectStatement) {
SQLSelectStatement createTable = (SQLSelectStatement) statement;
// Use methods like: createTable.getSelect().getQuery().
}
}

If you don't need to do it manually use SQL's included encryption and encoding operations.
If you need to do it manually split your SQL query string by spaces and ignore SQL key words as you loop to encrypt. Remember to encode your cipher results with base 64 or hex to ensure data integrity.
private String encryptSQLQuery(String plainSQLQuery){
StringBuilder cipherQuery = new StringBuilder();
String plainQuery = plainSQLQuery;
String[] splitQuery = plainQuery.split("\\s+");
for(String queryWord : splitQuery){
if(!isSQLKeyWord(queryWord))
queryWord = cryptoObject.encryptAndEncode(queryWord);
cipherQuery.append(queryWord);
cipherQuery.append(" ");
}
return cipherQuery.toString();
}
Note that you will have to implement the isSQLKeyWord() and CryptoObject.encryptAndEncode() methods.

Java.sql - Error while nesting CAST into MAX

I am trying to execute the following query:
select MAX(CAST(T.progress AS DECIMAL)) as maxProgress from default_APN_TRACKING T where T.smartlet = 'Life-Pension Selector' GROUP BY T.trackingId
Which throws me the following Exception:
java.lang.Exception: Exception on sql query:select MAX(CAST(T.progress AS DECIMAL)) as maxProgress from default_APN_TRACKING T where T.smartlet = 'Life-Pension Selector' GROUP BY T.trackingId
Basically, i got a table with tracking IDs, and a string representing the progresses ("100.0", "66.6", "33.3", ...). For each trackingId, i want to get the highest progress.
I Agree having Integers would make more sens, but it's a constraint I have to deal with. So i tried using the CAST. Also Attempted CAST(T.progress AS DECIMAL(10,5)) without success.
The same query without the CAST works just fine, but sorts alphanumerically ("66.6" > "100.0"). How should I tackle this problem?
Thanks for the help!
EDIT: Copying the same query directly inside SQL WorkBench works perfectly. Seems like Java.sql does not like my query for some reason.