Develop Reference

Cannot read special characters with JDBC

I'm having problem reading UTF-8 data from MySQL database by using MySQL Connector v. 8.0.19. Scandic letters, such as "äö" are replaced with unknown characters. I already made sure the database and its tables and columns are using utf8mb4. Then I added useUnicode=true&characterEncoding=UTF-8 to JDBC connection string, but the outcome is still unexpected. I'm running MySQL CE v. 8 in a Docker container. I can see the scandic letters fine when I run the SELECT queries in a command-line.

I solved this problem by passing --default-character-set=utf8mb4 to MySQL command-line before creating the schema from a separate file. I could add this option to MySQL server configuration as a default.

H2 Database Database to Lower Not Valid Connection Setting (2019)

I'm trying to connect to an embedded H2 database with the compatibility mode for PostgreSQL. I'm using the settings as described in the online documentation:
jdbc:h2:~/test.h2.db;MODE=PostgreSQL;DATABASE_TO_LOWER=TRUE
I'm getting the error:
Unsupported connection setting "DATABASE_TO_LOWER"
Is this still an experimental setting? Is there a solution and/or workaround for this error?
H2 Database v.1.4.199 + Java 11.0.2 (Oracle)

It looks like you have multiple versions of H2 in your classpath and wrong version is actually used. You can check the complete error message to be sure.
For example, version 1.4.197 throws
Exception in thread "main" org.h2.jdbc.JdbcSQLException: Unsupported connection setting "DATABASE_TO_LOWER" [90113-197]
A 197 indicates a database version.
1.4.199 accepts your URL properly.
Also note that database path in the URL shouldn't include a file name extension. With ~/test.h2.db the file test.h2.db.mv.db will be actually used.

Please visit http://www.h2database.com/html/changelog.html
PR #1776: Improve DATABASE_TO_LOWER handling
Try to update your h2 version from 1.4.199
Warning : By default H2 transforms all names into uppercase (ex: fromIP => FROMIP). This setting will pass everything in lowercase but will not keep the case (ex: fromIP => fromip)

How MySQL connector jar version can affect query performance?

Currently, in my application with mysql-connector-java:5.1.36, everything works fine. But when I upgrade connector to mysql-connector-java:5.1.47, a query starts to take minutes-2-hours time to execute. If I run the same query directly from the terminal or from the application with v5.1.36, it takes less than a few seconds to execute.
How MySQL connector jar version can affect query performance?

I found the reason. There is a change in mysql-connector-java:5.1.47 and above:
When UTF-8 is used for characterEncoding in the connection string, it maps to the MySQL character set name utf8mb4. While for mysql-connector-java:5.1.46 and below it corresponds to utf8 (or utf8mb3 more appropriately).
Link: https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-charsets.html
For my database character encoding is set to utf8 (or utf8mb3 more appropriately). Due to this index was not being used in query. After setting server property
character_set_server : utf8
it is working fine.

MySQL does not store properly some UTF8 chars

I'm using the mysql dbms to store pages from Wikipedia. I've set the character-set encoding to utf-8 (wikipedia encoding) in my.cnf file with the directive:
[mysqld]
character_set_server = utf8
And created my database with the 'chararacter set utf8' property definition.
I've also changed the charset-encoding for mysqld client by:
inserting the 'charSet=utf8' property when initializing my jdbc driver.
doing a query to 'set names utf8'
However I've noticed that mysql server replaces some characters with others.
For example it replaces á with a.
UPDATE
I've run the command show variables like '%char%' ensuring that both character_set_client and character_set_set are utf8.
How can I store the correct chars in my db? Thanks!

Try to specify the encoding in the DB URL :
url="jdbc:mysql://localhost:port/DBNAME?characterEncoding=UTF-8"
Here's some more information regarding my answer :
The following is taken from the MySQL documentation (http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-charsets.html) :
All strings sent from the JDBC driver to the server are converted
automatically from native Java Unicode form to the client character
encoding, including all queries sent using Statement.execute(),
Statement.executeUpdate(), Statement.executeQuery() as well as all
PreparedStatement and CallableStatement parameters with the exclusion
of parameters set using setBytes(), setBinaryStream(),
setAsciiStream(), setUnicodeStream() and setBlob().
Setting the Character Encoding
The character encoding between client
and server is automatically detected upon connection. You specify the
encoding on the server using the character_set_server for server
versions 4.1.0 and newer. The driver automatically uses the encoding
specified by the server. To override the automatically detected
encoding on the client side, use the characterEncoding property in the
URL used to connect to the server. To allow multiple character sets
to be sent from the client, use the UTF-8 encoding, either by
configuring utf8 as the default server character set, or by
configuring the JDBC driver to use UTF-8 through the characterEncoding
property.
I encountered a similar problem a few months ago. I checked the default value of character_set_server on my MySQL (using the “mysqld --verbose –help” command).
It was latin1.

UTF-8 encoding problem in unix machine

Im exporting a set of data to excel in java, the data has certain non ascii characters, while exporting in Windows machine the data are coming correctly in UTF-8 encoded format.But when i deploy my code in Unix machine it is not working properly.UTF-8 encoding is not working properly.
Im using Tomcat 5.5 server.I have also included URIencoding="UTF_8" parameter in server.xml. But still in unix machine it is not working properly

Running Tomcat with a
-Dfile.encoding=UTF8
option will force the VM to adopt UTF8 as it's default encoding regardless of your environment. I suspect that's your problem (and it's good practise nonetheless)

When you are working with UTF-8 data it can be very fragile. Every step in the chain needs to specify utf8.
(1) In the database, be sure the table has UTF8 encoding and collation. "show table status" will tell you this. Google for "mysql alter table utf8" for more info.
(2) The JDBC connection needs to specify the UTF8 encoding. For Connector/J this will be something similar to:
useUnicode=true&characterEncoding=UTF-8
(3) Every stream read/write, every string to byte conversion in the code needs to specify UTF-8 as the encoding. It's easy to miss one. If it's missed Java will use the system default which will vary server to server.
(4) Not applicable here, but for user submitted data from a form, you need to set the request character encoding. (I do this in a servlet filter)
request.setCharacterEncoding("UTF-8");
(5) Probably not applicable here, but if you output HTML/XML/text from your servlet, set the HTTP character encoding header. (This might apply if you are generating the Excel file as an XML file).
response.setContentType("text/html; charset=UTF-8");