Develop Reference

Table charset utf8mb4 not working mysql-connector-java 5.1.44 (and later)

I have mysql server 8.0.21 with configuration
character_set_client utf8
character_set_connection utf8
character_set_database utf8
character_set_filesystem binary
character_set_results utf8
character_set_server utf8
character_set_system utf8
And a table with charset utf8mb4
CREATE TABLE `test` (`c0` TEXT) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
This code is working with mysql-connector-java 5.1.18
Connection conn = DriverManager.getConnection("jdbc:mysql://hostxxxx/test?" +
"user=userxxx&password=passxxx");
String sql = "INSERT INTO test(c0) value ('🚘💕🚘💕🚘')";
conn.createStatement().execute(sql)
But when I use mysql-connector-java-5.1.44 (or higher) -> get the error java.io.IOException: Incorrect string value: '\xF0\x9F\x9A\x98\xF0\x9F...'
I am able to get this code to work with version 5.1.44++ by changing the server config character_set_server to utf8mb4, But because backward compatible, I could not change it
Anyone know how this code working with the old version (mysql-connector-java-5.1.18) but not work with the version (5.1.44 and later)
I found some related topics to this error but could not resolve my issue

utf8mb4 in MySQL Workbench and JDBC

I've been working with a UTF-8 encoded MySQL DB that now needs to be able to store 4-byte emojis, so I decided to change from utf8 encoding to utf8mb4:
ALTER DATABASE bstdb CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
ALTER TABLE HISTORY CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE HISTORY CHANGE SOURCE_CONTEXT SOURCE_CONTEXT VARCHAR(2000) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci;
And changed mysql.conf.d "character-set-server = utf8" to "character-set-server = utf8mb4"
After these steps, I am able to store emojis (as 💢),but only when the SQL query is executed in the MySQL console: When I try to launch the query from MySQL Workbench or from a Wildfly webapp, I am getting this error:
Error Code: 1366. Incorrect string value: '\xF0\x9F\x92\xA2' for column 'SOURCE_CONTEXT' at row 1
I assume I need to change the way the clients are connecting to the DB, but I have no clue on how. I've read something on using "useUnicode=yes" in JDBC, but does not work.
${bdpath:3306/bstdb?useUnicode=yes}
Edit:
As suggested in comments, I tried with:
${bdpath:3306/bstdb?characterEncoding=UTF-8}
but no luck, I am getting the same "Incorrect string value: '\xF0\x9F\x92\xA2'" error.
Also tried
${bdpath:3306/bstdb?useUnicode=true&characterEncoding=utf8mb4&}
but it refuses to stablish a connection.
Any idea on how to configure MySQL workbench and/or JDBC/Wildfly?
MySQL version is 5.7.18
MySQL WorkBench version is 6.0.8
JDBC driver version is 5.1.34
Thanks!

Use characterEncoding=utf8 for jdbc url
jdbc:mysql://x.x.x.x:3306/db?useUnicode=true&characterEncoding=utf8
Also check that you have configured MySQL to work with utf8mb4
[client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
See here

Starting from MySQL Connector/J 5.1.47,
When UTF-8 is used for characterEncoding in the connection string, it maps to the MySQL character set name utf8mb4.
You can check docs here

Finally, it works. It was an issue with stored procedures, that was still utf8 instead of utf8mb4 after the migration.
It was a 2-steps solution.
As suggested by #mike-adamenko set my.cnf to have the following
[client]
default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
Execute in mysql:
SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci;
Drop procedures involved, and create them again. They will be in utf8mb4.
Can be checked with
SHOW PROCEDURE STATUS where name LIKE 'procedure_name';

You can follow the documentation available for MySQL to resolve your problem. Here's the MySQL documentation, that you could refer to.
Basically, your ALTER TABLE scripts can be changed as per the documentation mentioned above and then you could use the following parameter in your connection string for the changes to take effect.
jdbc:mysql://localhost/yourdatabasename?useUnicode=true&characterEncoding=UTF-8
Please don't forget to restart your MySQL services after making the character set and the encoding changes.

persist arabic diacritics data

I want to insert arabic data that contains diacritics in my database but I got
this exception
I have already added this to the my.ini file
[client] default-character-set = utf8mb4
[mysql]
default-character-set = utf8mb4
[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
and I also added this to my hibernate.cfg.xml
<property name="hibernate.connection.properties">useUnicode=yes;
characterEncoding=utf8;
</property>
<property name="hibernate.connection.useUnicode">true</property>
<property name="hibernate.connection.characterEncoding">utf-8</property>
but in vain

In my case. I used to set the coding to utf-8. my.ini file has the following config
[client]
default-character-set = utf8
[mysql]
default-character-set = utf8
[mysqld]
character-set-server = utf8
collation-server = utf8_unicode_ci
I use the following url
jdbc:mysql://localhost:3306/test?useUnicode=true&characterEncoding=utf8&useSSL=false
cloud work fine.

JOOQ emoji (utf8mb4) support

We try to store and read emoji in our MySQL 5.6 database with JOOQ.
The database, table and column are using character set utf8mb4 and collation utf8mb4_unicode_ci. With MySQL Workbench I can create and select emojis. So the database should be ready.
But when I store an emoji with JOOQ I get:
Incorrect string value: '\xF0\x9F\x98\x80' for column 'test' at row 1SQL
DSLContext dslContext = DSL.using(dataSource, SQLDialect.MYSQL);
dslContext.insertInto(table)
.set(testRecord)
.returning()
.fetchOne();
Retrieving en emoji I stored with MySQL Workbench works fine.

To use utf8mb4 in the application make sure you set it on the server level or before performing the query.
There are 2 ways of doing it:
Server level: add character_set_server=utf8mb4 to my.cnf or "set global character_set_server=utf8mb4"
Before running query: "set names utf8mb4"

Can't store UTF-8 Content in MySQL Using Java PreparedStatement

For some strange reason I can't seem to add UTF-8 data to my MySQL database. When I enter a non-latin character, it's stored as ?????. Everything else is stored fine. So for example, "this is an example®™" is stored fine, but "和英辞典" is stored as "????".
The connection url is fine:
private DataSource getDB() throws PropertyVetoException {
ComboPooledDataSource db = new ComboPooledDataSource();
db.setDriverClass("com.mysql.jdbc.Driver");
db.setJdbcUrl("jdbc:mysql://domain.com:3306/db?useUnicode=true&characterEncoding=UTF-8");
db.setUser("...");
db.setPassword("...");
return db;
}
I'm using PreparedStatement as you would expect, I even tried entering "set names utf8" as someone suggested.
Connection conn = null;
PreparedStatement stmt = null;
ResultSet rs = null;
try {
conn = db.getConnection();
stmt = conn.prepareStatement("set names utf8");
stmt.execute();
stmt = conn.prepareStatement("set character set utf8");
stmt.execute();
... set title...
stmt = conn.prepareStatement("INSERT INTO Table (title) VALUES (?)");
stmt.setString(1,title);
stmt.execute();
} catch (final SQLException e) {
...
The table itself seems to be fine.
Default Character Set: utf8
Default Collation: utf8_general_ci
...
Field title:
Type text
Character Set: utf8
Collation: utf8_unicode_ci
I tested it by entering in Unicode ("和英辞典" specifically) through a GUI editor and then selecting from the table -- and it was returned just fine. So this seems to be an issue with JDBC.
What am I missing?

On your JDBC connection string, you just need set the charset encoding like this:
jdbc:mysql://localhost:3306/dbname?characterEncoding=utf8

There is 2 points in the mysql server to check in order to correctly set the UTF-8 charset.
Database Level
This is obtained by creating it :
CREATE DATABASE 'db' CHARACTER SET 'utf8';
Table Level
All of the tables need to be in UTF-8 also (which seems to be the case for you)
CREATE TABLE `Table1` (
[...]
) DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci;
The important part being DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci
Finally, if your code weren't handling utf8 correctly, you could have forced your JVM to use utf8 encoding by changing the settings by on startup :
java -Dfile.encoding=UTF-8 [...]
or changing the environment variable
"**JAVA_TOOLS_OPTIONS**" to -Dfile.encoding="UTF-8"
or programmatically by using :
System.setProperty("file.encoding" , "UTF-8");
(this last one may not have the desire effect since the JVM caches value of default character encoding on startup)
Hope that helped.

Use stmt.setNString(...) instead of stmt.setString(...).
Also don't forget to check column collation in database side.

If you log in to your mysql database and run show variables like 'character%';
this might provide some insight.
Since you're getting a one-to-one ratio of multi-byte characters to question marks then it's likely that the connection is doing a character set conversion and replacing the Chinese characters with the replacement character for the single-byte set.

Also check locale -a on ubuntu default Ubuntu works with en_us locale and doesn't have other locale installed.
must specify characterEncoding=utf8 while connecting through JDBC.

add at the end of your DB connection url - (nothing else needed)
ex.
spring.datasource.url = jdbc:mysql://localhost:3306/dbname?characterEncoding=utf8