Ive been stunk in working with java and mysql these days.
The problem is, ive got a mysql database. There is a column in one table which shows the chinese city names. One collegue changed the db to utf8 for every character(connection, db, results, server and system) The consequence is that the data before the change didn't show correctly any more only if i set the %character% back to latin1. In either character set i can only retrive half the data correctly. Could you please help me how to solve the problem?
Ive tried to use java to solve the problem but it doesn't work.
String sql = "SELECT * FROM customer_addresses";
ResultSet result = query.executeQuery(sql);
while (result.next()) {
byte b[] = result.getBytes("city");
c = new String(result.getBytes("city"), "UTF-8");
}
For example: there is one city in db like this 乌é²æœ¨é½å¸‚
the java print: 乌�?木�?市
it should be:乌鲁木齐市
Thanks in advance
Default charset of your MySQL server is probably not UTF8. Try to execute the following SQL queries before getting data from the database:
SET NAMES utf8
and
SET CHARACTER SET utf8
Add characterEncoding=UTF-8 to the connection string, where you connect to the database. For example:
"jdbc:mysql://servername:3306/databasename?characterEncoding=UTF-8"
Incidentally, the data in the database appears to be broken. If you want the database to store 乌鲁木齐市, that's what should be in the table, not 乌é²æœ¨é½å¸.
Update: The problem with the how the data is stored in the database is easier to solve using database's own tools, not Java. For each table that stores text do this:
ALTER TABLE tablename CONVERT TO CHARACTER SET binary;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8;
Related
I have problems when I insert into tables words with accent marks. So I think that I have to "activate" UTF-8 to fix that error.
I'm not using Class for name. That's my code:
miInitialContext = new InitialContext();
miDS = (DataSource) miInitialContext.lookup(InformacionProperties.getStrDataSource());
Connection conexion = miDS.getConnection();
Statement myStatement = conexion.createStatement();
myStatement.executeUpdate("INSERT INTO table values ......)
How can I "activate" that UTF8 with my code?
Use set names to set your connection charset. However, that won't matter much if the columns/tables/databases you interact with aren't configured with a compatible charset. For instance, with a latin1 column testcol, inserting utf8 data will result in an error like
INSERT INTO `test`.`table` (`testcol`) VALUES ('test_val'), ('Ídata');
ERROR 1366 (HY000): Incorrect string value: '\xE2\x88\x9A\xC3\xA7d...' for column 'testcol' at row 2
So you'll need to update the table structure
ALTER TABLE t MODIFY testcol CHAR(50) CHARACTER SET utf8;
Which then fixes the issue:
INSERT INTO `test`.`table` (`testcol`) VALUES ('test_val'), ('Ídata');
Query OK, 2 rows affected (0.00 sec)
Records: 2 Duplicates: 0 Warnings: 0
(See the mysql docs for details).
This post has good documentation on finding the character sets of various structures.
(Thanks to Shadow for the SET NAMES part)
I'm using mysql DB. The DB is amazone RDS DB.
When I execute this query: show global variables like '%character%';
I get the following result:
But when I execute this query: show variables like '%character%'; I get this result:
As you can see character_set_results is empty. I tried following queries and nothing changes the empty value:
ALTER DATABASE myDB CHARACTER SET utf8;
ALTER DATABASE myDB CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER DATABASE myDB DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
SET CHARACTER_SET_RESULTS=UTF8
SET NAMES 'utf8';
SET CHARACTER SET 'utf8';
SET session CHARACTER_SET_RESULTS = utf8;
As I understand the second query returns session parameters. Might this affect the results I'm getting?
I have two machines. Development machine where I run the application with eclipse (this is a java app with hibernate) and another one, the deployment machine. Everything works fine on the development machine but on the deployment machine sometimes I'm getting ???? or other strange characters when getting data from DB.
Both machines connect to the same DB and data is stored fine in the db itself.
Also the connection url is jdbc:mysql://myDB:3306/myApp?autoReconnect=true&useUnicode=true&createDatabaseIfNotExist=true&characterEncoding=utf-8.
Any ideas what can cause this?
Multiple question marks usually come from:
You are INSERTing Chinese (or any non-western-Europe text).
You said SET NAMES utf8 to declare that the client bytes are utf8-encoded (correct).
But the table columns are declared CHARSET latin1. <-- This is the problem.
Since there is no way to convert Chinese characters into latin1, '?' stored.
Please provide SHOW CREATE TABLE to confirm the above hypothesis.
If you are getting "other strange characters", please provide:
SELECT col, HEX(col) FROM ...
to show an example of what is stored for the "strange characters".
I'm trying to fill an SQLite database with data in my java program.
The data is read from an excel file using Apache POI. I have no trouble inserting the data into the db using normal methods.
However, when I check the database manually with the shell, all the Norwegian characters æ,ø,å are not displayed correctly. Whenever I fill out the database manually through the shell, they are displayed as they should.
Also, when printing out a java string in console containing these characters, they are displayed correctly.
The problem must occur when an action like this is performed:
String sql = "insert into db(name) values (æøå)";
stmt.executeUpdate(sql);
I have tried
byte[] b = sql.getBytes("utf-8");
sql = new String(b, "utf-8");
to no avail.
Any idea how to remedy the situation?
Thanks!
There is a very simple solution for you: Let Java and the SQLite driver do everything for you. You don't have to care about encodings and escaping of parameters.
How that is possible: Use a PreparedStatement:
String name = "æøå"
PreparedStatement prepStmt = conn.prepareStatement("insert into db(name) values (?)");
prepStmt.setString(1, name);
prepStmt.executeUpdate();
Furthermore this code fragment is secure against SQL injection attacks.
BTW: The second code fragment you posted is totally useless, it does nothing. Converting a String to byte[] and back to String does not change a single bit of the String.
Hi i am using Oracle DB to store string on Varchar2 column,
with using eclipselink my code is here,
pdescription = new String(this.description.getBytes("ISO-8859-9"));
sometimes its ok but, somtimes it only question marks, like that
it is taken "door" or "????"
I have column that is also string there is problem with that, their types are same both varchar2
I have an SQLite3 databse I created in python. And by default it writes the database in Unicode.
Now I am trying to query the database in a Java Applet using SQLite JDBC. And I cannot find tables, rows etc because I think Java &/or JDBC queries in ANSI.
Does anyone know how I can query my SQLite3 DB with a unicode query in Java? Something like the following doesn't work (in Java trying to execute a Unicode SQL query):
If I access the database in python I can print out the tables no problem & make updates BUT if I try to do the same in Java, I get no results returned from my query. Is this an encoding problem or some thing else
This works import sqlite3
def blah():
conn = sqlite3.connect( "a.db" )
cur = conn.cursor()
res = cur.execute( "SELECT name FROM sqlite_master WHERE type='table'" ).fetchal()
print res
blah()
This returns no tables when it should return the same tables as above
Class.forName("org.sqlite.JDBC");
Connection conn = DriverManager.getConnection("jdbc:sqlite:a.db");
Statement stat = conn.createStatement();
conn.setAutoCommit(false);
ResultSet tables = stat.executeQuery("SELECT name FROM sqlite_master WHERE type='table'");
String b = "";
while (tables.next()) { b+= "table= " + tables.getString("name"); }
Jim, that's very odd, it should work. Have you tried to open the DB from the console?
you can open it by running sqlite3 a.db
something that intrigues me, is that you're trying to open the db from a java applet. Have you given it the necessary permissions and signed it, so the apple can actually write to disk?
Have your tired opening the sqlite3 a.db and typing PRAGMA encoding; ? It should say UTF8 by default.
I am able to create a UTF8 sqlite database on the command line and read the contents of that file using sqlite jdbc.
Are you sure you are connecting to the database that you created with python? IF the database does not exist then sqlite will create the database and it will not have any tables in it. This is what is probably happening to you.
Are you running the java and python in the same directory?