Develop Reference

How to convert DB2 binary data to UTF-8 at query level

I am connected to IBM DB2 database with java but data is stored as binary format in database so when I fetch any value it comes as binary or hexdecimal format. How can I convert this in binary data in utf-8 at query level.
Sample code to fetch data -
String sql = "SELECT poMast.ORDNO from AMFLIBL.POMAST AS poMast ";
Class.forName("com.ddtek.jdbc.db2.DB2Driver");
String url = "jdbc:datadirect:db2://hostname:port;DatabaseName=dbName;";
Connection con = DriverManager.getConnection(url, "username","password");
PreparedStatement preparedStatement = con.prepareStatement(sql);
ResultSet rs = preparedStatement.executeQuery();
System.out.println("ResultSet : \n");
System.out.println(" VNDNO");
while (rs.next())
{
System.out.println(rs.getString("ORDNO"));
}

You probably need to use the CAST expression:
SELECT CAST(poMast.ORDNO as VARCHAR(50)) from AMFLIBL.POMAST AS poMast
Adjust the VARCHAR length to your needs. The string is in the database codepage (often UTF-8 these days) and converted to the client/application codepage when fetched.

you can "cast" the result from your select to utf8 like below.
String sql = "SELECT poMast.ORDNO, CAST(poMast.ORDNO AS VARCHAR(255) CCSID UNICODE) FROM AMFLIBL.POMAST AS poMast ";
src: cast db2

In my case, somehow bad UTF-8 data had gotten into varchars in a 1208/UTF-8 DB. Prior to conversion, when querying such data via the JDBC driver, the DB returned -4220 via the JDBC driver. This is fixable at the JDBC driver level by adding this property:
java -Ddb2.jcc.charsetDecoderEncoder=3 MyApp
see:
https://www.ibm.com/support/pages/sqlexception-message-caught-javaiocharconversionexception-and-errorcode-4220
The Db2 LUW Command Line Processor fixed it long ago as an APAR, so this error is only seen via the JDBC driver when the above property is not set.
But, if you want to fix the data in the db, this works:
update <table_name> set <bad_data_col> = cast(cast( <bad_data_col> as vargraphic) as varchar);
1st db2 treats (casts) the bad data as a binary where "anything goes" and then converts (casts) it back to valid UTF-8. After the casts, the JDBC driver shows the same result with or without the special property set and returns no errors.

Hive table sizes using Java/JDBC

Is it possible to find the sizes in GB of the hive tables using Java/jdbc? I don't want to depend on the hive warehouse folder in HDFS (as described in link) as different tables may have different locations

If you are mention 'totalSize' from 'tblproperties' then it is possible with similar approach:
String driverName = "org.apache.hive.jdbc.HiveDriver";
String connectionURL = "jdbc:hive2://HOSTNAME:PORT/default";
try {
Class.forName(driverName);
Connection connection = DriverManager.getConnection( connectionURL, "", "");
} catch (ClassNotFoundException e) {
e.printStackTrace();
System.exit(1);
}
Statement stmt = connection.createStatement("show tblproperties TABLENAME");
ResultSet rs = stmt.executeQuery(stmt);
while(rs.next()){
//doWhatYouWant
}

Currently not be possible but it can be done .
To get file size you have to run on file system( HDFS) commands .
In case of RDMS data bases ie sql server have encapsulated file system commands in SYS views and SYS functions (DMF and DMVs) .
If some one write or develop such UDF it will be possible but Internally the UDF will be calling same comand.

Reading Unicode characters from an Access database via JDBC-ODBC

I have some non-standard characters in my Access 2010 database. When I read them via
Connection con = null;
try{
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
java.util.Properties prop = new java.util.Properties();
prop.put("charSet", "UTF8");
String database = "jdbc:odbc:Lb";
con = DriverManager.getConnection(database, prop);
} catch (Exception ex) {
System.out.println("Error");
}
Statement stm = conn.createStatement();
ResultSet rs = stm.executeQuery("SELECT distinct forename, surname from PSN where isValid");
while (rs.next()) {
String forename = rs.getString("forename");
}
I receive question marks (?) where the character should be. Why is this?

I had question marks when DB contained polish characters. It was fixed when I set charecter encoding to windows-1250.
def establish(dbFile: File): Connection = {
val fileName = dbFile.getAbsolutePath
val database = s"${driver}DBQ=${fileName.trim};DriverID=22;READONLY=true}"
val props = new Properties()
props.put("charSet", "Cp1250")
val connection= DriverManager.getConnection(database,props)
connection
}

I expect your JDBC driver to handle reading and writing characters to your database transparently. Java's internal string representation is UTF-16.
Java(UTF-16) --JDBC--> Database(DbEncoding)
Database(DbEncoding) --JDBC--> Java(UTF-16)
Perhaps the problem is that you are trying to force reading them with UTF8 and the database uses another internal representation?
Also, how do you verify that you receive '?'
If System.out is involved, you should take into consideration that this PrintStream converts in memory Strings to the Charset that it uses. IIRC this Charset can be found with Charset.defaultcharset() and is a property of th JVM that runs the program.
It is preferable to inspect the hexadecimal value of the char and look up a Unicode table to be sure that information has been lost while reading from the database.
Hope this helps a bit.

It's not "utf8", "Cp1250" !
One must use : ISO-8859-1
java.util.Properties prop = new java.util.Properties();
prop.put("charSet", "ISO-8859-1");
String connURL = "jdbc:odbc:DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=" + accessFileName + ";uid=''; pwd='';";
sql = "SELECT * FROM enq_horaires;";'
con = DriverManager.getConnection(connURL, prop);
stmt = con.createStatement();
ResultSet rset = stmt.executeQuery(sql);

This is a long-standing interoperability issue between the Access ODBC driver and the JDBC-ODBC Bridge. Access stores Unicode characters using a variation of UTF-16LE encoding (not UTF-8) and the JDBC-ODBC bridge is unable to retrieve them.
(Note that this is not a problem with the Access ODBC driver per se because other tools like pyodbc for Python can retrieve the Unicode characters correctly. It is a compatibility issue between the JDBC-ODBC Bridge and the Access ODBC driver.)
A bug report was filed with Sun in November 2005 outlining the issue. That report was closed as "Won't Fix" in April 2013 with the comment
The bridge has been removed from Java SE 8 and is not supported
If you need to work with arbitrary Unicode characters in an Access database you should consider using UCanAccess. For more information, see
Manipulating an Access database from Java without ODBC

how the command 'use database-name' is using java code

I would like to make o program that would store data from excel files in databases. I have plenty of databases so in my program i have to choose in which one I will store the data.
I have made the code to be able to connect mysql with my program and to show the available databases. What I would like to do now is to say in which database i would store the data.
To be more specific I would like the user first of all to see tha available databases in his client and afterwards he would have the chance to say in which database the data would be stored.
Could anyone help me how I would do this?
The code to see all the available databases is the below:
Class.forName("com.mysql.jdbc.Driver");
Connection con = (Connection) DriverManager.getConnection(
"jdbc:mysql://localhost:3306/", "root", "root");
DatabaseMetaData meta = (DatabaseMetaData) con.getMetaData();
ResultSet res = meta.getCatalogs();
System.out.println("List of the databases: ");
while (res.next()){
System.out.println (" " +res.getString(1));
}
Thank you in advance!

I hope this SO link should help you . You can get all the data from the connection object

First do a create a simple connection
Connection con = (Connection) DriverManager.getConnection( "jdbc:mysql://localhost:3306/", "root", "root");
Look for an example here
see how he uses ResultSet Class to access the result.
Now,
retrieve information from information_schema.SCHEMATA so you can query like
SELECT schema_name FROM information_schema.SCHEMATA S;
to get all the schemas
Next after getting choice from user(maybe from console) you can set database according to uservalue using Connection#setCatalog()
If you want table information use this query
SELECT * FROM information_schema.TABLES T;
It would list all the tables in all schemas

Cannot read an SQLite Database made in another language

I have an SQLite3 databse I created in python. And by default it writes the database in Unicode.
Now I am trying to query the database in a Java Applet using SQLite JDBC. And I cannot find tables, rows etc because I think Java &/or JDBC queries in ANSI.
Does anyone know how I can query my SQLite3 DB with a unicode query in Java? Something like the following doesn't work (in Java trying to execute a Unicode SQL query):
If I access the database in python I can print out the tables no problem & make updates BUT if I try to do the same in Java, I get no results returned from my query. Is this an encoding problem or some thing else
This works import sqlite3
def blah():
conn = sqlite3.connect( "a.db" )
cur = conn.cursor()
res = cur.execute( "SELECT name FROM sqlite_master WHERE type='table'" ).fetchal()
print res
blah()
This returns no tables when it should return the same tables as above
Class.forName("org.sqlite.JDBC");
Connection conn = DriverManager.getConnection("jdbc:sqlite:a.db");
Statement stat = conn.createStatement();
conn.setAutoCommit(false);
ResultSet tables = stat.executeQuery("SELECT name FROM sqlite_master WHERE type='table'");
String b = "";
while (tables.next()) { b+= "table= " + tables.getString("name"); }

Jim, that's very odd, it should work. Have you tried to open the DB from the console?
you can open it by running sqlite3 a.db
something that intrigues me, is that you're trying to open the db from a java applet. Have you given it the necessary permissions and signed it, so the apple can actually write to disk?

Have your tired opening the sqlite3 a.db and typing PRAGMA encoding; ? It should say UTF8 by default.
I am able to create a UTF8 sqlite database on the command line and read the contents of that file using sqlite jdbc.
Are you sure you are connecting to the database that you created with python? IF the database does not exist then sqlite will create the database and it will not have any tables in it. This is what is probably happening to you.
Are you running the java and python in the same directory?