Develop Reference

Migrate ResultSet between two MySQL tables without specifying columns in Java

I have a MySQL database on server #1 and I need to migrate it to server #2. The database is very large and methods like mysqldump or MySQL Workbench Migration didn't work for me, so I wanted to write my own Java application, that would perform the following steps:
get all table names from the source database schema
for each table, it would select a batch of records (let's say 10.000 at a time) and insert them in the corresponding table in destination database. The schema is already prepared with correctly defined tables.
repeat until there are no rows left for current table.
repeat for each table.
The problem is, AFAIK, when using JDBC, it is needed to iterate through ResultSet and specify all column types and names, like this:
while (resultset.next()) {
System.out.println(resultset.getString("Col 1"));
System.out.println(resultset.getString("Col 2"));
System.out.println(resultset.getString("Col 3"));
System.out.println(resultset.getString("Col n"));
}
I want to do this for all tables and all their columns without specifying their names and types manually. I can't just type manually all these columns, as I have 150 tables and each of them has like 10-50 columns.
Is there any general way how to do this? Maybe taking advantage of the fact that both source and destination schemas are the same (same tables with same column names/types and same foreign keys)?

You can use ResultSetMetaData with ResultSet to get the columnNames, columnCount and few other details.
ResultSet resultSet = ps.executeQuery();
ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
while (resultSet.next()) {
for (int i = 1; i <= resultSetMetaData.getColumnCount(); i++) {
System.out.println(resultSet.getString(i));
}
}
if you need to fetch the type of Column also to make the decision you could use Switch conditions based on the type returned from resultSetMetaData.getColumnTypeName(i) for using something like
resultSet.getString()
resultSet.getInt()
resultSet.getBoolean()
resultSet.getDate()
Many more

On top of #koushlendra answer, you can also use these methods for additional informations(if needed)
resultSetMetaData.getTableName(int column) Returns the column’s table name.
resultSetMetaData.getSchemaName(int column) Returns the name of the schema of the column’s table.

Need to sync two tables from different databases in java

Usually to sync a table with in the network I can write a simple sql query and run it to sync the tables, but what if the source database is different(could be db2, mssql, mysql, oracle - from where I can get the data in to destination table).
How should I go about writing the code in java to achieve this?. I know I can create dblinks in java to fetch data from external network database but dblinks works only with database of same type. I am not able to figure out the implementation in my head. Please point me in the right direction so I can take baby steps and learn the same as I am new to java

You can do this with plain JDBC. Code example, not tested:
void migrate() throws SQLException {
Connection connA = DriverManager.getConnection("jdbc:oracle:thin:#myhost:1521:orcl");
Connection connB = DriverManager.getConnection("jdbc:mysql://192.168.1.33/test");
PreparedStatement stmA = connA.prepareStatement("select * from product where 1=1");
PreparedStatement stmB = connB.prepareStatement("insert into prod values(?,?,?,?)");
ResultSet rs = stmA.executeQuery();
while (rs.next()) {
for (int i = 0; i < rs.getMetaData().getColumnCount(); i++) {
stmB.setObject(i + 1, rs.getObject(i + 1));
}
stmB.executeUpdate();
}
connA.close();
connB.close();
}

Data Migration If you want to migrate data from one database to another database of different types,
Suppose you have A and B two different database, you have to export data from A database in XML format, table name is XML parent node (create structure as per your database).
Then parse that XML's and create objects of each table present in A, after that use any XML parser which will parse your XML, using hibernate save() api put each object in another database manage if tables has relationships in them then insert master table data first then insert child items data.
By this approach you no need to be dependent on destination (B) database each time, with the help of hibernate you can easily change your database.

How to create multiple schema connections at runtime?

I have a a project name drop-down list, and based on the selected value a schema drop-down list is generated.
After the schema list is generated, there is a 'choose file' option where a script name is chosen which is to be run on multiple schemas.
Both the project list values and the schema list values are stored in the database. There is another table in the database where the schema credentials are stored.
I want to know that how create connections on those multiple schemas, and how to run a script in multiple schemas selected from that schema drop-down list.
I have executed the code in one schema using iBatis framework using scriptrunner method.
Table 1: ProjectName
PROJECT_PK,
PROJECT_CODE,
PROJECT_NAME
Table2: ComponentName
COMPONENT_PK,
COMPONENT_CODE,
COMPONENT_NAME,
PROJECT_PK
Table 3: SchemaName (This table contains the credentials of Other Schemas)
SCHEMA_PK,
SCHEMA_NAME,
PASSWORD,
ORACLE_SID,
HOST_NAME,
PORT_ID
Table 4: PROJECT_DETAIL
PROJECT_DETAIL_PK,
COMPONENT_PK,
SCHEMA_PK
Table5: COMPONENT_DETAILS
COMPONENT_DETAILS_PK,
PROJECT_PK,
SCHEMA_PK
I am attaching the scenario image.

I propose you create a 'super-schema' with the appropiate grants on every other schema, and create your JDBC connection for this 'super-schema'. This will make you need some tampering with the sql script - you will need to write it with some marker to easily substitute the schema, something like
ALTER TABLE ${SCHEMA_MARKER}.INVOICES ADD CLIENT_ADRRESS Varchar2(35);
And your java code would substitute it for the schema you are targeting. With some imagination you can extend this idea to execute bulk DDL scripts on all your schemas.
By the way, I understand by your question that you have many schemas with the same structure. I was forced once to work with such a legacy structure, and so I know there is a lot of momentum in corporate structures; nonetheless I must recommend you to redesign such system. Think for example in creating materialized views on your super-schema, replicating your schema structure but with a new field in every table primary key (this new field would fill the gap that made whomever did it separe the data in many schemas for starters).
This will surely make your query cache suffer a lot less, and will make easier the development of any new software that needs to work with that 'distributed' data.

In Oracle, a schema is basically a user. So to create a table FOO in the schema SX, you just login as user SX and execute create table FOO (...) (without specifying a schema). The user == schema convention in Oracle will make sure this works.
To login as user SX, get hold of your DataSource and use getConnection(String username, String password) instead of the default getConnection()
Alternative ways are to add placeholders in your script that you process to generate valid SQL first. See the answer by Jorge_B for an example.
Lastly, you can change the default schema using ALTER SESSION (see Default Schema in Oracle Connection URL). The problem with this approach is that you must restore the schema before closing the connection when using a web container with a DataSource - in this scenario, connections are never really closed, so the next piece of code asking for a connection will get one with an unexpected default schema -> hard to find errors.

Here is the DAO class to create multiple schema connection on the same database at run time.
We need to enter the schema names at run time through and call that in some servlet.
The schema details i.e username, password,host Id,port Id and SID are to be stored in the database tables though which the connections will be created.
The created connections are stored here in the list which can be used at a later point of time.
The credential DTO here is the object which maps to the database tables in the database and then
Connection conn=DBUtil.getConnection(constants.DB_DS_NAME);
Statement stmt=null;
stmt=conn.createStatement();
ResultSet rs= null;
Connection [] con=new Connection[schemaname.length];
int i,j;
String [] url=new String[schemaname.length];
String [] username=new String[schemaname.length];
String [] password=new String[schemaname.length];
List<CredentialDTO> creDTOlist=new ArrayList<CredentialDTO>();
String query1="insert into dba_project_master VALUES(9,'abc','abc','abc','abc',40)";
String query2="CREATE TABLE EMP(EMPNO NUMBER(4,0) NOT NULL ENABLE,ENAME VARCHAR2(10 BYTE),JOB VARCHAR2(9 BYTE), SAL NUMBER(7,2),DEPTNO NUMBER(2,0))";
try
{
for(i=0;i<schemaname.length;i++){
String query=" select * from dba_schema_details where schema_name="+DBUtil.enquoteString(schemaname[i]);
rs=stmt.executeQuery(query);
while(rs.next()){
CredentialDTO creDTO=new CredentialDTO();
creDTO.setSCHEMA_PK(rs.getString("SCHEMA_PK"));
creDTO.setSCHEMA_NAME(rs.getString("SCHEMA_NAME"));
creDTO.setPASSWORD(rs.getString("PASSWORD"));
creDTO.setORACLE_SID(rs.getString("ORACLE_SID"));
creDTO.setHOST_NAME(rs.getString("HOST_NAME"));
creDTO.setPORT_ID(rs.getString("PORT_ID"));
creDTOlist.add(creDTO);
}
}
System.out.println("creDTOlist size:"+creDTOlist.size());
//create URL for the schema name
int m=creDTOlist.size();
Iterator<CredentialDTO> LItr= creDTOlist.iterator();
String [] username1=new String[m];
String [] password1=new String[m];
i=0;
while(LItr.hasNext()){
System.out.println("iteration "+i);
CredentialDTO temp = LItr.next();
String URL="jdbc:oracle:thin:#"+temp.getHOST_NAME()+":"+temp.getPORT_ID()+":"+temp.getORACLE_SID();
System.out.println("URL:"+URL);
username1[i]=temp.getSCHEMA_NAME();
System.out.println("iteartion "+i+" username="+username1[i]);
password1[i]=temp.getPASSWORD();
System.out.println("iteartion "+i+" password="+password1[i]);
Class.forName("oracle.jdbc.OracleDriver");
con[i]=DriverManager.getConnection(URL, username1[i], password1[i]);
System.out.println("Connection Name:" +con[i]);
Statement st1=con[i].createStatement();
con[i].setAutoCommit(false);
st1.addBatch(query1);
st1.addBatch(query2);
int [] update=st1.executeBatch();
i++;
}
}
catch(Exception ex){
ex.printStackTrace();
}finally
{
if (conn != null) try{conn.close();} catch(SQLException ignore) {}
if (stmt!= null) try{stmt.close();} catch(SQLException ignore) {}
if (rs != null) try{rs.close();} catch(SQLException ignore) {}
}
return creDTOlist;
}
}

Error while trying to test if a specific table exists in a MySQL database using JDBC

I want to check wether a table named basestation exists in a MySQL table from a Java program using JDBC. My code is:
conn = DriverManager.getConnection(url, username, password);
DatabaseMetaData dbm = conn.getMetaData();
tables = dbm.getTables(null, null, "basestations", null);
if (tables.next())
{
//Table exists in database
//...
}
else
{
//Table does not exist
//...
}
but although table basestations exists in my database it seems that my code never enters the first clause and always decides that table does not exist, causing naturally a table already exists in database SQL error when trying to create the table from the beginning.

Not the best way to do it below.. But you could do the following:
Use JDBC to send the following statement:
select count(*) from information_schema.tables where information_schema.table_schema='your_schema' and information_schema.table_name='basestations';
Parse the jdbc result set.
That should return either 1 or 0. If the resulting count is 1, then the table exists, otherwise, it does not.

I suspect your getTables() not returning anything here. As per getTables documentation Only table descriptions matching the catalog, schema, table name and type criteria are returned. In your case catalog and schema are null, which may not be correct (my guess), populate them with proper values.

Can't update Oracle IOT table with jbdc updateRow method

I have Oracle 10gR2 database with IOT table within:
create table countries (
id number primary key,
name varchar2(30) not null enable
) organization index;
I try to update table values with this Java (version 1.6) code:
Statement stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY,
ResultSet.CONCUR_UPDATABLE);
ResultSet src = stmt.executeQuery("select id, name from countries");
src.next();
src.updateString("name", "__test__");
src.updateRow();
But updateRow throws SQLException (ORA-01410: invalid ROWID). If I try to update a heap (ordinary) table - all works.
I have use this code with different versions of oracle drivers (from here http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/htdocs/jdbc_10201.html)
After some research I have detected that IOT and HEAP table have different format of rowids:
IOT example *BAJzKgwCwRb+
HEAP example AAAbgVAAJAAMyr8AAA
But I still don't know how to solve this problem. Are you have any ideas?

Can you get the results of an extended SQL trace of your query to see what JDBC's doing under the covers? I suspect it's attempting to do
UPDATE COUNTRIES SET NAME = '__TEST__' WHERE ROWID = :rowid_fetched
and ROWID means something totally different in Oracle IOT's; it's not the immutable address of a row, but a guess as to the path to the row.
My recommendation as to how to do this is to propagate a system-generated timestamp field onto all of your tables, and use that for concurrency control rather than declaring an updatable recordset -- which will take and hold locks for every record in the recordset.
Then your application would fetch the rowset as normal, but issue statements like:
UPDATE COUNTRIES SET NAME = '__TEST__' WHERE MOD_TS = :mod_ts_fetched
to give stateless optimistic locking.

Looks like your table does not really need to be IOT. I would suggest you recreate it as a normal table and add index on both ID and name. Same performance, same logic, no ROWID problem.