Develop Reference

Migrate ResultSet between two MySQL tables without specifying columns in Java

I have a MySQL database on server #1 and I need to migrate it to server #2. The database is very large and methods like mysqldump or MySQL Workbench Migration didn't work for me, so I wanted to write my own Java application, that would perform the following steps:
get all table names from the source database schema
for each table, it would select a batch of records (let's say 10.000 at a time) and insert them in the corresponding table in destination database. The schema is already prepared with correctly defined tables.
repeat until there are no rows left for current table.
repeat for each table.
The problem is, AFAIK, when using JDBC, it is needed to iterate through ResultSet and specify all column types and names, like this:
while (resultset.next()) {
System.out.println(resultset.getString("Col 1"));
System.out.println(resultset.getString("Col 2"));
System.out.println(resultset.getString("Col 3"));
System.out.println(resultset.getString("Col n"));
}
I want to do this for all tables and all their columns without specifying their names and types manually. I can't just type manually all these columns, as I have 150 tables and each of them has like 10-50 columns.
Is there any general way how to do this? Maybe taking advantage of the fact that both source and destination schemas are the same (same tables with same column names/types and same foreign keys)?

You can use ResultSetMetaData with ResultSet to get the columnNames, columnCount and few other details.
ResultSet resultSet = ps.executeQuery();
ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
while (resultSet.next()) {
for (int i = 1; i <= resultSetMetaData.getColumnCount(); i++) {
System.out.println(resultSet.getString(i));
}
}
if you need to fetch the type of Column also to make the decision you could use Switch conditions based on the type returned from resultSetMetaData.getColumnTypeName(i) for using something like
resultSet.getString()
resultSet.getInt()
resultSet.getBoolean()
resultSet.getDate()
Many more

On top of #koushlendra answer, you can also use these methods for additional informations(if needed)
resultSetMetaData.getTableName(int column) Returns the column’s table name.
resultSetMetaData.getSchemaName(int column) Returns the name of the schema of the column’s table.

Java DB Database, check for all table names

I've got an embedded Derby Database in my java application, and have multiple table's (that are created and deleted, so nothing is set in stone). I wanted to be able to return a list of names of all the tables currently in the database as I have to display the list in the application as well as get all the information from them.
Easiest way to do so? I don't need code just a method or methods. I'm a terrible google-fu user.
Currently my code works by grabbing a ResultSet from a specific table name entered, but it's only for testing purposes and I need to be able to display the full list of tables I have.
EDIT: My current workaround is actually different than posted. I simply have another table that holds all the table names created and updates when one is created/deleted. Obviously not the best approach but it works for me right now.

DatabaseMetaData metaData = connection.getMetaData();
ResultSet resultSet = metaData.getTables(null, "schenaName", "%" ,new String[] {"TABLE"} );
while (resultSet.next()) {
System.out.println(resultSet.getString(3));
}
Adding new answer:
Connection connection = getDBConnection();
DatabaseMetaData dbMetaData = connection.getMetaData();
//getting catalogs for mysql DB, if it is not working for your DB, try dbMetaData.getSchemas();
ResultSet catalogs = dbMetaData.getCatalogs();
while(catalogs.next()){
String catalogName = catalogs.getString(1);
//excluding table names from "mysql" schema from mysql DB.
if(!"mysql".equalsIgnoreCase(catalogName)){
ResultSet tables = dbMetaData.getTables(catalogName, null, null, null);
while(tables.next()){
System.out.println(catalogName + "::"+tables.getString(3));
}
}
}

Using metadata is the (somewhat) more portable solution. Note that you don't need the catalog stuff with Derby, as there are no catalogs. You can issue dmd.getTables(...) directly with null for the catalog. If all the tables you track are in a single schema, (and there aren't any other tables in that schema), getTables(null, "schemaName", null, null) should do the trick.
If need more fancy querying and you're not concerned about portability, you can check out
the dataBaseMetaData tool which gives you access to metadata as tables so that you can perform joins and other sophisticated queries on them.

Try this:
select tableName from sys.systables
You should get all the tables your system.

Connecting to Oracle with JDBC causes queries to return zero rows.

So I have been playing around with querying databases using the standard
Statement s = connection.createStatement();
ResultSet rs = s.executQuery(queryString);
ResultSetMetadata rsmd = rs.getMetaData();
while(rs.next)){
String code = "";
String value = "";
for(int i = 1; i <= rsmd.getColumnCount(); i++){
Object obj = rs.getObject(i);
if(i == 1){
code = obj.toString():
}
else{
label = obj.toString();
}
}
//Store code and labels in map
}
...go on to close statement and move on.
I am trying to select two columns from a table in each instance.
For the most part this works well. When working with MySql & Microsoft Sql databases I get a result set full of data in the table. However when I try to do this with an Oracle database I get an empty result set.
I have tested my query string in the SQL Developer application and it works fine, returns my data. But the result set doesnt contain anything. The resultSet metadata says that it has two columns though. Is there anything I need to do when interacting with an Oracle Database that is different from the other two? Thanks.

If your query works when you run it against the Oracle database, and you know the code works since you've run it against MySQL, then some other things to try are:
1.) Make sure your JDBC connection URL is correct. Are you sure you are connecting to the database that you intend to? (i.e. - the one that would return the rows you expect?)
2.) Take into account credentials. Make sure you are using the same credentials through JDBC that you are when connecting to Oracle directly.
3.) Make sure both connections are being made from the same machine and with the same environment. Oracle drivers rely on environment variables to find a file (I believe it is called tnsnames.ora, or something like that) that contains the alias & connection info. Getting different versions of that file could point you to different Oracle instances.
4.) Try manually specifying your schema name in the query. So instead of select * from my_table use select * from my_schema.my_table. Sometimes Oracle clients will configure their sessions to have default schemas set up in their preferences.
5.) If your are attempting to select data that you've inserted with your Oracle client, make sure you've committed the transaction in your Oracle client so that the data is visible to other sessions.
One last debugging tool to use is to try connecting via the Squirrel DB client. Squirrel is a 100% pure java SQL client that connects to any DB using JDBC. It would be a good test to make sure your JDBC Driver, Connection URL, etc. are all valid.

The database table has records but the JDBC client can't retrieve the records. Means the JDBC client doesn't have the select privileges. Please run the below query on command line:
grant all on emp to hr;

How to create multiple schema connections at runtime?

I have a a project name drop-down list, and based on the selected value a schema drop-down list is generated.
After the schema list is generated, there is a 'choose file' option where a script name is chosen which is to be run on multiple schemas.
Both the project list values and the schema list values are stored in the database. There is another table in the database where the schema credentials are stored.
I want to know that how create connections on those multiple schemas, and how to run a script in multiple schemas selected from that schema drop-down list.
I have executed the code in one schema using iBatis framework using scriptrunner method.
Table 1: ProjectName
PROJECT_PK,
PROJECT_CODE,
PROJECT_NAME
Table2: ComponentName
COMPONENT_PK,
COMPONENT_CODE,
COMPONENT_NAME,
PROJECT_PK
Table 3: SchemaName (This table contains the credentials of Other Schemas)
SCHEMA_PK,
SCHEMA_NAME,
PASSWORD,
ORACLE_SID,
HOST_NAME,
PORT_ID
Table 4: PROJECT_DETAIL
PROJECT_DETAIL_PK,
COMPONENT_PK,
SCHEMA_PK
Table5: COMPONENT_DETAILS
COMPONENT_DETAILS_PK,
PROJECT_PK,
SCHEMA_PK
I am attaching the scenario image.

I propose you create a 'super-schema' with the appropiate grants on every other schema, and create your JDBC connection for this 'super-schema'. This will make you need some tampering with the sql script - you will need to write it with some marker to easily substitute the schema, something like
ALTER TABLE ${SCHEMA_MARKER}.INVOICES ADD CLIENT_ADRRESS Varchar2(35);
And your java code would substitute it for the schema you are targeting. With some imagination you can extend this idea to execute bulk DDL scripts on all your schemas.
By the way, I understand by your question that you have many schemas with the same structure. I was forced once to work with such a legacy structure, and so I know there is a lot of momentum in corporate structures; nonetheless I must recommend you to redesign such system. Think for example in creating materialized views on your super-schema, replicating your schema structure but with a new field in every table primary key (this new field would fill the gap that made whomever did it separe the data in many schemas for starters).
This will surely make your query cache suffer a lot less, and will make easier the development of any new software that needs to work with that 'distributed' data.

In Oracle, a schema is basically a user. So to create a table FOO in the schema SX, you just login as user SX and execute create table FOO (...) (without specifying a schema). The user == schema convention in Oracle will make sure this works.
To login as user SX, get hold of your DataSource and use getConnection(String username, String password) instead of the default getConnection()
Alternative ways are to add placeholders in your script that you process to generate valid SQL first. See the answer by Jorge_B for an example.
Lastly, you can change the default schema using ALTER SESSION (see Default Schema in Oracle Connection URL). The problem with this approach is that you must restore the schema before closing the connection when using a web container with a DataSource - in this scenario, connections are never really closed, so the next piece of code asking for a connection will get one with an unexpected default schema -> hard to find errors.

Here is the DAO class to create multiple schema connection on the same database at run time.
We need to enter the schema names at run time through and call that in some servlet.
The schema details i.e username, password,host Id,port Id and SID are to be stored in the database tables though which the connections will be created.
The created connections are stored here in the list which can be used at a later point of time.
The credential DTO here is the object which maps to the database tables in the database and then
Connection conn=DBUtil.getConnection(constants.DB_DS_NAME);
Statement stmt=null;
stmt=conn.createStatement();
ResultSet rs= null;
Connection [] con=new Connection[schemaname.length];
int i,j;
String [] url=new String[schemaname.length];
String [] username=new String[schemaname.length];
String [] password=new String[schemaname.length];
List<CredentialDTO> creDTOlist=new ArrayList<CredentialDTO>();
String query1="insert into dba_project_master VALUES(9,'abc','abc','abc','abc',40)";
String query2="CREATE TABLE EMP(EMPNO NUMBER(4,0) NOT NULL ENABLE,ENAME VARCHAR2(10 BYTE),JOB VARCHAR2(9 BYTE), SAL NUMBER(7,2),DEPTNO NUMBER(2,0))";
try
{
for(i=0;i<schemaname.length;i++){
String query=" select * from dba_schema_details where schema_name="+DBUtil.enquoteString(schemaname[i]);
rs=stmt.executeQuery(query);
while(rs.next()){
CredentialDTO creDTO=new CredentialDTO();
creDTO.setSCHEMA_PK(rs.getString("SCHEMA_PK"));
creDTO.setSCHEMA_NAME(rs.getString("SCHEMA_NAME"));
creDTO.setPASSWORD(rs.getString("PASSWORD"));
creDTO.setORACLE_SID(rs.getString("ORACLE_SID"));
creDTO.setHOST_NAME(rs.getString("HOST_NAME"));
creDTO.setPORT_ID(rs.getString("PORT_ID"));
creDTOlist.add(creDTO);
}
}
System.out.println("creDTOlist size:"+creDTOlist.size());
//create URL for the schema name
int m=creDTOlist.size();
Iterator<CredentialDTO> LItr= creDTOlist.iterator();
String [] username1=new String[m];
String [] password1=new String[m];
i=0;
while(LItr.hasNext()){
System.out.println("iteration "+i);
CredentialDTO temp = LItr.next();
String URL="jdbc:oracle:thin:#"+temp.getHOST_NAME()+":"+temp.getPORT_ID()+":"+temp.getORACLE_SID();
System.out.println("URL:"+URL);
username1[i]=temp.getSCHEMA_NAME();
System.out.println("iteartion "+i+" username="+username1[i]);
password1[i]=temp.getPASSWORD();
System.out.println("iteartion "+i+" password="+password1[i]);
Class.forName("oracle.jdbc.OracleDriver");
con[i]=DriverManager.getConnection(URL, username1[i], password1[i]);
System.out.println("Connection Name:" +con[i]);
Statement st1=con[i].createStatement();
con[i].setAutoCommit(false);
st1.addBatch(query1);
st1.addBatch(query2);
int [] update=st1.executeBatch();
i++;
}
}
catch(Exception ex){
ex.printStackTrace();
}finally
{
if (conn != null) try{conn.close();} catch(SQLException ignore) {}
if (stmt!= null) try{stmt.close();} catch(SQLException ignore) {}
if (rs != null) try{rs.close();} catch(SQLException ignore) {}
}
return creDTOlist;
}
}

Connect to a SQL Server database

I am trying to connect to a SQL Server database, but I don't really know how to go about it using the info I was given. I was given the following:
Provider
DataSource
Persist Security Info
User ID
Initial Catalog
I have always connected via a web address or something, so I didn't really know how to go about using this. I am attempting to do this is Java using JDBC.

See here a wide list of examples, depending on which version you're using:
SQL Server 2000
SQL Server 2005
SQL Server 2008

To connect to MSSQL Server from a Java application, you need to use the JDBC API. The JDBC API provides classes and methods that connect to the database, load the appropriate driver, send SQL queries, retrieve results etc.
HOW TO CONNECT TO THE DATABASE: A ‘Connection’ object represents a connection with a database. To establish the connection, use the method ‘DriverManager.getConnection’. This method takes a string containing a URL which represents the database we are trying to connect to. Below is the sample code for establishing a connection:
private String DATABASE_URL = "jdbc:odbc:embedded_sql_app"; // establish connection to database
Connection connection = DriverManager.getConnection( DATABASE_URL,"sa","123" );
Detailed discussion about the Database URL and how to create it can be found in the resource provided at the end of this post.
QUERYING THE DATABASE: The JDBC API provides three interfaces for sending SQL statements to the database, and corresponding methods in the ‘Connection’ interface create instances of them. 1. Statement - created by the ‘Connection.createStatement’ methods. A ‘Statement’ object is used for sending SQL statements with no parameters.
2. PreparedStatement - created by the ‘Connection.prepareStatement methods’. A ‘PreparedStatement’ object is used for precompiled SQL statements. These can take one or more parameters as input arguments (IN parameters).
3. CallableStatement - created by the ‘Connection.prepareCall’ methods. ‘CallableStatement’ objects are used to execute SQL stored procedures from Java database applications.
RETRIEVING THE RESULT: A ‘ResultSet ‘is a Java object that contains the results of executing a SQL query. The data stored in a ‘ResultSet’ object is retrieved through a set of get methods that allows access to the various columns of the current row. The ‘ResultSet.next’ method is used to move to the next row of the ‘ResultSet’, making it the current row. The following code fragment executes a query that returns a collection of rows, with column ‘a’ as an ‘int’, column ‘b’ as a ‘String’, and column ‘c’ as a ‘float’:
java.sql.Statement stmt = con.createStatement();
ResultSet rs = stmt.executeQuery("SELECT a, b, c FROM Table1");
while (rs.next()) { // retrieve and print the values for the current row
int i = rs.getInt("a");
String s = rs.getString("b");
float f = rs.getFloat("c");
System.out.println("ROW = " + i + " " + s + " " + f);
}
This is just a brief introduction on how to interact with a database from Java. For more details on the items discussed above as well as information on passing parameters, executing stored procedures etc. please refer to the following resource: ( http://www.shahriarnk.com/Shahriar-N-K-Research-Embedding-SQL-in-C-Sharp-Java.html#Shahriar_N_Embedding_SQL_in_Java ) Here, you will also find information on how to interact with a database programmatically; i.e. without using SQL. Hope you find this useful.
Source:
http://www.shahriarnk.com/Shahriar-N-K-Research-Embedding-SQL-in-C-Sharp-Java.html