Any WorkAround For Java ResultSet Limitation - java

I am doing a database migration work. I have to copy a database in MSSQL to MySql database. It was possible to come up with a small java utility to copy table stucture from MSSQL to MySql Database. Now i have to copy all data from MSSQL to MySql. I tried using resultset in java to obtain all data from a table but then it could only fetch a small part of data. Is there any alternate solution to get all data from table to resultset or to some other similar structure which i could possibly use, to insert the same data into mysql Db. There are more than 25,00,000 records for a table.

A JDBC result set should in principle allow you to iterate the entirity of a large query result.
However going via Java may not be the most efficient approach. Bulk export to a file and bulk import may be the way to go. It appears that MS has a bcp utility that may do the export.

The best way to achieve a database migration like you describe is to use and ETL Tool - there's a good overview of ETL here:
http://en.wikipedia.org/wiki/Extract,_transform,_load
There's no reason why you wouldn't be able to do this with JDBC and so if you are set on rolling your own please elaborate on 'could only fetch a small part of data':
what is the query you are running?
are you getting an exception?
which JDBC driver are you using to connect to MS-SQL?

Related

Querying SQLite in Java - lightweight?

I need to execute one query in my program on a SQLite database. If I use the sqlite jdbc it is 3,5 Mb large. Is there no other simple solution for executing just one select query without needing a dependency that is so huge.
You may try SQLJet if you don't need to have full JDBC compliance. JAR is about 760Kb. And yes, technically, the library doesn't provide a SQL engine, it's an API which allows to select and fetch records from a sqlite database. Depending on your needs current and future it may or may not be suitable for you. I successfully use it inone small project to load records from a ~2.6GB sqlite DB and process them as needed.

How do I implement the UnityJDBC in my Java project?

i have a project am working on, its all about querying a data from multiple databases from different vendors (i mean querying databases like mysql, hsqldb, microsoft sql, oracle, etc at the same time using one query statement).
Though i have achieved this by loading each driver of the database connector sequentially and execute the query sequentially across the databases. But the project architecture is such that when i sent a query statement, it shouldgo simultaneously to each database and retrieve the item ifavailable in all databases involved.
I came across this unityjdbc software, a mediation software but dont know how to implement it in my java source file so that to achieve my aim. I have read the unityjdbc user manual but is not clear and straight-forward.
Please can anyone advise how toimplement this unityjdbc driver in my java application and use it to successful query multiple databases.
Suggestions for any other way to simultaneously query their multiple databases with a single statement would also be welcome.
UnityJDBC allows you to query multiple databases in one SQL query. You cannot do this using separate threads as you would then be responsible for merging the data from the multiple databases yourself in your Java program.
The setup steps are easy:
Use the SourceBuilder application to specify the JDBC connection information to your databases.
Test a sample query that accesses multiple databases. Standard SQL is supported. To reference tables in different databases use databaseName.tableName in your FROM clause.
For example:
SELECT *
FROM Database1.Table1 T1 INNER JOIN Database2.Table2 T2 ON T1.id = T2.id
The SourceBuilder application will provide an XML configuration file as output often called sources.xml. To use this in your own Java program or any software that supports JDBC the connection URL is: jdbc:unity://sources.xml You may specify an absolute or relative path to the sources.xml file.
There is documentation on their site at http://www.unityjdbc.com/support/ or contact them for free support.
Another way to get started quickly is to use the MultiSource SQL Plugin that comes with the open source query software SQuirreL SQL. The plugin will allow you to query any number of databases using SQL in SQuirreL and will generate the XML configuration files for you to use in other programs. The plugin is open source and free. The plugin also supports querying and joining NoSQL databases like MongoDB with relational databases such as MySQL and Postgres.
You don't need UnityJDBC for what you want to do, if you've already managed to load all the db-specific JDBC drivers.
Instead, you should look at doing each query in a separate thread. That way, you don't need to wait for one database to return its results before querying the next one.

How to copy resultset from MSSQL to MySQL? Maybe openquery?

I need to copy often the lasts rows of a single table in MSSQL to MySQL table.
I've created a java script and all works but I think that it's possible improve the performance.
I've replicate the table schema and import data with MySQL ODBC 5.1 Driver connector adding it in MSSQL and using Openquery.
It's possible use it directly on java?
What is this the best way to handle this task?
Thanks in advance.
Luca
MySQL Workbench has a utility to do this.
In the program, connect to your MySQL Database, and then choose Database -> Migrate.
~Christian

SQL Server To MY SQL

I got a new project from my teacher to convert database to another. How can I convert a MS SQL database into MySQL using Java?
You will want to keep in mind that there are two logical steps regardless of the Programming environment. Firstly, you will want to map the schema of the database to an equivalent schema in the target database. This means mapping data types and constraints. Sometimes there are cases where it is simply not possible. Secondly, mapping the data from one database to the other. Timestamps and date formats must be equivalent for example. Hope that helps you to get started.
How can I convert a MS SQL database into MYSQL using JAVA
Dump it ( from MSSQL) and execute the result in Mysql.
With a question like that it's the best answer you can give
You could also try to use the hibernate tools to create a mapping to the MS SQL tables. Then use hibernate again with the created mapping to create the tables in MySQL.
Look at the various metadata classes in JDBC:
java.sql.DatabaseMetaData
java.sql.ResultSetMetaData
These will allow you to get the structure from the MSSQL DB. You need to do a little experimentation as, if memory serves, what gets returned in the metadata can vary from database to database.
Once you've worked out how to get the metadata from the MSSQL DB you need to convert that into SQL commands to recreate the structure in the MySQL DB.
Then with the structure in place you can start copying the data between the two by creating insert queries that run against the MySQL DB from querying the contents of the MSSQL DB. You'll need to do this in the correct order so that the referential integrity isn't violated which again you'll be able to determine from the metadata.
It's an interesting academic exercise but in the real-world you'd use an Extract-Transform-Load (ETL) tool.

Bulk Row Transfer between Oracle Databases with a Select Filter

Basically, I'm trying to selectively copy a table from one database to another. I have two different [Oracle] databases (e.g., running on different hosts) with the same schema. I'm interested in a efficient way to load Table A in DB1 with the result of running a select on Table A in DB2. I'm using JDBC, if that's relevant.
Use a database link, and use create table as select.
create database link other_db connect to remote_user identified by remote_passwd using remote_tnsname;
create table a as select * from a#other_db;
If the databases are from the same vendor they usually provide a native way to make a view
of a table in another database. in which case, a "select into" query will do it no problem
Oracle, for example, has the database link which works pretty well.
Outside of that you are going to have to make a connection to each database and read in
from one connection and write out to the other.
There are tools like Oracle's ODI that can do the legwork, but they all use the same
read in, write out model
You may not even need to move that data. Maybe you can just select across the database link.

Categories

Resources