"Unreleased Resource: Database" forify issue - java

I have scanned my project using fortify tool, in the generated fortify report I got Unreleased Resource: Database issue in the following method.
public String handleInput( HttpServletRequest request, HtmlGenerator htmlGenObj ) {
HashMap old_child_list = new HashMap();
HashMap old_parent_list = new HashMap();
HashMap new_child_list = new HashMap();
HashMap new_parent_list = new HashMap();
logger.debug( "Start of handle Input" );
String dataSource = request.getParameter( "datasource" ) == null ? "" : request.getParameter( "datasource" );
String sortBy = request.getParameter( "sortby" ) == null ? "" : request.getParameter( "sortby" );
long sqlConsStartTime = System.currentTimeMillis();
//
// Get the sql query based on data source
//
String sqlQuery = this.getSql( dataSource, request );
// If any error then return
if( sqlQuery.startsWith( "ERROR" ) ) {
return sqlQuery;
}
if( sortBy.trim().length() > 0 ) {
logger.debug( "Sort by value = " + sortBy );
int orderByIndex = sqlQuery.indexOf( "order by" );
// If order by already exists then remove that value.
if( orderByIndex > 0 ) {
sqlQuery = sqlQuery.substring( 0, orderByIndex );
}
sqlQuery = sqlQuery + " order by " + sortBy;
}
plog.debug( getUniqueId( request ) + "SQL Construction Time = " + ( ( System.currentTimeMillis()- sqlConsStartTime )/1000.0 ) );
logger.info( "Final SQL = " + sqlQuery );
String results = "NONE";
Connection conn = null;
try {
Context initCtx = new InitialContext();
Context envCtx = ( Context ) initCtx.lookup( "java:comp/env" );
// Look up for the data source
DataSource ds = ( DataSource ) envCtx.lookup( "NavigatorCon" );
conn = ds.getConnection();
htmlGenObj.setSql( sqlQuery );
if( dataSource.equals( "compass" ) ) {
results = doCompassSearch( sqlQuery, conn, request, old_parent_list, old_child_list );
String matchType = request.getParameter( "coname_st" ) == null ? "" : request.getParameter( "coname_st" );
if( "fuzzy".equals( matchType ) ) {
long rollupSqlConsStartTime = System.currentTimeMillis();
String rollupSql = build_rollup_sql( conn, old_parent_list, old_child_list, new_parent_list, new_child_list ); //System.out.println( "Rollup SQL: " + rollupSql );
plog.debug(getUniqueId(request)+"Rollup SQL Construction Time = "+ ((System.currentTimeMillis()- rollupSqlConsStartTime)/1000.0) );
if( rollupSql.length() > 0 ) {
String rollupResult = doCompassSearch( rollupSql, conn, request, old_parent_list, old_child_list );
results = results + rollupResult;
// Build the new sql which returns both parent and child records.This is used in Excel download
sqlQuery = sqlQuery + " UNION " + rollupSql;
} //System.out.println( "Navigator Query is: \n" + sqlQuery + "\n" );
htmlGenObj.setSql( sqlQuery );
}
}
else if( dataSource.equals( "psar" ) ) {
results = getAMSearchResults( sqlQuery, conn, request, htmlGenObj );
}
else if( dataSource.equals( "knac" ) ) {
results = do_knac_search( sqlQuery, conn, request );
}
else if( dataSource.equals( "auth_user" ) ) {
results = do_auth_user_search( sqlQuery, conn, request );
}
else if( "docstore".equalsIgnoreCase( dataSource ) ) {
results = this.doDocStoreSearch( sqlQuery, conn, request );
}
}
catch( Exception sqle ) {
logger.fatal( "Error : " + sqle.toString(), sqle );
}
finally {
try {
conn.close();
}
catch( Exception e ) {
// giveup
}
}
old_child_list.clear();
old_parent_list.clear();
new_child_list.clear();
new_parent_list.clear();
logger.debug( "End of handle Input");
return results;
}
As per my knowledge this issue will come if the connection is not closed properly, but in this connection is closed inside finally block.
Can any one suggest me how to resolve it? xxxxxx

A NullPointerException maybe thrown, hence maybe change the second Exception into Throwable as NullPointerException is derived from the checked Exception class. Maybe the code analysis had a problem here.
And then, since Java 7 you can use try-with-resources to automatically close things:
try (Connection con = getDbConnection()) {
without finally. Also holds for the PreparedStatement and ResultSet.
As you named quality: it is deemed better style to use SQLException e, or something like SQLException | NullPointerException e.

You're not closing the ResultSet rs and the PreparedStatement ps. Maybe that's the issue.

In terms of fortify and security I would pay more attention o your SQL Injection...
String sortBy = request.getParameter...
[...]
sqlQuery = sqlQuery + " order by " + sortBy;
In general Fortify can generate a huge number of false positive results.

Related

Cannot insert data to Database Java without any error [closed]

Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I have tried to add stmt.executeUpdate(); to insert to the database, but it still not working!! After processing, no error is shown but no data is insert into the database.
try {
// do insert
String preparedSQL = "insert into abc (NUM, ID) VALUES (?,?)";
for(int i = 1; i < r.length; i++)
{
stmt = conn.prepareStatement(preparedSQL);
// SQL parameters
setString(stmt, i++, Num);
setString(stmt, i++, r[i]);
stmt.executeQuery();
}
stmt.executeUpdate();
stmt.close();
} catch (SQLException e) {
PrintWriter out = new PrintWriter(new OutputStreamWriter(res
.getOutputStream(), "Big5"), true);
res.setContentType("text/html; charset=utf-8");
res.setHeader("cache-control", "no-cache");
res.setHeader("Pragma", "no-cache");
res.setIntHeader("expiry", -1000);
out.println("..<!--" + RSCommonUtil.HTMLEncode(e.toString()) + "para in title not valid" +
RSCommonUtil.HTMLEncode(pDate.toString())
+ " " + RSCommonUtil.HTMLEncode(startPubDateAndEndPubDate[0].toString()) + " "
+ RSCommonUtil.HTMLEncode(startPubDateAndEndPubDate[1].toString()) + "-->");
debug.p("insert error" + e);
failed = true;
} finally {
try {
// finalize DB
if (stmt != null) {
stmt.close();
stmt = null;
}
} catch (SQLException sqle) {
}
}
executeUpdate is for create, drop, insert, update, delete etc., (i.e., updating your table), executeQuery is for select. It returns an instance of ResultSet.
See here for simple explanation: https://www.javatpoint.com/PreparedStatement-interface
Also, your setString has an incorrect number of parameters and it should be called as a method of the statement object, and the parameterIndex (first parameter of the setString method) should just be i, as you will otherwise increment three times for each iteration. I also changed the loop to start at 0, don't know why you would start at 1 as a normal array should start at 0.
So your main block would be something like this:
...
// do insert
String preparedSQL = "insert into abc (NUM, ID) VALUES (?,?)";
PreparedStatement stmt = conn.prepareStatement(preparedSQL);
for(int i = 0; i < r.length; i++)
{
// SQL parameters
stmt.setString(i, Num);
stmt.setString(i, r[i]);
stmt.executeUpdate();
}
stmt.close();
...
Inside the try block, try rewriting the code as follows:
String preparedSQL = "insert into abc (NUM, ID) VALUES (?,?)";
for(int i = 1; i < r.length; i++)
{
stmt = conn.prepareStatement(preparedSQL);
// SQL parameters
stmt.setString(1, Num);
stmt.setString(2, r[i]);
stmt.executeUpdate();
stmt.close();
}
The setString method takes only two parameters.
public void setString(int paramIndex, String value)
Here 'paramIndex' refers to the index of the parameter in your prepared statement and 'value' refers to which value you want to insert at that index.
The executeQuery() method of prepared statement interface is used for SELECT queries. But in your case, since the statement is an INSERT query, you will have to use executeUpdate() method. This method is used for CREATE, DROP, UPDATE, INSERT, etc. statements.
Refer this for more information regarding prepared statements and sample code.
The other Answers by yalda and by Nitish are both correct, with good advice. Ditto for some comments on the Question. I can add one more thought, plus a full working example app that writes and reads rows using the H2 Database Engine.
Try-With-Resources
Use the try-with-resources syntax to shorten your code, and guarantee your resources get closed appropriately.
Example app
package work.basil.example;
import org.h2.jdbcx.JdbcDataSource;
import java.sql.*;
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.util.UUID;
public class H2DateTimeExample
{
public static void main ( String[] args )
{
H2DateTimeExample app = new H2DateTimeExample ();
app.demo ();
}
private void demo ( )
{
JdbcDataSource dataSource = new JdbcDataSource ();
dataSource.setURL ( "jdbc:h2:mem:offsetdatetime_example_db;DB_CLOSE_DELAY=-1" ); // Set `DB_CLOSE_DELAY` to `-1` to keep in-memory database in existence after connection closes.
dataSource.setUser ( "scott" );
dataSource.setPassword ( "tiger" );
// Create table.
String sql = "CREATE TABLE person_ ( \n" +
" pkey_ UUID NOT NULL DEFAULT RANDOM_UUID() PRIMARY KEY , \n" +
" name_ VARCHAR NOT NULL , \n" +
"first_contacted_ TIMESTAMP WITH TIME ZONE NOT NULL " +
") ;";
// System.out.println ( sql );
try (
Connection conn = dataSource.getConnection () ;
Statement stmt = conn.createStatement () ;
)
{
stmt.execute ( sql );
} catch ( SQLException e )
{
e.printStackTrace ();
}
// Insert row.
sql = "INSERT INTO person_ ( name_ , first_contacted_ ) \n";
sql += "VALUES ( ? , ? ) \n";
sql += ";";
try (
Connection conn = dataSource.getConnection () ;
PreparedStatement pstmt = conn.prepareStatement ( sql , Statement.RETURN_GENERATED_KEYS ) ;
)
{
OffsetDateTime odt = OffsetDateTime.now ( ZoneOffset.UTC );
pstmt.setString ( 1 , "Jesse Johnson" );
pstmt.setObject ( 2 , odt );
pstmt.executeUpdate ();
ResultSet rs = pstmt.getGeneratedKeys ();
// System.out.println( "INFO - Reporting generated keys." );
// while ( rs.next() ) {
// UUID uuid = rs.getObject( 1 , UUID.class );
// System.out.println( "generated keys: " + uuid );
// }
} catch ( SQLException e )
{
e.printStackTrace ();
}
// Query table.
sql = "TABLE person_ ;";
try (
Connection conn = dataSource.getConnection () ;
PreparedStatement pstmt = conn.prepareStatement ( sql ) ;
)
{
try ( ResultSet rs = pstmt.executeQuery () ; )
{
while ( rs.next () )
{
UUID pkey = rs.getObject ( "pkey_" , UUID.class );
String name = rs.getString ( "name_" );
OffsetDateTime firstContacted = rs.getObject ( "first_contacted_" , OffsetDateTime.class );
System.out.println ( "pkey: " + pkey + " | name: " + name + " | firstContacted: " + firstContacted );
}
}
} catch ( SQLException e )
{
e.printStackTrace ();
}
System.out.println ( "Done." );
}
}
pkey: 0819123b-5c7f-4859-bd1b-f9dac82b855f | name: Jesse Johnson | firstContacted: 2019-08-03T18:16:30.367840Z
Done.

java - only reading and extracting one file to table

I have a program where I extract a text from PDF file and insert those text in a form of words into a table in MySQL.
I have two tables, a queue table and a word table.
The program retrieves a file from the queue table and build an index of words to the word table.
It works fine, but the problem is it only grabs one file but not the other files after that. Meaning, it only perform on the 1st file and then it closes. I am using a while loop and I disable autocommit so I have to commit it manually.
Did I make any mistake?
Code:
public void index1() throws Exception
{
try
{
// Connection con1=con.connect();
Connection con1 = con.connect();
Statement statement = con1.createStatement();
ResultSet rs = statement.executeQuery( "select * from filequeue where Status='Active' LIMIT 5" );
while ( rs.next() )
{
// get the filepath of the PDF document
String path1 = rs.getString( 2 );
int getNum = rs.getInt( 1 );
// while running the process, update status : Processing
//updateProcess_DB(getNum);
Statement test = con1.createStatement();
test.executeUpdate( "update filequeue SET STATUS ='Processing' where UniqueID=" + getNum );
try
{
// call the index function
/*Indexing process = new Indexing();
process.index(path1);*/
PDDocument document = PDDocument.load( new File( path1 ) );
if ( !document.isEncrypted() )
{
PDFTextStripper tStripper = new PDFTextStripper();
String pdfFileInText = tStripper.getText( document );
String lines[] = pdfFileInText.split( "\\r?\\n" );
for ( String line : lines )
{
String[] words = line.split( " " );
String sql = "insert IGNORE into test.indextable values (?,?);";
preparedStatement = con1.prepareStatement( sql );
int i = 0;
for ( String word : words )
{
// check if one or more special characters at end of string then remove OR
// check special characters in beginning of the string then remove
// insert every word directly to table db
word = word.replaceAll( "([\\W]+$)|(^[\\W]+)", "" );
preparedStatement.setString( 1, path1 );
preparedStatement.setString( 2, word );
preparedStatement.addBatch();
i++;
if ( i % 1000 == 0 )
{
preparedStatement.executeBatch();
System.out.print( "Add Thousand" );
}
}
if ( i > 0 )
{
preparedStatement.executeBatch();
System.out.print( "Add Remaining" );
}
}
}
// con1.commit();
// con1.close();
preparedStatement.close();
System.out.println( "Successfully commited changes to the database!" );
// After completing the process, update status: Complete
Statement pre = con1.createStatement();
pre.executeUpdate( "update filequeue SET STATUS ='Complete' where UniqueID=" + getNum );
con1.commit();
preparedStatement.close();
System.out.println( "Successfully commited changes to the database!" );
con1.close();
// updateComplete_DB(getNum);
}
catch ( Exception e )
{
try
{
System.err.println( e );
// updateError_DB(getNum);
// update status = Error in queue table if any error
Statement statement1 = con1.createStatement();
statement1.execute( "update filequeue SET STATUS ='Error' where UniqueID=" + getNum );
// add rollback function
rollbackEntries();
}
catch ( Exception e1 )
{
System.out.println( "Could not rollback updates :" + e1.getMessage() );
}
}
}
}
catch ( Exception e )
{
}
}

Possible to open two persistent connections to different remote MySQL servers using JAVA sql?

So I'm trying to run a code which creates a table in a remote MySQL server, by referencing tables located in a different MYSQL server. The server that I am trying to create a table in has space limitations, and the tables that are being referenced are very large, so they have to be kept on a different remote server.
I'm trying to find a way to set up persistent connections to both databases at the same time (using JDBC libraries) so I don't have to keep buffering small numbers of lines of data... I'd like to be able to just reference the data directly.
E.g.
Database A contains the data I am referencing, and Database B is where I am creating the new tables.
say the table I am referencing in Database A is 1,000,000 lines. Instead of, say, opening a connection to Database A, buffering 10,000 lines, closing the connection, opening a connection to Database B, writing the that database, deleting my buffer, and repeating...
I'd like to just have a persistent connection to database A, so every write to Database B can reference the data in Database A.
Is this possible? I've tried several ways (mostly by creating new connection objects that only refresh if the connection breaks), and I can't seem to get this idea working.
Has anyone done something similar to this using JDBC? If so, It would be much appreciated if you could point me in the right direction, or tell me how you managed to get it working.
You could create the data in Database A, and copy it to Database B via replication.
Alternatively, it sounds like you're implementing a queue of some kind. I once built a data-copying program in Java, which used a built-in implementation of the Queue interface. I had a thread that read the data from Database A and filled the queue, and a thread that read from the queue and wrote to Database B. I can try and dig up the classes I used if that's any use?
EDIT:
Here's the code, somewhat tweaked for publishing. I haven't include the config classes, but it should give you the idea of how to use the queue class;
package test;
import java.io.File;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.ResultSetMetaData;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
/**
* This class implements a JDBC bridge between databases, allowing data to be
* copied from one place to another.
* <p>This implementation is threaded, as it uses a {#link BlockingQueue} to pass
* data between a producer and a consumer.
*/
public class DBBridge {
public static void main( String[] args ) {
Adaptor fromAdaptor = null;
Adaptor toAdaptor = null;
BridgeConfig config = null;
try {
/* BridgeConfig is essentially a wrapper around the Simple XML serialisation library.
* http://simple.sourceforge.net/
*/
config = BridgeConfig.loadConfig( new File( "db-bridge.xml" ) );
}
catch ( Exception e ) {
System.err.println( "Failed to read or parse db-bridge.xml: " + e.getLocalizedMessage() );
System.exit( 1 );
}
BlockingQueue<Object> b = new ArrayBlockingQueue<Object>( config.getQueueSize() );
try {
HashMap<String, DatabaseConfig> dbs = config.getDbs();
System.err.println( "Configured DBs" );
final String sourceName = config.getSource();
final String destinationName = config.getDestination();
if ( !dbs.containsKey( sourceName ) ) {
System.err.println( sourceName + " is not a configured database connection" );
System.exit( 1 );
}
if ( !dbs.containsKey( destinationName ) ) {
System.err.println( destinationName + " is not a configured database connection" );
System.exit( 1 );
}
DatabaseConfig sourceConfig = dbs.get( sourceName );
DatabaseConfig destinationConfig = dbs.get( destinationName );
try {
/*
* Both adaptors must be created before attempting a connection,
* as otherwise I've seen DriverManager pick the wrong driver!
*/
fromAdaptor = AdaptorFactory.buildAdaptor( sourceConfig, sourceConfig );
toAdaptor = AdaptorFactory.buildAdaptor( destinationConfig, destinationConfig );
System.err.println( "Connecting to " + sourceName );
fromAdaptor.connect();
System.err.println( "Connecting to " + destinationName );
toAdaptor.connect();
/* We'll send our updates to the destination explicitly */
toAdaptor.getConn().setAutoCommit( false );
}
catch ( SQLException e ) {
System.err.println();
System.err.println( "Failed to create and configure adaptors" );
e.printStackTrace();
System.exit( 1 );
}
catch ( ClassNotFoundException e ) {
System.err.println( "Failed to load JDBC driver due to error: " + e.getLocalizedMessage() );
System.exit( 1 );
}
DataProducer producer = null;
DataConsumer consumer = null;
try {
producer = new DataProducer( config, fromAdaptor, b );
consumer = new DataConsumer( config, toAdaptor, b );
}
catch ( SQLException e ) {
System.err.println();
System.err.println( "Failed to create and configure data producer or consumer" );
e.printStackTrace();
System.exit( 1 );
}
consumer.start();
producer.start();
}
catch ( Exception e ) {
e.printStackTrace();
}
}
public static class DataProducer extends DataLogger {
private BridgeConfig config;
private Adaptor adaptor;
private BlockingQueue<Object> queue;
public DataProducer(BridgeConfig c, Adaptor a, BlockingQueue<Object> bq) {
super( "Producer" );
this.config = c;
this.adaptor = a;
this.queue = bq;
}
#Override
public void run() {
/* The tables to copy are listed in BridgeConfig */
for ( Table table : this.config.getManifest() ) {
PreparedStatement stmt = null;
ResultSet rs = null;
try {
String sql = table.buildSourceSelect();
this.log( "executing: " + sql );
stmt = this.adaptor.getConn().prepareStatement( sql );
stmt.execute();
rs = stmt.getResultSet();
ResultSetMetaData meta = rs.getMetaData();
/* Notify consumer that a new table is to be processed */
this.queue.put( table );
this.queue.put( meta );
final int columnCount = meta.getColumnCount();
while ( rs.next() ) {
ArrayList<Object> a = new ArrayList<Object>( columnCount );
for ( int i = 0; i < columnCount; i++ ) {
a.add( rs.getObject( i + 1 ) );
}
this.queue.put( a );
}
}
catch ( InterruptedException ex ) {
ex.printStackTrace();
}
catch ( SQLException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
/* refresh the connection */
/* Can't remember why I can this line - maybe the other
* end kept closing the connection. */
this.adaptor.reconnect();
}
catch ( SQLException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
try {
/* Use an object of a specific type to "poison" the queue
* and instruct the consumer to terminate. */
this.log( "putting finished object into queue" );
this.queue.put( new QueueFinished() );
this.adaptor.close();
}
catch ( InterruptedException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
catch ( SQLException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
/* Superclass for producer and consumer */
public static abstract class DataLogger extends Thread {
private String prefix;
public DataLogger(String p) {
this.prefix = p;
}
protected void log( String s ) {
System.err.printf( "%d %s: %s%n", System.currentTimeMillis(), this.prefix, s );
}
protected void log() {
System.err.println();
}
}
public static class DataConsumer extends DataLogger {
private BridgeConfig config;
private Adaptor adaptor;
private BlockingQueue<Object> queue;
private int currentRowNumber = 0;
private int currentBatchSize = 0;
private long tableStartTs = -1;
public DataConsumer(BridgeConfig c, Adaptor a, BlockingQueue<Object> bq) throws SQLException {
super( "Consumer" );
this.config = c;
this.adaptor = a;
this.queue = bq;
/* We'll send our updates to the destination explicitly */
this.adaptor.getConn().setAutoCommit( false );
}
public void printThroughput() {
double duration = ( System.currentTimeMillis() - this.tableStartTs ) / 1000.0;
long rowsPerSec = Math.round( this.currentRowNumber / duration );
this.log( String.format( "%d rows processed, %d rows/s", this.currentRowNumber, rowsPerSec ) );
}
#Override
public void run() {
this.log( "running" );
Table currentTable = null;
ResultSetMetaData meta = null;
int columnCount = -1;
PreparedStatement stmt = null;
while ( true ) {
try {
Object o = this.queue.take();
if ( o instanceof Table ) {
currentTable = (Table) o;
this.log( "processing" + currentTable );
if ( this.currentBatchSize > 0 ) {
/* Commit outstanding rows from previous table */
this.adaptor.getConn().commit();
this.printThroughput();
this.currentBatchSize = 0;
}
/* refresh the connection */
this.adaptor.reconnect();
this.adaptor.getConn().setAutoCommit( false );
/*
* Arguably, there's no need to flush the commit buffer
* after every table, but I like it because it feels
* tidy.
*/
this.currentBatchSize = 0;
this.currentRowNumber = 0;
if ( currentTable.isTruncate() ) {
this.log( "truncating " + currentTable );
stmt = this.adaptor.getConn().prepareStatement( "TRUNCATE TABLE " + currentTable );
stmt.execute();
}
this.tableStartTs = System.currentTimeMillis();
}
else if ( o instanceof ResultSetMetaData ) {
this.log( "received metadata for " + currentTable );
meta = (ResultSetMetaData) o;
columnCount = meta.getColumnCount();
String sql = currentTable.buildDestinationInsert( columnCount );
stmt = this.adaptor.getConn().prepareStatement( sql );
}
else if ( o instanceof ArrayList ) {
ArrayList<?> a = (ArrayList<?>) o;
/* One counter for ArrayList access, one for JDBC access */
for ( int i = 0, j = 1; i < columnCount; i++, j++ ) {
try {
stmt.setObject( j, a.get( i ), meta.getColumnType( j ) );
}
catch ( SQLException e ) {
/* Sometimes data in a shonky remote system
* is rejected by a more sane destination
* system. Translate this data into
* something that will fit. */
if ( e.getMessage().contains( "Only dates between" ) ) {
if ( meta.isNullable( j ) == ResultSetMetaData.columnNullable ) {
this.log( "Casting bad data to null: " + a.get( i ) );
stmt.setObject( j, null, meta.getColumnType( j ) );
}
else {
this.log( "Casting bad data to 0000-01-01: " + a.get( i ) );
stmt.setObject( j, new java.sql.Date( -64376208000L ), meta.getColumnType( j ) );
}
}
else {
throw e;
}
}
}
stmt.execute();
this.currentBatchSize++;
this.currentRowNumber++;
if ( this.currentBatchSize == this.config.getBatchSize() ) {
/*
* We've reached our non-committed limit. Send the
* requests to the destination server.
*/
this.adaptor.getConn().commit();
this.printThroughput();
this.currentBatchSize = 0;
}
}
else if ( o instanceof QueueFinished ) {
if ( this.currentBatchSize > 0 ) {
/* Commit outstanding rows from previous table */
this.adaptor.getConn().commit();
this.printThroughput();
this.log();
this.log( "completed" );
}
/* Exit while loop */
break;
}
else {
throw new RuntimeException( "Unexpected obeject in queue: " + o.getClass() );
}
}
catch ( InterruptedException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
catch ( SQLException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
try {
this.adaptor.close();
}
catch ( SQLException e ) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public static final class QueueFinished {
/*
* This only exists as a completely type-safe value in "instanceof"
* expressions
*/
}
}
In a program I wrote a while back for work I had two simultaneous connections. Without giving the code away you'll want
public void initialize() {
String dbUrl, dbUrl2, dbClass, dbClass2, user, user2, password, password2;
Connection con, con2;
Statement stmt, stmt2;
ResultSet rs, rs2;
try {
Class.forName(dbClass);
con = DriverManager.getConnection(dbUrl,user,password);
con2 = DriverManager.getConnection(dbUrl2,user2,password2);
stmt = con.createStatement();
} catch(ClassNotFoundException e) {
e.printStackTrace();
}
catch(SQLException e) {
e.printStackTrace();
}
}
Then once you have your two connections running,
rs = stmt.executeQuery("query");
rs2 = stmt2.executeQuery("second query");
I don't know how to specifically address your problem, but this code may be a little taxing on your system (assuming you don't have a high-end personal/company machine) and could take a while. That should give you enough to get started at least, I'd post more if I could, sadly it's slightly too complicated to mock up a version. Good luck though!
I think you'll be best served by having two separate connections, a read connection and a write connection, and pass the data through your Java application using a small buffer of sorts.
Another more complicated but possibly more elegant solution is to use a FEDERATED table. It makes a table on a remote server appear to be local. The queries are passed to the remote server and results are sent back. You have to be careful about indexes or it'll be really slow, but it may work for what you want to do.
http://dev.mysql.com/doc/refman/5.5/en/federated-description.html
I have done this before, and I recommend that you do what I did, which is get the data you need from DB A and write it out into 1 or more files as SQL 'set' statements. When I did this I had to split into about 10 files, because of restrictions on the size of the file being loaded in DB B

MySQL commit and rollback failed

I have a problem.
try {
jdbcConnect( ); //get mysql connect
conn.setAutoCommit( false );
pstmt = conn.prepareStatement (
"INSERT INTO member ( member_name, member_introduce ) VALUES ( ?, ? )", Statement.RETURN_GENERATED_KEYS );
pstmt.setString( 1, "something" );
pstmt.setString( 2, "something" );
pstmt.executeUpdate( );
rs = pstmt.getGeneratedKeys( );
rs.next( );
String no = Integer.toString( rs.getInt( 1 );
pstmt = conn.prepareStatement ( "UPDATE account SET account_name = ? WHERE account_no = ?" );
pstmt.setString( 1, "something");
pstmt.setString( 2, no );
pstmt.executeUpdate( );
conn.commit( );
conn.setAutoCommit( true );
} catch ( SQLException t ) {
try {
if (conn != null) {
conn.rollback();
conn.setAutoCommit( true );
}
} catch ( SQLException e ) {
}
}//close conn and prepareStatement
I was expect working commit.
But If an error occurs update statement, insert statement is running.
What is wrong?
I think that you are using MyISAM table instead of InnoDB. MyISAM does not support transaction at all.

Table access - SQL (Java)

Suppose I have a table MYtable as given below:
ID A B C
1 100 APPLE CAKE
2 200 BANANA PIE
I want to be able to save all these table records into a list (of some kind) and iterate through each record.
Query would be :
select * from Mytable where ID in (1,2,3)
So,the list should have 2 records:
record1 should contain 1,100,APPLE,CAKE
and record2 should contain 2,200,BANANA,PIE
I also want to be able to iterate through each record
So for record1 - I want to getColumnA i.e 100, getColumnB i.e APPLE, and so on
List<YourObject> listOfObjects = new ArrayList<YourObject>();
while(rs.next()){
int id = rs.getInt(1);
int A = rs.getInt(2);
String B= rs.getString(3);
String C = rs.getString(4);
YourObject ob = new YourObject (id, A, B, C);
listOfObjects.add(ob); //now you have the list of objects .. iterate trough it
}
First you will need JDBC to connect to the database.
You can use Statement or PreparedStatement to fetch data from database and then you can get that data stored in ResultSet.
Now you can iterate through the results in ResultSet.
For that you need to implement JDBC.
After creating JDBC connection. Execute query and the fetch records from the resultset. From a record you can fetch particular columns as well.
see following example how to work with jdbc basic
http://www.jdbc-tutorial.com/
import java.sql.* ;
class JDBCQuery
{
public static void main( String args[] )
{
try
{
// Load the database driver
Class.forName( "sun.jdbc.odbc.JdbcOdbcDriver" ) ;
// Get a connection to the database
Connection conn = DriverManager.getConnection( "jdbc:odbc:Database" ) ;
// Print all warnings
for( SQLWarning warn = conn.getWarnings(); warn != null; warn = warn.getNextWarning() )
{
System.out.println( "SQL Warning:" ) ;
System.out.println( "State : " + warn.getSQLState() ) ;
System.out.println( "Message: " + warn.getMessage() ) ;
System.out.println( "Error : " + warn.getErrorCode() ) ;
}
// Get a statement from the connection
Statement stmt = conn.createStatement() ;
// Execute the query
ResultSet rs = stmt.executeQuery( "select * from Mytable where ID in (1,2,3)" ) ;
// Loop through the result set
while( rs.next() )
{
System.out.println( rs.getString('ID') ) ;
System.out.println( rs.getString('A') ) ;
System.out.println( rs.getString('B') ) ;
System.out.println( rs.getString('C') ) ;
}
// Close the result set, statement and the connection
rs.close() ;
stmt.close() ;
conn.close() ;
}
catch( SQLException se )
{
System.out.println( "SQL Exception:" ) ;
// Loop through the SQL Exceptions
while( se != null )
{
System.out.println( "State : " + se.getSQLState() ) ;
System.out.println( "Message: " + se.getMessage() ) ;
System.out.println( "Error : " + se.getErrorCode() ) ;
se = se.getNextException() ;
}
}
catch( Exception e )
{
System.out.println( e ) ;
}
}
}

Categories

Resources