I want to add records I database table with some interval of time using jdbc.
For ex., I want to add 100000 records in 10 sec interval so it'll insert 10000/sec.
My code of MySQL as below :
String url1 = "jdbc:mysql://localhost:3306/xyz";
String user = "root";
String password = "root";
conn1 = DriverManager.getConnection(url1, user, password);
if (conn1 != null) {
System.out.println("Connected to the database xyz");
for(int i=0;i<=n;i++){ // where n is no. of record that I want to insert
// Here is my insert logic
}
}
#yogesh-jalodara In my comments I meant something like that
final long loopDuration = 1;//second
final long totalSize = 100000;
final long timeInterval = 10;
final AtomicLong batchNumber = new AtomicLong((long)Math.ceil((double) timeInterval / loopDuration));
Timer timer = new Timer();
timer.schedule(new TimerTask() {
#Override
public void run() {
//insert logic
if (batchNumber.decrementAndGet() == 0) {
timer.cancel();
timer.purge();
}
}
}, 0, loopDuration * 1000);
Using batch queries, You effectively reduce database round-trip, You save a lot of time spent on network latency, which results in better performance of Java application.
public class MySQLJdbcExample {
public static void main(String args[]) throws SQLException {
String url="jdbc:mysql://localhost:3306/test";
Connection conn = DriverManager.getConnection(url, "root", "root");
String query = "insert into employee values (?,?,NULL)";
PreparedStatement pStatement = conn.prepareStatement(query);
int batchSize = 100;
long startTime = System.currentTimeMillis();
for (int count = 0; count < 1000; count++) {
pStatement.setString(1, Integer.toString(count));
pStatement.setString(2, "Employee" + count);
pStatement.addBatch();
if (count % batchSize == 0) {
pStatement.executeBatch();
}
}
pStatement.executeBatch() ; //for remaining batch queries if total record is odd no.
// conn.commit();
pStatement.close();
conn.close();
long endTime = System.currentTimeMillis();
long elapsedTime = (endTime - startTime)/1000; //in seconds
System.out.println("Total time required to execute 1000 queries using PreparedStatement with JDBC batch insert is :" + elapsedTime);
}
}
Read more: http://javarevisited.blogspot.com/2013/01/jdbc-batch-insert-and-update-example-java-prepared-statement.html#ixzz47Bgqyx64
Related
This is using MySQL 5.7.28 as a Docker image, in case that's relevant. The client is JDBC 5.1.48, running on GraalVM 11.20 on Mac.
When I run the following query in the information_schema schema, I get 3,105 rows:
PreparedStatement ps = conn.prepareStatement("select * from COLUMNS");
ResultSet rs = ps.executeQuery();
int numRows = 0;
while (rs.next()) {
numRows++;
}
But if I do this repeatedly in 100 parallel threads, each thread using its own connection, I occasionally get a slightly different result set. About 0.5% of the time, the number of rows is slightly lower, like 3,098. It's inconsistent, but easily reproducible.
Notice that this is not select count(*) -- this is select * and then iterate over all the rows.
This only happens against tables in the information_schema catalog -- for instance the TABLES table.
When doing the same thing against tables in the mysql catalog, or any other catalog actually, this does not happen, so my best guess is that this is due to the fact that the information_schema schema is special, like maybe it's cached in the JDBC driver, or the server, or something like that.
Can anyone shed any light on this?
Here is the complete code, which should run against any MySQL once you change the URL, user name and password:
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.util.List;
import java.util.ArrayList;
import java.util.concurrent.atomic.AtomicInteger;
public class ManyQueryThreads {
private static final List<Thread> threads = new ArrayList<>();
private static final List<QueryThread> queryThreads = new ArrayList<>();
private static final int NUM_THREADS = 100;
private static final int NUM_ITERS = 20;
/**
* This gets set by the first thread to complete the query. All subsequent queries are expected
* to return the same number of rows.
*/
private static final AtomicInteger expectedNumRows = new AtomicInteger(-1);
private static class QueryThread implements Runnable {
public boolean done = false;
#Override
public void run() {
Connection conn;
try {
conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/information_schema?useSSL=false",
"root", "Password1");
}
catch(Exception ex) {
throw new RuntimeException(ex);
}
for (int i = 0; i < NUM_ITERS; i++) {
try {
PreparedStatement ps = conn.prepareStatement("select * from COLUMNS");
ResultSet rs = ps.executeQuery();
int numRows = 0;
while (rs.next()) {
numRows++;
}
expectedNumRows.compareAndExchange(-1, numRows);
if (numRows != expectedNumRows.get()) {
System.out.println("Incorrect number of rows, expected " +
expectedNumRows.get() + ", but got " + numRows + ", thread " + Thread.currentThread().getId() +
", iteration " + i);
}
rs.close();
ps.close();
}
catch (Exception ex) {
throw new RuntimeException(ex);
}
}
try {
conn.close();
}
catch(Exception ex) {
throw new RuntimeException(ex);
}
done = true;
}
}
public static void main(String[] args) throws Exception {
System.out.println("Creating " + NUM_THREADS + " threads...");
long startTime = System.currentTimeMillis();
for (int i = 0; i < NUM_THREADS; i++) {
QueryThread qt = new QueryThread();
queryThreads.add(qt);
Thread t = new Thread(qt);
threads.add(t);
}
System.out.println("Threads created, starting them for " + NUM_ITERS + " iterations...");
for (int i = 0; i < NUM_THREADS; i++) {
threads.get(i).start();
}
// Now wait for everyone to be done
System.out.println("Threads are running, waiting for them to end...");
deathWatch:
while (true) {
for (int i = 0; i < NUM_THREADS; i++) {
QueryThread qt = queryThreads.get(i);
if ( ! qt.done) {
Thread.sleep(100);
continue deathWatch;
}
}
break deathWatch;
}
System.out.println("All done, time elapsed: " + (System.currentTimeMillis() - startTime) + "ms.");
}
}
Typical output:
Creating 100 threads...
Threads created, starting them for 20 iterations...
Threads are running, waiting for them to end...
Incorrect number of rows, expected 3105, but got 3099, thread 49, iteration 4
Incorrect number of rows, expected 3105, but got 3086, thread 17, iteration 6
Incorrect number of rows, expected 3105, but got 3101, thread 64, iteration 8
Incorrect number of rows, expected 3105, but got 2853, thread 57, iteration 12
etc...
package Simple;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Time;
import java.sql.Timestamp;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
import java.text.*;
public class CurrentProg {
//connecting to the database
private static final String DB_DRIVER = "com.mysql.jdbc.Driver";
private static final String DB_CONNECTION ="jdbc:mysql://localhost:3306/db?autoReconnect=true";
private static final String DB_USER = "root";
private static final String DB_PASSWORD = "root";
static Connection dbConnection = null;
static Statement statement = null;
static int total=1;
//Searching between startdate and enddate
public static java.util.LinkedList searchBetweenDates(java.util.Date startDate, java.util.Date endDate) {
java.util.Date begin = new Date(startDate.getTime());
java.util.LinkedList list = new java.util.LinkedList();
list.add(new Date(begin.getTime()));
java.util.Date end = new Date(endDate.getTime());
endDate.setTime(endDate.getTime() + 24*3600*1000);
Calendar cal = Calendar.getInstance();
cal.setTime(begin);
dbConnection = getDBConnection();
while(begin.compareTo(endDate)<0){
begin = new Date(begin.getTime() + 86400000);
list.add(new Date(begin.getTime()));
Timestamp timestamp = new Timestamp(new Date().getTime());
//For a single day calculation: 24hours*60mins=1440 /2 (2 mins time difference as per the requirement) = 720
for (int j = 0; j < 720; j++) {
cal.add(Calendar.MINUTE, 2);
timestamp = new Timestamp(cal.getTime().getTime());
String S = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(timestamp);
String[] parts = S.split(" ");
String date=parts[0];
String time=parts[1];
cal.getTime().toString();
// To create data loop into a List
List<String> records = new ArrayList<String>();
StringBuffer record = new StringBuffer();
for (int i = 1; i <= total; i++) {
records = new ArrayList<String>(total);
int a2 = 220 + j % 31; // 230 - 244 by 1
String wString = Integer.toString(a2);
String a = String.valueOf(a2);
double b2 = 0.00 + j % 3.7 ; // 1.3 - 3.9 by 0.1
String aString = Double.toString(b2);
String b = String.valueOf(b2);
b = b.substring(0, Math.min(b.length(), 5));
record.delete(0, record.length());
record.append(a + "," + b + ",'"+ date + "', '"+ time + "'");
record.append("\t\t");
record.append("\n");
records.add(record.toString());
//Insert Query
String insertTableSQL = "INSERT INTO cmd1"
+ "(a, b, date, time) " + "VALUES"
+ "("+record.toString()+")";
System.out.println("insertTableSQL - " + insertTableSQL); // Statement.executeUpdate(insertTableSQL);
try {
statement = dbConnection.createStatement();
statement.executeUpdate(insertTableSQL);
System.out.println("Record is inserted into Db table!");
} catch (SQLException e) {
System.out.println(e.getMessage());
}
try {
// dbConnection = getDBConnection();
statement = dbConnection.createStatement();
statement.executeUpdate(insertTableSQL);
System.out.println("Record is inserted into Db table!");
} catch (SQLException e) {
System.out.println(e.getMessage());
}
finally {
// httpPost.releaseConnection()
try{
if(statement!=null)
statement.close();
}
finally{
}
try{
if(dbConnection!=null)
dbConnection.close();
}
finally{
}
}
}
}
}
return list;
}
#SuppressWarnings("unused")
public static void main(String[] args) throws Exception {
//To enter startDate and enddate
// EntityManagerFactory.getCache().evictAll;
SimpleDateFormat startDate=new java.text.SimpleDateFormat("yyyy-MM-dd");
SimpleDateFormat endDate=new java.text.SimpleDateFormat("yyyy-MM-dd");
java.util.LinkedList hitList = searchBetweenDates(
startDate.parse("2016-01-01"),
endDate.parse("2016-03-01"));
String[] combo = new String[hitList.size()];
for(int i=0; i<hitList.size(); i++)
combo[i] = new java.text.SimpleDateFormat("yyyy-MM-dd").format(((java.util.Date)hitList.get(i)));
}
private static void insertRecordIntodb() {
//
}
private static Connection getDBConnection() {
Connection dbConnection = null;
try {
Class.forName(DB_DRIVER);
} catch (ClassNotFoundException e) {
System.out.println(e.getMessage());
}
try {
dbConnection = DriverManager.getConnection(DB_CONNECTION, DB_USER, DB_PASSWORD);
return dbConnection;
} catch (SQLException e) {
System.out.println(e.getMessage());
}
return dbConnection;
}
}
Server connection failure during transaction. Due to underlying exception: 'java.net.SocketException: java.net.SocketException: No buffer space available (maximum connections reached?): connect'.
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: java.net.SocketException: No buffer space available (maximum connections reached?): connect
STACKTRACE:
java.net.SocketException: java.net.SocketException: No buffer space available (maximum connections reached?): connect
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:156)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:276)
at com.mysql.jdbc.Connection.createNewIO(Connection.java:2717)
at com.mysql.jdbc.Connection.<init>(Connection.java:1509)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at Simple.CurrentProg.getDBConnection(CurrentProg.java:186)
at Simple.CurrentProg.searchBetweenDates(CurrentProg.java:126)
at Simple.CurrentProg.main(CurrentProg.java:164)
** END NESTED EXCEPTION **
Attempted reconnect 3 times. Giving up.
Exception in thread "main" java.lang.NullPointerException
at Simple.CurrentProg.searchBetweenDates(CurrentProg.java:137)
at Simple.CurrentProg.main(CurrentProg.java:164)
Here I am trying to connect java program with database but when i am trying to insert large data say for 1 month so its only fetching 16000 records not more than that i want the data should be inserted as per the given date range what should i do to get that . In stacktrace its showing an exception as no buffer space available maximum connection reached. Thanks In advance
Here is the modified code. I removed a lot of code which is not necessary:
public static LinkedList<Date> searchBetweenDates(Date startDate, Date endDate) throws SQLException {
Date begin = new Date(startDate.getTime());
LinkedList<Date> list = new LinkedList<Date>();
list.add(new Date(begin.getTime()));
endDate.setTime(endDate.getTime() + 24 * 3600 * 1000);
Calendar cal = Calendar.getInstance();
cal.setTime(begin);
dbConnection = getDBConnection();
PreparedStatement ps = dbConnection.prepareStatement("INSERT INTO cmd1(aaaa, bbbb, datee, timee) VALUES(?, ?, ?, ?)");
while (begin.compareTo(endDate) < 0) {
begin = new Date(begin.getTime() + 86400000);
list.add(new Date(begin.getTime()));
Timestamp timestamp = new Timestamp(new Date().getTime());
// For a single day calculation: 24hours*60mins=1440 /2 (2 mins time
// difference as per the requirement) = 720
for (int j = 0; j < 720; j++) {
cal.add(Calendar.MINUTE, 2);
timestamp = new Timestamp(cal.getTime().getTime());
String S = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(timestamp);
String[] parts = S.split(" ");
String date = parts[0];
String time = parts[1];
cal.getTime().toString();
// To create data loop into a List
for (int i = 1; i <= total; i++) {
int a2 = 220 + j % 31; // 230 - 244 by 1
String a = String.valueOf(a2);
double b2 = 0.00 + j % 3.7; // 1.3 - 3.9 by 0.1
String b = String.valueOf(b2);
b = b.substring(0, Math.min(b.length(), 5));
ps.setString(1, a);
ps.setString(2, b);
ps.setString(3, date);
ps.setString(4, time);
ps.execute();
}
}
}
if (ps != null)
ps.close();
if (dbConnection != null)
dbConnection.close();
return list;
}
What I changed:
I removed your try/catch's because I wanted to make the code short so that I can edit it here easily. But you should handle exception correctly. And don't swallow exceptions ever. I mean the following is no go:
} catch (SQLException e) {
System.out.println(e.getMessage());
}
It is very bad Idea because you'll not see the real cause of the problem in this case; at least do e.printStackTrace() eventhough it is not recommended in a real project.
I exchanged the Statement object for PreparedStatement because it is much more efficient.
Removed package names and put import statements instead because it is not necessary to do so unless you have different classes from different packages with the same name.
I changed column names because my DB does not want to accept them. Column names like a are very bad. Choose instead descriptive names so that you might understand what it is for a couple of months later. Don't use column names like date because they are reserved words for some databases systems I know.
Don't create database resources like Connection in a loop unless it is absolutely needed! Otherwise, your program will go out of resources because these resources are very expensive! That is exactly what you are experiencing right now.
Hope it helps, otherwise, drop me a comment.
I'm trying to code a system for a large IRC channel (Twitch Channel)
One of the things I'm trying to do is log every user and give them points for being in the chat. For all intents and purposes the chat is just a large IRC channel. I'm retrieving the users in a big list from the twitch API, I put all the usernames in a large Array and running the following timer with a while loop:
timer = new Timer(900000, new ActionListener() {
public void actionPerformed(ActionEvent evt) {
updating = true;
try {
Arraynumber = 0;
TwitchBot.getDate();
arrayused = false;
System.out.println("trying to save users if theres enough stuff");
while(Arraynumber < TwitchBot.words.length){
TwitchBot.CheckUpdateUserSQL(TwitchBot.words[Arraynumber]);
Arraynumber++;
System.out.println("updating database");
}
} catch (ClassNotFoundException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
As you can see it's a simple timer that picks the name from a String[] and runs every name through the script individually.
The updateuser looks like such:
public static void CheckUpdateUserSQL(String sqluser) throws ClassNotFoundException{
selectSQL(sqluser);
if (id == "thisuserdoesntexistforsure"){
InsertSQL(sqluser);
}
else{
int progress = CurrentTime - lastlogin;
int totalprogress = progress + totaltime;
if(progress < 60 && progress > 0){
c15 = null;
Statement stmt = null;
if(isonline == 1) {
coins = progress / 4;
}
else{
coins = progress / 5;
}
int coinsincrease = (int) Math.ceil(coins);
int coinstotal = coinsamount + coinsincrease;
Class.forName("org.sqlite.JDBC");
try {
c15 = DriverManager.getConnection("jdbc:sqlite:users.db");
c15.setAutoCommit(false);
stmt = c15.createStatement();
String sql = "UPDATE USERS set TOTALTIME = " + totalprogress + " where NAME='" + sqluser + "';";
stmt.executeUpdate(sql);
c15.commit();
String sql2 = "UPDATE USERS set LASTLOGIN = " + CurrentTime + " where NAME='" + sqluser + "';";
stmt.executeUpdate(sql2);
c15.commit();
String sql3 = "UPDATE USERS set TOTALCOIN = " + coinstotal + " where NAME='" + sqluser + "';";
stmt.executeUpdate(sql3);
c15.commit();
String sql4 = "UPDATE USERS set ISONLINE = 0 where NAME='" + sqluser + "';";
stmt.executeUpdate(sql4);
c15.commit();
stmt.close();
c15.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
else {
Connection c = null;
Statement stmt = null;
try {
c = DriverManager.getConnection("jdbc:sqlite:users.db");
c.setAutoCommit(false);
stmt = c.createStatement();
String sql2 = "UPDATE USERS set LASTLOGIN = " + CurrentTime + " where NAME='" + sqluser + "';";
stmt.executeUpdate(sql2);
c.commit();
stmt.close();
c15.close();
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
This code checks whether an user exists. (using the select method, which is as concise as I can get it, it only search for an username and returns the id, which will be 'thisuderdoesntexistforsure' if nothing returns)
If the user exists it will run the code to calculate their online time and the increase in online time and points since the last time they visited. Then updates the code. If they were not online or if the time somehow returns a negative value (or one that's too high) it will instead only update the timestamp and skip the rest of the updates. This makes sure that users who leave for a day don't just get 1.400 minutes of online time when they log on five minutes the next day.
Anyway. My question is; How can I trim it down? I'm running into an issue where it will take 6 minutes to update the entire userlist. having 2000 users online is not rare and it would take 2,000 loops through that while loop to update them all. The program is updating more often then not. I've tried cutting down the code to be as condensed as possible, but I have no idea where to start to speed things up.
Sorry if I'm coming over as moronic, I'm relatively new to SQL and this is my biggest project yet in JAVA.
You can use batching to perform your updates, but in your given code a simpler optimization would be to update the values with one update call (instead of 4). Also, you could use PreparedStatement and try-with-resources close. Something like,
public static void CheckUpdateUserSQL(String sqluser) throws ClassNotFoundException {
selectSQL(sqluser);
if (id.equals("thisuserdoesntexistforsure")) {
InsertSQL(sqluser);
} else {
String sql = "UPDATE USERS set TOTALTIME = ?, LASTLOGIN = ?, "
+ "TOTALCOIN = ?, ISONLINE = 0 where NAME = ?";
String sql2 = "UPDATE USERS set LASTLOGIN = ? where NAME=?";
int progress = CurrentTime - lastlogin;
int totalprogress = progress + totaltime;
if (progress < 60 && progress > 0) {
if (isonline == 1) {
coins = progress / 4;
} else {
coins = progress / 5;
}
int coinsincrease = (int) Math.ceil(coins);
int coinstotal = coinsamount + coinsincrease;
Class.forName("org.sqlite.JDBC");
try (Connection conn = DriverManager.getConnection("jdbc:sqlite:users.db");
PreparedStatement ps = conn.prepareStatement(sql)) {
ps.setInt(1, totalprogress);
ps.setInt(2, CurrentTime);
ps.setInt(3, coinstotal);
ps.setString(4, sqluser);
ps.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
} else {
Class.forName("org.sqlite.JDBC");
try (Connection conn = DriverManager.getConnection("jdbc:sqlite:users.db");
PreparedStatement ps = conn.prepareStatement(sql2)) {
ps.setInt(1, CurrentTime);
ps.setString(2, sqluser);
ps.executeUpdate();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
What you need is batchupdate. Some good tutorial can be found on the internet.
An example can be the following:
stm = db.prepareStatement("INSERT INTO ITEM (ID, TYPE, TITEL, UITGELEEND) VALUES (?, ?, ?, ?)");
db.setAutoCommit(false);
for (int n = 0; n < ItemLijst.getItems().size(); n++) {
Item huidigItem = ItemLijst.getItemObvIdx(n);
stm.setString(1, huidigItem.getID().toString());
stm.setString(2, huidigItem.getType().toString());
stm.setString(3, huidigItem.getTitel());
stm.setString(4,String.valueOf(huidigItem.isUitgeleend()));
stm.addBatch();
}
String SQL = "UPDATE Employees SET age = 35 " +
"WHERE id = 100";
// Add above SQL statement in the batch.
stm.addBatch(SQL);
stm.executeBatch();
db.commit();
Also try avoiding joining strings, instead use '?', otherwise it will be subjected to sql injection attacks.
http://tutorials.jenkov.com/jdbc/batchupdate.html
http://www.tutorialspoint.com/jdbc/jdbc-batch-processing.htm
May I know How to insert a stop watch for this piece of code from Poll() method...I have to make start count such that before the database starts and amount of time it took for polling.
public void poll() throws Exception {
st = conn.createStatement();
for (int i=0; i<10; i++)
{
Timestamp start;
rs = st.executeQuery( "select * from msg_new_to_bde" );
Timestamp end;
//speed = end - start;
Collection<KpiMessage> pojoCol = new ArrayList<KpiMessage>();
while (rs.next()) {
KpiMessage filedClass = convertRecordsetToPojo(rs);
pojoCol.add(filedClass);
}
for (KpiMessage pojoClass : pojoCol) {
System.out.println("=== Iteratioin Nr. " + i + "====");
System.out.print(pojoClass.getSequence());
System.out.print(pojoClass.getTableName());
System.out.print(pojoClass.getEntryTime());
System.out.print(pojoClass.getProcessingTime());
System.out.println(pojoClass.getStatus());
// System.out.println(pojoClass.getprocessDuration());
}
System.out.print(pojoCol.size());
}
}
You have to use currentTimeMillis() function:
Before launch polling:
long start = System.currentTimeMillis();
After Polling execution:
long stop= System.currentTimeMillis();
Execution time is stop - start in milliseconds.
I believe System.currentTimeMillis is what you looking for.
long startTime = System.currentTimeMillis();
//
long endTime = System.currentTimeMillis();
System.out.println((endTime - startTime) + "ms");
java.util.Date date = new java.util.Date();
Timestamp start = new Timestamp(date.getTime());
//process
java.util.Date date1 = new java.util.Date();
Timestamp end = new Timestamp(date1.getTime());
long start = System.currentTimeMillis();
rs = st.executeQuery( "select * from msg_new_to_bde" );
long stop= System.currentTimeMillis();
System.out.println("execution time: " +stop-start + " ms");
long start = System.nanoTime();
timeThisMethod();
long end = System.nanoTime();
long howLongDidItTake = end - start;
This method is more precise then System.currentTimeMillis()
Citation from java API :
Returns the current value of the most precise available system timer,
in nanoseconds.
Thank you in advance for your help. I am developing a java based tool that is preforming some database work. I have a very simple problem. For some reason the time reported to complete the task is incorrect.
public static void makeDatabaseThreaded() throws IOException, InterruptedException {
final long startTime = System.nanoTime();
ArrayList<String> tablesMade = new ArrayList<>();
File rootDirectory = root;
String[] files = rootDirectory.list();
double percentDone = 0;
double numOfTablesMade = 0;
double numberOfTables = 62.0;
DatabaseBuilderThread lastThread = null;
for (int i = 0; i <= files.length - 1; i++) {
if (!files[i].contains(".csv")) {
continue;
}
File file = new File(rootDirectory + File.separator + files[i]);
String tableName = getTableNameFromFile(file);
if (!tablesMade.contains(tableName)) {
tablesMade.add(tableName);
DatabaseBuilderThread thread = new DatabaseBuilderThread(i, file);
lastThread = thread;
thread.start();
threadsRunning++;
numOfTablesMade++;
percentDone = (int) (100.0 * (numOfTablesMade) / (numberOfTables));
while (threadsRunning > 10) {
Thread.sleep(1000);
}
System.out.println(percentDone + "% done. Making Table For File: " + file.getName());
}
}
//Make Sure all threads are done
lastThread.join();
final long endTime = System.nanoTime();
final long duration = endTime - startTime;
Time time = new Time(duration);
System.out.println("Done Making The Database. It took " + time.toString());
}
The program reports that it worked about twice as long at it truly did for the cases that I ran.
Thanks
System.nanoTime() returns time values in nanoseconds. Time() takes a value in milliseconds as a parameter. This would throw your time value off by a factor of 10^-6.
Time takes milliseconds as a constructor parameter, where as nanoTime() gives you nanoseconds precision, could that be the problem?
discussion here: System.currentTimeMillis vs System.nanoTime