I have developed a MIS Report in my swing application. Means a table
in which the first column contains date column followed by 7 columns of invoice status like pending for payment, paid, pending for park, post etc..
Just trying to show the EXCEL PIVOT LIKE table report in swing, I have successfully implemented it.
but the problem is: day by day the row count is increasing and I have developed a query like
select count(invoice_No)
from MyTable
where ClaimStatus='columnHeader'
AND date='row1stColumn'
and it will loop through all rows and column to get desired count and will display it into jtable.
But as I said the row count gets increased day by day and SQL Server table data is also increasing a lot.
So my count query gets lot of time to populate that table
Is there any way to make above query faster?
If anyone wants to see my code I will provide it.
I have implemented my application but because of huge data its taking time to show MIS Report
Please see the picture i have attached which is the output of my code.
And The code is,
1) for distinct dates in 1st column
q="Select distinct(Inward_Date_Short) from Inward_Master";
PreparedStatement ps=con.prepareStatement(q);
ResultSet rs=ps.executeQuery();
while(rs.next())
{
inwardDateList.add(rs.getString(1));
}
2) static columns of JTable
headers.add("Pending For Digitization");
headers.add("Pending For Claim Creation");
headers.add("Resolution - Pending For Claim Creation");
headers.add("Pending For Approval");
headers.add("Pending For Parking");
headers.add("Pending For Posting");
headers.add("Objection");
headers.add("Pending For Payment");
headers.add("Paid");
headers.add("Rejected");
headers.add("Outward");
3) now this is most important code which i want to make more faster
for(int i=0;i<inwardDateList.size();i++)
{
Vector varsha=new Vector();
varsha.add(inwardDateList.get(i).toString());
for(int c=1;c<headers.size();c++)
{
try(Connection con=dbConnection.dbConnector();)
{
String q="";
q=headers.get(c).toString();
PreparedStatement ps=con.prepareStatement("Select COUNT_BIG(Inward_No) from Inward_Master where Inward_Date_Short='"+inwardDateList.get(i).toString()+"' AND Claim_Status='"+q+"'");
//PreparedStatement ps=con.prepareStatement("Select count(Inward_No) from(Select Inward_No from Inward_Master where Inward_Date_Short='"+inwardDateList.get(i).toString()+"' AND Claim_Status='"+q+"') X");
ResultSet rs=ps.executeQuery();
rs.next();
data.add(rs.getInt(1));
}
catch(Exception e)
{
e.printStackTrace();
}
}
rowdata.add(data);
}
Related
Hi I am trying to fetch 50K + rows from one of the table in MYSQL DB. It is taking more than 20 minutes to retrieve all the data and writing it to text file. Can I use multi threading to reduce this fetching time and make the code more efficient. Any help will be appreciated.
I have used normal JDBC connection and ResultSetMetaData to fetch rows from the Table.
String row = "";
stmt = conn.createStatement();
ResultSet rs = stmt.executeQuery("select * from employee_details");
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
while (rs.next()) {
for (int i = 1; i < columnCount; i++) {
row = row + rs.getObject(i) + "|";// check
}
row = row + "\r\n";
}
And I am writing the fetched values in text file as below.
BufferedWriter writer = new BufferedWriter(new FileWriter(
"C:/Users/430398/Desktop/file/abcd.txt"));
writer.write(row);
writer.close();
Remember that rs.next will fetch Results from the DB in n-batches. Where n is a number defined by the JDBC-Implementation. I assume it's at 10 right now. So for every 10 batches it will again query the DB, hence there'll be an network-overhead - even if it's on the very same machine.
Just increasing that number will result in a faster loading time.
edit:
adding this
stmt.setFetchSize(50000);
might be it.
Be aware, that this results in heavy memory consumption.
First you need to identify where the bottleneck is. Is it the SQL query? Or the fetching of the rows via the ResultSet? Or the building of the huge string? Or perhaps writing the file?
You need to measure the duration of the above mentioned individual parts of your algorithm and tells us the results. Without this knowledge is not possible to tell how to speed the algorithm.
I am trying to Insert large amount of data from one table to another table. The two tables are in different regions. When I Insert data, the ID (I am using to create connection)is able to insert data for less number of rows. If It inserts data for more than 500 rows it throws exception
com.ibm.db2.jcc.b.SqlException: DB2 SQL error: SQLCODE: -551, SQLSTATE: 42501, SQLERRMC: DB2GCS;EXECUTE PACKAGE;NULLID.SYSLH203.
I am not able to find why it shows authorization exception for the same id if data is more.
My Code Snippet :
while(RSet.next()){
stmt=test_conn.prepareStatement("Insert Query");
for(int i=1;i<=columnCount;i++){
stmt.setString(i,RSet.getString(i));
}
stmt.addBatch();;
}
stmt.executeBatch();
Thanks in Advance for Your Help.
Your code is actually not batching correctly and this could be the reason why it's breaking. The problem is that you're preparing your insert query over and over again needlessly.
You need to prepare it just once outside the loop like
test_conn.setAutoCommit(false);
stmt = test_conn.prepareStatement("INSERT INTO ...");
while(RSet.next()){
for(int i = 1; i <= columnCount; i++){
stmt.setString(i, RSet.getString(i));
}
stmt.addBatch();
}
stmt.executeBatch();
test_conn.commit();
This question already has answers here:
java.sql.SQLException: Column count doesn't match value count at row 1
(3 answers)
Closed 6 years ago.
I'm using java netbeans & mysql and i used "insert into select statement".
But when I run the program and input the data it always says "Column count doesn't match the value count at row 1"
Connection conn = null;
ResultSet rs = null;
PreparedStatement pst = null;
private void jButton5ActionPerformed(java.awt.event.ActionEvent evt) {
String sql = "insert into tbl_addcharityroom1 values ('"+ jTextField10aw.getText() +"', (select charityWardID from tbl_addcharityward where diseaseCategory='"+ jComboBox1.getSelectedItem().toString() +"'))";
try {
pst = conn.prepareStatement(sql);
JOptionPane.showMessageDialog(null, "Saved");
pst.execute();
}
catch(Exception e){
JOptionPane.showMessageDialog(null, e);
}
}
You have 3 column in your table and you need to enter in 2 columns, so try this:
INSERT INTO tbl_addcharityroom1 (charityRoomStatus, charityWardID)
VALUES ('"+ jTextField10aw.getText() +"', (select charityWardID from tbl_addcharityward where diseaseCategory='"+ jComboBox1.getSelectedItem().toString() +"'))";
The number of columns in the destination table does not match the number of columns you're selecting, which is 2.
That is what it says. So conclusion is that tbl_addcharityroom1 does not have 2 columns.
MySQL is telling you that the number of columns in your query do not match the number of columns in the table. That should be a fairly easy fix but after you fix that you're still going to have a potentially nasty bug in this code.
What that bug is will depend on exactly how you have defined the tbl_addcharityroom1 table. I would assume by looking at this that you have multiple charity wards and those have a many to one relation with the categories table. This means that the select inside your insert statement may have multiple returned values. When this happens the query will fail. You are going to want to modify this code to handle that case. You can either look them all up ahead of time and perform an insert for each or you can use the CONCAT MySql command to put them all into a string which can be save as a discrete value.
This is my first time making an application and I am quite new with connecting the netbeans IDE to the MySQL database. I have a delete button in a Jpanel, and I want to be able to delete added rows. At the moment I can delete the added rows but they are of course not delete within the SQL DB which means they will remain there when I restart the application.
This is what I have to delete the rows so far
private void removeProductBtnActionPerformed(java.awt.event.ActionEvent evt) {
// TODO add your handling code here:
DefaultTableModel model = (DefaultTableModel) this.productTable.getModel();
int[] rows = productTable.getSelectedRows();
for(int i=0;i<rows.length;i++){
model.removeRow(rows[i]-i);
// If I highlight the rows and delte the, they are still in the SQL database.
//How to remove the complete data from the row in the SQL database? What method to write?
String sql = "DELETE FROM Product WHERE ProductID = ?";
Below I have my query to put it inside the database. (so you know what variables I am using)
public ResultSet insertQuery(String query) {
ResultSet result = null;
try {
Statement statement = connection.createStatement();
statement.executeUpdate(query);
result = statement.getGeneratedKeys();
} catch (SQLException e) {
System.err.println(e);
}
return result;
You have a '?' in your sql but you aren't using prepared statement.
For example's sake try:
String sql = "DELETE FROM Product WHERE ProductID = " + rows[i]-i
Question marks are used for prepared statements in java which (among other things) is used for sanitising inputs. Once you get the rest of your code working you should investigate it.
My problem is i have set a table auto commit false. I need to get the max id from that table(the currently inserted value of auto increment id). But i am getting the id of the previous committed process. Is it possible to get the value
My real problem is i need to insert into table some values, And need to take the id of the last inserted record from the first table and insert it to the second. The second insertion includes some image upload as well(as part of the code). so it take some delay or can have exceptions. I need to undo all insertions(both in the first and second) by occurring any exceptions. I tried to use commit-roll back method for this. But its not properly working as i mentioned above. main portion of my code is written below
try
{
//getting connection and setting auto commit false
dbHandler dbGetConnect=new dbHandler();
Connection conRegPlot=null;
conRegPlot=dbGetConnect.getconn();
conRegPlot.setAutoCommit(false);
String qryInsertPlot="INSERT INTO property_table(name,message,owner,locality,lattitude,longitude,country,state,city,type,catagory,created,modified,creted_date,zoompoint,mob_no,title,img_path,expire_date,lease_term) VALUES('none','"+description+"','"+sessionUserId+"','"+locality+"','"+lattitude+"','"+longitude+"','"+country+"','"+state+"','"+city+"','"+type+"','"+catagory+"',NOW(),NOW(),CURDATE(),'"+zoom+"','"+mob_no+"','"+title+"','NOT IN USE',"+expireDate+",'"+termsAndConditions+"')";//insertion into the first table
Statement stCrs=conRegPlot.createStatement();
int resp=stCrs.executeUpdate(qryInsertPlot);
String qryGetMaxProperty="SELECT MAX(l_id)"+
" FROM property_table"+
" WHERE stat='active'"+
" AND CURDATE()<=expire_date";
propertyId1=dbInsertPropert.returnSingleValue(qryGetMaxProperty);// get the max id
String qryInsertImage="INSERT INTO image_table(plot_id,ownr_id,created_date,created,modified,stat,img_path) VALUES('"+propertyId1+"','"+sessionUserId+"',CURDATE(),NOW(),NOW(),'active','"+img_pth+"')";
Statement stImg=conRegPlot.createStatement();
stImg.executeUpdate(qryInsertImage);// inserting the image
conRegPlot.commit();
}
catch(Exception exc)
{
conRegPlot.rollback();
}
finally{
conRegPlot.close();
}
Since
And need to take the id of the last inserted record from the first
table and insert it to the second.
You could the use of the new JDBC 3.0 method getGeneratedKeys() for get the generated ID. In ahother hand, you should use PreparedStatement for avoid SQL Injection.
PreparedStatement ps = null;
try {
conn = getConnection();
ps = conn.prepareStatement(myQuery, PreparedStatement.RETURN_GENERATED_KEYS);
ps.setInt(1, anInteger);
...
int rows = ps.executeUpdate();
if (rows == 1) {
ResultSet keysRS = ps.getGeneratedKeys();
if (keysRS.next())
int id = keysRS.getInt(1) // Get generated id
For MySQL, see more in http://dev.mysql.com/doc/refman/5.1/en/connector-j-usagenotes-last-insert-id.html#connector-j-examples-autoincrement-getgeneratedkeys