I am using java to execute some SQL Queries. Some of them are Getting data from one database(A) and storing in a table in another database(B).After process is done i am deleting all data from table in database(B). I am repeating this process every 5 mins.
Code:
String sql = "delete from newtable";
stmt5 = conn.prepareStatement(sql);
stmt5.executeUpdate(sql);
String sql_1 = "select distinct tbl_alm_log_2000000000.Csn, tbl_alm_log_2000000000.IsCleared, tbl_alm_log_2000000000.Id,tbl_alm_log_2000000000.NEType, tbl_alm_log_2000000000.OccurTime, tbl_alm_log_2000000000.hostIP, tbl_alm_log_2000000000.ExtendInfo From fmdb.dbo.tbl_alm_log_2000000000 Where IsCleared = 0";
ResultSet rs = stmt_1.executeQuery(sql_1);
String sql_2 = "insert into newtable (CSN, IsCleared, Id, NEType, OccurTime, hostIP) values(?,?,?,?,?,?)";
conn.setAutoCommit(false);
PreparedStatement ps = conn.prepareStatement(sql_2);
final int batchSize = 1000;
int count = 0;
while (rs.next()){
ps.setString(1, rs.getString(1)); //csn
ps.setString(2, rs.getString(2)); //iscleared
ps.setString(3, rs.getString(3));//id
ps.setString(4, rs.getString(4));//netype
ps.setString(5, rs.getString(5));//occurtime
ps.setString(6, rs.getString(6));//hostip
ps.addBatch();
if(++count % batchSize == 0) {
ps.executeBatch();
}
}
ps.executeBatch(); // insert remaining records
conn.commit();
ps.close();
It runs perfectly for 10 -20 runs and then gives "duplicate entry error for "value" in Csn as it is Primary key".
I added Distinct keyword in query and it is still giving this error after 10-20 runs.
Note: I m deleting data from newtable befor start of process so it is always adding in a empty table.
Suggest where i am going wrong.
Looks like you have misunderstanding about how does distinct work. In query with several selected columns it will search for distinct tuples of values, not for distinct Csn column only.
There are different ways how to select distinct values by one column only. It generally depends on particular DBMS you use and logic you want to apply for multiply tuples found for same Csn column values. Consider for instance this question: DISTINCT for only one Column
One of general ideas: select distinct single values for Csn column only, then iterate through this list and select first tuple of values with this Csn value (I don't know is it suitable for you select first tuple or not).
when you insert the data , you can add if not exists not make sure your data is unique ( i considered CSN only column in PK)
if not exists(select 1 from tbl_alm_log_2000000000 where CSN=? )
insert into newtable (CSN, IsCleared, Id, NEType, OccurTime, hostIP) values(?,?,?,?,?,?)
Related
Is there any way to detect in Java that a H2 merge into statement was doing insert instead of update?
In both cases the result is the number of affected rows but I need to know if the record was already there and updated or it was just created.
The result is interesting for a Rest service where I want to return 204 instead of 201 - Created.
Yes, there is:
Connection c = DriverManager.getConnection("jdbc:h2:mem:1")) {
Statement s = c.createStatement();
s.execute("CREATE TABLE TEST(ID INT PRIMARY KEY, V INT)");
PreparedStatement ps = c.prepareStatement(
"SELECT ID FROM OLD TABLE (MERGE INTO TEST(ID, V) KEY(ID) VALUES (?, ?))");
ResultSet rs;
ps.setInt(1, 1);
ps.setInt(2, 10);
rs = ps.executeQuery();
System.out.println(rs.next() ? "UPDATE" : "INSERT");
ps.setInt(1, 1);
ps.setInt(2, 20);
rs = ps.executeQuery();
System.out.println(rs.next() ? "UPDATE" : "INSERT");
You need a recent version of H2, however, because historic versions don't support data change delta tables.
In the sample code I used non-standard and non-portable variant of MERGE command for simplicity. You can either use it or its standard-compliant version, you can find syntax of both of them in documentation of H2.
I created 2 types as OBJECT and a type of table and a nested table in my Oracle database. I know how to inset a row in table in cmd but I have no idea that how I can insert a row in this table with PreparedStatement in Java.
This is my table and type:
CREATE Or REPLACE TYPE item AS OBJECT
(
id number,
counter number
);/
CREATE Or REPLACE TYPE time AS OBJECT
(
year number,
month number,
second number
);
CREATE Or REPLACE TYPE Shoping_list AS TABLE OF item;
create table invoice(
id number NOT NULL PRIMARY KEY ,
list shoping_list,
dateIn date,
timeIn time,
totalCost number )
nested table list store as list_of_product;
I can insert with this code in cmd
insert into invoice(id,list,dateIn,timeIn,totalCost) values(seq_ID_invoice.nextval,
shoping_list(item(1,2),item(26,1)),TO_DATE('1396-04-11','YYYY-MM-DD'),time(11,25,40),7000);
and this my java code
preparedStatement = connection.prepareStatement("insert into invoice (id,list,dateIn,timeIn,totalCost) values ("+id+",?,?,?,?)");
seq_ID_invoice is Oracle Sequence. So oracle takes its value automatically. For rest you can use setInt, setString, setFloat, etc. methods available in PreparedStatement based on the types of columns defined inside your table.
Following sample code may help you.
PreparedStatement ps = connection.prepareStatement(
"insert into invoice(id, list, dateIn, timeIn, totalCost)"+
" values (seq_ID_invoice.nextval, shoping_list(item(?,?), item(?,?)), TO_DATE(?,'YYYY-MM-DD'), time(?, ?, ?), ?)");
ps.setInt(1, int_value_for_id_of_item_1);
ps.setInt(2, int_value_for_counter_of_item_1);
ps.setInt(3, int_value_for_id_of_item_2);
ps.setInt(4, int_value_for_counter_of_item_2);
ps.setString(5, string_value_for_date);
ps.setInt(6, int_value_for_year_of_time);
ps.setInt(7, int_value_for_month_of_time);
ps.setInt(8, int_value_for_second_of_time);
ps.setInt(9, int_bvalue_for_total_cost);
int rowCnt = ps.executeUpdate();
System.out.println(rowCnt+" rows affected.");
int_value_for_id_of_item_1, int_value_for_counter_of_item_1, ... these are java variables. You can defined variables of your choice or you can directly mention values in place of these variables.
Currently we are selecting data from one database and inserting it into a backup database(SQL SERVER).
This data always contains more than 15K records in one select.
We are using Enumeration to iterate over the data selected.
We are using JDBC PreparedStatement to insert data as:
Enumeration values = ht.elements(); -- ht is HashTable containing selected data.
while(values.hasMoreElements())
{
pstmt = conn.prepareStatement("insert query");
pstmt.executeUpdate();
}
I am not sure if this is the correct or efficient way to do the faster insert.
For inserting 10k rows it takes near about 30 min or more.
Is there any efficient way to make it fast?
Note: Not using any indexes on the table.
Use a batch insert, but commit after a few entris, don't try to send all 10K at once. Try investigating to get the best size, it' a trade off to memory vs network trips.
Connection connection = new getConnection();
Statement statement = connection.createStatement();
int i = 0;
for (String query : queries) {
statement.addBatch("insert query");
if ((i++ % 500) == 0) {
// Do an execute now and again, don't send too many at once
statement.executeBatch();
}
}
statement.executeBatch();
statement.close();
connection.close();
Also, from your code I'm not sure what you are doing, but use paramaterised queries rather than sending 10K insert statements as text. Something like:
String q= "INSERT INTO data_table (id) values (?)";
Connection connection = new getConnection();
PreparedStatement ps = connection.prepareStatement(q);
for (Data d: data) {
ps.setString(1, d.getId());
ps.addBatch();
}
ps.executeBatch();
ps.close();
connection.close();
You can insert all the values in one sql command:
INSERT INTO Table1 ( Column1, Column2 ) VALUES
( V1, V2 ), ( V3, V4 ), .......
You may also insert the values by bulks of 500 records, for example, if the query would become very big. It is not efficient at all to insert on row per statement remotely (using a connection). Another solution is to do the inserts using a stored procedure. You just pass the values to it as parameters.
Here is how you can do it using the INSERT command above:
Enumeration values = ht.elements(); -- ht is HashTable containing selected data.
int i=0;
String sql="";
while(values.hasMoreElements())
{
sql+="(" + values + ")"; //better use StringBuffer here
i++;
if(i % 500 == 0) {
pstmt = conn.prepareStatement("insert query "+sql);
pstmt.executeUpdate();
sql="";
}
else
sql += " , ";
}
I want to retrieve all the data from database, and at the same time, I want to know how many rows of data I get. And this is my SQL:
rs = s.executeQuery("SELECT COUNT(*), * FROM tblUser");
Is this a valid SQL statement? and after I retrieved all the data, how to set them into different variables? For example, I have a column called UserIDin the database, I can simply get it by using rs.getString('UserID'), but how to get the result of the COUNT(*)?
Your SQL is not valid. The ANSI standard way to do what you want uses window functions:
select count(*) over () as total_cnt,
u.*
from tblUser u;
This adds a new column to every row -- which seems to be what you want. There are other mechanisms, depending on the underlying database for doing this.
The results you request are not interrelated, so run two queries:
rs1 = s.executeQuery("SELECT COUNT(*) FROM tblUser");
rs2 = s.executeQuery("SELECT * FROM tblUser");
and retrieve the values (one only for rs1) the usual way.
You can do this to count the rows in resultset
String query = "Select * from tblUser";
rs = s.executeQuery(query);
public int getCount(ResultSet rs) {
int rows = 0;
while(rs.next()) {
i++;
}
return i;
}
This way you can get the resultset as well as count
Since you are already accessing the recordset within VBA probably the simplest was to return the count of the record set is to:
rs = s.executeQuery("SELECT * FROM tblUser");
If Not rs.EOF Then
' Important: You must move to the last record to
' obtain the count of the full recordset
rs.MoveLast
rsCount = rs.RecordCount
' Remember to Return to the First Record so that you can
' continue to use the recordset
rs.MoveFirst
End If
An alternative if your RDBMS doesn't support window functions
rs = s.executeQuery("SELECT B.cnt, U.*
FROM tblUser U,
(SELECT count(*) cnt FROM tblUser) B");
How can I use a prepared statement to delete entries from a database? I have found that I must write the following code
String deleteSQL = "DELETE DBUSER WHERE USER_ID = ?
but I want to specify a clause with more than one variable. I have used the AND operator but it doesn't seem to work.
Here is an example if your syntax is not correct..
DELETE DBUSER WHERE USER_ID = ? and USER_NAME = ?;
you can append more conditions in where clause by using more AND ... operators.
OR if you have more than one USER_IDs to delete in a single query..
DELETE DBUSER WHERE USER_ID in (?, ?, ?, ?);
It's must work/ for example
Select from Employee e where e.ID < ? and e.ID >= ? order by e.ID
to set values use this:
int id1 = 1;
int id2 = 10;
preparedStatement.setInt(2, id1);
preparedStatement.setInt(1, id2);
for delete I use this code:
public synchronized boolean deleteNewsById(Integer[] idList)
throws NewsManagerException {
DatabaseConnection connection = pool.getConnection();
StringBuffer buffer = new StringBuffer();
buffer.append("(");
buffer.append(idList[0]);
for (int i = 1; i < idList.length; i++) {
buffer.append(",");
buffer.append(idList[i]);
}
buffer.append(")");
PreparedStatement statement = connection
.getPreparedStatement(DELETE_NEWS_BY_ID + buffer);
}
and sql query looks like this
private static final String DELETE_NEWS_BY_ID = "delete from NEWS where ID in ";
or simple write delete from NEWS where ID in (?,?,?) and set values like in first example
I think the response from Aleksei Bulgak is correct, but to perhaps more straightforwardly word it...you can set your parameters like this:
String stmt = "DELETE DBUSER WHERE USER_ID = ? and (USER_NAME = ? or USER_NAME = ?)";
preparedStatement.setInt(1, firstParam);
preparedStatement.setString(2, secondParam);
preparedStatement.setString(3, thirdParam);
...and for however many parameters(question marks) in your SQL (no matter if you're using IN or whatever you want), you should set that many parameters here(using setInt for ints, setString for Strings, etc). This goes for select and delete queries.
Are you looking for the IN operator which allows you to specify multiple values in the WHERE clause such as in my example.
String deleteSQL = "DELETE DBUSER WHERE USER_ID IN (?)"
Though in PreparedStatement IN clause alternatives there are some useful answers and links that you may want to take a look at such as Batch Statements in JDBC which discuss the pros and cons of different batching approaches. The IN approach I'm suggesting is part of that discussion. The end result is that you make just one trip to the database, rather than one per delete and that's better performing because of the reduced network activity required.