Develop Reference

How do I add a new row of data into my Derby database?

Using Netbeans, I have my database and table set up, and have added my data manually, in which I am able to see within my application I am building, as intended.
I would like the user to add their own data in which will be appended to a new row on the table. However, I am having trouble trying to write code in order to do this.
Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
Connection conn = DriverManager.getConnection("jdbc:derby://localhost:1527/stockApplication");
Statement stat = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
String insertDerbyData = "INSERT INTO TIGER_INFO"
+ "(TIGER_ID, TIGER_LOCATION)"
+ "VALUES (123456, Store)";
stat.executeUpdate(insertDerbyData);
I cannot execute the above code as I'm returned with an error mentioning that 'STORE' is not in any table. 'STORE' is meant to be a value for my 'TIGER_LOCATION' column. What's going on here?
In theory, I have two columns, and I would like to add both values, '123456' and 'Store' into their respective columns. How do I go about correctly doing so?

If TIGER_LOCATION is a string/varchar column, and Store is a string literal, then the value must be enclosed in single quotes, as in most SQL-based databases:
INSERT INTO TIGER_INFO (TIGER_ID, TIGER_LOCATION) VALUES (123456, 'Store')

Strings should be between '...' you have to use :
VALUES (123456, 'Store')
//--------------^-----^

my server resultset is taking long time while looping through

I am facing performance issue with my sql server when I am trying to connect using a java application. sqljdbc4-4.0 is my driver version. Looks like this is the latest available for this.
The issue comes when I use the PreparedStatemnt with the set paramerter option and when looping trough the result set after a specific number of iterations the execution stops amost 5min and start iterating again. This stopping occuring on some fixed iterations.
This behavior occurs even for very simple query like below.
PreparedStatemnt ps = null;
ResultSet rs = null;
ps = con.prepareStatemnt("select id,name,phone from Table1 where name = ?");
ps.setString(1, "Raj");
rs = ps.executeQuery(); // OK here
int i=0;
while(rs.next()){ // hangs here
System.out.println(i++);
}
However, it works fine if I do not set the parameter and concatenate the parameter to the query like this,
PreparedStatemnt ps = null;
ResultSet rs = null;
ps = con.prepareStatemnt("select id,name,phone from Table1 where name = 'Bob' ");
rs = ps.executeQuery(); // OK here
int i=0;
while(rs.next()){ // OK now
System.out.println(i++);
}
when I use the query with no where condition it runs perfectly with no hanging.
This tell me that the issue comes when I use the set parameter option . I have tried to use the top rows to see whether the issue is with the large number of rows. But even when I set the top with 100. For these 100 rows itself the iterations stopped 2 times at 28th iteration and 66th iterations. I think on which iteration it hangs is depends on size of the number of columns.
can some one help me to solve the issue.
Thanks in Advance,
Syam.

You have more, many more, Rajs than Bobs in your database.
It has nothing to do with whether you use PreparedStatement arguments or not.

Get Rows By Group MySQL Java

I have a large amount of data and I need to process them by group.
Having the table structure in the image
How do I retrieve them by group? (First get group 1, process. Next, get group 2, process.. and so on.) The grouping is based on the pos column which should be equal.
I have been reading up on references on doing a basic join of the same table but it is not possible for me as it returns a very large set of data which will result to OutOfMemoryError.

You'll need two cursors, one to select distinct groups, another to process each group separately.
//Assuming you have a Connection conn;
PreparedStatement groupsPS =
conn.prepareStatement("SELECT distinct pos from yourtable");
ResultSet groupsRS = groupsPS.executeQuery();
PreparedStatement groupdataPS =
conn.prepareStatement("SELECT * from yourtable where pos = ?");
ResultSet groupdataRS = null;
while(groupsRS.next())
{
groupdataPS.setString(1, groupsRS.getString("pos"));
groupdataRS = groupdataPS.executeQuery();
while(groupsRS.next())
{
//process your data here
}
groupdataRS.close();
}
groupsRS.close();
groupdataPS.close();
groupsPS.close();

You have two options:
Rewrite your sql to make sure you get smaller resultsets and read the whole data in junks.
Use a ScrollableResultset. This allows you to get row by row without loading the whole resultset into memory.

Just like what #TimeKiller and #BetaRide mentioned, I restructured my SQL tables and added an indexing table for my pos with its frequency as another column.
Using this, I can just iterate through each pos with a frequency >= 2, get the rows in the table above with the selected pos, and process them.
Thanks!

Prepared Statement insert into Multiple Tables using Try-with-resources , java DB and NetBeans 8.0 [duplicate]

I need to insert a couple hundreds of millions of records into the mysql db. I'm batch inserting it 1 million at a time. Please see my code below. It seems to be slow. Is there any way to optimize it?
try {
// Disable auto-commit
connection.setAutoCommit(false);
// Create a prepared statement
String sql = "INSERT INTO mytable (xxx), VALUES(?)";
PreparedStatement pstmt = connection.prepareStatement(sql);
Object[] vals=set.toArray();
for (int i=0; i<vals.length; i++) {
pstmt.setString(1, vals[i].toString());
pstmt.addBatch();
}
// Execute the batch
int [] updateCounts = pstmt.executeBatch();
System.out.append("inserted "+updateCounts.length);

I had a similar performance issue with mysql and solved it by setting the useServerPrepStmts and the rewriteBatchedStatements properties in the connection url.
Connection c = DriverManager.getConnection("jdbc:mysql://host:3306/db?useServerPrepStmts=false&rewriteBatchedStatements=true", "username", "password");

I'd like to expand on Bertil's answer, as I've been experimenting with the connection URL parameters.
rewriteBatchedStatements=true is the important parameter. useServerPrepStmts is already false by default, and even changing it to true doesn't make much difference in terms of batch insert performance.
Now I think is the time to write how rewriteBatchedStatements=true improves the performance so dramatically. It does so by rewriting of prepared statements for INSERT into multi-value inserts when executeBatch() (Source). That means that instead of sending the following n INSERT statements to the mysql server each time executeBatch() is called :
INSERT INTO X VALUES (A1,B1,C1)
INSERT INTO X VALUES (A2,B2,C2)
...
INSERT INTO X VALUES (An,Bn,Cn)
It would send a single INSERT statement :
INSERT INTO X VALUES (A1,B1,C1),(A2,B2,C2),...,(An,Bn,Cn)
You can observe it by toggling on the mysql logging (by SET global general_log = 1) which would log into a file each statement sent to the mysql server.

You can insert multiple rows with one insert statement, doing a few thousands at a time can greatly speed things up, that is, instead of doing e.g. 3 inserts of the form INSERT INTO tbl_name (a,b,c) VALUES(1,2,3); , you do INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(1,2,3),(1,2,3); (It might be JDBC .addBatch() does similar optimization now - though the mysql addBatch used to be entierly un-optimized and just issuing individual queries anyhow - I don't know if that's still the case with recent drivers)
If you really need speed, load your data from a comma separated file with LOAD DATA INFILE , we get around 7-8 times speedup doing that vs doing tens of millions of inserts.

If:
It's a new table, or the amount to be inserted is greater then the already inserted data
There are indexes on the table
You do not need other access to the table during the insert
Then ALTER TABLE tbl_name DISABLE KEYS can greatly improve the speed of your inserts. When you're done, run ALTER TABLE tbl_name ENABLE KEYS to start building the indexes, which can take a while, but not nearly as long as doing it for every insert.

You may try using DDBulkLoad object.
// Get a DDBulkLoad object
DDBulkLoad bulkLoad = DDBulkLoadFactory.getInstance(connection);
bulkLoad.setTableName(“mytable”);
bulkLoad.load(“data.csv”);

try {
// Disable auto-commit
connection.setAutoCommit(false);
int maxInsertBatch = 10000;
// Create a prepared statement
String sql = "INSERT INTO mytable (xxx), VALUES(?)";
PreparedStatement pstmt = connection.prepareStatement(sql);
Object[] vals=set.toArray();
int count = 1;
for (int i=0; i<vals.length; i++) {
pstmt.setString(1, vals[i].toString());
pstmt.addBatch();
if(count%maxInsertBatch == 0){
pstmt.executeBatch();
}
count++;
}
// Execute the batch
pstmt.executeBatch();
System.out.append("inserted "+count);

MySQL connection in Java problem [duplicate]

This question already has answers here:
ResultSet exception - before start of result set
(6 answers)
Closed 5 years ago.
I try to get some data form database. The connection method works for sure, but I have a problem getting any data form DB:
SQLConnect s = new SQLConnect();
Connection c = s.getConnection();
Statement st = c.createStatement();
ResultSet rs = st.executeQuery("select * from produkty");
System.out.println(rs.getString(2));
The problem is with the last line (when I comment it, no error appears).
Error message:
Connected to database
Exception in thread "main" java.sql.SQLException: Before start of result set
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1073)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
at com.mysql.jdbc.ResultSetImpl.checkRowPos(ResultSetImpl.java:841)
at com.mysql.jdbc.ResultSetImpl.getStringInternal(ResultSetImpl.java:5656)
at com.mysql.jdbc.ResultSetImpl.getString(ResultSetImpl.java:5576)
at antmedic.Main.main(Main.java:85)
Java Result: 1
BUILD SUCCESSFUL (total time: 1 second)
Thanks for any help

You need to call ResultSet#next() to shift the resultset cursor to the next row. Usually, when there's means of multiple rows, do this in a while loop.
while (rs.next()) {
System.out.println(rs.getString(2));
}
Or when you expect zero or one row, use an if statement.
if (rs.next()) {
System.out.println(rs.getString(2));
}
See also:
JDBC tutorial
Examples of how to traverse the ResultSet correctly

As you get the ResultSet object, the cursor points to the row before the first row, So after calling
while (rs.next()) {
//your code
}
the cursor points to the next row
i.e. the first row.

Remember, whenever select query fires for retrieving the data from database into ResultSet,
so the structure of ResultSet is
-> Zero Record Area
-> Database Record Area
-> No Record Area
that's why alwayz we must put next() with ResultSet object so it can move from Zero Record Area to Database Record Area.

while(rs.next())
{
System.out.println(rs.getString(1));
System.out.println(rs.getString(2));
}
at the place of 1, 2, .... we can use the database columns name also. But technically always we use indexing like 1,2,3,.... its reason for any updation happening in future at our database like changes the column name so it can't be occur any problem for us because we haven't used the columns names.