Develop Reference

Insert data into Database - What is the best way to do it

I have to insert anywhere between 50 - 500 contact's information into the database. I have 4 arraylists that contain image, name, number, bool variable respectively.
Each row in the data is made up of a combination of all the 4 arraylists along with a SNO. Please refer to the image below.
My question is, let's say i have 500 contacts that I retrieve from the User's Contacts list. Is it a good thing that, I have a function that inserts each row at a time into the table and call it 500 times? or is there any other way? A mean idea would be to combine all the arraylists together, pass it to the function and retrieve the data there and repeat the insert statement 500 times.
What is better in terms of performance?
for(int i =0; i < 500; i++)
{
dbObj.insert_row(par1, par2, par3, par4, ...);
}
OR
function insert_row(Combined ArrayLists)
{
for(int i=0; i<500; i++)
{
db.execSql(//Insert Statement);
}
}

Insert data into Database - What is the best way to do it
I suggest you to create own object that will represent your table where properties of object will be equal to columns in table, e.q.
public class Contact {
private String name;
private String number;
private String image;
private boolean conn;
//getters and setters
}
Now your approach sounds like "spaghetti code". There is not need to have four ArrayLists and this design is not efficient and correct.
Now, you will have one ArrayList of Contact object with 500 childs.
What is the best way to insert?
For sure wrap your insertion to one TRANSACTION that speed up your inserts rapidly and what is the main your dealing with database becomes much more safer and then you don't need to care about possibility of losing database integrity.
Example of transaction(one method from my personal example project):
public boolean insertTransaction(int count) throws SQLException {
boolean result = false;
try {
db = openWrite(DataSource.getInstance(mContext));
ContentValues values = new ContentValues();
if (db != null) {
db.beginTransaction();
for (int i = 0; i < count; i++) {
values.put(SQLConstants.KEY_TYPE, "type" + i);
values.put(SQLConstants.KEY_DATE, new Date().toString());
db.insertOrThrow(SQLConstants.TEST_TABLE_NAME, SQLConstants.KEY_TYPE, values);
values.clear();
}
db.setTransactionSuccessful();
result = true;
}
return result;
}
finally {
if (db != null) {
db.endTransaction();
}
close(db);
}
}

If you are going to insert 500 records into database you should use transaction
database.beginTransaction();
try {
// perform inserts
database.setTransactionSuccessful();
finally {
database.endTranasction();
}

As mentioned before create Your own class to represent one row or use ContentValues class
SQlite doesn't provide possibility to insert many rows in one query like in MySQL but there is some way around it You can read here.
If You decide to use method described in this link it is better if You create a function to generate this query and run it just once. Otherwise As others mentioned already you may use transaction to improve a performance of many inserts.

Converting 4 arrays into one object array makes the code better. but you can create these objects without doing it like this.
Prepare the sql statement with bind variables (? or :vars), then execute the statement multiple times in a loop by setting the bind variables for each row.
String sql = "insert into..... values (?,?,?,?)";
// Get connection etc
PreparedStatement stmt = conn.prepareStatement(sql);
for(int i =0; i < 500; i++)
{
stmt.setString(1, name.get(i));
stmt.setNumber(2, number.get(i));
...
stmt.executeUpdate();
}

ResultSet reset when using recursion

I have a question regarding ResultSet objects in Java and recursion.
I was working on some code for university and whenever I found a descendant I recursed on with that new node but when I came out of the recursion and tried to rs.next() the pointer had gone from pointing to row 1 back to row 0 and when it hit row 0 the rs.next() failed and it returned! I knew there was one thing in there that it hadn't read yet! What is it that causes this?
The only way I got round that problem was to go through the resultset and get every element and add it into an array list, then loop through the arraylist doing the recursion on each element in the array! Surely this must be a better way around this?
This is the new code I'm using
private Vector<String> getDescendents(String dogname, Vector<String> anc) {
if (anc == null) anc = new Vector<String>();
ArrayList<String> tempList = new ArrayList<String>(2);
try {
System.out.println("Inside ");
childStmt.setString(1,dogname);
childStmt.setString(2,dogname);
ResultSet rs = childStmt.executeQuery();
System.out.println("Before while "+rs.getRow());
while (rs.next()){
String col1 = rs.getString(1);
tempList.add(col1);
anc.add(col1);
}
for (String s:tempList){
getDescendents(s,anc);
}
}
catch(Exception e) {
doError(e, "Failed to execute ancestor query in getBreeding");
}
return anc;
}
However before this, I had the getDescendents call inside the while loop and thus no for loop and no arraylist either, but whenever it actually recursed it would loose track of the resultset when it returned out of the recursion.
Further details :
When I used the debugger (nearly said gdb there lol far too much C) the ID of the result set was the same but the row pointer had returned to 0 and the rs.next call failed!
Once again any explanation is appreciated!
p.s it previously looked like
private Vector<String> getDescendents(String dogname, Vector<String> anc) {
if (anc == null) anc = new Vector<String>();
ArrayList<String> tempList = new ArrayList<String>(2);
try {
System.out.println("Inside ");
childStmt.setString(1,dogname);
childStmt.setString(2,dogname);
ResultSet rs = childStmt.executeQuery();
System.out.println("Before while "+rs.getRow());
while (rs.next()){
String col1 = rs.getString(1);
anc.add(col1);
getDescendendts(col1,anc);
}
}
catch(Exception e) {
doError(e, "Failed to execute ancestor query in getBreeding");
}
return anc;
}

It looks like you're re-using childStmt; don't do this. From the Statement javadoc:
By default, only one ResultSet object per Statement object can be open
at the same time. Therefore, if the reading of one ResultSet object is
interleaved with the reading of another, each must have been generated
by different Statement objects. All execution methods in the Statement
interface implicitly close a statment's current ResultSet object if an
open one exists.
You'll have to either save all the rows first, then do the recursive query, or create a new Statement for each ResultSet you want to fetch.

Total Number of Row Resultset getRow Method

Read the Following Code:
public class selectTable {
public static ResultSet rSet;
public static int total=0;
public static ResultSet onLoad_Opetations(Connection Conn, int rownum,String sql)
{
int rowNum=rownum;
int totalrec=0;
try
{
Conn=ConnectionODBC.getConnection();
Statement stmt = Conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
String sqlStmt = sql;
rSet = stmt.executeQuery(sqlStmt);
total = rSet.getRow();
}
catch(Exception e)
{
System.out.println(e.getMessage());
}
System.out.println("Total Number of Records="+totalrec);
return rSet;
}
}
The folowing code dos't show actual total:
total = rSet.getRow();
my jTable display 4 record in jTable but total = 0; when I evaluate through debug, it shows:
total=(int)0;
rather than total=(int)4
And if I use
rSet=last(); above from the code total = rSet.getRow();
then total shows accurate value = 4 but rSet return nothing. then jTable is empty.
Update me!

BalusC's answer is right! but I have to mention according to the user instance variable such as:
rSet.last();
total = rSet.getRow();
and then which you are missing
rSet.beforeFirst();
the remaining code is same you will get your desire result.

You need to call ResultSet#beforeFirst() to put the cursor back to before the first row before you return the ResultSet object. This way the user will be able to use next() the usual way.
resultSet.last();
rows = resultSet.getRow();
resultSet.beforeFirst();
return resultSet;
However, you have bigger problems with the code given as far. It's leaking DB resources and it is also not a proper OOP approach. Lookup the DAO pattern. Ultimately you'd like to end up as
public List<Operations> list() throws SQLException {
// Declare Connection, Statement, ResultSet, List<Operation>.
try {
// Use Connection, Statement, ResultSet.
while (resultSet.next()) {
// Add new Operation to list.
}
} finally {
// Close ResultSet, Statement, Connection.
}
return list;
}
This way the caller has just to use List#size() to know about the number of records.

The getRow() method retrieves the current row number, not the number of rows. So before starting to iterate over the ResultSet, getRow() returns 0.
To get the actual number of rows returned after executing your query, there is no free method: you are supposed to iterate over it.
Yet, if you really need to retrieve the total number of rows before processing them, you can:
ResultSet.last()
ResultSet.getRow() to get the total number of rows
ResultSet.beforeFirst()
Process the ResultSet normally

As others have answered there is no way to get the count of rows without iterating till the end. You could do that, but you may not want to, note the following points:
For a many RDBMS systems ResultSet is a streaming API, this means
that it does not load (or maybe even fetch) all the rows from the
database server. See this question on SO. By iterating to the
end of the ResultSet you may add significantly to the time taken to
execute in certain cases.
A default ResultSet object is not updatable and has a cursor
that moves forward only. I think this means that unless you
execute
the query with ResultSet.TYPE_SCROLL_INSENSITIVE rSet.beforeFirst() will throw
SQLException. The reason it is this way is because there is cost
with scrollable cursor. According to the documentation, it may throw SQLFeatureNotSupportedException even if you create a scrollable cursor.
Populating and returning a List<Operations> means that you will
also need extra memory. For very large resultsets this will not
work
at all.
So the big question is which RDBMS?. All in all I would suggest not logging the number of records.

One better way would be to use SELECT COUNT statement of SQL.
Just when you need the count of number of rows returned, execute another query returning the exact number of result of that query.
try
{
Conn=ConnectionODBC.getConnection();
Statement stmt = Conn.createStatement();
String sqlStmt = sql;
String sqlrow = SELECT COUNT(*) from (sql) rowquery;
String total = stmt.executeQuery(sqlrow);
int rowcount = total.getInt(1);
}

The getRow() method will always yield 0 after a query:
ResultSet.getRow()
Retrieves the current row number.
Second, you output totalrec but never assign anything to it.

You can't get the number of rows returned in a ResultSet without iterating through it. And why would you return a ResultSet without iterating through it? There'd be no point in executing the query in the first place.
A better solution would be to separate persistence from view. Create a separate Data Access Object that handles all the database queries for you. Let it get the values to be displayed in the JTable, load them into a data structure, and then return it to the UI for display. The UI will have all the information it needs then.

I have solved that problem. The only I do is:
private int num_rows;
And then in your method using the resultset put this code
while (this.rs.next())
{
this.num_rows++;
}
That's all

The best way to get number of rows from resultset is using count function query for database access and then rs.getInt(1) method to get number of rows.
from my code look it:
String query = "SELECT COUNT() FROM table";
ResultSet rs = new DatabaseConnection().selectData(query);
rs.getInt(1);
this will return int value number of rows fetched from database.
Here DatabaseConnection().selectData() is my code for accessing database.
I was also stuck here but then solved...

How to protect against SQL injection when the WHERE clause is built dynamically from search form?

I know that the only really correct way to protect SQL queries against SQL injection in Java is using PreparedStatements.
However, such a statement requires that the basic structure (selected attributes, joined tables, the structure of the WHERE condition) will not vary.
I have here a JSP application that contains a search form with about a dozen fields. But the user does not have to fill in all of them - just the one he needs. Thus my WHERE condition is different every time.
What should I do to still prevent SQL injection?
Escape the user-supplied values? Write a wrapper class that builds a PreparedStatement each time? Or something else?
The database is PostgreSQL 8.4, but I would prefer a general solution.
Thanks a lot in advance.

Have you seen the JDBC NamedParameterJDBCTemplate ?
The NamedParameterJdbcTemplate class
adds support for programming JDBC
statements using named parameters (as
opposed to programming JDBC statements
using only classic placeholder ('?')
arguments.
You can do stuff like:
String sql = "select count(0) from T_ACTOR where first_name = :first_name";
SqlParameterSource namedParameters = new MapSqlParameterSource("first_name", firstName);
return namedParameterJdbcTemplate.queryForInt(sql, namedParameters);
and build your query string dynamically, and then build your SqlParameterSource similarly.

I think that fundamentally, this question is the same as the other questions that I referred to in my comment above, but I do see why you disagree — you're changing what's in your where clause based on what the user supplied.
That still isn't the same as using user-supplied data in the SQL query, though, which you definitely want to use PreparedStatement for. It's actually very similar to the standard problem of needing to use an in statement with PreparedStatement (e.g., where fieldName in (?, ?, ?) but you don't know in advance how many ? you'll need). You just need to build the query dynamically, and add the parameters dynamically, based on information the user supplied (but not directly including that information in the query).
Here's an example of what I mean:
// You'd have just the one instance of this map somewhere:
Map<String,String> fieldNameToColumnName = new HashMap<String,String>();
// You'd actually load these from configuration somewhere rather than hard-coding them
fieldNameToColumnName.put("title", "TITLE");
fieldNameToColumnName.put("firstname", "FNAME");
fieldNameToColumnName.put("lastname", "LNAME");
// ...etc.
// Then in a class somewhere that's used by the JSP, have the code that
// processes requests from users:
public AppropriateResultBean[] doSearch(Map<String,String> parameters)
throws SQLException, IllegalArgumentException
{
StringBuilder sql;
String columnName;
List<String> paramValues;
AppropriateResultBean[] rv;
// Start the SQL statement; again you'd probably load the prefix SQL
// from configuration somewhere rather than hard-coding it here.
sql = new StringBuilder(2000);
sql.append("select appropriate,fields from mytable where ");
// Loop through the given parameters.
// This loop assumes you don't need to preserve some sort of order
// in the params, but is easily adjusted if you do.
paramValues = new ArrayList<String>(parameters.size());
for (Map.Entry<String,String> entry : parameters.entrySet())
{
// Only process fields that aren't blank.
if (entry.getValue().length() > 0)
{
// Get the DB column name that corresponds to this form
// field name.
columnName = fieldNameToColumnName.get(entry.getKey());
// ^-- You'll probably need to prefix this with something, it's not likely to be part of this instance
if (columnName == null)
{
// Somehow, the user got an unknown field into the request
// and that got past the code calling us (perhaps the code
// calling us just used `request.getParameterMap` directly).
// We don't allow unknown fields.
throw new IllegalArgumentException(/* ... */);
}
if (paramValues.size() > 0)
{
sql.append("and ");
}
sql.append(columnName);
sql.append(" = ? ");
paramValues.add(entry.getValue());
}
}
// I'll assume no parameters is an invalid case, but you can adjust the
// below if that's not correct.
if (paramValues.size() == 0)
{
// My read of the problem being solved suggests this is not an
// exceptional condition (users frequently forget to fill things
// in), and so I'd use a flag value (null) for this case. But you
// might go with an exception (you'd know best), either way.
rv = null;
}
else
{
// Do the DB work (below)
rv = this.buildBeansFor(sql.toString(), paramValues);
}
// Done
return rv;
}
private AppropriateResultBean[] buildBeansFor(
String sql,
List<String> paramValues
)
throws SQLException
{
PreparedStatement ps = null;
Connection con = null;
int index;
AppropriateResultBean[] rv;
assert sql != null && sql.length() > 0);
assert paramValues != null && paramValues.size() > 0;
try
{
// Get a connection
con = /* ...however you get connections, whether it's JNDI or some conn pool or ... */;
// Prepare the statement
ps = con.prepareStatement(sql);
// Fill in the values
index = 0;
for (String value : paramValues)
{
ps.setString(++index, value);
}
// Execute the query
rs = ps.executeQuery();
/* ...loop through results, creating AppropriateResultBean instances
* and filling in your array/list/whatever...
*/
rv = /* ...convert the result to what we'll return */;
// Close the DB resources (you probably have utility code for this)
rs.close();
rs = null;
ps.close();
ps = null;
con.close(); // ...assuming pool overrides `close` and expects it to mean "release back to pool", most good pools do
con = null;
// Done
return rv;
}
finally
{
/* If `rs`, `ps`, or `con` is !null, we're processing an exception.
* Clean up the DB resources *without* allowing any exception to be
* thrown, as we don't want to hide the original exception.
*/
}
}
Note how we use information the user supplied us (the fields they filled in), but we didn't ever put anything they actually supplied directly in the SQL we executed, we always ran it through PreparedStatement.

The best solution is to use a middle that does data validation and binding and acts as an intermediary between the JSP and the database.
There might be a list of column names, but it's finite and countable. Let the JSP worry about making the user's selection known to the middle tier; let the middle tier bind and validate before sending it on to the database.

Here is a useful technique for this particular case, where you have a number of clauses in your WHERE but you don't know in advance which ones you need to apply.
Will your user search by title?
select id, title, author from book where title = :title
Or by author?
select id, title, author from book where author = :author
Or both?
select id, title, author from book where title = :title and author = :author
Bad enough with only 2 fields. The number of combinations (and therefore of distinct PreparedStatements) goes up exponentially with the number of conditions. True, chances are you have enough room in your PreparedStatement pool for all those combinations, and to build the clauses programatically in Java, you just need one if branch per condition. Still, it's not that pretty.
You can fix this in a neat way by simply composing a SELECT that looks the same regardless of whether each individual condition is needed.
I hardly need mention that you use a PreparedStatement as suggested by the other answers, and a NamedParameterJdbcTemplate is nice if you're using Spring.
Here it is:
select id, title, author
from book
where coalesce(:title, title) = title
and coalesce(:author, author) = author
Then you supply NULL for each unused condition. coalesce() is a function that returns its first non-null argument. Thus if you pass NULL for :title, the first clause is where coalesce(NULL, title) = title which evaluates to where title = title which, being always true, has no effect on the results.
Depending on how the optimiser handles such queries, you may take a performance hit. But probably not in a modern database.
(Though similar, this problem is not the same as the IN (?, ?, ?) clause problem where you don't know the number of values in the list, since here you do have a fixed number of possible clauses and you just need to activate/disactivate them individually.)

I'm not confident if there is a quote() method, which was widely used in PHP's PDO. This would allow you a more flexible query building approach.
Also, one of the possible ideas could be creating special class, which would process filter criterias and would save into a stack all placeholders and their values.

Strange behaviour of ArrayList

I am writing a website using JSP, JSTL, Servlets and JavaBeans.
At one point of my code, I am trying to use an ArrayList of objects, and a strange thing is happening: when I add the first object it is fine, and when I add a second object it adds it in the second place, but the object at index(0) gets the same values as the object at index(1).
Maybe a problem is in the
ArrayList<Article> articleList = new ArrayList<Article>();
Article newArticle = new Article();
Since articleList is ArrayList of Article class.
Can somebody point me to what I am doing wrong?
Below is my code:
public ArrayList<Article> getArticles()
{
baseIO mySql = new baseIO();
ArrayList<Article> articleList = new ArrayList<Article>();
int articleId = 0;
try
{
String sql =
"select * from jsp_blog_article order by article_id Desc Limit 3";
con = (Connection)mySql.getConnection();
pstmt = (PreparedStatement) con.prepareStatement(sql);
ResultSet rs = pstmt.executeQuery();
while (rs.next()) {
Article newArticle = new Article();
newArticle.setArticleAuthor(rs.getString("article_name"));
newArticle.setArticleBody(rs.getString("article_body"));
newArticle.setArticleAuthor(rs.getString("article_author"));
newArticle.setArticleDate(rs.getString("article_date"));
articleId = Integer.parseInt(rs.getString("article_id"));
newArticle.setArticleId(String.valueOf(articleId));
newArticle.setArticleComments(this.getCommentsNum(articleId));
articleList.add(newArticle);
}
con.close();
pstmt.close();
}
catch(Exception e)
{
return null;
}
return articleList;
}
And the Article class
package objects;
import java.io.Serializable;
public class Article implements Serializable{
private String articleName;
private String articleBody;
private String articleAuthor;
private String articleComments;
private String articleDate;
private String articleId;
public Article()
{
}
// all the getters and setters in place, but it is too long
// so i am not going to post them in forum
}

I would try it this way and see what this does.
int x = 0;
while (rs.next()) {
articleList.add(new Article());
articleList.get(x).setArticleName(rs.getString("article_name"));
articleList.get(x).setArticleBody(rs.getString("article_body"));
articleList.get(x).setArticleAuthor(rs.getString("article_author"));
articleList.get(x).setArticleDate(rs.getString("article_date"));
articleList.get(x).setArticleId(rs.getString("article_id"));
articleList.get(x).setArticleComments(this.getCommentsNum(articleId));
x++;
}

You are calling newArticle.setArticleAuthor twice...I know that's not part of your list problem, but that is an observation.

The code should be cleaned up per the other comments, but functionally, it should work.
Here's what I think is happening.
Your code has the following two lines in it:
newArticle.setArticleAuthor(rs.getString("article_name"));
newArticle.setArticleAuthor(rs.getString("article_author"));
and there is no corresponding call to:
newArticle.setArticleName(rs.getString("article_name"));
this means that your object has no article name specified (even though the author is specified). I'll bet that you are then doing some sort of processing before you display the list that somehow merges articles with the same name.
As a general approach to debugging, I recommend that you mock up your code so you can run it in a debugger and see what's actually going on (right now, your system has so many moving parts that it's going to be difficult for you to hone in on the actual problem).
In the current case, this would be as simple as running the one method outside of your web container, and using a debugger to take a look at the objects in the list that gets returned. You will find that the objects in the list are, indeed, separate objects - just having the same articleName property.

Code looks fine, how are you displaying the list that makes you think the same value is at both indexes? Perhaps your problem is with that code.

Are you adding articles to the database concurrently while reading them? I think that it is possible, depending on your storage engine, that you'd have problems reading while updating is going on.