Related
I wrote a piece of JDBC template code, which inserts the record in the table, but the problem is my execution is stuck on this particular snippet, it seems some kind of hang up. I didn't figure out the cause as query properly running in sqldeveloper
List<SalaryDetailReport> reports = salaryDetailReportDAO.findAll(tableSuffix, regionId, circleId);
// the above line find the required data, if data is found then it proceeds
if (reports != null && reports.size() > 0) {
for (SalaryDetailReport salaryDetail : reports) {
try {
SalaryDetail sd = new SalaryDetail();
sd.setDetailReport(salaryDetail);
salaryDetailDAO.save(sd, tableSuffix);
} catch (Exception e) {
log.error("Error occured", e);
e.printStackTrace();
throw new MyExceptionHandler(" Error :" + e.getMessage());
}
}
System.out.println("data found");
} else {
log.error("Salary Record Not Found.");
throw new MyExceptionHandler("No record Found.");
}
You people saw try-catch , my execution stuck inside try and catch and here is the insertion code in my implementation class. when i commented the above code then my application works fine, but why my application stuck here, I am not able to figure it out, kindly help me
#Override
public void save(SalaryDetail details, String tableSuffix) {
String tabName = "SALARY_DETAIL_" + tableSuffix;
// String q = "INSERT INTO " + tabName + "(ID "
String q = "INSERT INTO SALARY_DETAIL_TBL "
+ " (ID "
+ " ,EMP_NAME "
+ " ,EMP_CODE "
+ " ,NET_SALARY "
+ " ,YYYYMM "
+ " ,PAY_CODE "
+ " ,EMP_ID "
+ " ,PAY_CODE_DESC "
+ " ,REMARK "
+ " ,PAY_MODE ) "
+ " (SELECT (sd.SALARY_REPORT_ID) ID "
+ " ,(sd.emp_name) emp_name "
+ " ,(sd.EMP_CODE) EMP_CODE "
+ " ,(sd.amount) NET_SALARY "
+ " ,(sd.YYYYMM) YYYYMM "
+ " ,(sd.pay_code) pay_code "
+ " ,(sd.emp_id) emp_id "
+ " ,(sd.PAY_CODE_DESC) PAY_CODE_DESC "
+ " ,(sd.REMARK) REMARK "
+ " ,(sd.PAY_MODE)PAY_MODE "
// + " FROM SALARY_DETAIL_REPORT_" + tableSuffix + " sd "
+ " FROM SALARY_DETAIL_REPORT_TBL sd "
+ " WHERE sd.PAY_CODE = 999 "
+ " AND sd.EMP_ID IS NOT NULL "
// + " AND sd.EMP_ID NOT IN (SELECT EMP_ID FROM SALARY_DETAIL_" + tableSuffix + ") "
+ " AND sd.EMP_ID NOT IN (SELECT EMP_ID FROM SALARY_DETAIL_TBL) "
+ " ) ";
MapSqlParameterSource param = new MapSqlParameterSource();
param.addValue("id", details.getId());
param.addValue("EMP_NAME", details.getEmpName());
param.addValue("EMP_CODE", details.getEmpCode());
param.addValue("NET_SALARY", details.getNetSalary());
param.addValue("GROSS_EARNING", details.getGrossEarning());
param.addValue("GROSS_DEDUCTION", details.getGrossDeduction());
param.addValue("YYYYMM", details.getYyyymm());
param.addValue("EMP_ID", details.getEmployee() != null ? details.getEmployee().getEmpId() : null);
KeyHolder keyHolder = new GeneratedKeyHolder();
getNamedParameterJdbcTemplate().update(q, param);
// details.setId(((BigDecimal) keyHolder.getKeys().get("ID")).longValue());
}
The main problem is in your query is Not In condition. It will degrade your performance. Try to fetch the "SELECT EMP_ID FROM SALARY_DETAIL_TB" in a separate query and pass in the Not in block in the main query. This will increase the performance of your query. Every time a save is performed this will fire the select query every time.
You have to decide whether you will insert records from SELECT or from the application.
If you don't need to manipulate with data after their select then you can simply call one INSERT INTO SELECT statement without any for cycle. It will be fast because of the only one INSERT statement call.
So you will implement method like copyAllInSalaryDetail(tableSuffix, regionId, circleId) in your SalaryDetailReportDAO and that method will execute INSERT INTO salary_detail_tbl... (...) (SELECT ... WHERE ...) using the same WHERE condition as you have in findAll() method. All inserts will be done only on the Database layer.
If you want to manipulate with data before their insert you can continue with your approach using SalaryDetail bean and for cycle, but you should remove the SELECT part from the INSERT statement and use values from the provided bean. Then the save() method can look like:
#Override
public void save(SalaryDetail details, String tableSuffix) {
// use tableSuffix if it is really needed
String q = "INSERT INTO SALARY_DETAIL_TBL "
+ " (ID "
+ " ,EMP_NAME "
+ " ,EMP_CODE "
+ " ,NET_SALARY "
+ " ,YYYYMM "
+ " ,PAY_CODE "
+ " ,EMP_ID "
+ " ,PAY_CODE_DESC "
+ " ,REMARK "
+ " ,PAY_MODE ) "
+ " VALUES (:id "
+ " ,:emp_name "
+ " ,:emp_code "
+ " ,:net_salary "
+ " ,:yyyymm "
+ " ,:pay_code "
+ " ,:emp_id "
+ " ,:pay_code_desc "
+ " ,:remark "
+ " ,:pay_mode)";
MapSqlParameterSource param = new MapSqlParameterSource();
// KeyHolder keyHolder = new GeneratedKeyHolder();
// details.setId(((BigDecimal) keyHolder.getKeys().get("ID")).longValue());
param.addValue("id", details.getId());
param.addValue("emp_name", details.getEmpName());
param.addValue("emp_code", details.getEmpCode());
param.addValue("net_salary", details.getNetSalary());
param.addValue("pay_code", details.getPayCode());
param.addValue("pay_code_desc", details.getPayCodeDesc());
param.addValue("pay_mode", details.getPayMode());
param.addValue("remark", details.getPayRemark());
param.addValue("yyyymm", details.getYyyymm());
param.addValue("emp_id", details.getEmployee() != null ? details.getEmployee().getEmpId() : null);
getNamedParameterJdbcTemplate().update(q, param);
}
This code serves to update a customer's data in sql. How I can simplify this code? Is there another way to do this?
if (!clienteOld.getNome().equalsIgnoreCase(clienteNew.getNome())) {
stmt.executeUpdate("UPDATE CLIENTES SET NOME = '" + clienteNew.getNome() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- NOME=" + clienteNew.getNome());
}
if (!clienteOld.getDataNascimento().equalsIgnoreCase(clienteNew.getDataNascimento())) {
stmt.executeUpdate("UPDATE CLIENTES SET DATA_NASCI = '" + clienteNew.getDataNascimento() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- DATA NASCIMENTO=" + clienteNew.getDataNascimento());
}
if (!clienteOld.getMorada().equalsIgnoreCase(clienteNew.getMorada())) {
stmt.executeUpdate("UPDATE CLIENTES SET MORADA = '" + clienteNew.getMorada() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- MORADA=" + clienteNew.getMorada());
}
if (!clienteOld.getPais().equalsIgnoreCase(clienteNew.getPais())) {
stmt.executeUpdate("UPDATE CLIENTES SET PAIS = '" + clienteNew.getPais() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- PAIS=" + clienteNew.getPais());
}
if (!clienteOld.getNacionalidade().equalsIgnoreCase(clienteNew.getNacionalidade())) {
stmt.executeUpdate("UPDATE CLIENTES SET NACIONALIDADE = '" + clienteNew.getNacionalidade() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- NACIONALIDADE=" + clienteNew.getNacionalidade());
}
if (!clienteOld.getBI().equalsIgnoreCase(clienteNew.getBI())) {
stmt.executeUpdate("UPDATE CLIENTES SET BI = '" + clienteNew.getBI() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- BI=" + clienteNew.getBI());
}
if (!clienteOld.getTipoIndentificaçao().equalsIgnoreCase(clienteNew.getTipoIndentificaçao())) {
stmt.executeUpdate("UPDATE CLIENTES SET TIPO_IDENT = '" + clienteNew.getTipoIndentificaçao() + "' WHERE ID = " + id1 + ";");
criarLog("Ficha do cliente: " + clienteOld.getNome() + " foi atualizada -- TIPO IDENTIFICAÇAO=" + clienteNew.getTipoIndentificaçao());
}
Try this logic:
StringBuilder sql = new StringBuilder("UPDATE CLIENTES SET ");
Map<String, String> cols = new HashMap<>();
if (!clienteOld.getNome().equalsIgnoreCase(clienteNew.getNome())) {
cols.put("NOME", clienteNew.getNome());
}
if (!clienteOld.getDataNascimento().equalsIgnoreCase(clienteNew.getDataNascimento())) {
cols.put("DATA_NASCI", clienteNew.getDataNascimento());
}
// and the other if statements
Then you can iterate the map and build your actual update statement:
int cnt = 0;
for (Map.Entry<Integer, Integer> entry : cols.entrySet()) {
if (cnt > 0) sql.append(", ");
sql.append(entry.getKey()).append(" = '").append(entry.getValue()).append("'");
++cnt;
}
sql.append(" WHERE ID = ").append(id1).append(";");
But note that this approach is not SQL injection safe. If these values are coming from the outside, e.g. a UI, then you should absolutely be using a prepared statement. There is nothing inherently wrong with using a separate statement for each if condition. I only answered to show that you can cleanup your current approach, should it be appropriate.
if (!oldClient.getName().equalsIgnoreCase(newClient.getName())) {
stmt.executeUpdate("UPDATE CLIENTS SET NAME = '" + newClient.getName() +
"' WHERE ID = " + id1 + ";");
}
if (!oldClient.getBirthDate().equalsIgnoreCase(newClient.getBirthDate())) {
stmt.executeUpdate("UPDATE CLIENTS SET BIRTH = '" + newClient.getBirthDate() +
"' WHERE ID = " + id1 + ";");
}
can be rewritten as
if (!oldClient.getName().equalsIgnoreCase(newClient.getName()) ||
!oldClient.getBirthDate().equalsIgnoreCase(newClient.getBirthDate())) {
stmt.executeUpdate("UPDATE CLIENTS SET NAME = '" + newClient.getName() +
"', BIRTH = '" + newClient.getBirthDate() +
"' WHERE ID = " + id1 + ";");
}
This will perform better because it executes one SQL statement instead of two. The fact that you are possibly setting two columns when only one needs to be set is probably of little consequence, compared with that.
Notes:
If you try to "optimize" the number of columns set, the code is more complicated; see Tim's answer.
This should probably be done with a PreparedStatement and statement parameters to avoid SQL injection. If you follow the above pattern, the changes needed to use a PreparedStatement are straight forward.
First, you can use one query to update every fields :
"UPDATE CLIENTES SET DATA_NASCI "
+ "MORADA = '" + clienteNew.getMorada() + "'"
+ "PAIS = '" + clienteNew.getPais() + "'"
+ "NACIONALIDADE = '" + clienteNew.getNacionalidade() + "'"
+ "BI = '" + clienteNew.getBI() + "'"
+ "TIPO_IDENT = '" + clienteNew.getTipoIndentificaçao() + "'"
+ "WHERE ID = " + id1 + ";"
Please use a PreparedStatement instead of this, this would be much safer !
All you have to do is check if only one field have changed (to prevent the transaction doing nothing) using a condition check each field :
if (!clienteOld.getNome().equalsIgnoreCase(clienteNew.getNome()))
|| (clienteOld.getDataNascimento().equalsIgnoreCase(clienteNew.getDataNascimento()))
|| (!clienteOld.getMorada().equalsIgnoreCase(clienteNew.getMorada()))
|| (!clienteOld.getPais().equalsIgnoreCase(clienteNew.getPais()))
|| (!clienteOld.getNacionalidade().equalsIgnoreCase(clienteNew.getNacionalidade()))
|| (!clienteOld.getBI().equalsIgnoreCase(clienteNew.getBI()))
|| (!clienteOld.getTipoIndentificaçao().equalsIgnoreCase(clienteNew.getTipoIndentificaçao()))
This can be a bit verbose ... so why not reduce this code a bit using Stream and some function references to compare and build a map of value to update :
First, the class Bean for our example :
class Bean {
String firstname, lastname;
public Bean(String firstname, String lastname) {
this.firstname = firstname;
this.lastname = lastname;
}
public String getFirstname() {
return firstname;
}
public String getLastname() {
return lastname;
}
}
Then, let's create a mapping of Column name and Function, the function will allow us to use the getters of Bean :
Map<String, Function<Bean, String>> functions = new HashMap<>();
functions.put("FIRSTNAME", Bean::getFirstname);
functions.put("LASTNAME", Bean::getLastname);
Then, using final instance (to be used in a Predicate)
final Bean clienteNew = new Bean("Foo", "Bar");
final Bean clienteOld = new Bean("Foo", "Boo");
Map<String, String> values = functions.entrySet()
.stream()
//filter only the value that changed between clienteOld and clienteNew
.filter(entry -> !entry.getValue().apply(clienteOld).equals(entry.getValue().apply(clienteNew)))
//then collect the map `name -> new value`
.collect(Collectors.toMap(e -> e.getKey(), e -> e.getValue().apply(clienteNew)));
System.out.println(values);
{LASTNAME=Bar}
This will give you a Map<String, String> that can be used to create a PreparedStatement with only the column we want/need to edit.
If you have a new field, just need to add the mapping in the functions map and you are good to go. (this become a bit more complex with primitive type...)
I have got a webapp(JSP/Servlet) with Tomcat8 + SQL Server2012
JDBC Driver Type 4: JTDS old version 1.2.5 (http://jtds.sourceforge.net/)
I change this kind of query, adding Prepared Statement (server pagining)
Sting DDXsql = "SELECT '?' *, ( DDX_RECORD_COUNT / '?' + 1 ) AS DDX_PAGE_COUNT
FROM
( SELECT '?' *
FROM ( SELECT '?' *,
(SELECT COUNT(*) " + "FROM "
+ session.getAttribute("DatabaseName") + ".G1_grid "
+ sqlFrom
+ sqlWhere + " "
+ " ) AS DDX_RECORD_COUNT "
+ "FROM " + session.getAttribute("DatabaseName") + ".G1_grid "
+ sqlFrom
+ sqlWhere + " "
+ " ORDER BY '?' '?' , '?' '?' ) AS TMP1 ORDER
BY '?' '?', '?' '?') AS r ORDER BY '?' '?', '?' '?'";
Parameters:
String top1 = DBManager.getTOP(request, "TOP " + Integer.valueOf((String)ResourceManager.findData("pageSize", request)));
Integer pagesizeInt = Integer.valueOf((String)ResourceManager.findData("pageSize", request));
String top2 = DBManager.getTOP(request, "TOP " + Integer.valueOf((String)ResourceManager.findData("ddxrecordcount", request)));
String top3= DBManager.getTOP(request, "TOP " + Integer.valueOf((String)ResourceManager.findData("toRange", request)));
String notSortStr = (String)ResourceManager.findData("notSort", request);
Object[] values = new Object[] {
top1,
pagesizeInt,
top2,
top3,
SortKey,
Sort,
TotalSortKey,
Sort,
SortKey,
notSortStr,
TotalSortKey ,
notSortStr,
SortKey,
Sort,
TotalSortKey,
Sort
};
Before, I didint use PreparedStatement I have this kind of query (replace "?" with the Object array values, without StringEscapeUtils):
String DDXsql = "SELECT " +
DBManager.getTOP(request, "TOP "
+ Integer.valueOf(StringEscapeUtils.escapeSql((String)ResourceManager.findData("pageSize", request)))) + " *,
( DDX_RECORD_COUNT / " + Integer.valueOf(StringEscapeUtils.escapeSql((String)ResourceManager.findData("pageSize", request))) + " + 1 ) AS DDX_PAGE_COUNT FROM
( SELECT "
+ DBManager.getTOP(request, "TOP "
+ Integer.valueOf(StringEscapeUtils.escapeSql((String)ResourceManager.findData("ddxrecordcount", request))))
+ " * FROM ( SELECT " + DBManager.getTOP(request, "TOP " + Integer.valueOf(StringEscapeUtils.escapeSql((String)ResourceManager.findData("toRange", request))))
+ " *, (SELECT COUNT(*) "
+ "FROM " + session.getAttribute("DatabaseName") + ".G1_grid " + sqlFrom + sqlWhere + " " + " ) AS DDX_RECORD_COUNT "
+ "FROM " + session.getAttribute("DatabaseName")
+ ".G1_grid " + sqlFrom + sqlWhere + " " + " ORDER BY "
+ StringEscapeUtils.escapeSql(SortKey) + " " + StringEscapeUtils.escapeSql(Sort) + ", "
+ StringEscapeUtils.escapeSql(TotalSortKey) + " "
+ StringEscapeUtils.escapeSql(Sort) + ") AS TMP1 ORDER BY "
+ StringEscapeUtils.escapeSql(SortKey) + " "
+ StringEscapeUtils.escapeSql((String)ResourceManager.findData("notSort", request))
+ ", " + StringEscapeUtils.escapeSql(TotalSortKey) + " "
+ StringEscapeUtils.escapeSql((String)ResourceManager.findData("notSort", request)) + " ) AS r ORDER BY "
+ StringEscapeUtils.escapeSql(SortKey) + " "
+ StringEscapeUtils.escapeSql(Sort) + ", "
+ StringEscapeUtils.escapeSql(TotalSortKey)
+ " " + StringEscapeUtils.escapeSql(Sort) + " ";
The last query runs without error, System.out of this query give this for example:
SELECT TOP 20 *, ( DDX_RECORD_COUNT / 20 + 1 ) AS DDX_PAGE_COUNT
FROM
( SELECT TOP 20 * FROM
( SELECT TOP 20 *,
(SELECT COUNT(*)
FROM SuiteMA_DIP.dbo.G1_grid
WHERE 1 = 1 ) AS DDX_RECORD_COUNT
FROM SuiteMA_DIP.dbo.G1_grid WHERE 1 = 0 ORDER BY DATA_ISCRIZIONE_ORDER DESC, SOGGETTO_RILEVANTE_PAID DESC) AS TMP1 ORDER BY DATA_ISCRIZIONE_ORDER ASC, SOGGETTO_RILEVANTE_PAID ASC ) AS r ORDER BY DATA_ISCRIZIONE_ORDER DESC, SOGGETTO_RILEVANTE_PAID DESC
But when i run sql with preparedStatement:
java.sql.SQLException: Invalid parameter index 1.
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.getParameter(JtdsPreparedStatement.java:340)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setParameter(JtdsPreparedStatement.java:409)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setObjectBase(JtdsPreparedStatement.java:395)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.setObject(JtdsPreparedStatement.java:667)
at org.apache.tomcat.dbcp.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:188)
at org.apache.tomcat.dbcp.dbcp2.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:188)
at it.netbureau.jfx.db.SQLDBManager.execSQL(SQLDBManager.java:57)
at it.netbureau.jfx.db.SQLDBManager.execSQL(SQLDBManager.java:78)
at org.apache.jsp.G1.select_jsp._jspService(select_jsp.java:691)
The java method execute the query :
class jfx.db.SQLDBManager.execSQL:
public Object execSQL(PreparedStatement stmt, Object values[], String xmlId)
throws SQLException
{
Object result = null;
if(stmt == null)
return null;
try
{
for(int i = 0; i < values.length; i++)
if(values[i] == null)
stmt.setNull(i + 1, 4);
else
stmt.setObject(i + 1, values[i]); <--this give exception!
if(stmt.execute()) result = transform(stmt.getResultSet(), xmlId);
}
catch(SQLException ex)
{
rollback();
throw ex;
}
return result;
}
What's wrong?
Thank you very much
roby
Your query does not contain any parameters, a '?' is just a literal string with a question mark in it, it is not a parameter.
You also can't parameterize object names like column names and clauses (like a TOP 20), so even if you'd change it to - for example - order by ?, ... it wouldn't work, as you'd be sorting by the string value (which would be the same for all rows, so effectively you wouldn't be sorting at all).
To do what you want to do you will need to concatenate the column names (and other clauses) into the query string. This also means that you might open yourself up to SQL injection: be sure to check the values carefully (for example against a whitelist of allowed values).
public boolean saveHighScore(Client c) throws SQLException {
long totallvlexp = (long) (c.playerXP[0]) + (c.playerXP[1]) + (c.playerXP[2]) + (c.playerXP[3]) + (c.playerXP[4]) + (c.playerXP[5]) + (c.playerXP[6]) + (c.playerXP[7]) + (c.playerXP[8]) + (c.playerXP[9]) + (c.playerXP[10]) + (c.playerXP[11]) + (c.playerXP[12]) + (c.playerXP[13]) + (c.playerXP[14]) + (c.playerXP[15]) + (c.playerXP[16]) + (c.playerXP[17]) + (c.playerXP[18]) + (c.playerXP[19]) + (c.playerXP[20]);
int totallevell = (int) (c.getLevelForXP(c.playerXP[0]) + c.getLevelForXP(c.playerXP[1]) + c.getLevelForXP(c.playerXP[2]) + c.getLevelForXP(c.playerXP[3]) + c.getLevelForXP(c.playerXP[4]) + c.getLevelForXP(c.playerXP[5]) + c.getLevelForXP(c.playerXP[6]) + c.getLevelForXP(c.playerXP[7]) + c.getLevelForXP(c.playerXP[8]) + c.getLevelForXP(c.playerXP[9]) + c.getLevelForXP(c.playerXP[10]) + c.getLevelForXP(c.playerXP[11]) + c.getLevelForXP(c.playerXP[12]) + c.getLevelForXP(c.playerXP[13]) + c.getLevelForXP(c.playerXP[14]) + c.getLevelForXP(c.playerXP[15]) + c.getLevelForXP(c.playerXP[16]) + c.getLevelForXP(c.playerXP[17]) + c.getLevelForXP(c.playerXP[18]) + c.getLevelForXP(c.playerXP[19]) + c.getLevelForXP(c.playerXP[20]));
String delQuery1 = "DELETE FROM `skills` WHERE playerName = '"+c.playerName+"';";
String delQuery2 = "DELETE FROM `skillsoverall` WHERE playerName = '"+c.playerName+"';";
String delQuery3 = "DELETE FROM `playerrights` WHERE playerName = '"+c.playerName+"';";
String insQuery1 = "INSERT INTO `skills` (`playerName`,`Attacklvl`,`Attackxp`,`Defencelvl`,`Defencexp`,`Strengthlvl`,`Strengthxp`,`Hitpointslvl`,`Hitpointsxp`,`Rangelvl`,`Rangexp`,`Prayerlvl`,`Prayerxp`,`Magiclvl`,`Magicxp`,`Cookinglvl`,`Cookingxp`,`Woodcuttinglvl`,`Woodcuttingxp`,`Fletchinglvl`,`Fletchingxp`,`Fishinglvl`,`Fishingxp`,`Firemakinglvl`,`Firemakingxp`,`Craftinglvl`,`Craftingxp`,`Smithinglvl`,`Smithingxp`,`Mininglvl`,`Miningxp`,`Herblorelvl`,`Herblorexp`,`Agilitylvl`,`Agilityxp`,`Thievinglvl`,`Thievingxp`,`Slayerlvl`,`Slayerxp`,`Farminglvl`,`Farmingxp`,`Runecraftlvl`,`Runecraftxp`) VALUES ('"+c.playerName+"',"+c.playerLevel[0]+","+c.playerXP[0]+","+c.playerLevel[1]+","+c.playerXP[1]+","+c.playerLevel[2]+","+c.playerXP[2]+","+c.playerLevel[3]+","+c.playerXP[3]+","+c.playerLevel[4]+","+c.playerXP[4]+","+c.playerLevel[5]+","+c.playerXP[5]+","+c.playerLevel[6]+","+c.playerXP[6]+","+c.playerLevel[7]+","+c.playerXP[7]+","+c.playerLevel[8]+","+c.playerXP[8]+","+c.playerLevel[9]+","+c.playerXP[9]+","+c.playerLevel[10]+","+c.playerXP[10]+","+c.playerLevel[11]+","+c.playerXP[11]+","+c.playerLevel[12]+","+c.playerXP[12]+","+c.playerLevel[13]+","+c.playerXP[13]+","+c.playerLevel[14]+","+c.playerXP[14]+","+c.playerLevel[15]+","+c.playerXP[15]+","+c.playerLevel[16]+","+c.playerXP[16]+","+c.playerLevel[17]+","+c.playerXP[17]+","+c.playerLevel[18]+","+c.playerXP[18]+","+c.playerLevel[19]+","+c.playerXP[19]+","+c.playerLevel[20]+","+c.playerXP[20]+");";
String insQuery2 = "INSERT INTO `skillsoverall` (`playerName`,`lvl`,`xp`) VALUES ('"+c.playerName+"',"+totallevell+","+totallvlexp+");";
String insQuery3 = "INSERT INTO `playerrights` (`playerName`,`rank`) VALUES ('"+c.playerName+"',"+c.playerRights+");";
String[] delQueries = {delQuery1, delQuery2, delQuery3};
String[] insQueries = {insQuery1, insQuery2, insQuery3};
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
try(Connection conX = DriverManager.getConnection("jdbc:mysql://localhost/highscores","root","root")) {
try(Statement stmtX = conX.createStatement()) {
for(int i = 0; i < delQueries.length; i++)
{
stmtX.executeUpdate(delQueries[i]);
stmtX.executeUpdate(insQueries[i]);
}
} catch (Exception e) {
e.printStackTrace();
return false;
}
} catch (Exception e) {
e.printStackTrace();
return false;
}
} catch (Exception e) {
e.printStackTrace();
return false;
}
return true;
}
}
Using this code how could I make it faster? If I used UPDATE instead would it make it faster? If it would please show me how to do that with the code I provided. I'm not very good with SQL, again just helping out a friend.
Some ways to improve this:
Use UPDATE statement and use the value of the primary key in the WHERE clause. This is in case you know the id. If you don't know it, then
Use UPDATE statement properly. Add a proper index on the columns you will use for the WHERE clause, in case is a single column and it's not the primary key and this statement may be executed several times.
Update the relevant columns, not all the columns (unless you are updating all the values for that row, which is something odd but not impossible).
Since you want/need to update everything or nothing, then you should use a transaction. You can start a transaction by using Connection#setAutoCommit(false); and close it by using Connection#commit. In case of errors, use Connection#rollback() and none of the operations will affect the database.
Stop using Statement so naively. If your queries need to be parameterized, then use PreparedStatement instead.
In your database, do this:
ALTER TABLE skills ADD INDEX idx_playerName (playerName);
Here's part of your Java code updated using the statements above:
public boolean saveHighScore(Client c) throws SQLException {
long totallvlexp = (long) (c.playerXP[0]) + (c.playerXP[1]) + (c.playerXP[2]) + (c.playerXP[3]) + (c.playerXP[4]) + (c.playerXP[5]) + (c.playerXP[6]) + (c.playerXP[7]) + (c.playerXP[8]) + (c.playerXP[9]) + (c.playerXP[10]) + (c.playerXP[11]) + (c.playerXP[12]) + (c.playerXP[13]) + (c.playerXP[14]) + (c.playerXP[15]) + (c.playerXP[16]) + (c.playerXP[17]) + (c.playerXP[18]) + (c.playerXP[19]) + (c.playerXP[20]);
int totallevell = (int) (c.getLevelForXP(c.playerXP[0]) + c.getLevelForXP(c.playerXP[1]) + c.getLevelForXP(c.playerXP[2]) + c.getLevelForXP(c.playerXP[3]) + c.getLevelForXP(c.playerXP[4]) + c.getLevelForXP(c.playerXP[5]) + c.getLevelForXP(c.playerXP[6]) + c.getLevelForXP(c.playerXP[7]) + c.getLevelForXP(c.playerXP[8]) + c.getLevelForXP(c.playerXP[9]) + c.getLevelForXP(c.playerXP[10]) + c.getLevelForXP(c.playerXP[11]) + c.getLevelForXP(c.playerXP[12]) + c.getLevelForXP(c.playerXP[13]) + c.getLevelForXP(c.playerXP[14]) + c.getLevelForXP(c.playerXP[15]) + c.getLevelForXP(c.playerXP[16]) + c.getLevelForXP(c.playerXP[17]) + c.getLevelForXP(c.playerXP[18]) + c.getLevelForXP(c.playerXP[19]) + c.getLevelForXP(c.playerXP[20]));
boolean result = true;
String updateSkillsSql = "UPDATE skills"
+ " SET `Attacklvl`=?,"
+ " `Attackxp`=?,"
+ " `Defencelvl`=?,"
+ " `Defencexp`=?,"
+ " `Strengthlvl`=?,"
+ " `Strengthxp`=?,"
+ " `Hitpointslvl`=?,"
+ " `Hitpointsxp`=?,"
+ " `Rangelvl`=?,"
+ " `Rangexp`=?,"
+ " `Prayerlvl`=?,"
+ " `Prayerxp`=?,"
+ " `Magiclvl`=?,"
+ " `Magicxp`=?,"
+ " `Cookinglvl`=?,"
+ " `Cookingxp`=?,"
+ " `Woodcuttinglvl`=?,"
+ " `Woodcuttingxp`=?,"
+ " `Fletchinglvl`=?,"
+ " `Fletchingxp`=?,"
+ " `Fishinglvl`=?,"
+ " `Fishingxp`=?,"
+ " `Firemakinglvl`=?,"
+ " `Firemakingxp`=?,"
+ " `Craftinglvl`=?,"
+ " `Craftingxp`=?,"
+ " `Smithinglvl`=?,"
+ " `Smithingxp`=?,"
+ " `Mininglvl`=?,"
+ " `Miningxp`=?,"
+ " `Herblorelvl`=?,"
+ " `Herblorexp`=?,"
+ " `Agilitylvl`=?,"
+ " `Agilityxp`=?,"
+ " `Thievinglvl`=?,"
+ " `Thievingxp`=?,"
+ " `Slayerlvl`=?,"
+ " `Slayerxp`=?,"
+ " `Farminglvl`=?,"
+ " `Farmingxp`=?,"
+ " `Runecraftlvl`=?,"
+ " `Runecraftxp`=?"
+ " WHERE playerName = ?";
//do similar for the other queries...
//not needed since JDBC 4, noted since you're using Java 7
//Class.forName("com.mysql.jdbc.Driver").newInstance();
try(Connection con = DriverManager.getConnection("jdbc:mysql://localhost/highscores","root","root")) {
//when storing multiple data, it's better to use a transaction
con.setAutoCommit(false);
try(PreparedStatement pstmt = con.prepareStatement(updateSkillsSql);
//declare the other PreparedStatements for each update sql statement here...
) {
//do something like this for every PreparedStatement
setParameters(pstmt,
c.playerLevel[0], c.playerXP[0],
c.playerLevel[1], c.playerXP[1],
c.playerLevel[2], c.playerXP[2],
c.playerLevel[3], c.playerXP[3],
c.playerLevel[4], c.playerXP[4],
c.playerLevel[5], c.playerXP[5],
c.playerLevel[6], c.playerXP[6],
c.playerLevel[7], c.playerXP[7],
c.playerLevel[8], c.playerXP[8],
c.playerLevel[9], c.playerXP[9],
c.playerLevel[10], c.playerXP[10],
c.playerLevel[11], c.playerXP[11],
c.playerLevel[12], c.playerXP[12],
c.playerLevel[13], c.playerXP[13],
c.playerLevel[14], c.playerXP[14],
c.playerLevel[15], c.playerXP[15],
c.playerLevel[16], c.playerXP[16],
c.playerLevel[17], c.playerXP[17],
c.playerLevel[18], c.playerXP[18],
c.playerLevel[19], c.playerXP[19],
c.playerLevel[20], c.playerXP[20],
c.playerName);
pstmt.executeUpdate();
} catch (Exception e) {
System.out.println(String.format("There's a problem when saving the data of player %s.", c.playerName));
e.printStackTrace(System.out);
con.rollback();
result = false;
}
if (result) {
con.commit();
con.setAutoCommit(true);
}
} catch (Exception e) {
System.out.println(String.format("There's a problem when saving the data of player %s.", c.playerName));
e.printStackTrace(System.out);
result = false;
}
return result;
}
//created method to add parameters despite its type
private void setParameters(PreparedStatement pstmt, Object ... args) {
int i = 1;
for (Object arg : args) {
pstmt.setObject(i++, arg);
}
}
The biggest cost in either your DELETE / INSERT approach or an UPDATE approach is probably your WHERE clause.
Doing the UPDATE or DELETE with a WHERE pointing at playerName = 'FriendlyNameOfPlayer' is brutal. You could make it a little better if you created an index on that field.
You'd create the index with something along the lines of...
CREATE INDEX playerName_ix ON skills (playerName)
CREATE INDEX playerName_ix ON skillsoverall (playerName)
Read more on some additional options that may be relevant on a case by case basis here.
A better practice would be to target the rows by their PK (maybe an auto-incrementing integer?). Ideally, the schema behind this would be a table called Player_Master with an autoincrementing integer as the PK, then the friendly name of the player stored in another column. Tables like those referenced in your code snippit should be storing the PK values from Player_Master (instead of the friendly name), which would be FK's. Your DELETE statement would instead look like...
DELETE FROM `skills` WHERE skills.playerID = 37
Redoing the schema would be the "correct" way to do it, but I get that every project may not have the time or resources required for a total teardown / redo. If your buddy is saying "can you make this faster?", you could do the index and walk away. If the question were "how do I do this right?", I'd start the clock on billable hours, because from the small glimpse into the code and schema provided, it's not a quick fix.
I'll also wristslap you for calling raw executes of cobbled together TSQL statements. There are better ways to do this.
I was wondering if someone here could help me, I can't find a solution for my problem and I have tried everything.
What I am trying to do is read and parse lines in a csv file into java objects and I have succeeded in doing that but after it reads all the lines it should insert the lines into the database but it only inserts the 1st line the entire time and I don't no why. When I do a print it shows that it is reading all the lines and placing them in the objects but as soon as I do the insert it wants to insert only the 1st line.
Please see my code below:
public boolean lineReader(File file){
BufferedReader br = null;
String line= "";
String splitBy = ",";
storeList = new ArrayList<StoreFile>();
try {
br = new BufferedReader(new FileReader(file));
while((line = br.readLine())!=null){
line = line.replace('|', ',');
//split on pipe ( | )
String[] array = line.split(splitBy, 14);
//Add values from csv to store object
//Add values from csv to storeF objects
StoreFile StoreF = new StoreFile();
if (array[0].equals("H") || array[0].equals("T")) {
return false;
} else {
StoreF.setRetailID(array[1].replaceAll("/", ""));
StoreF.setChain(array[2].replaceAll("/",""));
StoreF.setStoreID(array[3].replaceAll("/", ""));
StoreF.setStoreName(array[4].replaceAll("/", ""));
StoreF.setAddress1(array[5].replaceAll("/", ""));
StoreF.setAddress2(array[6].replaceAll("/", ""));
StoreF.setAddress3(array[7].replaceAll("/", ""));
StoreF.setProvince(array[8].replaceAll("/", ""));
StoreF.setAddress4(array[9].replaceAll("/", ""));
StoreF.setCountry(array[10].replaceAll("/", ""));
StoreF.setCurrency(array[11].replaceAll("/", ""));
StoreF.setAddress5(array[12].replaceAll("/", ""));
StoreF.setTelNo(array[13].replaceAll("/", ""));
//Add stores to list
storeList.add(StoreF);
}
} //print list stores in file
printStoreList(storeList);
executeStoredPro(storeList);
} catch (Exception ex) {
nmtbatchservice.NMTBatchService2.LOG.error("An exception accoured: " + ex.getMessage(), ex);
//copy to error folder
//email
}
return false;
}
public void printStoreList(List<StoreFile> storeListToPrint) {
for(int i = 0; i <storeListToPrint.size();i++){
System.out.println( storeListToPrint.get(i).getRetailID()
+ storeListToPrint.get(i).getChain()
+ storeListToPrint.get(i).getStoreID()
+ storeListToPrint.get(i).getStoreName()
+ storeListToPrint.get(i).getAddress1()
+ storeListToPrint.get(i).getAddress2()
+ storeListToPrint.get(i).getAddress3()
+ storeListToPrint.get(i).getProvince()
+ storeListToPrint.get(i).getAddress4()
+ storeListToPrint.get(i).getCountry()
+ storeListToPrint.get(i).getCurrency()
+ storeListToPrint.get(i).getAddress5()
+ storeListToPrint.get(i).getTelNo());
}
}
public void unzip(String source, String destination) {
try {
ZipFile zipFile = new ZipFile(source);
zipFile.extractAll(destination);
deleteStoreFile(source);
} catch (ZipException ex) {
nmtbatchservice.NMTBatchService2.LOG.error("Error unzipping file : " + ex.getMessage(), ex);
}
}
public void deleteStoreFile(String directory) {
try {
File file = new File(directory);
file.delete();
} catch (Exception ex) {
nmtbatchservice.NMTBatchService2.LOG.error("An exception accoured when trying to delete file " + directory + " : " + ex.getMessage(), ex);
}
}
public void executeStoredPro(List<StoreFile> storeListToInsert) {
Connection con = null;
CallableStatement st = null;
try {
String connectionURL = MSSQLConnectionURL;
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver").newInstance();
con = DriverManager.getConnection(connectionURL, MSSQLUsername, MSSQLPassword);
for(int i = 0; i <storeListToInsert.size();i++){
st = con.prepareCall( "IF EXISTS (SELECT * FROM tblPay#RetailStores WHERE StoreID = " + storeListToInsert.get(i).getStoreID() + " AND RetailID = "+ storeListToInsert.get(i).getRetailID() + ")"
+ " UPDATE tblPay#RetailStores "
+ " SET RetailID = '" + storeListToInsert.get(i).getRetailID() + "',"
+ " StoreID = '" + storeListToInsert.get(i).getStoreID() + "',"
+ " StoreName = '" + storeListToInsert.get(i).getStoreName() + "',"
+ " TestStore = 0,"
+ " Address1 = '" + storeListToInsert.get(i).getAddress1() + "',"
+ " Address2 = '" + storeListToInsert.get(i).getAddress2() + "',"
+ " Address3 = '" + storeListToInsert.get(i).getAddress3() + "',"
+ " Address4 = '" + storeListToInsert.get(i).getAddress4() + "',"
+ " Address5 = '" + storeListToInsert.get(i).getAddress5() + "',"
+ " Province = '" + storeListToInsert.get(i).getProvince() + "',"
+ " TelNo = '" + storeListToInsert.get(i).getTelNo() + "',"
+ " Enabled = 1"
+ " ELSE "
+ " INSERT INTO tblPay#RetailStores ( [RetailID], [StoreID], [StoreName], [TestStore], [Address1], [Address2], [Address3], [Address4], [Address5], [Province], [TelNo] , [Enabled] ) "
+ " VALUES "
+ "('" + storeListToInsert.get(i).getRetailID() + "',"
+ "'" + storeListToInsert.get(i).getStoreID() + "',"
+ "'" + storeListToInsert.get(i).getStoreName() + "',"
+ "0,"
+ "'" + storeListToInsert.get(i).getAddress1() + "',"
+ "'" + storeListToInsert.get(i).getAddress2() + "',"
+ "'" + storeListToInsert.get(i).getAddress3() + "',"
+ "'" + storeListToInsert.get(i).getAddress4() + "',"
+ "'" + storeListToInsert.get(i).getAddress5() + "',"
+ "'" + storeListToInsert.get(i).getProvince() + "',"
+ "'" + storeListToInsert.get(i).getTelNo() + "',"
+ "1)");
st.executeUpdate();
}
con.close();
} catch (Exception ex) {
nmtbatchservice.NMTBatchService2.LOG.error("Error executing Stored proc with error : " + ex.getMessage(), ex);
nmtbatchservice.NMTBatchService2.mailingQueue.addToQueue(new Mail("support#nmt-it.co.za", "Service Email Error", "An error occurred during Store Import failed with error : " + ex.getMessage()));
}
}
Any advise would be appreciated.
Thanks
Formatting aside, your code is wrong (I truncated the part of the query):
for(int i = 0; i <storeListToInsert.size();i++){
st = con.prepareCall( "IF EXISTS (SELECT * FROM tblPay#RetailStores ...
+ "'" + storeListToInsert.get(i).getTelNo() + "',"
+ "1)");
st.executeUpdate();
}
Don't do a classical for loop while foreach exists and can be better to use, and even if you do a classical for loop, use local variables, eg:
for(int i = 0; i <storeListToInsert.size();i++){
StoreFile item = storeListToInsert.get(i);
st = con.prepareCall( "IF EXISTS (SELECT * FROM tblPay#RetailStores ...
+ "'" + item.getTelNo() + "',"
+ "1)");
st.executeUpdate();
}
Which could translate as:
for (StoreFile item : storeListToInsert) {
st = con.prepareCall( "IF EXISTS (SELECT * FROM tblPay#RetailStores ...
+ "'" + item.getTelNo() + "',"
+ "1)");
st.executeUpdate();
}
Now, the second problem is your PreparedStatement. A PreparedStatement allow reusing, which means you don't need to create PreparedStatement per item which is what you are doing.
Also, you need to close the statement otherwise, you will exhaust resources..
You must not create it in the for loop, but before, like this:
PreparedStatement st = null;
try {
st = con.prepareCall( "IF EXISTS (SELECT * FROM tblPay#RetailStores ...
+ "SET RetailID = :RetailID ,"
+ "1)");
for (StoreFile item : storeListToInsert) {
st.setString(":RetailID", item.getRetailID());
st.executeUpdate();
}
} finally {
if (null != st) {st.close();}
}
In brief:
You need to close the PreparedStatement after usage, because it is a memory leak otherwise.
You need to rewrite your query using either named parameters, either positional parameter (like: ? or ?1 for first parameter, and so on). I favor named parameters, but they are not always available. The example I linked all use positional parameters.
You need to set the value for each parameters in the for loop, and care about the type. I expected here that getRetailID() is a String, but it might be a Long in that case that would be st.setLong.
Your query is reusable, avoiding the need to reparse it/resend it to the SQL Server. You just send the parameter's values. Beside, you can also batch update.
A PreparedStatement for a statement that you generate (like you are doing) is overkill, and beside, it is missing SQL escapement to protect the String you inject to your query to avoid it being badly interpreted (aka SQL errors) or worst, to do what it was not intended for (like, even if it is far fetched, dropping the whole database, etc).
The executeUpdate() return the number of updated rows. You can check it to see if there was updates.
You can also use Batch statement, which can help performances.
And finally, you can use opencsv to parse common CSV files.