I had a situation where my code was getting hit by the deadlock issue with SQL Server for some transactions. So, I implemented a retry logic to overcome the same. Now, I'm facing a new problem. The problem is, whenever it retries, the batch which was tried to execute will be empty/cleared after recovering from the exception. This is causing missing data inserts/updates. Please help me with this.
Summary:
Retry the preparedstatement batches if an exception (SQLException | BatchUpdateException) occurs
Current Implementation:
do {
try {
if (isInsert) {
int[] insertRows = psInsert.executeBatch();
psInsert.clearBatch();
System.out.println("insertRowsSuccess:" + Arrays.toString(insertRows));
} else {
int[] updateRows = psUpdate.executeBatch();
psUpdate.clearBatch();
System.out.println("updateRowsSuccess:" + Arrays.toString(updateRows));
}
break;
} catch (BatchUpdateException e) {
conn.rollback();
if (++count == maxTries) {
System.out.println(e.getMessage());
getFailedRecords(e, operation);
}
} catch (SQLException e) {
if (++count == maxTries) {
System.err.format("SQL State: %s\n%s", e.getSQLState(), e.getMessage());
}
}
System.out.println("Tries:" + count);
} while (true);
private static void getFailedRecords(BatchUpdateException ex, String operation) {
int[] updateCount = ex.getUpdateCounts();
ArrayList<Integer> failedRecsList = new ArrayList<Integer>();
int failCount = 0;
for (int i = 0; i < updateCount.length; i++) {
if (updateCount[i] == Statement.EXECUTE_FAILED) {
failCount++;
failedRecsList.add(i);
}
}
System.out.println(operation + " Failed Count: " + failCount);
System.out.println(operation + " FailedRecordsIndex:" + failedRecsList);
}
After execute, irrespective of success or failure of the batch, the batch will be cleared. You will need to repopulate the batch before you can retry.
I think you should just move the clearBatch() outside of the try. If you have to recheck whether it's an insert or update, so be it. If psInsert and psUdate are the same class, or derive from the same base class (and it sounds like they should), you can easily call it in a separate one liner method.
I also think if you submit this question to code review, you'd get some good suggestions to improve the code. I'm not saying it's terrible, but I think there's room for improvement in the underlying model. I'm not familiar enough with Java, though.
Related
I got an error during the following process. I'm aware that it seems like this error is thrown because it tried to read the whole records in the partition (rec) but trying to assign it to string (Str=jsonArray.toJSONString();) at the same time I'm using 5-sec batch interval in spark streaming configuration. Any suggestions for this code? Please kindly help. Thanks
The error is in this line :
Str=jsonArray.toJSONString();
Below is my full function :
MapRowRDD.foreachRDD(rdd ->{
rdd.foreachPartition(
rec-> {
while(rec.hasNext()) {
JSONObject record = rec.next();
i=i+1;
if(TimeUnit.MINUTES.convert(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
.parse((String) record.get("DATE_TRANSACTION"))
.getTime()-DateUtils.addMinutes(new Date(), -5)
.getTime(),TimeUnit.MILLISECONDS)>=0 || Integer.valueOf((String) record.get("EVENT_TYPE"))<0) {
jsonArray.add(record);
if(i % v_BATCH_WINDOW == 0)
{
try {
Str=jsonArray.toJSONString();
HttpResponse<String> Response = ui.post(v_REST_API_ENDPOINT).body(Str).asString();
out_JSON=Response.getBody();
log.warn("Response : " + out_JSON.toString());
}
catch(UnirestConfigException e){
System.out.println("UnirestConfigException occured "+ e.toString());
e.printStackTrace();
}
jsonArray.clear();
i=0;
}
}
publishToKafka(record.toString(), outputTopic, props);
}
Str=jsonArray.toJSONString();
if (!Str.equals("[]") && Str!=null && !Str.isEmpty()) {
HttpResponse<String> Response = ui.post(v_REST_API_ENDPOINT).body(Str).asString();
}
jsonArray.clear();
i=0;
}
);
});
As you know this exception occurs when you modify and iterate the same collection at the same time via different threads. jsonArray is not thread-safe replace that with some thread-safe collections like Vector and see this works
i try with this, but it is some times not working correctly..i used a while loop for loop the code. can i add some listner for this? any one can give me the correct answer for this? in need to get responce real time
while (true) {
msgList = new ArrayList<InboundMessage>();
Service.getInstance().readMessages(msgList, InboundMessage.MessageClasses.ALL);
for (InboundMessage im : msgList) {
if (last < im.getMemIndex()) {
ResultSet rs = DB.getConnection().createStatement().executeQuery("Select * From codes where code='" + im.getText() + "'");
if (rs.next()) {
ResultSet rs2 = DB.getConnection().createStatement().executeQuery("Select * From sms_log where code='" + im.getText() + "' AND tel_no='" + im.getOriginator() + "'");
if (rs2.next()) {
if (m == null) {
m = new SMSClient(1);
}
m.sendMessage(im.getOriginator(), "The Code is Already Sent... Thank You!.");
System.out.println("The Code is Already Sent... Thank You!.");
} else {
System.out.println("The Code Verified... Thank You!.");
if (m == null) {
m = new SMSClient(1);
}
m.sendMessage(im.getOriginator(), "The Code Verified... Thank You!.");
DB.getConnection().createStatement().execute("INSERT INTO sms_log (tel_no,code,status) values('" + im.getOriginator() + "','" + im.getText() + "',1)");
}
} else {
if (m == null) {
m = new SMSClient(1);
}
m.sendMessage(im.getOriginator(), "Invalid Code... Thank You!.");
System.out.println("Invalid Code... Thank You!.");
}
}
}
Thread.sleep(10000);
System.out.println("start");
}
I think IInboundMessageNotification is the interface you are looking for
public class InboundNotification implements IInboundMessageNotification {
#Override
public void process(AGateway aGateway, Message.MessageTypes messageTypes, InboundMessage inboundMessage) {
//add you logic for received messages here
}
}
Add notification class to smsLib service
Service.getInstance().setInboundMessageNotification(new InboundNotification())
From now on, process() method will be called every time your modem receives a message.
As far as I remember, smslib (version 3.5.x) does not delete received messages so it needs to be done manually
#Override
public void process(AGateway aGateway, Message.MessageTypes messageTypes, InboundMessage inboundMessage) {
try {
aGateway.deleteMessage(inboundMessage);
} catch (TimeoutException | GatewayException | InterruptedException | IOException e) {
e.printStackTrace();
}
// your logic here
}
otherwise you will keep receiving not deleted messages every time you receive a new one.
Hope you will find this useful.
OK this is going to be a bit long and complex, I'm hoping for some rubber ducking here.
I have this code, which is actually a bit more complex, but I think i did a reasonable job of simplifying it:
private Result getResult(Request request, RequestType type) {
final String date = request.getData(); // marker 1
final DataSource jdbcDataSource = getDataSource();
final JdbcOperations jdbcTemplate = newJdbcOperations(jdbcDataSource);
final TransactionTemplate transactionTemplate = createTransactionTemplate(jdbcDataSource);
Supplier<Integer> createResult = () -> transactionTemplate.execute(transactionStatus -> {
List<Map<String, Object>> rs = jdbcTemplate.queryForList("SELECT * FROM table");
if (rs.size() > 0) {
return ((Number) rs.get(0).get("count")).intValue();
} else {
log.info(request + " not found in table");
}
if (type == TYPE1) {
//...
} else {
throw new RuntimeException("unexpected type:" + type); // marker2
}
return 0;
});
int retryCount = 0;
while (retryCount < 5) {
try {
totalCount = createResult.get();
break;
} catch (DuplicateKeyException | DeadlockLoserDataAccessException e) {
// log
}
retryCount++;
}
}
This works fine, except that today the app server got into a state where it stopped working. The method enters, gets past line "marker1" so we know request is not null, then proceeds to log "null not found in table" suggesting that request turned into null and then throws runtimeException "marker2" because type has turned into null as well.
I hate to say it but this really just smells like a bug in JVM, but this really should be the last to consider, so I'm hoping that you, my dear rubber duck, would have some ideas.
I'm new with java and sql query and for the user connexion, I connect to the DB and check if the login exists. Here is what I do :
requete = "SELECT Login,Password,DroitModifAnnuaire,DroitRecepteurDem,DroitResponsableDem,PiloteIso,Administrateur,DroitNews,DroitTenues,DroitEssai,Nom,Prenom FROM Annuaire WHERE Login='"
+ (request.getParameter("login") + "'");
instruction = connexion.createStatement();
jeuResultats = instruction.executeQuery(requete);
try{
jeuResultats.next();
} catch (SQLException e) {
e.printStackTrace();
}
if (jeuResultats.next() == false) {
loadJSP("/index.jsp", request, reponse);
}else {
loadJSP("/views/menu.jsp", request, reponse);
}
The login that I enter is good but it redirect me to index.jspand I have the error : the result set has no current row
I tried to search answer to this error but I didn't found. So why it returns me false ? While when I do System.out.println(jeuResultats.getString(1)); the login is printed.
jeuResultats.next(); moves your result to the next row. You start with 0th row, i.e. when you call .next() it reads the first row, then when you call it again, it tries to read the 2nd row, which does not exist.
Some additional hints, not directly related to the question:
Java Docs are a good place to start Java 8 ResultSet, for e.x., perhaps ResultSet.first() method may be more suited for your use.
Since you are working with resources, take a look at try-with-resources syntax. Official tutorials are a good starting point for that.
Also take a look at prepared statement vs Statement. Again, official guide is a good place to start
Make the below changes in you code. Currently the next() method is shifting result list to fetch the data at 1st index, whereas the data is at the 0th Index:
boolean result = false;
try{
result = jeuResultats.next();
} catch (SQLException e) {
e.printStackTrace();
}
if (!result) {
loadJSP("/index.jsp", request, reponse);
}else {
loadJSP("/views/menu.jsp", request, reponse);
}
Replace your code by below code:
requete = "SELECT Login, Password, DroitModifAnnuaire, DroitRecepteurDem, DroitResponsableDem, PiloteIso, Administrateur, DroitNews, DroitTenues, DroitEssai, Nom, Prenom FROM Annuaire WHERE Login = '"
+ (request.getParameter("login") + "'");
instruction = connexion.createStatement();
jeuResultats = instruction.executeQuery(requete);
try{
if (jeuResultats.next()) {
loadJSP("/index.jsp", request, reponse);
} else {
loadJSP("/views/menu.jsp", request, reponse);
}
} catch (SQLException e) {
e.printStackTrace();
}
I have a java code that generates a request number based on the data received from database, and then updates the database for newly generated
synchronized (this.getClass()) {
counter++;
System.out.println(counter);
System.out.println("start " + System.identityHashCode(this));
certRequest
.setRequestNbr(generateRequestNumber(certInsuranceRequestAddRq
.getAccountInfo().getAccountNumberId()));
System.out.println("outside funcvtion"+certRequest.getRequestNbr());
reqId = Utils.getUniqueId();
certRequest.setRequestId(reqId);
System.out.println(reqId);
ItemIdInfo itemIdInfo = new ItemIdInfo();
itemIdInfo.setInsurerId(certRequest.getRequestId());
certRequest.setItemIdInfo(itemIdInfo);
dao.insert(certRequest);
addAccountRel();
counter++;
System.out.println(counter);
System.out.println("end");
}
the output for System.out.println() statements is `
1
start 27907101
com.csc.exceed.certificate.domain.CertRequest#a042cb
inside function request number66
outside funcvtion66
AF88172D-C8B0-4DCD-9AC6-12296EF8728D
2
end
3
start 21695531
com.csc.exceed.certificate.domain.CertRequest#f98690
inside function request number66
outside funcvtion66
F3200106-6033-4AEC-8DC3-B23FCD3CA380
4
end
In my case I get a call from two threads for this code.
If you observe both the threads run independently. However the data for request number is same in both the cases.
is it possible that before the database updation for first thread completes the second thread starts execution.
`
the code for generateRequestNumber() is as follows:
public String generateRequestNumber(String accNumber) throws Exception {
String requestNumber = null;
if (accNumber != null) {
String SQL_QUERY = "select CERTREQUEST.requestNbr from CertRequest as CERTREQUEST, "
+ "CertActObjRel as certActObjRel where certActObjRel.certificateObjkeyId=CERTREQUEST.requestId "
+ " and certActObjRel.certObjTypeCd=:certObjTypeCd "
+ " and certActObjRel.certAccountId=:accNumber ";
String[] parameterNames = { "certObjTypeCd", "accNumber" };
Object[] parameterVaues = new Object[] {
Constants.REQUEST_RELATION_CODE, accNumber };
List<?> resultSet = dao.executeNamedQuery(SQL_QUERY,
parameterNames, parameterVaues);
// List<?> resultSet = dao.retrieveTableData(SQL_QUERY);
if (resultSet != null && resultSet.size() > 0) {
requestNumber = (String) resultSet.get(0);
}
int maxRequestNumber = -1;
if (requestNumber != null && requestNumber.length() > 0) {
maxRequestNumber = maxValue(resultSet.toArray());
requestNumber = Integer.toString(maxRequestNumber + 1);
} else {
requestNumber = Integer.toString(1);
}
System.out.println("inside function request number"+requestNumber);
return requestNumber;
}
return null;
}
Databases allow multiple simultaneous connections, so unless you write your code properly you can mess up the data.
Since you only seem to require a unique growing integer, you can easily generate one safely inside the database with for example a sequence (if supported by the database). Databases not supporting sequences usually provide some other way (such as auto increment columns in MySQL).