Develop Reference

How to know that rollback has been executed ? [#Transactional]

I have the following case:
I'm iterating over my Affiliate entities and for each of them I need to persist and update data in one unique transaction. So I have a service with a method annotated with Spring #Transactional annotation (where data is created and updated) but I don't know how can I see that the transaction has been rollback for an affiliate ?
I would like to know that for a special Affiliate the transaction has been rollback and retrieve a custom error code from my service..
This was my service before using Spring:
public void savePostingPaymentDetails(List<Posting> postingsToUpdate, List<PaymentPostingDetail> detailsToInsert, Payment payment) {
logger.info("DB ACCESS : INSERT PAYMENT DETAILS & UPDATE POSTINGS");
long begin = System.nanoTime();
this.em.getTransaction().begin();
try {
// TEST
// 1 - Save Payments
this.em.persist(payment);
// 2 - Save Details
for (PaymentPostingDetail ppd : detailsToInsert) {
this.em.persist(ppd);
}
// 3 - Update Postings
for (Posting p : postingsToUpdate) {
if(p.getSignature() != null)
{
p.getSignature().setModification("withholding-tax.pay", new Date());
}
else
{
logger.error("The Posting with id = " + p.getIdentifier() + " has no PersistenceSignature ?!");
}
this.em.merge(p);
}
}
catch (Exception e)
{
logger.error("Unexpected error on saving/updating the DB.", e);
this.em.getTransaction().rollback();
logger.info("RollBack done.");
e.printStackTrace();
System.exit(JobStatus.ABNORMAL_END_OF_EXECUTION_ERROR.getCode());
}
this.em.getTransaction().commit();
logger.info("Details inserted & Postings updated.");
long end = System.nanoTime();
logger.info("Execution time = " + ((end-begin) / 1000000) + " milliseconds.");
logger.info("----------------------------------------------------------");
}
Now I have this:
#Transactional
public void savePostingPaymentDetails(List<Posting> postings, List<PaymentPostingDetail> paymentDetails, Payment payment)
{
logger.info("DB ACCESS : INSERT PAYMENT DETAILS & UPDATE POSTINGS");
long begin = System.nanoTime();
this.paymentRepository.save(payment);
this.ppdRepository.save(paymentDetails);
for(Posting p : postings){
if(p.getSignature() != null)
{
p.getSignature().setModifiedAt(LocalDate.now());
p.getSignature().setModifiedBy(PayCopyrightWithholdingTaxProcess.SIGNATURE);
}
else{
p.setSignature(new PersistenceSignature(LocalDate.now(), PayCopyrightWithholdingTaxProcess.SIGNATURE));
}
this.postingRepository.save(p);
}
long end = System.nanoTime();
logger.info("Execution time = " + ((end-begin) / 1000000) + " milliseconds.");
logger.info("----------------------------------------------------------");
}
But how can I return let us say a special integer (instead of System.exit()) if the transaction has been rollback ?

There is something called User managed Transaction(UMT) and Container managed Transaction (CMT)
When you are using #Transactional you are actually delegating the transaction management to your Spring container (CMT), which is responsible for e.g opening and closing the transaction for you. It
rolls back automatically when unchecked Exception is thrown like NullPointerException, or RuntimeException ). For checked
exceptions you have to specify when the rollback is supposed to occured #Transactional(rollbackFor=myCheckedException.class).
You can also listen, observe how the transaction is doing with a TransactionalEventListener and react with some AOP listening code like shown here. But You are not ultimately managing the Transaction, Spring is doing for you. The client code can't react with some custom code, when something special happens, because the management of the transaction is delegated to Spring.
Therefore you have to fall back on the User managed Transaction, where you open your transaction, commit it and react in case of a rollback. That is exactly the purpose of UMT: giving total control of your transaction.
from your old code you may get something like:
public int savePostingPaymentDetails(List<Posting> postingsToUpdate, List<PaymentPostingDetail> detailsToInsert, Payment payment) {
int returnCode = 1 // 1 -> "success" , 0 -> "failure"
logger.info("DB ACCESS : INSERT PAYMENT DETAILS & UPDATE POSTINGS");
long begin = System.nanoTime();
long end = 0;
this.em.getTransaction().begin();
try {
// TEST
// 1 - Save Payments
this.em.persist(payment);
// 2 - Save Details
for (PaymentPostingDetail ppd : detailsToInsert) {
this.em.persist(ppd);
}
// 3 - Update Postings
for (Posting p : postingsToUpdate) {
if(p.getSignature() != null)
{
p.getSignature().setModification("withholding-tax.pay", new Date());
}
else
{
logger.error("The Posting with id = " + p.getIdentifier() + " has no PersistenceSignature ?!");
}
this.em.merge(p);
}
this.em.getTransaction().commit();
end = System.nanoTime();
}
catch (Exception e)
{
returnCode = 0;
logger.error("Unexpected error on saving/updating the DB.", e);
this.em.getTransaction().rollback();
logger.info("RollBack done.");
// e.printStackTrace();
System.exit(JobStatus.ABNORMAL_END_OF_EXECUTION_ERROR.getCode());
return returnCode;
}
//this.em.getTransaction().commit();
logger.info("Details inserted & Postings updated.");
//long end = System.nanoTime();
logger.info("Execution time = " + ((end-begin) / 1000000) + " milliseconds.");
logger.info("----------------------------------------------------------");
return returnCode = 1;
}
PS: on a side note, best practice would have you to throw an Exception when your commit fails, instead of special code.
your new method signature could be:
public void savePostingPaymentDetails(List<Posting> postingsToUpdate, List<PaymentPostingDetail> detailsToInsert, Payment payment)
throws MyFailedDbOperationException, OtherException {
}
and Throw the exception on your catch block
catch (Exception e)
{
logger.error("Unexpected error on saving/updating the DB.", e);
this.em.getTransaction().rollback();
logger.info("RollBack done.");
throw MyFailedDbOperationException("my db operation failed");
}

You can use a listener (#TransactionalEventListener) to be informed of a rolled back transaction (the listener can be bound to the different phases of a transaction). See section 16.8 of https://docs.spring.io/spring/docs/4.2.x/spring-framework-reference/html/transaction.html for more information (requires Spring >= 4.2)

Java executor.scheduleAtFixedRate adds up Delay

I am experiencing some issues with the delay of the executor. My thread looks like this:
public void run() {
logger.log("Starting " + tso.getName() + " for " + dataType);
List<String[]> allRows = new ArrayList<String[]>();
String[] lastRow;
LocalDateTime lastDate;;
LocalDateTime lastQuarterHour;
while (true) {
try {
// Make attempt to take CSV data for today. If there is no data wait several seconds and try again
allRows = getCSVRows();
if (allRows == null || allRows.isEmpty()) {
logger.log("Sleeping thread due to empty list for TSO " + tso.getName() + " and data type " + dataType);
Thread.sleep(MILISECONDS_TO_WAIT);
continue;
}
lastRow = allRows.get(allRows.size() - 1);
lastDate = convertStringToUTC(lastRow[0] + " " + lastRow[2]);
lastQuarterHour = takeLastQuorterHourTime();
// If CSV data is available take the last record if it is before last quarter hour wait SEVERAL seconds and try again
if (lastDate.isBefore(lastQuarterHour)) {
logger.log("Sleeping due to lack of information for the current quarter for TSO " + tso.getName() + " and data type " + dataType);
Thread.sleep(MILISECONDS_TO_WAIT);
} else {
break;
}
} catch (InterruptedException e) {
logger.log(e.getMessage());
}
}
}
}
The first time i run my thread the delay is OK, but when the thread sleeps 2 or 3 times, the delay when the next thread cycle starts, is not what is defined in:
executor.scheduleAtFixedRate(extractorThread, 0, WAIT_INTERVAL_MINUTES, TimeUnit.SECONDS);
So, when does the delay start, once the thread finishes its 2-3 sleeps and terminates or when the thread itself is started, no matter how long it works?

It might have something to do with the fact that you're sleeping inside the Thread
Thread.sleep(MILISECONDS_TO_WAIT);
Try removing those and see if the problem still occurs
[EDIT]
The answer to your question is that it will run every X amount of seconds and it counts from when the initial thread/next thread starts but if the thread is not available at the time of the scheduled execution (Because of the thread sleeping or for instance doing very heavy calculations) it will wait for it to become available.

Synchronizing Hibernate inserts using guarded block

I am trying to solve a problem with conflicting concurrent database inserts via Hibernate in MySQL.
I have a piece of code that can easily be executed by multiple threads at the same time. It is checking the database for an existence of a record and if it does not exist a new record gets inserted. This same insert-if-nonexistent operation is performed on a related child record. I get a ConstraintViolationException if two threads try to persist the child record at the same time, because both threads see the record does not exist at the moment they are querying it, so both threads attempt to save the same record which violates a unique constraint, and one of them fails.
I am trying to synchronise the query-insert operations on the application level using a guarded block, so that a thread is waiting for another thread to finish inserting the records before querying the database. But even though I see the synchronisation works, querying for the record still returns no results, even if the record has been persisted in another thread. So the constraint violation still happens.
I am using Hibernate 5.1.0
I am managing database transactions manually
I have enabled query cache and second-level cache globally, but am using CacheMode.REFRESH for the SELECT queries
I am not using optimistic or pessimistic database locking or row versioning.
Here is a code example:
In each synchronized operation I try to persist a Product if it does not exist, and a related parent Supplier if it does not exist.
public class UpdateProcessor extends HttpServlet {
// Indicator used for synchronizing read-insert operations
public static Boolean newInsertInProgress = false;
#Override
public void doPost(HttpServletRequest request, HttpServletResponse response) {
Session hbSession = null;
Transaction tx = null;
try {
hbSession = HibernateUtils.getNewSession();
UpdateProcessor.waitForInsert(); // if there is an insert in progress, wait for it to finish
UpdateProcessor.notifyInsertStarted(); // obtain lock
tx = hbSession.beginTransaction();
Product existingProduct = findProductBySKU(sku);
if(existingProduct == null) {
Product newProduct = new Product();
newProduct.setSKU(sku);
Supplier existingSupplier = findSupplierByName(name);
if(existingSupplier == null) {
Supplier newSupplier = new Supplier();
newSupplier.setName(name);
db.save(newSupplier);
newProduct.setSupplier(newSupplier);
} else {
newProduct.setSupplier(existingSupplier);
}
db.save(newProduct);
}
tx.commit();
} catch (Exception t) {
// <rollback transaction>
response.sendError(500);
} finally {
// Safeguard to avoid thread deadlock - release lock always, if obtained
if(UpdateProcessor.newInsertInProgress) {
UpdateProcessor.notifyInsertFinished(); // release lock and notify next thread
}
// <close session>
}
}
private static synchronized void waitForInsert() {
if(!UpdateProcessor.newInsertInProgress) {
log("Skipping wait - thread " + Thread.currentThread().getId() + " - " + System.currentTimeMillis());
return;
}
while(UpdateProcessor.newInsertInProgress) {
boolean loggedEntering = false;
if(!loggedEntering) {
log("Entering wait - thread " + Thread.currentThread().getId() + " - " + System.currentTimeMillis());
loggedEntering = true;
}
try {
UpdateProcessor.class.wait();
} catch (InterruptedException e) {}
}
log("Exiting wait - thread " + Thread.currentThread().getId() + " - " + System.currentTimeMillis());
}
private static synchronized void notifyInsertStarted() {
UpdateProcessor.newInsertInProgress = true;
UpdateProcessor.class.notify();
log("Notify start - thread " + Thread.currentThread().getId() + " - " + System.currentTimeMillis());
}
private static synchronized void notifyInsertFinished() {
UpdateProcessor.newInsertInProgress = false;
UpdateProcessor.class.notify();
log("Notify finish - thread " + Thread.currentThread().getId() + " - " + System.currentTimeMillis());
}
}
The output after concurrently making the request:
Skipping wait - thread 254 - 1478171162713
Notify start - thread 254 - 1478171162713
Entering wait - thread 255 - 1478171162713
Entering wait - thread 256 - 1478171162849
Notify finish - thread 254 - 1478171163050
Exiting wait - thread 255 - 1478171163051
Notify start - thread 255 - 1478171163051
Entering wait - thread 256 - 1478171163051
Error - thread 255:
org.hibernate.exception.ConstraintViolationException: could not execute statement
...
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '532-supplier-name-1' for key 'supplier_name_uniq'
Persisting the new supplier record still threw an exception in thread 255 because the unique constraint (id, name) is violated.
Why is the SELECT still not returning any records after a synchronized insert? Is guarded lock a correct way to avoid the multi-insert problem?

Based on Mechkov's answer above:
Short answer: I needed to include the Hibernate session creation in the synchronised piece of code.
Long answer:
The guarded block properly synchronised the query-insert block but the problem was that even though one thread finishes persisting the records, the second thread cannot see the change in the database until a fresh Hibernate session is created. So the effects of concurrent database modifications are not immediately visible to all threads. An up-to-date database state is obtained via creating a session AFTER an insert is made in some other thread. Including the session creation in the synchronised code ensures that is the case.

Using a Commonj Work Manager to send Asynchronous HTTP calls

I switched from making sequential HTTP calls to 4 REST services, to making 4 simultaneous calls using a commonj4 work manager task executor. I'm using WebLogic 12c. This new code works on my development environment, but in our test environment under load conditions, and occasionally while not under load, the results map is not populated with all of the results. The logging suggests that each work item did receive back the results though. Could this be a problem with the ConcurrentHashMap? In this example from IBM, they use their own version of Work and there's a getData() method, although it doesn't like that method really exists in their class definition. I had followed a different example that just used the Work class but didn't demonstrate how to get the data out of those threads into the main thread. Should I be using execute() instead of schedule()? The API doesn't appear to be well documented. The stuckthreadtimeout is sufficiently high. component.processInbound() actually contains the code for the HTTP call, but I the problem isn't there because I can switch back to the synchronous version of the class below and not have any issues.
http://publib.boulder.ibm.com/infocenter/wsdoc400/v6r0/index.jsp?topic=/com.ibm.websphere.iseries.doc/info/ae/asyncbns/concepts/casb_workmgr.html
My code:
public class WorkManagerAsyncLinkedComponentRouter implements
MessageDispatcher<Object, Object> {
private List<Component<Object, Object>> components;
protected ConcurrentHashMap<String, Object> workItemsResultsMap;
protected ConcurrentHashMap<String, Exception> componentExceptionsInThreads;
...
//components is populated at this point with one component for each REST call to be made.
public Object route(final Object message) throws RouterException {
...
try {
workItemsResultsMap = new ConcurrentHashMap<String, Object>();
componentExceptionsInThreads = new ConcurrentHashMap<String, Exception>();
final String parentThreadID = Thread.currentThread().getName();
List<WorkItem> producerWorkItems = new ArrayList<WorkItem>();
for (final Component<Object, Object> component : this.components) {
producerWorkItems.add(workManagerTaskExecutor.schedule(new Work() {
public void run() {
//ExecuteThread th = (ExecuteThread) Thread.currentThread();
//th.setName(component.getName());
LOG.info("Child thread " + Thread.currentThread().getName() +" Parent thread: " + parentThreadID + " Executing work item for: " + component.getName());
try {
Object returnObj = component.processInbound(message);
if (returnObj == null)
LOG.info("Object returned to work item is null, not adding to producer components results map, for this producer: "
+ component.getName());
else {
LOG.info("Added producer component thread result for: "
+ component.getName());
workItemsResultsMap.put(component.getName(), returnObj);
}
LOG.info("Finished executing work item for: " + component.getName());
} catch (Exception e) {
componentExceptionsInThreads.put(component.getName(), e);
}
}
...
}));
} // end loop over producer components
// Block until all items are done
workManagerTaskExecutor.waitForAll(producerWorkItems, stuckThreadTimeout);
LOG.info("Finished waiting for all producer component threads.");
if (componentExceptionsInThreads != null
&& componentExceptionsInThreads.size() > 0) {
...
}
List<Object> resultsList = new ArrayList<Object>(workItemsResultsMap.values());
if (resultsList.size() == 0)
throw new RouterException(
"The producer thread results are all empty. The threads were likely not created. In testing this was observed when either 1)the system was almost out of memory (Perhaps the there is not enough memory to create a new thread for each producer, for this REST request), or 2)Timeouts were reached for all producers.");
//** The problem is identified here. The results in the ConcurrentHashMap aren't the number expected .
if (workItemsResultsMap.size() != this.components.size()) {
StringBuilder sb = new StringBuilder();
for (String str : workItemsResultsMap.keySet()) {
sb.append(str + " ");
}
throw new RouterException(
"Did not receive results from all threads within the thread timeout period. Only retrieved:"
+ sb.toString());
}
LOG.info("Returning " + String.valueOf(resultsList.size()) + " results.");
LOG.debug("List of returned feeds: " + String.valueOf(resultsList));
return resultsList;
}
...
}
}

I ended up cloning the DOM document used as a parameter. There must be some downstream code that has side effects on the parameter.

Neo4J create relationship hangs on remote, but node creation succeeds

My relationship creation hangs, yet the nodes underneath manage to persist to my remote client.
public class Baz
{
private static enum CustomRelationships implements RelationshipType {
CATEGORY
}
public void foo()
{
RestGraphDatabse db = new RestGraphDatabase("http://remoteIp:7474/db/data",username,password);
Transaction tx = db.beginTx();
try{
Node a = db.createNode();
a.setProperty("foo", "foo"); // finishes
Node b = db.createNode();
b.setProperty("bar", "bar"); //finishes
a.createRelationshipTo(b, CustomRelationships .CATEGORY); // hangs
System.out.println("Finished relationship");
tx.success();
} finally {
tx.finish();
}
}
}
And I cannot figure out why. There is no stack and the connection doesn't time out.
a.createRelationshipTo(b, DynamicRelationshipType.withName("CATEGORY"));
also hangs
This query executes correctly from the admin shell:
start first=node(19), second=node(20) Create first-[r:RELTYPE {
linkage : first.Baz + '<-->' + second.BazCat }]->second return r
Yet when run in this fashion:
ExecutionResult result = engine.execute("start first=node("
+ entityNode.getId() + "), second=node("
+ categoryNode.getId() + ") "
+ " Create first-[r:RELTYPE { linkage : first.Baz"
+ " + '<-->' + second.BazCat" + " }]->second return r");
Also hangs.

There are no real transactions over rest.
It is a bug in the Java-Rest-Binding that internal threads are not started as daemon threads. It actually doesn't hang just the program is not ended.
You can System.exit(0) to end the program as a workaround.