I'm going through the Sharded Counters example in Java:
http://code.google.com/appengine/articles/sharding_counters.html
I have a question about the implementation of the increment method. In python it explicitly wraps the get() and increment in a transaction. In the Java example it just retrieves it and sets it. I'm not sure I fully understand the Datastore and transactions but it seems like the critical update section should be wrapped in a datastore transaction. Am I missing something?
Original code:
public void increment() {
PersistenceManager pm = PMF.get().getPersistenceManager();
Random generator = new Random();
int shardNum = generator.nextInt(NUM_SHARDS);
try {
Query shardQuery = pm.newQuery(SimpleCounterShard.class);
shardQuery.setFilter("shardNumber == numParam");
shardQuery.declareParameters("int numParam");
List<SimpleCounterShard> shards =
(List<SimpleCounterShard>) shardQuery.execute(shardNum);
SimpleCounterShard shard;
// If the shard with the passed shard number exists, increment its count
// by 1. Otherwise, create a new shard object, set its count to 1, and
// persist it.
if (shards != null && !shards.isEmpty()) {
shard = shards.get(0);
shard.setCount(shard.getCount() + 1);
} else {
shard = new SimpleCounterShard();
shard.setShardNumber(shardNum);
shard.setCount(1);
}
pm.makePersistent(shard);
} finally {
pm.close();
}
}
}
Transactional code (I believe you need to run this in a transaction to gurantee correctness under concurrent transactions?) :
public void increment() {
PersistenceManager pm = PMF.get().getPersistenceManager();
Random generator = new Random();
int shardNum = generator.nextInt(NUM_SHARDS);
try {
Query shardQuery = pm.newQuery(SimpleCounterShard.class);
shardQuery.setFilter("shardNumber == numParam");
shardQuery.declareParameters("int numParam");
List<SimpleCounterShard> shards =
(List<SimpleCounterShard>) shardQuery.execute(shardNum);
SimpleCounterShard shard;
// If the shard with the passed shard number exists, increment its count
// by 1. Otherwise, create a new shard object, set its count to 1, and
// persist it.
if (shards != null && !shards.isEmpty()) {
Transaction tx = pm.currentTransaction();
try {
tx.begin();
//I believe in a transaction objects need to be loaded by ID (can't use the outside queried entity)
Key shardKey = KeyFactory.Builder(SimpleCounterShard.class.getSimpleName(), shards.get(0).getID())
shard = pm.getObjectById(SimpleCounterShard.class, shardKey);
shard.setCount(shard.getCount() + 1);
tx.commit();
} finally {
if (tx.isActive()) {
tx.rollback();
}
}
} else {
shard = new SimpleCounterShard();
shard.setShardNumber(shardNum);
shard.setCount(1);
}
pm.makePersistent(shard);
} finally {
pm.close();
}
}
This section straight out of the docs shows that you are exactly right about needing a transaction:
http://code.google.com/appengine/docs/java/datastore/transactions.html#Uses_For_Transactions
This example demonstrates one use of transactions: updating an entity with a new property value relative to its current value.
Key k = KeyFactory.createKey("Employee", "k12345");
Employee e = pm.getObjectById(Employee.class, k);
e.counter += 1;
pm.makePersistent(e);
This requires a transaction because the value may be updated by another user after this code fetches the object, but before it saves the modified object. Without a transaction, the user's request will use the value of counter prior to the other user's update, and the save will overwrite the new value. With a transaction, the application is told about the other user's update. If the entity is updated during the transaction, then the transaction fails with an exception. The application can repeat the transaction to use the new data.
Its very close to what that sharded example is doing and, like you, I was unable to find any reason why sharded counters would be different.
Related
i have an issue that a freshly created payment transaction has no ID.
#Override
#Transactional("blTransactionManager")
public PaymentTransaction getNewTemporaryOrderPayment(Order cart, PaymentType paymentType) {
OrderPayment tempPayment = null;
if (CollectionUtils.isNotEmpty(cart.getPayments())) {
Optional<OrderPayment> optionalPayment = NmcPaymentUtils.getPaymentForOrder(cart);
if (optionalPayment.isPresent()) {
tempPayment = optionalPayment.get();
invalidateTemporaryPaymentTransactions(tempPayment);
}else {
throw new IllegalStateException("Missing payment");
}
} else {
tempPayment = this.orderPaymentService.create();
}
tempPayment = this.populateOrderPayment(tempPayment, cart, paymentType);
//its necessary to create every time a new transaction because the ID needs to be unique in the parameter passed to 24pay
PaymentTransaction transaction = createPendingTransaction(cart);
transaction.setOrderPayment(tempPayment);
tempPayment.addTransaction(transaction);
tempPayment = orderService.addPaymentToOrder(cart, tempPayment, null);
orderPaymentService.save(transaction);
orderPaymentService.save(tempPayment);
return transaction;
}
even if i do an explicit save on the returned PaymentTransaction, the ID is still null. It is correctly persisted and has an ID in the database.
PaymentTransaction paymentTransaction = paymentService.getNewTemporaryOrderPayment(cart, PaymentType.CREDIT_CARD);
orderPaymentService.save(paymentTransaction);
how can i explicitly refresh this entity ? or any other suggestions how to solve this? I can do something like this to find my pending transaction
OrderPayment orderPayment = paymentTransaction.getOrderPayment();
Optional<PaymentTransaction> any = orderPayment.getTransactions().stream().filter(t -> t.isActive()).findFirst();
but that seems like an extra step which should not be needed. Any suggestions how to solve this in an elegant way ?
The transaction object has a null id because that variable is not updated when the order is saved.
Calls to save() methods return a new object, and that new object will have its id set.
Consider the following example:
Transaction transaction1 = createTransaction(...);
Transaction transaction2 = orderPaymentService.save(transaction1);
After this code executes, transaction1 will not have been changed in save(), so its id will still be null. Transaction2 will be a new object with the id set.
Therefore, the variable transaction, created with PaymentTransaction transaction = createPendingTransaction(cart);, is never updated with the saved value, so the id is still null at the end.
Further, the save() calls at the end for the transaction and payment probably won't work as you intend. This is because the orderService.addPaymentToOrder(cart, tempPayment, null); will save the order, which should also cascade to save the transaction and payment. I'm pretty sure that calling save again would result in new objects that are not connected to the saved order.
So what do you do about this?
The call to tempPayment = orderService.addPaymentToOrder(cart, tempPayment, null); returns a persisted OrderPayment. Read the transactions from that object to find the one you just created. It is very similar to the extra step you are trying to avoid, but you can at least cut out one line.
OrderPayment persistedPayment = orderService.addPaymentToOrder(cart, tempPayment, null);
Optional<PaymentTransaction> persistedTransaction = persistedPayment.getTransactions().stream().filter(t -> t.isActive()).findFirst();
I am new to Java and Hibernate.
I have implemented a functionality where I generate request nos. based on already saved request no. This is done by finding the maximum request no. and incrementing it by 1,and then again save i it to database.
However I am facing issues with multithreading. When two threads access my code at the same time both generate same request no. My code is already synchronized. Please suggest some solution.
synchronized (this.getClass()) {
System.out.println("start");
certRequest.setRequestNbr(generateRequestNumber(certInsuranceRequestAddRq.getAccountInfo().getAccountNumberId()));
reqId = Utils.getUniqueId();
certRequest.setRequestId(reqId);
ItemIdInfo itemIdInfo = new ItemIdInfo();
itemIdInfo.setInsurerId(certRequest.getRequestId());
certRequest.setItemIdInfo(itemIdInfo);
dao.insert(certRequest);
addAccountRel();
System.out.println("end");
}
Following is the output showing my synchronization:
start
end
start
end
Is it some Hibernate issue.
Does the use of transactional attribute in Spring affects the code commit in my Case?
I am using the following Transactional Attribute:
#Transactional(readOnly = false, propagation = Propagation.REQUIRED, rollbackFor = Exception.class)
EDIT: code for generateRequestNumber() shown in chat room.
public String generateRequestNumber(String accNumber) throws Exception {
String requestNumber = null;
if (accNumber != null) {
String SQL_QUERY = "select CERTREQUEST.requestNbr from CertRequest as CERTREQUEST, "
+ "CertActObjRel as certActObjRel where certActObjRel.certificateObjkeyId=CERTREQUEST.requestId "
+ " and certActObjRel.certObjTypeCd=:certObjTypeCd "
+ " and certActObjRel.certAccountId=:accNumber ";
String[] parameterNames = {"certObjTypeCd", "accNumber"};
Object[] parameterVaues = new Object[]
{
Constants.REQUEST_RELATION_CODE, accNumber
};
List<?> resultSet = dao.executeNamedQuery(SQL_QUERY,
parameterNames, parameterVaues);
// List<?> resultSet = dao.retrieveTableData(SQL_QUERY);
if (resultSet != null && resultSet.size() > 0) {
requestNumber = (String) resultSet.get(0);
}
int maxRequestNumber = -1;
if (requestNumber != null && requestNumber.length() > 0) {
maxRequestNumber = maxValue(resultSet.toArray());
requestNumber = Integer.toString(maxRequestNumber + 1);
} else {
requestNumber = Integer.toString(1);
}
System.out.println("inside function request number" + requestNumber);
return requestNumber;
}
return null;
}
Don't synchronize on the Class instance obtained via getClass(). It can have some strange side effects. See https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=43647087
For example use:
synchronize(this) {
// synchronized code
}
or
private synchronized void myMethod() {
// synchronized code
}
To synchronize on the object instance.
Or do:
private static final Object lock = new Object();
private void myMethod() {
synchronize(lock) {
// synchronized code
}
}
Like #diwakar suggested. This uses a constant field to synchronize on to guarantee that this code is synchronizing on the same lock.
EDIT: Based on information from chat, you are using a SELECT to get the maximum requestNumber and increasing the value in your code. Then this value is set on the CertRequest which is then persisted in the database via a DAO. If this persist action is not committed (e.g. by making the method #Transactional or some other means) then another thread will still see the old requestNumber value. So you could solve this by making the code transactional (how depends on which frameworks you use etc.). But I agree with #VA31's answer which states that you should use a database sequence for this instead of incrementing the value in code. Instead of a sequence you could also consider using an auto-incement field in CertRequest, something like:
#GeneratedValue(strategy=GenerationType.AUTO)
private int requestNumber;
For getting the next value from a sequence you can look at this question.
You mentioned this information in your question.
I have implemented a functionality where I generate request nos. based on already saved request no. This is done by finding the maximum request no. and incrementing it by 1,and then again save i it to database.
On a first look, it seems the problem caused by multi appserver code. Threads are synchronised inside one JVM(appserver). If you are using more than one appserver then you have to do it differently using more robust approach by using server to server communication or by batch allocation of request no to each appserver.
But, if you are using only one appserver and multiple threads accessing the same code then you can put a lock on the instance of the class rather then the class itself.
synchronized(this) {
lastName = name;
nameCount++;
}
Or you can use the locks private to the class instance
private Object lock = new Object();
.
.
synchronized(lock) {
System.out.println("start");
certRequest.setRequestNbr(generateRequestNumber(certInsuranceRequestAddRq.getAccountInfo().getAccountNumberId()));
reqId = Utils.getUniqueId();
certRequest.setRequestId(reqId);
ItemIdInfo itemIdInfo = new ItemIdInfo();
itemIdInfo.setInsurerId(certRequest.getRequestId());
certRequest.setItemIdInfo(itemIdInfo);
dao.insert(certRequest);
addAccountRel();
System.out.println("end");
}
But make sure that your DB is updated by the new sequence no before the next thread is accessing it to get new one.
It is a good practice to generate "the request number (Unique Id)" by using the DATABASE SEQUENCE so that you don't need to synchronize your Service/DAO methods.
First thing:
Why are you getting the thread inside the method. I is not required here.
Also, one thing;
Can you try like this once:
final static Object lock = new Object();
synchronized (lock)
{
.....
}
what I feel is that object what you are calling is different so try this once.
I have a java servlet which interacts with hibernate . It is necessary to generate a check id on the system thus:
for (long j = 0; j < soldProductQuantity.longValue(); j++) {
checkId = Hotel.getNextMcsCheckAndIncrement();
checkIdString = checkId.toString();
McsCheck.generateCheck(checkIdString);
}
where getNextMcsCheckAndIncrement() is defined as
static public synchronized Long getNextMcsCheckAndIncrement()
It pulls a value from the database using hibernate, does some operations on it, stores the modified value back, then returns the number.
Because getNextMcsCheckAndIncrement is synchronized, I would expect no two checks to have the same number, because no two threads could enter that method at the same time.
Yet I can see in my data repeated instances of multiple check ids. So clearly this isn't working. What am I missing?
The implementation of getNext as asked:
// Increment FIRST, then return the resulting value as the current MCS check value.
static public synchronized Long getNextMcsCheckAndIncrement() {
Hotel theHotel = null;
Long checkCounter;
#SuppressWarnings("unchecked")
List<Hotel> hotelList = Hotel.returnAllObjects();
for (Hotel currentHotel : hotelList) { // there should be only one.
theHotel = currentHotel;
}
checkCounter = theHotel.getMcsCheckCounter()+1;
theHotel.setMcsCheckCounter(checkCounter);
theHotel.update();
return checkCounter;
}
static public List returnAllObjects() {
return Hotel.query ("from Hotel");
}
static public List query(String queryString) {
Session session = HibernateUtil.getSessionFactory().openSession();
List result = session.createQuery(queryString).list();
session.close();
return result;
}
public void update() {
Session session = HibernateUtil.getSessionFactory().openSession();
Transaction transaction = session.beginTransaction();
session.update(this);
transaction.commit();
session.close();
}
Yes, I know it's not the best way to do it, and I'll solve that in time. But the immediate issue is why concurrency fails.
Anonymous in comments gave the correct answer: The problem must be the Hotel object in the hibernate database, not the synchronization method. Although the counter method is correctly synchronized, if the hotel object is being modified outside of this algorithm, those accesses will NOT be synchronized.
The correct answer, therefore, is to closely check all database accesses to the hotel object and ensure that the object is not modified elsewhere when this loop is in progress, or to refactor the counter out of the Hotel object into dedicated storage.
I am writing code for a webapp that runs a bowling game. The players scores are saved in a database, so that the top ten can be retrieved to be seen on the homepage. Problem is, I cant access the values in the arraylist that is returned from the method that retrieves the scores from the database.
This is a JUnit test method: (CDR is the top player, should assert to true)
<!-- language: c# -->
public class DatabaseTest {
ScoreDB sdbTest = new ScoreDB();
#Test
public void testTopPl(){
assertTrue(sdbTest.listTopPlayers().get(0).getName() == "CDR");
}
}
And this is the database retrieval code:
public List<Player> listTopPlayers( ){
Transaction trns = null;
Session session = HibernateUtil.getSessionFactory().openSession();
List<Player> topPlayers = new ArrayList<Player>();
try {
trns = session.beginTransaction();
List<Player> players = session.createQuery("FROM Player").list();
Collections.sort(players, new ScoreComparator());
for (int i=0; i<10; i++){
topPlayers.add(players.get(i));
}
// System.out.print(topPlayers.get(0).getName() + " test");
trns.commit();
trns = session.beginTransaction();
}catch (RuntimeException e) {
if(trns != null){
trns.rollback();
}
e.printStackTrace();
} finally{
session.flush();
session.close();
}
return topPlayers;
}
}
First of all you should use order by in the database and setMaxResults in your query to only retrive the top 10. Otherwise you may have memory problems.
Second you are calling begin transaction twice. Do it only once, besides, you are in a read only operation that doesn't neeed a transaction.
Finnaly post your problem, are you getting some null pointer or detached entitity exception? If you are getting detatched entity use Hibernate.initialize in the property that you are reading.
And of course, use EQUALS to compare String.
This might be your real problem. Never compare Strings in Java using "==" operator.
Hope these tips would be usefull.
This syntax does not produce unique values across different Fetch Groups:
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private long id;
So I've written the method below to simulate generating a database sequence for my Order model. But what I'm not sure about is what the transactional state needs to be configured as here (isolation level / nontransactional read / nontransactional write / optimistic / etc.):
public long getNextId()
{
PersistenceManager pm = this.getPm();
Transaction tx = pm.currentTransaction();
tx.begin();
long nextId = 0;
Query query = pm.newQuery("select id from orders order by id desc");
query.setRange(0, 1);
try
{
List<Order> results = (List<Order>) query.execute();
if (results.iterator().hasNext())
{
for (Order r : results)
{
nextId = r.getId() + 1;
}
}
else
{
return 0;
}
}
catch (Exception e)
{
return 0;
}
tx.commit();
return nextId;
}
Does the scope of the transaction need to be broader than just this method? In other words, should it also include the insert action for the new Order?
I want to make sure that no two Orders that I insert can have the same id value across the entire application.
IDs generated with IdGeneratorStrategy.SEQUENCE are unique for all entities with the same parent. If your entities are root entities (Eg, no parent), then they will all get unique IDs. What is your use case where you have child entities, but need a unique ID across all of them?
Whats wrong with IdGeneratorStrategy.SEQUENCE ? since GAE/J claims to support it