I have to make a very simplified Joint Bank Account program (in this example with 3 users who all have access to the bank accounts resources) but I'm having trouble correctly using Java threads.
Here is how the program should work. There are "users" that all have access to one Joint Bank Account with an arbitrary set initial balance(I used 5000). They can each withdraw or deposit money (whether they withdraw or deposit is randomly generated each time) three times over one run of the program.
The amount they deposit or withdraw is also randomly generated, with the only rule for that being that the amount can never exceed 1/3rd of the current balance.
Finaly after each transaction the current Thread has to "wait" a random amount of seconds between 1 and 10.
Now here is the confusing part. Our teacher asked us to make a unique NotEnoughBalance exception class in case one of the users somehow withdraws more money than what is currently in the account (but here is my first point of confusion: in theory this could never occur due to the 1/3rd rule).
Here is the full code posted on pastebin:
http://pastebin.com/Upam56NF
Currently, when I run the main:
public class BankAccount{
public static void main(String[] args) throws InterruptedException{
int capital = 5000;
JointBankAccount acc = new JointBankAccount(capital);
Thread t1 = new Thread(new Owner("Josh", acc));
Thread t2 = new Thread(new Owner("Wade", acc));
Thread t3 = new Thread(new Owner("Ben", acc));
System.out.println(capital);
String tname = Thread.currentThread().getName();
System.out.println(tname);
t1.start();
t2.start();
t3.start();
t1.join();
t2.join();
t3.join();
for(AccountTransaction s : acc.history){
System.out.println(s.toString());
}
System.out.println(acc.getBalance());
}
}
I randomly sometimes get a NPE exception at System.out.println(s.toString()).
This is completely fixable if I make both deposit and withdraw function Synchronized.
The problem is, somehow I think makign them synchronized defeats the purpose of what our teacher is asking. If I make them synchronized, then I feel like I'm ensuring that the 1/3rd rule gets sucessfuly followed with each withdrawal correctly, and the not enough Balance exception can never exist.
The fact that I get NPEs when I remove synchronized also makes me think possibly the error is in me not properly handling the exception when it does occur. I don't know.
Any help would be greatly appreciated.
This looks to me like a fine exercise in understanding atomic transactions.
But first the NPE: one JointBankAccount instance is shared between several threads, and thus anything contained by the JointBankAccount like the history list. The history list is of type ArrayList which is not thread-safe, i.e. if two threads call the add method at the same time, the list breaks. This is easy to fix though: List<AccountTransaction> history = Collections.synchronizedList(new ArrayList<AccountTransaction>());
Now for the atomic transactions. Since the balance can be changed at any time by any thread, the moment you read the balance, the balance is already outdated. I.e. a statement like if (balance > 1000) then updateBalance() is not valid since the balance could have changed after then. One way to circumvent this is to synchronize everything (although even then you have to be careful, e.g. use AomticInteger to register the balance). But the NotEnoughBalanceException implies a different way of working using a ReentrantReadWriteLock. Before the balance is updated, use a read-lock to read the latest balance and apply any rules to determine if the balance can be updated. This allows you to inform the "owner" that the balance can probably be updated. But since you used a read-lock you cannot be certain: the balance might have already been updated. Now use a write-lock and apply the rules again. This time you can be certain about the balance value (it can only be updated by the code that has the single write-lock) and you might find the balance is no good (one of the rules fails). This is where the NotEnoughBalanceException comes into play: you promised the "owner" the balance could be updated but now that you have the final write-lock you find the balance cannot be updated (the atomic transaction cannot be completed).
This follows a pretty common pattern: use "cheap" locks to determine if a transaction can be done (perform pre-checks) and then use an "expensive" lock to perform the transaction but always assume the transaction might fail and retry the transaction when appropriate.
Related
I found out that put and get with CacheLoader operations use Reentrant lock under the hood, but why this is not implemented for getIfPresent operation?
get which is used by getIfPresent
#Nullable
V get(Object key, int hash) {
try {
if (this.count != 0) {
long now = this.map.ticker.read();
ReferenceEntry<K, V> e = this.getLiveEntry(key, hash, now);
Object value;
if (e == null) {
value = null;
return value;
}
value = e.getValueReference().get();
if (value != null) {
this.recordRead(e, now);
Object var7 = this.scheduleRefresh(e, e.getKey(), hash, value, now, this.map.defaultLoader);
return var7;
}
this.tryDrainReferenceQueues();
}
Object var11 = null;
return var11;
} finally {
this.postReadCleanup();
}
}
put
#Nullable
V put(K key, int hash, V value, boolean onlyIfAbsent) {
this.lock();
.....
Is the only thing I can do to reach thread-safety in basic get/put operations is to use synchronization on client ?
Even if getIfPresent did use locks, that won't help. It's more fundamental than that.
Let me put that differently: Define 'threadsafe'.
Here's an example of what can happen in a non-threadsafe implementation:
You invoke .put on a plain jane j.u.HashMap, not holding any locks.
Simultaneously, a different thread also does that.
The map is now in a broken state. If you iterate through the elements, the first put statement doesn't show at all, the second put statement shows up in your iteration, and a completely unrelated key has disappeared. But calling .get(k) on that map with the second thread's key doesn't find it eventhough it is returned in the .entrySet(). This makes no sense and breaks all rules of j.u.HashMap. The spec of hashmap does not explain any of this, other than 'I am not threadsafe' and leaves it at that.
That's an example of NOT thread safe.
Here is an example of perfectly fine:
2 threads begin.
Some external event (e.g. a log) shows that thread 1 is very very very slightly ahead of thread 2, but the notion of 'ahead', if it is relevant, means your code is broken. That's just not how multicore works.
Thread 1 adds a thing to a concurrency-capable map, and logs that it has done so.
Thread 2 logs that it starts an operation. (From the few things you have observed, it seems to be running slightly 'later') so I guess we're "after" the point where T1 added the thing) now queries for the thing and does not get a result.1
That's fine. That's still thread safe. Thread safe doesn't mean every interaction with an instance of that data type can be understood in terms of 'first this thing happened, then that thing happened'. Wanting that is very problematic, because the only way the computer can really give you that kind of guarantee is to disable all but a single core and run everything very very slowly. The point of a cache is to speed things up, not slow things down!
The problem with the lack of guarantees here is that if you run multiple separate operations on the same object, you run into trouble. Here's some pseudocode for a bank ATM machine that will go epically wrong in the long run:
Ask user how much money they want (say, €50,-).
Retrieve account balance from a 'threadsafe' Map<Account, Integer> (maps account ID to cents in account).
Check if €50,-. If no, show error. If yes...
Spit out €50,-, and update the threadsafe map with .put(acct, balance - 5000).
Everything perfectly threadsafe. And yet this is going to go very very wrong - if the user uses their card at the same time they are in the bank withdrawing money via the teller, either the bank or the user is going to get very lucky here. I'd hope it's obvious to see how and why.
The upshot is: If you have dependencies between operations there is nothing you can do with 'threadsafe' concepts that can possibly fix it; the only way is to actually write code that explicitly marks off these dependencies.
The only way to write that bank code is to either use some form of locking. Basic locking, or optimistic locking, either way is fine, but locking of some sort. It has to look like2:
start some sort of transaction;
fetch account balance;
deal with insufficient funds;
spit out cash;
update account balance;
end transaction;
Now guava's code makes perfect sense:
There is no such thing as 'earlier' and 'later'. You need to stop thinking about multicore in that way. Unless you explicitly write primitives that establish these things. The cache interface does have these. Use the right operation! getIfPresent will get you the cache if it is possible for your current thread to get at that data. If it is not, it returns null, that's what that call does.
If instead you want this common operation: "Get me the cached value. However, if it is not available, then run this code to calculate the cached value, cache the result, and return it to me. In addition, ensure that if 2 threads simultaneously end up running this exact operation, only one thread runs the calculation, and the other will wait for the other one (don't say 'first' one, that's not how you should think about threads) to finish, and use that result instead".. then, use the right call for that: .cache.get(key, k -> calculateValueForKey(k)). As the docs explicitly call out this will wait for another thread that is also 'loading' the value (that's what guava cache calls the calculation process).
No matter what you invoke from the Cache API, you can't 'break it', in the sense that I broke that HashMap. The cache API does this partly by using locks (such as ReentrantLock for mutating operations on it), and partly by using a ConcurrentHashMap under the hood.
[1] Often log frameworks end up injecting an actual explicit lock in the proceedings and thus you do often get guarantees in this case, but only 'by accident' because of the log framework. This isn't a guarantee (maybe you're logging to separate log files, for example!) and often what you 'witness' may be a lie. For example, maybe you have 2 log statements that both log to separate files (and don't lock each other out at all), and they log the timestamp as part of the log. The fact that one log line says '12:00:05' and the other says '12:00:06' means nothing - the log thread fetches the current time, creates a string describing the message, and tells the OS to write it to the file. You obviously get absolutely no guarantee that the 2 log threads run at identical speed. Maybe one thread fetches the time (12:00:05), creates the string, wants to write to the disk but the OS switches to the other thread before the write goes through, the other thread is the other logger, it reads time (12:00:06), makes the string, writes it out, finishes up, and then the first logger continues, writes its context. Tada: 2 threads where you 'observe' one thread is 'earlier' but that is incorrect. Perhaps this example will further highlight why thinking about threads in terms of which one is 'first' steers you wrong.
[2] This code has the additional complication that you're interacting with systems that cannot be transactional. The point of a transaction is that you can abort it; you cannot abort the user grabbing a bill from the ATM. You solve that by logging that you're about to spit out the money, then spit out the money, then log that you have spit out the money. And finally write to this log that it has been processed in the user's account balance. Other code needs to check this log and act accordingly. For example, on startup the bank's DB machine needs to flag 'dangling' ATM transactions and will have to get a human to check the video feed. This solves the problem where someone trips over the power cable of the bank DB machine juuust as the user is about to grab the banknote from the machine.
Seems like guava cache is implementing ConcurrentMap api
class LocalCache<K, V> extends AbstractMap<K, V> implements ConcurrentMap<K, V>
so the base get and put operations should be thread safe by nature
Suppose I am working in a banking domain and I have three customers say A,B,C.
Balance of A= Rs.100 Balance of B= Rs.0 Balance of C=Rs.100
Now both A and C are sending money to B at same time. The code for increasing the balance runs concurrently.
When A sends money to B, a call is made to DB which gets its current balance i.e Rs.0
At the same time C sends money to B and call is made to DB which also returns current balance as Rs.0.
So when control is back from DB for "A" and money is added in B's account,
it will be
Balance=Current Balance+ Money Received
so balance= 100.
Again when control is back from DB for "C",
balance = Rs.100 since current balance fetched by this request was also Rs.0
How to handle such scenarios?
A simple UPDATE account SET balance=balance+100 WHERE userId = ? would make sure that the balance is increased properly even in concurrent scenarios.
Only if you retrieve the balance to increase it in Java code you need to make special precautions, but that would be the more complex solution.
First of all, Concurrency is achieved by implementing it into the send method. So in your example there is nothing to achive, because there is already a concurrent setup. It would not be concurrent if the send commands go into a pipeline and being processed one after one.
The problem you experience is a dirty write/dirty read to/from the database Therefore a transaction isolation is needed. Usually a database checks this circumstance and locks the row that is written to. Theoretically you could trust the database, when it offers transactions. It would be a bad practice and unnecessary to double check this. When not you should implement locks (which could get a performance problem, because of intensive locking) and synchronize the access to customer accounts.
To syncronize you could take the addBalance method:
synchronized void addBalance(int amount, CustomerAccountEntity customer) {
//getCustomerBalance
balance = balance+amount;
}
I'm trying to implement pessimistic locking using select for update, as I want other threads to wait until the lock on the selected row is released.
The part that I have understood is after going through multiple threads Spring JDBC select for update and various similar threads is it is achievable in case select and update are happening within same method and hence they are part of same transaction.
The issue in my case is I have a JAR for DAO functionality where in a selectforUpdate method is available and a separate update method is available, both method has a finally block which contains
resultSet.close();
statement.close();
connection.close();
Now I'm struggling to find out is there a way in which I can use both the methods from outside of the JAR, maybe by annotating my method with #Transactional annotation and make it work in some way. So that lock is only released once update method has been executed.
You're making a mistake. Using the wrong tool for the job. Transaction levels and FOR UPDATE has the purpose of ensuring data integrity. Period. It it isn't designed for control flow and if you use it for this, it will bite you in the butt sooner rather than later.
Let me try to explain what SELECT FOR UPDATE is for, so that, when later I tell you that it is most definitely not for what you're trying to do with it, it is easier to follow.
Imagine a bank. Simple enough. The bank has some ATMs out front and a website where you can see your transactions and transfer money to other accounts.
Imagine you (ABC) and I (Reinier) are trying to fleece the bank some. Here is our plan: We set it up so that you have €1000,- in your account and I have nothing.
Then, you log into the website from your phone, and start a transfer, transferring €1000,- to my account. But, while you're doing that, right in the middle, you withdraw €10,- from the ATM.
If the bank messed up their transactions, it's possible you end up with €990,- in your account and I have €1000,- in my account, and we fleeced the bank. This is how that could happen (and if halfway through the example you think: I already know this stuff, I know what FOR UPDATE does! - I'm not so sure you do, read it carefully)
ATM code
startTransaction();
int currentBalance = sql("SELECT balance FROM account WHERE user = ?", abc);
if (currentBalance < requestedWithdrawal) throw new InsufficientFundsEx();
sql("UPDATE account SET balance = ? WHERE user = ?", currentBalance - requestedWithdrawal, abc);
commit();
moneyHopper.spitOut(requestedWithdrawal();
Website code
startTransaction();
int balanceTo = sql("SELECT balance FROM account WHERE user = ?", reinier);
int balanceFrom = sql("SELECT balance FROM account WHERE user = ?", abc);
if (transfer > balanceFrom) throw new InsufficientFundsEx();
sql("UPDATE account SET balance = ? WHERE user = ?", balanceTo + transfer, reinier);
sql("UPDATE account SET balance = ? WHERE user = ?", balanceFrom - transfer, abc);
commit();
controller.notifyTransferSucceeded();
How it can go wrong
The way it goes wrong is if the balanceTo and balanceFrom are 'locked in', then the ATM withdrawal goes through, and then the update SQL statements from the website transaction go through (this wipes out the ATM withdrawal, effectively - whatever the ATM spit out is free money), or if the ATM's balance check locks in, then the transfer goes through, and then the ATM's update goes through (which gives the recipient, i.e. me their €1000,-, and ensures that the ATM code's update, setting your balance to 990, is the last thing that happens, giving us €990,- of free money.
So what's the fix? Hint: Not FOR UPDATE
The fix is to consider what a transaction means. The purpose of transactions is to turn operations into atomic notions. Either both your account is reduced by the transfer amount and mine is raised by the same, or nothing happens.
It's obvious enough with statements that change things (UPDATE and INSERT). It's a bit more wonky when we talk about reading data. Should those reads be considered part of the transaction?
One way to go is to say: No, unless you add FOR UPDATE at the end of it all, in which case, yes - i.e. lock those rows only if FOR UPDATE is applied until the transaction ends.
But that is not the only way to ensure data integrity.
Optimistic locking to the rescue - or rather, to your doom
A much more common way is called MVCC (MultiVersion Concurrency Control) and is far faster. The idea behind MVCC (also called optimistic locking), is to just assume no clashes ever occur. Nothing is ever locked. Instead, [A] all changes made within a transaction are completely invisible to things running in any other transaction until you commit, and [B] when you COMMIT a transaction, the database checks if everything you have done within the span of this transaction still 'holds up' - for example, if you updated a row within this transaction that was also modified by another transaction that has committed already, you get an error when you commit, not when you ran the UPDATE statement.
In this framework, we can still talk about what SELECT even means. This, in java/JDBC, is called the Transaction Isolation Level and is configurable on a DB connection. The best level, the level the bank should be using to avoid this issue, is called the TransactionLevel.SERIALIZABLE. Serializable effectively means everything dirties everything else: If during a transaction you read some data, and when you commit, that same SELECT statement would have produced different results because some other transaction modified something, then the COMMIT just fails.
They fail with a so-called 'RetryException'. This means literally what it says: Just start your transaction over, from the top. It makes sense if you think about that bank example: What WOULD have happened, had the bank done it right and set up serializable transaction isolation level, is that either the ATM machine's transaction or the transfer transaction would get the retryexception. Assuming the bank wrote their code right and they actually do what the exception tells you to (start over), then they would start over, and that includes re-reading the balances out. No cheating of the bank can occur now.
Crucially, in the SERIALIZABLE model, locking NEVER occurs, and FOR UPDATE does not mean anything at all.
Thus, usually, FOR UPDATE does literal stone cold nothing, a complete no-op, depending on how the db is setup.
FOR UPDATE does not mean 'lock other transactions that touch this row'. No matter how much you want it to.
Some DB implementations, or even some combination of DB engine and connection configuration may be implemented in that fashion, but that is an extremely finicky setup, and your app should include documentation that strongly recommends the operator to never change the db settings, never switch db engines, never update the db engine, never update the JDBC driver, and never mess with the connection settings.
That's the kind of silly caveat you really, really don't want to put on your code.
The solution is to stop buttering your toast with that chainsaw. Even if you think you can manage to get some butter on that toast with it, it's just not what it was made for, like at all, and we're all just waiting until you lose a thumb here. Just stop doing it. Get a butterknife, please.
If you want to have one thread wait for another, don't use the database, use a lock object. If you want to have one process wait for another, don't use the database, don't use a lock object (you can't; processes don't share memory); use a file. the new java file IO has an option to make a file atomically (meaning, if the file already exists, throw an exception, otherwise make the file, and do so atomically, meaning if two processes both run this 'create atomically new file' code, you have a guarantee that one succeeds and one throws).
If you want data integrity and that's the only reason you wanted pessimistic locking in the first place, stop thinking that way - it's the DBs job, not your job, to guarantee data integrity. MVCC/Optimistic locking DBs guarantee that the bank will never get fleeced no matter how hard you try with the shenanigans at the top of this answer and nevertheless, pessimistic locking just isn't involved.
JDBC itself sucks (intentionally, a bit too much to get into) for 'end use' like what you are doing here. Get yourself an abstraction that makes it nice such as JDBI or JOOQ. These tools also have the only proper way to interact with databases, which is that all DB code must be in a lambda. That's because you don't want to manually handle those retry exceptions, you want your DB access framework to take care of it. This is what the bank code should really look like:
dbAccess.run(db -> {
int balance = db.sql("SELECT balance FROM account WHERE user =?", abc);
if (balance < requested) throw new InsufficientBalanceEx();
db.update("UPDATE account SET balance = ? WHERE user = ?", balance - requested, abc);
return requested;
};
This way, the 'framework' (the code behind that run method) can catch the retryex and just rerun the lambda as often as it needs to. rerunning is tricky - if two threads on a server both cause the other to retry, which is not that hard to do, then you can get into an endless loop where they both restart and both again cause the other to retry, at infinitum. The solution is literally dicethrowing. When retrying, you should roll a random number and wait that many milliseconds, and for every further retry, the range on which you're rolling should increase. If this sounds dumb to you, know that you're currently using it: It's how Ethernet works, too (ethernet uses randomized backoff when collisions occur on the wire). Ethernet won, token ring lost. It's the exact same principle at work (token ring is pessimistic locking, ethernet is optimistic 'eh just try it and detect if it went wrong, then just redo it, with some randomized exponential backoff sprinkled in to ensure you don't get 2 systems in lock-step forever screwing up the other's attempt).
I am not sure how to go about this in Java, For fun, Assume I am building a bank model, and I would like to calculate the total interest accumulated for a customer when his account matures. I can check if an account is mature or not and then do the math, but is there a way to have this done automatically, I assume a real bank software would do this automatically, or this a database trigger?
Since you want elegance for your hypothetical bank; then you can use a thread whose main job is to calculate the total interest accumulated for a customer when his account matures.
Your bank software will be doing other things, adding new accounts, transferring money, but the thread will always be there checking.
Now the thread would not run for ever, it could wake up everyday at midnight and start its search/task. Or it could wake up a couple of key times during the day and do a portion of the task. You can wake it up using a timer and a synchronized block.
synchronized(timeObj)
{
// Check all bank accounts for "maturity" and interests.
}
This is of course an abstraction of what you will built, once you get started you can tweak it to your liking.
Let's say customer has a Credit Card. And he has balance of x amount of money and he is buying y valued item(y<x). And again he is going to buy another item witch will cost z.
(y+z>x but z<x) .
Now I am going to simulate this scenario in Java. If all transaction happens in sequential, there is no need to panic. Customer can buy y valued item and then he don't have enough credit to buy other one.
But when we come in to multi-threaded environment we have to deal with some locking mechanism or some strategy. Because if some other thread read credit card object before reflect changes by previous thread serious issues will rise.
As far as I can see one way is we can keep a copy of original balance and we can check current value just before update the balance. If value is same as original one then we can make sure other threads doesn't change the balance. If balance different then we have to undo our calculation.
And again Java Synchronization also a good solution. Now my question is what will be the best approach to implement in such a scenario?
Additionally if we are going to see this in big picture. Synchronization hits the performance of the system. Since it is locked the object and other thread has to wait.
I will prefer to have a ReadWriteLock, this helps to to lock it for reading and writing, this is nice because you can have separate read and write lock for each resource:
ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
readWriteLock.readLock().lock();
// multiple readers can enter this section
// if not locked for writing, and not writers waiting
// to lock for writing.
readWriteLock.readLock().unlock();
readWriteLock.writeLock().lock();
// only one writer can enter this section,
// and only if no threads are currently reading.
readWriteLock.writeLock().unlock();
ReadWriteLock internally keeps two Lock instances. One guarding read access, and one guarding write access.
Your proposal doesn't fit. You can't be sure the context switch doesn't happen between check and update.
The only way is synchronization.
What you are talking about sounds like software transactional memory. You optimistically assume that no other threads will modify the data upon which your transaction depends, but you have a mechanism to detect if they have.
The types in the java.util.concurrent.atomic package can help build a lock-free solution. They implement efficient compare-and-swap operations. For example, an AtomicInteger reference would allow you to to something like this:
AtomicInteger balance = new AtomicInteger();
…
void update(int change) throws InsufficientFundsException {
int original, updated;
do {
original = balance.get();
updated = original + change;
if (updated < 0)
throw new InsufficientFundsException();
} while (!balance.compareAndSet(original, update));
}
As you can see, such an approach is subject to a livelocked thread condition, where other threads continually change the balance, causing one thread to loop forever. In practice, specifics of your application determine how likely a livelock is.
Obviously, this approach is complex and loaded with pitfalls. If you aren't a concurrency expert, it's safer to use a lock to provide atomicity. Locking usually performs well enough if code inside the synchronized block doesn't perform any blocking operations, like I/O. If code in critical sections has a definite execution time, you are probably better off using a lock.
As far as I can see one way is we can keep a copy of original balance
and we can check current value just before update the balance. If
value is same as original one then we can make sure other threads
doesn't change the balance. If balance different then we have to undo
our calculation.
Sounds like what AtomicInteger.compareAndSet() and AtomicLong.compareAndSet() do.
An easier-to-understand approach would involve using synchronized methods on your CreditCard class that your code would call to update the balance. (Only one synchronized method on an object can execute at any one time.)
In this case, it sounds like you want a public synchronized boolean makePurchase(int cost) method that returns true on success and false on failure. The goal is that no transaction on your object should require more than one method call - as you've realized, you won't want to make two method calls on CreditCard (getBalance() and later setBalance()) to do the transaction, because of potential race conditions.