Develop Reference

Is it necessary to begin a transaction to find an object in the data base?

I know the following is necessary to interact with objects in the database:
db.getTransaction().begin();
//Code to modify objects in the data base
db.getTransaction().commit();
But is it necessary when I use a function find(object.class, object.id)?

Is it necessary to begin a transaction to find an object in the data base?
Depends on what you mean by the word 'transaction'.
Databases do everything in transactions. Period. Therefore, trivially, 'yes', it is neccessary to begin a transaction, because you can't interact with a database without doing this.
However, databases have a 'convenience' feature where you can work as if transactions do not exist. This is a lie; it merely means every statement you send to the DB is silently wrapped in 'START TRANSACTION;' and 'COMMIT;' calls. This is called 'auto commit' mode (and not 'transactionless mode'), because that is the proper word for it: It commits after every transaction automatically.
Thus, in basic JDBC (the base layer upon which all SQL-based DB interaction in java is generally built, but note that you're using some sort of library built on top of it!), you can just run a select query, no 'need' to bother with transactions, by using auto-commit mode.
But, auto commit mode is generally a bad idea.
transactions are fundamental to database design. Even if you are only making 'read' queries, transactions are still important. They guarantee consistency and atomicity. Here is an example:
Let's say that you want to know the bank balance of Jane, and the bank balance of Jack, for example to check if the balance is sufficient to give out a lease for a home or whatnot.
Seems simple:
SELECT balance FROM bankaccounts WHERE user = 'Jack';
SELECT balance FROM bankaccounts WHERE user = 'Jane';
and add em up, right?
Wrong.
If you run that in auto-commit mode, then it is possible that Jack has 50k, Jane has 100k, and nevertheless, the sum appears to be 200k. This is done by jack managing to transfer his 50k to jane right in between the 2 statements, giving you a sum of 50k+150k = 200k, even though that is a lie.
transactions therefore also apply to read-only sessions; and they can even cause retry assuming you use the correct isolation level (SERIALIZABLE is the sensible level). With transactions, you'd get the right answer, of 150k, every time, no matter when or how often jane and jack transfer funds back and forth to try to fake you out.
JDBC itself does not have this 'getTransaction()' thing, or 'begin', and in general that is not the right way to 'do' transactions. Thus, it is not clear what third party library you're using here.
A proper database abstraction would look something like:
int sum = dbAccess.executeInt(db -> {
// query or modify things here:
return
db.select("SELECT balance FROM bankaccount WHERE user = ?", "Jack").singleInt() +
db.select("SELECT balance FROM bankaccount WHERE user = ?", "Jane").singleInt();
};
It needs to be in a lambda to deal with retries, and it meshes well with the truth that all interactions with a database are transactional.
You may want to peruse the docs some more; this strategy (of using lambdas) is may post-date the beginnings of your DB library (lambdas were added in java8; admittedly, that's a decade ago).

last item in the cart

I am learning microservices and trying to design an e-commerce website. I can't figure out how big shopping sites take care of the last item in the cart problem.
For example, I selected an item from Amazon which had just a single item available in stock. I logged in from two different accounts and placed the item in cart. I even reached the payment page from both the account and the site didn't restrict me anywhere saying that the item is not available. I am not sure after the payment page when payment from both the account is in progress, how Amazon handles it.
Few solutions which come to my mind are like:
Accept payment from both the accounts and later cancel transaction for one of them which paid later than the first. This will not be a good practice though as it will result in bas customer experience.
Keep few items in reserve and use them in case of overbooking.
I forget what Amazon is doing and implement quantity checks in Order service from Item service via REST calls, at every stage of the order. But these checks sometimes can fail when a lot of people are ordering the same item. for e.g. in flash sales
Please share if you guys have worked on similar problem and solved it even with few limitations. If I need to put any in more details, let me know.

I cannot answer how Amaozon does it, nor I think anyone could on a public forum I can tell you how I think this could be managed.
So you have to take lock on your inventory if you want to make sure you precisely map inventory to an order. If you intend to do that, question will be where you take lock. When an item gets added to the cart, when user goes for payment or when payment is done. But the problem with lock is that it will make you system slow.
So that is something you should avoid.
Rest all the options you have already covered in your question and it boils down to tradeoffs.
First point, user experience will suffer and you also need to incur the cost of the transaction.
Second option ask you to be ready to undersell or oversell.
When you keep reserves, you are basically saying that I will be underselling. This can also backfire because say you decide to reserve 5 items but you get 20 concurrent request foir checkout and payment, you will be back to the square one. But it can help in most scenarios, given you are willing to take a hit.
Doing inventory check at checkout can help you get better resolution on inventory but it will not help when you literally have last item in inventory and 10 people doing a checkout on it. Read calls even for two such request coincides you will give them inventory and back to square one.
So what I do in such scenarios, is
1. My inventory goes not as number but enum i.e critical, low, med, high, very high
Depending on some analytics we configure inventory check. For high and very high we will not do any check and book the item. for critical and we take the lock. (not exactly a db lock but we reserve the inventory for them), for low and medium we check the inventory and proceed if we have enough. All these values are configurable and help us mitigate the scenarios we have.
Another thing that we are trying is to distribute inventory to inventory brokers and assign inventory broker to some set of services to see this inventory. Even if we reserve the inventory on one broker others can continue selling freely. And there brokers regularly update the inventory master about the status of inventory. Its like Inventory master has 50 items, it distributes 5 each to all ten. After 10 mins they come back and if they need more inventory they ask for it, if they have left over (in case of failure) they drop back inventory to the master for it to be assigned to others.
The above approach will not help you resolve the issue precisely but it gives you certain degree of freedom as to how you can manage the inventory.

Consider doing:
On the payment page, you should re-check if the product is still available. This can be a simple HTTP GET.
If the GET call is slow for you, consider caching recent product added by user to some in-memory databases (eg. REDIS). Now if first users successfully processes the payment, decrease counter for that product-id in redis. And before proceeding payment for second user, check the counter of that product-id in redis.
(BONUS: Redis offers atomic operations, so you can successfully handle the race condition in ordering the product as well.)

Is checksum a good way to see if table has been modified in MySQL?

I'm currently developing an application in Java that connects to a MySQL database using JDBC, and displays records in jTable. The application is going to be run by more than one user at a time and I'm trying to implement a way to see if the table has been modified. EG if user one modifies a column such as stock level, and then user two tries to access the same record tries to change it based on level before user one interacts.
At the moment I'm storing the checksum of the table that's being displayed as a variable and when a user tries to modify a record it will do a check whether the stored checksum is the same as the one generated before the edit.
As I'm new to this I'm not sure if this a correct way to do it or not; as I have no experience in this matter.

Calculating the checksum of an entire table seems like a very heavy-handed solution and definitely something that wouldn't scale in the long term. There are multiple ways of handling this but the core theme is to do as little work as possible to ensure that you can scale as the number of users increase. Imagine implementing the checksum based solution on table with million rows continuously updated by hundreds of users!
One of the solutions (which requires minimal re-work) would be to "check" the stock name against which the value is updated. In the background, you'll fire across a query to the table to see if the data for "that particular stock" has been updated after the table was populated. If yes, you can warn the user or mark the updated cell as dirty to indicate that that value has changed. The problem here is that the query won't be fired off till the user tries to save the updated value. Or you could poll the database to avoid that but again hardly an efficient solution.
As a more robust solution, I would recommend using a database which implements native "push notifications" to all the connected clients. Redis is a NoSQL database which comes to mind for this.
Another tried and tested technique would be to forgo direct database connection and use a middleware layer like a messaging queue (e.g. RabbitMQ). Message queues enable design of systems which communicate using message. So for e.g. every update the stock value in the JTable would be sent across as a message to an "update database queue". Once the update is done, a message would be sent across to a "update notification queue" to which all clients would be connected. This will enable all of them to know that the value of a given stock has been updated and act accordingly. The advantage to this solution is that you get to keep your existing stack (Java, MySQL) and can implement notifications without polling the DB and killing it.

Checksum is a way to see if data has changed.
Anyway I would suggest you store a column "last_update_date", this column is supposed to be always updated at every update of the record.
So you juste have to store this date (precision date time) and do the check with that.
You can also add a column version number : a simple counter incremented by 1 at each update.
Note:
You can add a trigger on update for updating last_update_date, it should be 100% reliable, maybe you don't need a trigger if you control all updates.

When using in network communication:
A checksum is a count of the number of bits in a transmission unit
that is included with the unit so that the receiver can check to see
whether the same number of bits arrived. If the counts match, it's
assumed that the complete transmission was received.
So it can be translated to check 2 objects are different, your approach is correct.

Concurrency control for inventory control

There are many similar questions related to my question. But I didnt find any satisfactory ans for my question. So I am putting this question in this forum.
I have a question in concurrency control in inventory management system.
Say I have products A, B, C with quantity 2,3,4. And my application is multi user.
I have product page where user see the list of products and available quantity.
and I have check out and payment page which may take some time to reach after product page.
Now if it is multi user web application and say user 1 has ordered 2 quantity product A but order is not yet placed, user 2 can still see A with 2 quantities.
Should I temporarily(configurable time) lock the 2 quantities of product A until order is placed? Is it a good design. If yes, should I lock in java or in the database?

No, it's not a good design because it lend itself for abuse and problems. What if a user (competitor?) locks all your products during the whole month? What if another person fills up his/her shopping cart with product and then decide it's too much money and just turns off his/her device?
Best alternatives are:
1) Tell the user how much availability is of each product, but also tell him/her it could be gone if an order is not placed soon. This should also incentive sales. Do lock/unlock your products after the payment page, i.e. when there is a business commit to the operation. If available quantities are not enough anymore, send the user back a previous page, updating the amounts to whetever is available.
2) Similar to 1), but you could also update availability from time to time. Or send warnings like "the stock of some items in your cart is running low". Again, this also could prompt sales.
3) Reserve ("lock") items as they are moved to a shopping cart, but not forever. Release the items after being locked for certain amount of time. Keep informed the user. The time-out can be per the whole shopping cart or per item.
It is very important to notice that any "lock" mentioned above is a "business lock/reservation". It doesn't need to be implemented in the form of locks or any other particular technical solution. For example, 3) above can be implemented by adding fields locked_by and locked_until. While you check/update/manipulate these fields you will probably need to do it within a "technical lock". After the checks/updates are done, the technical locks are released. However, a "business lock" could still in place because locked_until has not elapsed yet and any other code will check this field to consider the product available or not. Why? Because business rules mandate so (not because there is a technical lock in place, which in fact is not).
"Technical locks" should be very quick. "Business locks" can be much longer (but never forever; always define a time limit for them).
It's hard to tell if you should lock "in Java" on "in the database" with the very little information you give. Are you using Entity Beans, for example? What are your fault tolerance requirements? Etc.
That said, in the general case it's probably better to keep locks (as long as they are "business locks") in the database, for these main reasons:
Persistence (in case of a power failure). You should also provide a mechanism to recover shopping carts. Perhaps also storing them in the DB?
Ability to interface with other environments (i.e. a corporate ERP or fullfillment system).

Should I temporarily(configurable time) lock the 2 quantities of product A until order is placed? Is it a good design.
It depends on the conversation rate of your site, i.e. the number of checkouts/the number of payments. If you have a high conversation rate, you can pre lock the quantities to get a better user experience.
If yes, should I lock in java or in the database?
You need a global lock to guarantee the correctness. If you have multiple application servers, you have to put the lock in the database.

Inventory Management should be handled by a Custom System Built from ground up. You cannot rely simply on ACID compliance for performance reasons. Implementing Transactions on Inventory during the order is a very bad idea and is not scalable. I propose the following solution.
An Inventory Management Backend App that updates the inventory as new items come in. Use row lock to update the inventory.
An Inventory Management Micro Service App to give the inventory to Order Management System as it needs the inventory to finish the order and keep track of time out.
a. If the order finished with Acknowledgement, The given inventory is already deducted by management system. We are good.
b. If the order is not finished and no Acknowledgement is received, the given inventory is added back to the inventory system after the timeout (Typically 2 - 3 minutes).
c. If the order has failed and acknowledgement is received that the order is failed, the given inventory is added back to the inventory system.
So, you don't lock the product rows until the order is finished. Instead, you do a soft lock on inventory and release if the order was not successful or failed due to an exception/App Failure.

Designing a count based access control

I would like to get some advice on designing a count based access control. For example I want to restrict the number of users that a customer can create in my system based on their account. So by default a customer can create 2 users but if the upgrade their account they get to create 5 users and so on.
There are a few more features that I need to restrict on a similar basis.
The application follows a generic model so every feature exposed has a backing table and we have a class which handles the CRUD operation on that table. Also the application runs on multiple nodes and has a distributed cache.
The approach that I am taking to implement this is as follows
- I have a new table which captures the functionality to control and the allowed limit (stored per customer).
- I intercept the create method for all tables and check if the table in question needs to have access control applied. If so I fetch the count of created entities and compare against the limit to decide if I should allow the creation or not.
- I am using the database to handle synchronization in case of concurrent requests. So after the create method is called I update the table using the following where clause
where ( count_column + 1 ) = #countInMemory#
. i.e. the update will succeed only if the value stored in the DB + 1 = value in memory. This will ensure that even if two threads attempt a create at the same time, only one of them will be able to successfully update. The thread that successfully updates wins and the other one is rolled back. This way I do not need to synchronize any code in the application.
I would like to know if there is any other / better way of doing this. My application runs on Oracle and MySQL DB.
Thanks for the help.

When you roll back, do you retry (after fetching the new user count) or do you fail? I recommend the former, assuming that the new fetched user count would permit another user.
I've dealt with a similar system recently, and a few things to consider: do you want CustomerA to be able to transfer their users to CustomerB? (This assumes that customers are not independent, for example in our system CustomerA might be an IT manager and CustomerB might be an accounting manager working for the same company, and when one of CustomerA's employees moves to accounting he wants this to be reflected by CustomerB's account.) What happens to a customer's users when the customer is deleted? (In our case another customer/manager would need to adopt them, or else they would be deleted.) How are you storing the customer's user limit - in a separate table (e.g. a customer has type "Level2," and the customer-type table says that "Level2" customers can create 5 users), or in the customer's row (which is more error prone, but would also allow a per-customer override on their max user count), or a combination (a customer has a type column that says they can have 5 users, and an override column that says they can have an additional 3 users)?
But that's beside the point. Your DB synchronization is fine.