My aim is to built a trusted synchronization between both local DB and server DB with NO conflictions or deadlock.
I suppose that a user may request his transaction using DML (insert, update and delete) from his local DB where there is NO access to the Interent and when he wants he asks for sync and send all the transaction with the data i need to the server via RESTful API.
My problem when tow sync operations has performed in a different time and different user which may cause a conflict (ex. the first user update a row and the second user tries to update the same row but another column while they were both offline before sync operations).
In the local DB i track every transaction on another table for every type of transaction (inserted_table , updated_table and deleted_table) every one of them contain table_id,row_id and time.
My question when i send these information to the server for the first user when he asks for sync and then after while the second user makes sync too how i can know the information for the first sync process for the first user(like time of the transaction in the local for the first user) to make rollback if i need and how to mange the conflict resolution?
The whole synchronization logic is up to you to decide how it should work!
One solution to your problem is to add columns STATUS(varchar) and LASTMODIFICATION(datetime). When syncing the databases, you can choose the most recent by comparing the LASTMODIFICATION field and overwrite the data with the newest update.
Related
I would like to understand how Firestore works for counting read operations.
firestore lecture operations
I have a database and an application for mobile devices so to make Firestore go offline, nothing needs to be done.
When my application is opened the user receives data from a collection of | n | documents.
I wanted to understand if a user who opens the application with the internet will perform read operations from the database even if there are no changes in the database?
Or every time it accesses if there are no changes, no read operations are added since the data still resides in the cache?
if a user who opens the application with the internet on will perform read operations from the database even if there are no changes in the database?
If you are using get() call, the Firestore SDK will always try to get the data from the server, no matter if the data is changed or not. You'll be able to use the data from the cache, only if the device goes offline. However, if you trying to listen for changes in real-time using a SnapShotListener, according to the official documentation:
The initial state can come from the server directly, or from a local cache. If there is a state available in a local cache, the query snapshot will be initially populated with the cached data, then updated with the server's data when the client has caught up with the server's state.
In other words, if the server says that you have no new added/updated/deleted documents, you get the data from the cache. However, there is something else you should take care regarding the duration of time you can get the data from cache:
Also, if the listener is disconnected for more than 30 minutes (for example, if the user goes offline), you will be charged for reads as if you had issued a brand-new query.
Another possible charge might also come from:
There is a minimum charge of one document read for each query that you perform, even if the query returns no results.
But all these charges are normal according to how is Firestore designed.
I'm currently developing an application in Java that connects to a MySQL database using JDBC, and displays records in jTable. The application is going to be run by more than one user at a time and I'm trying to implement a way to see if the table has been modified. EG if user one modifies a column such as stock level, and then user two tries to access the same record tries to change it based on level before user one interacts.
At the moment I'm storing the checksum of the table that's being displayed as a variable and when a user tries to modify a record it will do a check whether the stored checksum is the same as the one generated before the edit.
As I'm new to this I'm not sure if this a correct way to do it or not; as I have no experience in this matter.
Calculating the checksum of an entire table seems like a very heavy-handed solution and definitely something that wouldn't scale in the long term. There are multiple ways of handling this but the core theme is to do as little work as possible to ensure that you can scale as the number of users increase. Imagine implementing the checksum based solution on table with million rows continuously updated by hundreds of users!
One of the solutions (which requires minimal re-work) would be to "check" the stock name against which the value is updated. In the background, you'll fire across a query to the table to see if the data for "that particular stock" has been updated after the table was populated. If yes, you can warn the user or mark the updated cell as dirty to indicate that that value has changed. The problem here is that the query won't be fired off till the user tries to save the updated value. Or you could poll the database to avoid that but again hardly an efficient solution.
As a more robust solution, I would recommend using a database which implements native "push notifications" to all the connected clients. Redis is a NoSQL database which comes to mind for this.
Another tried and tested technique would be to forgo direct database connection and use a middleware layer like a messaging queue (e.g. RabbitMQ). Message queues enable design of systems which communicate using message. So for e.g. every update the stock value in the JTable would be sent across as a message to an "update database queue". Once the update is done, a message would be sent across to a "update notification queue" to which all clients would be connected. This will enable all of them to know that the value of a given stock has been updated and act accordingly. The advantage to this solution is that you get to keep your existing stack (Java, MySQL) and can implement notifications without polling the DB and killing it.
Checksum is a way to see if data has changed.
Anyway I would suggest you store a column "last_update_date", this column is supposed to be always updated at every update of the record.
So you juste have to store this date (precision date time) and do the check with that.
You can also add a column version number : a simple counter incremented by 1 at each update.
Note:
You can add a trigger on update for updating last_update_date, it should be 100% reliable, maybe you don't need a trigger if you control all updates.
When using in network communication:
A checksum is a count of the number of bits in a transmission unit
that is included with the unit so that the receiver can check to see
whether the same number of bits arrived. If the counts match, it's
assumed that the complete transmission was received.
So it can be translated to check 2 objects are different, your approach is correct.
The problem I have right now deals with the SQL UPDATE and DELETE statements concurrently. If the program is only called one after the other then there is no problems, however, if two people decide to run the program it might fail.
What my program does:
A program about food which all has a description and a date of when that description was made. As people enter the description of the food it gets entered into a database where you can quickly retrieve the description. If the description is lets say 7 days old then we delete it cause its outdated. However, if a user enters a food already in the database with a different description then we update it and change the date. The deletion happens after the update/insertion (those that dont need updating will be inserted and then the program checks for outdated things in the database and deletes them).
The problem:
Two people run the program and right as one person is trying to update a food, the other clears it out with the deletion cause it just finished. The update will not happen, and the program will continue with the rest of the updates (<- I read that this is because my driver doesn't stop. Some drivers stop updating if there is an error).
What I want to do:
I want my program to stop at the bad update or grab that food position and restart the process/thread. The restarting will include sorting out which foods needs to be updated or inserted. Therefore, the bad record will be moved into the inserting method and not the update. The update will continue where it left off. And all's well.
I know this is not the only way, so different methods on how to solve this problem is welcome. I have looked up that you can use an upsert statement, but that also has race conditions. (Question about the upsert statement: If I make the upsert method synchronized will it not have race conditions?)
Thanks
There are different pratical solutions to your problem depending on jout jdbc connection management.
If the application is a client server one and it uses a dedicated persistent connection (i.e. it opens a jdbc connection at program startup and it closes when the program shutdowns) for each client you can use a select for update statement.
You must issue a select for update when displaying records to the user and when the user does its action you do what is needed and commit.
This approach serializes the dabatabase operations and if you show and lock multiple records it may not be feasible.
A second approach is usable when you have a web application with a connection pool or when you don't have a dedicated connection you can use for the read and update/delete operation. In this case you have this scenario
user 1 selects its data with jdbc connection 1
user 2 selects its data (the same as user 1) with jdbc connection 2
user 2 submit data causing some deletions with jdbc connection 3
user 1 submit data and lot an update beacuse the data was deleted with jdbc connection 2
Since you cannot realy on the same jdbc connection to lock the data you read, you can issue a select for update before updating the data and check if there are data. If you have the data you can update them (and they will not be deleted by other sessions since every delete command on the same data is waiting your select for update to terminate); if you don't have the data because they where deleted during user display you must reinsert them. You delete statement must have a filter on the date column that represent the last update.
You can use other approaches and avoid the select for update using for example an
update food-table set last_update=? where id=? and last_update=<the last update you have in java program>
and you must check that the update statement did update a row (in jdbc executeUpdate returns the number of rows modified, but you did not specifiy if you are using "plain" JDBC or some sort of framework) and if it did not update a row you must isse the insert statement.
Set transaction level to serializable in java code. Then your statements should look like:
update food_table set update_time = ? where ....
delete from food_table where update_time < ?
You may get an serializable exception in either case. In the case of the update you will need to reinsert the entry. In the second case, just ignore and run again.
I have a soap-based web service with Java + Mysql.
The web services consist in save and send as a response generated documents. Each user has a limited number of documents available. This service provide documents to external systems, so, i have to know the documents available any time for an specific user.
To improve this a build a trigger that updates the user row when a new document is created.
CREATE TRIGGER `Service`.`discount_doc_fromplan`
AFTER INSERT ON `Service`.`Doc` FOR EACH ROW
UPDATE `Service`.`User` SET User.DocAvailable = User.DocAvailable - 1 where User.id = NEW.idUser
The problem comes when an user tries to create 2 or more documents at the same time because of their systems. This give me a "Deadlock found when trying to get lock".
Somebody has an idea to improve this without the deadlock problem and at the same time the right number of documents available?. This is my first web service. Thanks.
You are trying to implement your business logic inside a database trigger. Instead of trigger, you can implement this logic in either (1) your web service application middleware or (2) a stored procedure. I prefer approach (1) though. The basic code in either will collect all inserts in Doc table by a user in a cumulative counter and at the end of all inserts, update the User table DocAvailable = DocAvailable -counter in one go. You can do this in a transaction so that you can rollback in case of a problem. You will have to read the available Doc quota for the user before starting the transaction.
I am just trying to make a project in which there might be many employee and doctors may access. it will contain a database and interface and i want to do it in java.But now i am facing a problem -i am not sure how to make the database be able to update at a time ,i mean when an employee update the database the other can access the updated database.i have previously use database in the local host where user is one.
You can use synchronized blocks or monitorfunctions with semaphors in Java.
Put this around the method calling the database and you can control number of simultaneous access to the database.