HELP!
the case is when two or more transactions are trying to affect the same client monetary account in some external system. I need the second transaction be performed until the first one has finished.
consider:
- there are two or more transactions trying to affect the same balance
- there are multiple clients at the same time
- working with 1000 TPS with 100ms avg per transaction
ideas:
- as we are working with multi threads to support 1000TPS i'm trying to create Queues based on the Client ID. using some kind of workmanager that limit one thread by Client. So if i have 2 request with the same clientID at same time dynamically can queue the second.
tools
i'm trying to use Oracle tools for example:
- Fusion Middleware: using the Workmanager based on message context [not sure if possible because looks like context can be based only on session data] i like WorkManager because has no performance issues
- Oracle OCEP: creating a dynamic queue using the CQL [ not sure if possible and performance]
- Oracle Advance Queuing: maybe possible with transactions group.
thanks for any idea
I hope I got your problem.
In you question you asked, wether it is possible to perform a second transaction on a row before the first is completed. This is impossible! A database which follows the ACID paradigm has to be Consistent! So you can't "overtake" the first transaction!!! If you want to do that, you should use NoSQL Databases (like MongoDB, ...) where consistency is not that strong.
But maybe you want to know, if there is a Oracle view to figure out, wether a row is locked or not? Let's assume, that there is a view like that. You would check this view and if there is no lock, you start your update/delete. But you can't be sure that this will work because even 1ms after you checked it, another process can put a lock on it.
The only thing you can do is, to put a "select ... for update NOWAIT" before you UPDATE/DELETE statement.
If the row is locked, you will get a exception (ORA-00054: Resource busy). This is the recommended/"out of the box way" to let the database manage row-level-locking for you!
See the following example with the emp table. Consider: to check this out, start this code in two different sessions at the same time.
declare
l_sal number;
resource_busy exception; -- declare your own exception
pragma exception_init (resource_busy, -54); -- connect your exception with ORA-00054
begin
select sal
into l_sal
from emp
where empno = 7934
for update NOWAIT;
update emp
set sal = sal + 100
where empno = 7934;
exception
when resource_busy then
null; -- in your case, simply do nothing, if the row is locked
end;
Related
I have a application which needs to aware of latest number of some records from a table from database, the solution should be applicable without changing the database code or add triggers or functions to it ,so I need a database vendor independent solution.
My program written in java but database could be (SQLite,MySQL,PostgreSQL or MSSQL),for now I'm doing Like that:
In a separate thread that is set as a daemon my application sends a simple command through JDBC to database to be aware of latest number of the records with condition:
while(true){
SELECT COUNT(*) FROM Mytable WHERE exited='1'
}
and this sort of coding causes DATABASE To lock,slows down the whole system and generates huge DB Logs which finally brings down the whole thing!
how can i do it in a right way to always have latest number of certain records or only counting when the number changed?
A SELECT statement should not -- by itself -- have the behavior that you are describing. For instance, nothing is logged with a SELECT. Now, it is possible that concurrent insert/update/delete statements are going on, and that these cause problems because the SELECT locks the table.
Two general things you can do:
Be sure that the comparison is of the same type. So, if exited is a number, do not use single quotes (mixing of types can confuse some databases).
Create an index on (exited). In basically all databases, this is a single command: create index idx_mytable_exited on mytable(exited).
If locking and concurrent transactions are an issue, then you will need to do more database specific things, to avoid that problem.
As others have said, make sure that exited is indexed.
Also, you can set the transaction isolation on your query to do a "dirty read"; this indicates to the database server that you do not need to wait for other processes' transactions to commit, and instead you wish to read the current value of exited on rows that are being updated by those other processes.
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED is the standard syntax for using "dirty read".
I have a situation in my java, spring based web app. My server generates coupons ( a number mixed with alphabets , all random but unique) , each coupon can be applied or used by only one and only on logged in customer. They are shown on the front end to all the users, which then gets accepted/selected by the customers.But once accepted by one customer it gets assigned to him and not available to anyone else.
I tried to do synchronization of code block which checks if the coupon is already applied / availed, it worked but , cases like when two users click avail it at exact same time, it fails ( get allocated to both)
Please help.
Do not use synchronization for this. You can store the state of the coupons in a database, and work on these data in a DB transaction, using locks. So:
User tries the coupon, you get the ID
Start a DB transaction, get the coupon row from it, and lock it
Do what you need to, then invalidate the coupon
End the DB transaction, release the lock
The database do not necessarly need to be a standalone RDMS, in a simple case, even SQLite is sufficient. Anyway, DBs most certainly handle race conditions betten than you (or most of us) can.
If you prefer avoid database transactions you can use a Set with all the generated coupons and a set referencing only available coupons. When a user select a coupon in a synch block remove the coupon from available ones. The second user then fail to obtain it
In my web-service i have a user table which contains column called "HITS",now as a user in this user class/table consumes the web-service,"HITS" count is incremented by 1.which involves write operation and hence comes in optimistic/pessimistic locks.
Issue am facing is high concurrency scenario,in which 100 hits come simultaneously for same user.
Can anyone suggest me a strategy to avoid delays caused by locking a user row to update "HITS" when web-service is used simultaneously from same user's account ?
OR
you can also suggest a way to update some variable which i can monitor after every 12 hours,to generate the "HITS" value,by summing up the variable(s) or something like that ?
Am using hibernate and EJB stateless session bean.
just insert one record for every hit into table all_hits...
then on some schedule - select from that table, and insert into hits_summary table, and delete the originals.
I'm looking for a high level answer, but here are some specifics in case it helps, I'm deploying a J2EE app to a cluster in WebLogic. There's one Oracle database at the backend.
A normal flow of the app is
- users feed data (to be inserted as rows) to the app
- the app waits for the data to reach a certain size and does a batch insert into the database (only 1 commit)
There's a constraint in the database preventing "duplicate" data insertions. If the app gets a constraint violation, it will have to rollback and re-insert one row at a time, so the duplicate rows can be "renamed" and inserted.
Suppose I had 2 running instances of the app. Each of the instances is about to insert 1000 rows. Even if there is only 1 duplicate, one instance will have to rollback and insert rows one by one.
I can easily see that it would be smarter to re-insert the non-conflicting 999 rows as a batch in this instance, but what if I had 3 running apps and the 999 rows also had a chance of duplicates?
So my question is this: is there a design pattern for this kind of situation?
This is a long question, so please let me know where to clarify. Thank you for your time.
EDIT:
The 1000 rows of data is in memory for each instance, but they cannot see the rows of each other. The only way they know if a row is a duplicate is when it's inserted into the database.
And if the current application design doesn't make sense, feel free to suggest better ways of tackling this problem. I would appreciate it very much.
http://www.oracle-developer.net/display.php?id=329
The simplest would be to avoid parallel processing of the same data. For example, your size or time based event could run only on one node or post a massage to a JMS queue, so only one of the nodes would process it (for instance, by using similar duplicate-check, e.g. based on a timestamp of the message/batch).
I am stuck at some point wherein I need to get database changes in a Java code. Request is to get any record updated, added, deleted in any table of db; should be recognized by Java program. How could it be implemented JMS? or a Java thread?
Update: Thanks guys for your support i am actually using Oracle as DB and Weblogic 10.3 workshop. Actually I want to get the updates from a table in which I have only read permission so guys what do you all suggest. I can't update the DB. Only thing I can do is just read the DB and if there is any change in the table I have to get the information/notification that certain data rows has been added/deleted or updated.
Unless the database can send a message to Java, you'll have to have a thread that polls.
A better, more efficient model would be one that fires events on changes. A database that has Java running inside (e.g., Oracle) could do it.
We do it by polling the DB using an EJB timer task. In essence, we have a status filed which we update when we have processed that row.
So the EJB timer thread calls a procedure that grabs rows which are flagged "un-treated".
Dirty, but also very simple and robust. Especially, after a crash or something, it can still pick up from where it crashed without too much complexity.
The disadvantage is the wasted load on the DB, and also response time will be limited (probably requires seconds).
We have accomplished this in our firm by adding triggers to database tables that call an executable to issue a Tib Rendezvous message, which is received by all interested Java applications.
However, the ideal way to do this IMHO is to be in complete control of all database writes at the application level, and to notify any interested parties at this point (via multi-cast, Tib, etc). In reality this isn't always possible where you have a number of disparate systems.
You're indeed dependent on whether the database in question supports it. You'll also need to take the overhead into account. Lot of inserts/updates also means a lot of notifications and your Java code has to handle them consistently, else it will bubble up.
If the datamodel allows it, just add an extra column which holds a timestamp which get updated on every insert/update. Most major DB's supports an auto-update of the column on every insert/update. I don't know which DB server you're using, so I'll give only a MySQL-targeted example:
CREATE TABLE mytable (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
somevalue VARCHAR(255) NOT NULL,
lastupdate TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX (lastupdate)
)
This way you don't need to worry about inserting/updating the lastupdate yourself. You can just do an INSERT INTO mytable (somevalue) VALUES (?) or UPDATE mytable SET somevalue = ? WHERE id = ? and the DB will do the magic.
After ensuring that the DB server's time and Java application's time are the same, you can just fire a background thread (using either Timer with TimerTask, or ScheduledExecutorService with Runnable or Callable) which does roughly this:
Date now = new Date();
statement = connection.prepareStatement("SELECT id FROM mytable WHERE lastupdate BETWEEN ? AND ?");
statement.setDate(1, this.lastTimeChecked);
statement.setDate(2, now);
resultSet = statement.executeQuery();
while (resultSet.next()) {
// Handle accordingly.
}
this.lastTimeChecked = now;
Update: as per the question update it turns out that you have no control over the DB. Well, then you don't have much good/efficient options. Either just refresh the entire list in Java memory with entire data from DB without checking/comparing for changes (probably the fastest way), or dynamically generate a SQL query based on the current data which excludes the current data from the results.
I assume that you're talking about a situation where anything can update a table. If for some reason you're instead talking about a situation where only the Java application will be updating the table that's different. If you're using Java only you can put this code in your DAO or EJB doing the update (it's much cleaner than using a trigger in this case).
An alternative way to do this is to funnel all database calls through a web service API, or perhaps a JMS API, which does the actual database calls. Processes could register there to get a notification of a database update.
We have a similar requirement. In our case we have a legacy system that we do not want to adversely impact performance on the existing transaction table.
Here's my proposal:
A new work table with pk to transaction and insert timestamp
A new audit table that has same columns as transaction table + audit columns
Trigger on transaction table to dump all insert/update/deletes to an audit table
Java process to poll the work table, join to the audit table, publish the event in question and delete from the work table.
Question is: What do you use for polling? Is quartz overkill? How can you scale back the polling frequency based on the current DB load?