How to make sure that a particular DB transaction happens only once. I am making a payment request from my mobile (more than once), but the backend should only execute only one. Once the request is executed its status is marked as COMPLETED. But in case of multiple request, before one request gets completed another starts is execution so the payment is done twice before the status to be marked COMPLETED. How to solve this problem? I am using Java as backend. How can synchronize() help to solve this problem?
You can try to add a lock around the code. That way only one thread can enter at any given time.
If you make one request, then the other request have to wait until the request is finish.
This is a known issue as Double Post.
Preventing parallel access to the method with Synchronize and lock will not help you, as the requests will be proceed in series.
Using client Side methods may help, but is not enough, as many things may happen at client side.
If you want to prevent it at Server Side (this is the correct way to do), you can add a hidden field to the client form (some unique hash string) and send it to the server with every request. In the Server side component, you can check if a request with that hash is already received, and if so, return an error code to client.
You can also persist the hash with your Data and make it a unique field, so the first request that reach your database will be persisted, and the others will see unique field errors.
Related
My programme is a notification service, it basically receives http requests(client sends notifications) and forwards them to a device.
I want it to work the following way:
receive client notification request
save it to the database(yes, i need this step, its mandatory)
async threads watch new requests in database
async threads forward them to the destination(device).
In this case the programme can send client confirmation straight away after the step 2).
Thus, not waiting for the destination to respond(device response time can be too long).
If I stored client notification in memory i would use BlockingQueue. But I need to persist my notifications in db. Also, I cannot use Message Queues, because clients want rest endpoints to send notifications.
Help me to work out the architecture of such a mechanism.
PS In Java, Postgresql
Here are some ideas that can lead to the solution:
Probably the step 2 is mandatory to make sure that the request is persisted so that rather it will be queried. So we're talking about some "data model" here.
With this in mind, if you "send" the confirmation "right away after the step 2" - what if later you want to do some action with this data (say, send it somewhere) and this action doesn't succeed. You store it on disk? what happens if the disk is full?
The most important question is what happens to your data model (in the database) in this case? Should the entry in the database still be there or the whole "logical" action has failed? This is something you should figure out depending on the actual system the answers can be different.
The most "strict" solution would use transactions in the following (schematic) way:
tr = openTransaction()
try {
saveRequestIntoDB(data);
forwardToDestination(data);
tr.commit();
} catch(SomeException ex) {
tr.rollback();
}
With this design, if something goes wrong during the "saveRequest" step - well, nothing will happen. If the data is stored in db, but then forwardToDestination fails - then the transaction will be rolled back and the record won't be stored in DB.
If all the operations succeed - the transaction will be committed.
Now It looks like you still can use the messaging system in step 4. Sending message can be fast and won't add any significant overhead to the whole request.
On the other hand, the benefits are obvious:
- Who listens to these "notifications"? If you send something and only one service should receive and process the notification how do you make sure that others won't get it? How would you implement the opposite - what if all the services should get the notification and process it independently?
These facilities are already implemented by any descent messaging system.
I can't really understand the statement:
I cannot use Message Queues, because clients want rest endpoints to send notifications.
Since the whole flow is originated by the client's request I don't see any contradication here. The code that is called from rest endpoint (which is after all is a logic entrypoint that should be implemented by you) can call the database, persist the data and then send the notification...
Consider user cart and checkout: a customer can perform addItemToCart action which will be handled by main DB instance. However, getUserCartItems action might be performed on Read Replica and it might not contain result of the first action yet due to Replica Lag. Even if we try to minimize this lag, still it's possible to hit this case, so I'm wondering what solutions have you tried in production?
According to #Henrik answer, we have 3 options:
1. Wait at user till consistent.
This means we need to perform polling (regular or long polling) on the client and wait until Replica will receive update. However, I assume Replica Lag shouldn't be longer than 1-5 secs. Also, the less Replica Lag, the more performance down we will have.
2. Ensure consistency through 2PC.
If I understood correctly, we need to combine both addItemToCart insert and getUserCartItems select into one aggregate operation on backend and return getUserCartItems as addItemToCart response. However, the next request might still not get updated info due to lag… Yes it returns immediate confirmation about successful operation and the application can continue, however proceeding to checkout requires user cart items in order to show price correctly, so we are not fixing the problem anyway.
3. Fool the client.
Application stores/caches all successfully send data and uses it for showing. Yes, this is a solution, but it definitely requires additional business logic to be implemented:
Perform getUserCartItems request;
if (getUserCartItems returned success)
Store addItemToCart in local storage;
else
Show error and retry;
Perform getUserCartItems request;
if (getUserCartItems contains addItemToCart ID)
Update local storage / cache and proceed with it.
else
Use existing data from local storage;
How do you deal with eventual inconsistency?
The correct answer is to NOT send SELECT queries to a read slave if the data needs to be immediately available.
You should structure your application such that all real-time requests hit your master, and all other requests hit one of your read slaves.
For things where you don't need real-time results, you can fool the user quite well using something like AJAX requests or websockets (websockets is going to make your application a lot more resource friendly as you won't be hammering your backend servers with multiple AJAX requests).
Hy, I have made a java application using asterisk-java and from it I can receive a call and can initiate outbound as well. But I am facing one problem that whenever I bulk outbound calls to say 50k users, the application can handle only those who answered the calls not those who unanswered or didn't responded. Also as I have set the value of
OriginateAction.setAsync(true)
in my outbound calling application so I am getting success response to every call, which means call is successfully initiated, but if this value is not set then although I can check the response (error: incase the user didn't responded the call), but then in this case outbound bulk drops from 50k to 3k.
My Ideal solution would be if the call is not answered and is hung up eventually then I can redirect it to some AGI script, which would show its record (hangup cause, Answered/Busy/etc)
Kindly guide.
You have manage number of call yourself.
Asterisk not designed to know how many calls at once your hardware/trunks can support,that interface just for place SINGLE call.
Check vicidial dialler or other project writed before.
So, to explain this, I'll start out by going through the application stack.
The system is running JSP with jQuery on top, talking through a controller layer with a service layer, which in turn utilizes a persistence layer implemented in Hibernate.
Now, traditionally, errors like having overlapping contracts has been handled through throwing exceptions up through the layers until they're translated into an error message for the user.
Now I have an object that at any given time can only be tied to one contract. At the moment, when I save a contract, I look at all of these objects and check if they're already covered by an existing contract. However, since multiple clients can be saving at any given time, this introduces the risk of getting past the check on two separate contracts, leading to one object being tied to two contracts at the same time.
To combat this, the idea was to use a queue, put objects into the queue from the main thread, and then have a separate thread take them out one by one, saving them.
However, here's the problem. For one, I would like the user to know that the saving is currently happening, for another, if by accident the scenario before happens, and two contracts with the same object covering the same time is in the queue, the second one will fail, and this needs to be sent back to the user.
My initial attempt was to keep data fields on the object put into the queue, and then check against those in a blocking wait, and then throw an exception or report success based on what happens. That deadlocked the system completely.
Anyone able to point me in the right direction with regards to techniques and patterns I should be using for this?
I can't really tell why you have a deadlock without seeing your code. I can think of some other options though:
Poll the thread to see its state (not as good).
Use some kind of eventing system. You would have an event listener (OverlappingContractEventListener perhaps) and then you would trigger the event from the thread when the scenario happens. The event handler would need to persist this information somehow.
If you are going for this approach, then on the client side you will need to poll.
You can poll a specific controller (using setInterval and AJAX) that looks up the corresponding information for the object to see what state its in. This information should have been persisted by your event listener.
You can use web workers (this is supported in Chrome, Firefox, Safari, and Opera. IE will support it in 10) and perform the polling in the background.
There is one other way that doesn't involve eventing. It depends on you figuring out the source of your deadlock though. Once you fix the source of your deadlock you can do one of two things:
Perform an AJAX call to the controller. The controller will wait for the service to return information. The code to issue feedback to the user will be inside the success handler of your controller.
Use a web worker to perform the call in the background. The web worker would also perform an AJAX call and wait for the response.
Shouldn't you be doing the check for duplicate contracts in the database? Depending on the case, you can do this with a constraint, trigger, o stored procedure. If it fails, send an exception up the stack. That's normally the way to handle things like this. You can then catch the exception in jQuery and display an error:
jQuery Ajax error handling, show custom exception messages
Hope this helps.
I want to make an AJAX call to my Java webapp. The Java webapp will in turn make an asynchronous return call elsewhere. The result of that call will then be returned as the result of AJAX request.
The crux of my question is what would I do with the HttpRequest whilst I'm waiting for the second call to return?
Do I just block and wait for the call within the AJAX handler method or do I store the request somewhere and wait for a callback? How would I handle errors / timeouts?
For those who care further information as to how I arrived at this situation follows:
This is part of an XMPP based instant messaging system. There is one global support user which is displayed as an icon on every page in our webapp. I also want to display the presence of this user, so, I could just use the IM system to request this users presence on every single page load for every user and eventually DDOS myself. Instead I want to have a single user query the presence from the webapp periodically and cache the result.
The AJAX call is therefore to the server which will then either return the cached presence or query the XMPP server asynchronously.
You shouldn't have to block and wait for the AJAX call. That is, don't make the call synchronously. What you should do on the Java side is figure out a way to block while you wait for the response to come back from your asynchronous call (i.e., figure out to a way to make the request synchronously. The performance hit will be on the first call for any new data. Subsequent calls will hit the cache, so you should be good). You can maintain a cache for this data, so you can check the cache first to see if the data exists. If it doesn't make the call and store the result in the cache. Otherwise, grab the data from the cache and send it back to the view. Since AJAX is asynchronous, your callback will be called as soon as the data comes back from the server.
here is what i would do:
when the page startup, init an job to retrieve data array you need for that specific page, you need to identify the job and the job result for later usage
use ajax from the page to poll for the job result, once the job is done, the poll finishes and returned with data
cache the entries you have requested as Vivin indicated
cache the job result on your server and give it a time-out option
HTTP requests, i.e. HttpServletRequest objects are not serializable. Therefore you cannot store them in a persistent store of any sort, for the duration of the call. It doesn't make sense anyway to store the request, for its life is limited to the duration of the HTTP request itself, given the stateless nature of the HTTP protocol.
This effectively means that you have to hold on to the HttpServletResponse object for the duration of the call. The HttpServletRequest object is no longer needed, once the parsing of the HTTP request is performed, and once all the data is available to your application; it is the response object that is of importance in your context.
The response could be populated with the cached copy of the user status. If the copy in the cache is stale, you might want to refresh it synchronously from the XMPP server (after all, it affects the performance of just one page load). You could query asynchronously from within the application server, but some result must be returned to the browser (so there might be a few edges cases that need to be taken care of).