i am going to call a function which will retrieve some data value from database. but before that i am sending those data. i am just checking whether those data properly inserted or not with this function call. but inserting data taking some time to insert into the database but my function calling starts before it actually inserts the data into the database. As because of that it is finding that no data is inserted in the database. Can any one tell how do i resolve this issue. How to synchronize this. whether i should get the proper result after the proper insertion into the database. i cant use here runnable interface or thread class. the think that i have to do is to call the data access function after certain time so that data gets enough time to get inserted into the database. please help me out.
Don't know what language you are using, but maybe the function has a parameter which causes it to wait until the query finishes before returning? Something mentioning the word "synchronous?"
Use the Database Driver
I'm not familiar with JDBC so I'm not sure which tools are/aren't available to you, but it seems like you're doing more work than you need to.
Typically the database driver will inform you whether the query was executed successfully, so you should not need to have your application query the data afterward to verify that the data is there. Instead, ask the driver for errors to see whether there was a problem with the query.
If you're inserting a large about of data and your database supports it, you may want to use a transaction to perform your insert. This will pass all of the data into the db, attempt the insert, and warn you of any problems.
If there are problems with the transaction, you can roll back, and the database state will be the same as when you started (obviously you will need to handle the errors to save your data). If there are no problems, you can finish committing the transaction, and rest assured that the database state matches the application state.
Alternatives
If for some reason the above methods won't work, you can try to resolve the race condition using an event pattern. In simple terms, you want to raise an event when the data is done inserting to alert the validator that it can start reading data. The validator will listen for that event and trigger when it hears it.
Related
I am developing a java application that loads certain things from a database, such as client records and product info. When the user navigates to say the 'products' tab, I query for products in the database and update a table with that information.
I am wondering if there is a way to see if the query results have changed since the last check, in order to avoid querying and loading all info from the database, and instead just load updates. Is there a way to do this, or perhaps just load changes only from a query into my table list? My goal is to make the program run faster when switching between tabs.
I am wondering if there is a way to see if the query results have changed since the last check
Stated differently, you want a way to automatically answer the question “is this the same result?” without retrieving the entire result.
The general approach to this problem would be to come up with some fast-to-query proxy for the entire state of the result set, and query that instead.
Once you have determined a stable fast computation for the entire result set, you can compute that any time the relevant data changes; and only poll that stored proxy to see whether the data has changed.
For example, you could say that “the SHA-256 hash of fields lorem, ipsum, and dolor” is your proxy. You can now:
Implement that computation inside the database as a function, maybe products_hash.
Create a latest_products_hash table, that stores created timestamp and products_hash that was computed at that time.
In your application, retrieve the most recent record from latest_products_hash and keep it for reference.
In the database, have a scheduled job, or a trigger on some event you decide makes sense, that will compute and store the products_hash in latest_products_hash automatically without any action from the application.
To determine whether there have been updates yet, the application will query the latest_products_hash table again and compare its most recent record with the one the application stored for reference.
Only if the latest_products_hash most-recent value is different, then query the products table and get the full result set.
That way, the application is polling a much faster query (the most-recent record in latest_products_hash) frequently, and avoiding the full products query until it knows the result set will be new.
I have an Oracle DB with a lot of tables ... How can I write something which listens to the Oracle DB and if any changes are made to the DB, it calls another program which does some processing
It would be asnync
I just need to trigger a java program if there is any kind of DML happening .. I dont want details about which table or what rows or new/old values ...
If there is a DML on any table in a DB call a java program thats it
http://docs.oracle.com/cd/B14117_01/win.101/b10118/o4o00118.htm
OracleDB has some events. Just try to use them; For example, when the db is updated - And to call another java program:
Runtime.getRuntime().exec(/Here the argument/);
I fear you're setting yourself up for failure here. I suggest a related, but slightly different course of action.
Rather than trigging your processing on every data change, consider instead having your processing run every X minutes/hours/whatever using something like Cron or Quartz.
If you're worried about having it run when no changes have been made, you can add triggers to your tables to update a "last updated" table and you can abort if no changes have been made since the last run.
This avoids the biggest concern you would have with an "on-update-processor", namely what do you do if there's an update, which triggers a process, and while that process is running another update happens. Do you trigger another process? What if they conflict? I think you'll find it better to allow there to be a slight delay between the update and the synchronization process.
That's a pretty tall order for a question.
For starters, you are going to need a way to detect that a DML operation (INSERT, UPDATE, DELETE) has occurred.
For auditing on individual tables, we can use TRIGGERS.
CREATE TRIGGER [BEFORE|AFTER] [INSERT][UPDATE][DELETE] ... [FOR EACH ROW]
The normative action is to record information about the information (whatever is needed later) in an "audit log" table. You have to make a decision whether the operations you need to perform are synchronous (should happen before the DML completes) or whether those operations can be asynchronous (can happen anytime after the DML completes).
Normally, synchronous processing is done in PL/SQL, but Oracle does provide a mechanism for calling external procedures (EXTPROC), though you wouldn't want to do that from a TRIGGER.
Oracle also provides "fine grained auditing". The information gathered by FGA is not at a "row by row" level, like we can get with a FOR EACH ROW trigger.
So, it really depends on what you are trying to achieve.
The next step is figuring out how you are going to get this information to your external process. Is that process going to periodically poll table (or set of tables), are you going to use AQ Advanced Queueing, or some other queue mechanism.
I am trying to sync multiple databases whose items have GUID for IDs, meaning that one item has the same ID on all databases.
My question is:
If i modify or create on item on 1 database, and want to synchronize this change to the other database should i:
1.) Check if the item is new or just modified, if its new then use the save() function, if its modified then use the update() function
or
2.)Do not check if its new or modified and just use the saveOrUpdate() function?
After seeing your use case in the comments, I think the best approach is to track (on both the client and server) when the last updated/last synced time was. In the event that the last sync time is null, or comes before the last updated time, you know that the data needs to be synced.
Now, on to the heart of your question: how to sync it. The client need not know the state of a server when it sends an object to you. In fact, it shouldn't. Consider the case where the client posts an object, your server receives it and process it, but the connection dies before your client receives the response. This is a very valid scenario and will result in a mis-match of data. As a result, any way that you try to determine whether or not the server has received an object (from the client) is likely to end up in a bad state.
The best solution is really to create an idempotent endpoint on the server (an upsert method, or saveOrUpdate as you referred to it in your question) which is able to determine what to do with the object. The server can query it's database by primary key to determine if it has the object or not. If it does, it can update, if not, it can insert.
Understandably, performance is important as well as the data. But, stick with primary keys in the database and that one additional select query you add should be extremely minimal (sub-10ms). If you really want to squeeze some more performance out, you could always use memcache or redis as a caching layer to determine if you have a certain GUID in your database. This way, you only have to hit memory (not your database) to determine if an object exists or not. The overhead of that would be measured only in the latency between your web server and cache server (since a memory read is incredibly cheap).
tl;dr
Upsert (or saveOrUpdate) is the way to go. Try not to track the state of one machine on another.
I have some problem like this.
I am accessing a database which is currently having over 100,000 data in new entry table.
Now I want to write a listener, means if any new record insert to table from somewhere else I have to get a notification.
My question is: What is best and fastest way to do this? because for a day there should have around 500 new data in the new entry table. Is is suitable to check the database every time using a thread?
Im using Java to do this with MySQL.
Please advice me.
I am not sure whether there is any listener that exists for Mysql changes. So it wouldn't be straight forward to get these details.
But there is something called 'The Binary Log' in mysql, which contains “events” that describe database changes such as table creation operations or changes to table data.
So one way to track the changes can be polling these logs. The challenge is that these logs are written in binary format. Mysql provides a utility called mysqlbinlog to process these logs in text format.
Here is one java parser for your rescue, which can read the mysql binary logs:
https://github.com/tangfl/jbinlog
Integrating all this bits and pieces , you may be able to get what you need.
try out this...
numero = stmt.executeUpdate(query, Statement.RETURN_GENERATED_KEYS);
Take a look at the documentation for the JDBC Statement interface.
I used java timer class for as an alternative to this solution. Now it works fine. It checks the database in every 10 seconds and if the condition true, it will execute what I want.
I have created a database application using java swing. My program retrieves data from the database whenever I call the find class. The find class simply creates a statement, get the current database connection, then execute the statement. The returned values from the database will be placed on a ResultSet then will be displayed on a jTable. The problem is this:
I opened the find class, the result was displayed. Then I go to SQLyog or HeidiSQL (applications to manipulate the database), edit the values which was displayed on my program, then save. I went back to my program, close the find class then reopen it, I still get the previous data, not the edited one. Please help. Updates must be displayed once the find class is open. The only way for me to get the updated data is to close my entire program then reopen it, which I dont want to do.
EDIT:
This is what I tried. Basically, once my program creates a connection to the database the first time, I save it to another class which makes the connection always open(I assume). So whenever I want to create a query, I'll just call the class to get the connection. What I did now is, after executing the query from the find class, I close the connection with the .close() function. It works, but do I really need to do close the connection every time? Again this is just a desktop application, not a web program.
I think you're having a transaction isolation issue. Some drivers start an implicit transaction when it grabs the connection so you're seeing a snapshot of the database at that time. You probably want a READ COMMITTED level which should show the database with all committed transactions applied.
Looks like either you have not commit the edit or you need to refresh data by refreshing query.