I need to get the only data which is updated, I am getting complete data which is affecting the performance. So whenever a client hit the API I want to send the only changes (updated data) from DB which is related to client.
There is a scenario just like facebook. If user goes offline at 12 o'clock for 1 hour i.e he will be get online at 1 o'clock. Now I need to send the notifications to him that which activities are done within offline time period.
I am using timestamp right now but Is their any other better option ?
I don't want to use timestamp or flag for last session ending time.
Thanks in advance
Try to use Versioning concept there u can get latest record .. or try to get latest record from database of that client.
There must be a column which might be an identification for updates, like last_modified_date or something. Apply the same in where clause and execute the query.
I am not sure but i know in mongo db oplog can be use to monitored changes in database , i guess like that mysql log can be use to do such monitoring if you can access it with your code. may be i am still not sure you can give it a try
Related
I'm currently developing an application in Java that connects to a MySQL database using JDBC, and displays records in jTable. The application is going to be run by more than one user at a time and I'm trying to implement a way to see if the table has been modified. EG if user one modifies a column such as stock level, and then user two tries to access the same record tries to change it based on level before user one interacts.
At the moment I'm storing the checksum of the table that's being displayed as a variable and when a user tries to modify a record it will do a check whether the stored checksum is the same as the one generated before the edit.
As I'm new to this I'm not sure if this a correct way to do it or not; as I have no experience in this matter.
Calculating the checksum of an entire table seems like a very heavy-handed solution and definitely something that wouldn't scale in the long term. There are multiple ways of handling this but the core theme is to do as little work as possible to ensure that you can scale as the number of users increase. Imagine implementing the checksum based solution on table with million rows continuously updated by hundreds of users!
One of the solutions (which requires minimal re-work) would be to "check" the stock name against which the value is updated. In the background, you'll fire across a query to the table to see if the data for "that particular stock" has been updated after the table was populated. If yes, you can warn the user or mark the updated cell as dirty to indicate that that value has changed. The problem here is that the query won't be fired off till the user tries to save the updated value. Or you could poll the database to avoid that but again hardly an efficient solution.
As a more robust solution, I would recommend using a database which implements native "push notifications" to all the connected clients. Redis is a NoSQL database which comes to mind for this.
Another tried and tested technique would be to forgo direct database connection and use a middleware layer like a messaging queue (e.g. RabbitMQ). Message queues enable design of systems which communicate using message. So for e.g. every update the stock value in the JTable would be sent across as a message to an "update database queue". Once the update is done, a message would be sent across to a "update notification queue" to which all clients would be connected. This will enable all of them to know that the value of a given stock has been updated and act accordingly. The advantage to this solution is that you get to keep your existing stack (Java, MySQL) and can implement notifications without polling the DB and killing it.
Checksum is a way to see if data has changed.
Anyway I would suggest you store a column "last_update_date", this column is supposed to be always updated at every update of the record.
So you juste have to store this date (precision date time) and do the check with that.
You can also add a column version number : a simple counter incremented by 1 at each update.
Note:
You can add a trigger on update for updating last_update_date, it should be 100% reliable, maybe you don't need a trigger if you control all updates.
When using in network communication:
A checksum is a count of the number of bits in a transmission unit
that is included with the unit so that the receiver can check to see
whether the same number of bits arrived. If the counts match, it's
assumed that the complete transmission was received.
So it can be translated to check 2 objects are different, your approach is correct.
I have a task to change the status of users in the IDM. The solution I chose is naive: looping each one and calling KeyCloak's service using REST.
However, I've noticed that this consumes a lot of time. I thought that something like bulk update (equivalent to SQL) might solve the issue, but I didn't find it in KeyCloak's API.
Does anyone know how to fix it? Thanks for help!
Do you have access to Keycloak's database? If that's the case, you may update users' data with SQL sentences. The schema is pretty straightforward to understand, I've made bulk updates in this fashion before.
What do you mean by "status"? If you mean the "enabled" status, your update will look like this:
UPDATE user_entity SET enabled = (value) WHERE (your conditions)
AFAIK, there's no way to bulk update through REST or the admin console.
Good luck!
i have an application that receive Id from Web service and store it into database if received Id for any reason like connection failed could not store in database i want to know what id was.
is there any way to do it or is there any sample code?
i want use java language.
Use triggers. If you use triggers than you can revoke the transactions in sql.
Do you want a way to make sure that you don't lose the ids even if there is failure in the connection?
If so, why don't you save the last Id you receive in your code that way, you can check if there are more IDs after the last one in case of an outage, Or in your code you can make sure that the web service is reachable, and if it isn't store the IDs till the connection is restored.
I have some problem like this.
I am accessing a database which is currently having over 100,000 data in new entry table.
Now I want to write a listener, means if any new record insert to table from somewhere else I have to get a notification.
My question is: What is best and fastest way to do this? because for a day there should have around 500 new data in the new entry table. Is is suitable to check the database every time using a thread?
Im using Java to do this with MySQL.
Please advice me.
I am not sure whether there is any listener that exists for Mysql changes. So it wouldn't be straight forward to get these details.
But there is something called 'The Binary Log' in mysql, which contains “events” that describe database changes such as table creation operations or changes to table data.
So one way to track the changes can be polling these logs. The challenge is that these logs are written in binary format. Mysql provides a utility called mysqlbinlog to process these logs in text format.
Here is one java parser for your rescue, which can read the mysql binary logs:
https://github.com/tangfl/jbinlog
Integrating all this bits and pieces , you may be able to get what you need.
try out this...
numero = stmt.executeUpdate(query, Statement.RETURN_GENERATED_KEYS);
Take a look at the documentation for the JDBC Statement interface.
I used java timer class for as an alternative to this solution. Now it works fine. It checks the database in every 10 seconds and if the condition true, it will execute what I want.
I have ran into a small issue with an Android App that I am currently working on.
I have a list of points of interest declared in an XML file. The POI's look like this:
<hotel>
<name>Hotel Atlas</name>
<latitude>45.612079</latitude>
<longitude>25.660756</longitude>
<thumb_url>/hotels/atlas.jpg</thumb_url>
<phone>039999999</phone>
</hotel>
The problem that I have is that there are about 200 points of interest and every time the users access the app the XML is read every time from an online location and that means high volumes of internet traffic. The XML is stored online so I can update it every time I want to add another POI and not force users to update the app. Is there any way to send this data to the app and not require to download the entire XML?
I have not yet found an optimum sollution for this and decided to ask for some opinions.
Thanks, Vlad
There are only 2 possible ways:
Either give static database with all POI or
Load data from web whenever required.
Now you can do one thing, keep static database with App. Make web call once app starts and check for the updated data if any changes you made in server database. If there is any change on server database, then update only those data to the local(static) database.
For implementing above step, you must have to pass Last Updated date to/from server. I mean whenever you make web call, you need to pass last updated date from the client, server will check for new data to be updated post this date. Server will store client date as updated date and return back to client.
Maybe it's possible to store POIs like "One POI - One file"? In this case you can easily read only new points from server