Develop Reference

Retrieve data and wait time to check if something new appeared in database

I would like to retrieve data from my in-memory H2 database via rest endpoint using Spring and Java8. I have 2 endpoints: one to retrieve data, and second one to add data to database.
How can I achieve something like it is described below in easiest way? I am not sure what solution can be better I thought about JMS Queue or CompletableFuture (if it is possible). It should work for few users, they will call to retrieve data saved under their id number.
Scenario:
User calls rest-endpoint to retrieve data.
If data is present in database then it is retrieved and returned to user.
If data is not present in database then connection is hold for 60 seconds and if during that time something will appear in database (added via endpoint to add new data) then data will be returned.
If data is not present in database and new data won’t appear in 60 seconds then endpoint returns no content.

There were multiple ways of doing that and if requirements is clear then i suggest below two approaches.
Approach 1:
Find and retrieve if data available without waiting.
If data not available set resource id and retrieveTime in header and respond to consumer.
Based resource id you can be ready with data if available.
In this way you can sure about your endpoint service time always consistent and ideally it shouldn't be more than 3 seconds.
Approach 2.
if data not available then out sleep in 60 seconds (not in database connection scope) and then again try with same thread.
Don't need any queue or ansyc process here.
Here your loosing resources and service time would take more.
Apart from other approaches, if your systems using eventing then use eventing approach when there is record persistent then send event to consumer (all database has the feature to send event to source system today).

Set some trigger that removes rows older than few minutes from DB with Spring Boot

I have a Java app on Spring Boot with Cassandra DB, where I'm writing to DB Person entities.
Each row of person in DB must be deleted when get 5 minute old, so the concept is easy:
Some person is added to DB with timestamp and this person must be removed after exactly 5 minutes.
The only idea that comes to mind is setting Spring Scheduler which runs every second and checks every row if it's expired and if it is, then it is deleted.

Since you are using Cassandra as a DB you could leverage the Cassandra TTL feature.
During data insertion, you have to specify 'ttl' value in seconds. 'ttl' value is the time to live value for the data. After that particular amount of time, data will be automatically removed.
TTL syntax in cql would be like
INSERT INTO person (name, age) VALUES ('ExampleName', '39') USING TTL 300;

instead of running a Spring Scheduler every second, you could easily create a timer task after each record and using TimerTask(Core java), you create a task that will execute after the set interval and delete the record.
PFB some useful link and example:
https://www.baeldung.com/java-timer-and-timertask

trigger java function on mysql datetime

I have several datetime column in my MySQL DB. I want to trigger a java function when the date is reached. At worst, trigger a MySQL function can do the job as well. How to have a trigger datetime based on MySQL without doing cron job on every minute ?

Even a trigger wouldn't do the job, there must be a process to check (in your case if the date was reached)
Like Thomas said job or a task (CRON) that sets the trigger or an application to do what you wish with the database.

It is not ideal to do this in database, but if no better choice, you can achieve it by creating a MySQL event, which is a scheduled task.
You need to add a insert and/or update trigger to the database table and create the event based on the datetime value of the column
You can create the event in the way that drops itself after it is executed at the specified time.

PostgreSQL: How to make updates readable from other transactions while first transaction not finished

I do simple schedule service.
One table with jobs
CREATE TABLE system_jobs (
id bigserial,
job_time timestamp without time zone,
job_function text,
run_on text,
CONSTRAINT system_jobs_pri PRIMARY KEY (id)
)
Multiple JAVA daemons select all rows where job_time < now() and execute job_function (pass their id as argument)
job_function sample
CREATE OR REPLACE FUNCTION public.sjob_test(in_id bigint)
RETURNS text AS
$BODY$DECLARE
utc timestamp without time zone;
BEGIN
utc := timezone('UTC', now());
-- This changes not avail from other transactions
UPDATE system_jobs SET run_on='hello' WHERE id = in_id;
PERFORM pl_delay(60); -- Delay 1 minute
UPDATE system_jobs SET job_time = now() + interval '10 seconds', run_on = '' WHERE id = in_id;
RETURN 'done';
END;$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
Inside function before 60 seconds delay I update run_on field and reset it after delay.
I expect run_on contains 'hello' while delay (60 sec) and will available for reading from other transactions, but it is not.
My task - prevent execute same job_function by different JAVA daemons simultaneous. I want check run_on before execute.
I read many docs and blogs about transaction levels, but I don't understand how can I use it in practice.
How can I configure my function or table or external process to allow other transaction see this changes?

PostgreSQL doesn't support dirty reads. See PostgreSQL documentation:
PostgreSQL's Read Uncommitted mode behaves like Read Committed. This is because it is the only sensible way to map the standard isolation levels to PostgreSQL's multiversion concurrency control architecture.
But looks like there is a workaround, called autonomous transactions, than might help you. There are at least two ways to implement it. See more info here and here.
Using these autonomous transactions, you can commit the change of run_on inside your function, so other transactions will be able to read it.

Only one way to do this - via dblink. Something like:
PERFORM dblink('your server config', 'UPDATE ...');

database polling using Java

I am stuck at some point wherein I need to get database changes in a Java code. Request is to get any record updated, added, deleted in any table of db; should be recognized by Java program. How could it be implemented JMS? or a Java thread?
Update: Thanks guys for your support i am actually using Oracle as DB and Weblogic 10.3 workshop. Actually I want to get the updates from a table in which I have only read permission so guys what do you all suggest. I can't update the DB. Only thing I can do is just read the DB and if there is any change in the table I have to get the information/notification that certain data rows has been added/deleted or updated.

Unless the database can send a message to Java, you'll have to have a thread that polls.
A better, more efficient model would be one that fires events on changes. A database that has Java running inside (e.g., Oracle) could do it.

We do it by polling the DB using an EJB timer task. In essence, we have a status filed which we update when we have processed that row.
So the EJB timer thread calls a procedure that grabs rows which are flagged "un-treated".
Dirty, but also very simple and robust. Especially, after a crash or something, it can still pick up from where it crashed without too much complexity.
The disadvantage is the wasted load on the DB, and also response time will be limited (probably requires seconds).

We have accomplished this in our firm by adding triggers to database tables that call an executable to issue a Tib Rendezvous message, which is received by all interested Java applications.
However, the ideal way to do this IMHO is to be in complete control of all database writes at the application level, and to notify any interested parties at this point (via multi-cast, Tib, etc). In reality this isn't always possible where you have a number of disparate systems.

You're indeed dependent on whether the database in question supports it. You'll also need to take the overhead into account. Lot of inserts/updates also means a lot of notifications and your Java code has to handle them consistently, else it will bubble up.
If the datamodel allows it, just add an extra column which holds a timestamp which get updated on every insert/update. Most major DB's supports an auto-update of the column on every insert/update. I don't know which DB server you're using, so I'll give only a MySQL-targeted example:
CREATE TABLE mytable (
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
somevalue VARCHAR(255) NOT NULL,
lastupdate TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
INDEX (lastupdate)
)
This way you don't need to worry about inserting/updating the lastupdate yourself. You can just do an INSERT INTO mytable (somevalue) VALUES (?) or UPDATE mytable SET somevalue = ? WHERE id = ? and the DB will do the magic.
After ensuring that the DB server's time and Java application's time are the same, you can just fire a background thread (using either Timer with TimerTask, or ScheduledExecutorService with Runnable or Callable) which does roughly this:
Date now = new Date();
statement = connection.prepareStatement("SELECT id FROM mytable WHERE lastupdate BETWEEN ? AND ?");
statement.setDate(1, this.lastTimeChecked);
statement.setDate(2, now);
resultSet = statement.executeQuery();
while (resultSet.next()) {
// Handle accordingly.
}
this.lastTimeChecked = now;
Update: as per the question update it turns out that you have no control over the DB. Well, then you don't have much good/efficient options. Either just refresh the entire list in Java memory with entire data from DB without checking/comparing for changes (probably the fastest way), or dynamically generate a SQL query based on the current data which excludes the current data from the results.

I assume that you're talking about a situation where anything can update a table. If for some reason you're instead talking about a situation where only the Java application will be updating the table that's different. If you're using Java only you can put this code in your DAO or EJB doing the update (it's much cleaner than using a trigger in this case).

An alternative way to do this is to funnel all database calls through a web service API, or perhaps a JMS API, which does the actual database calls. Processes could register there to get a notification of a database update.

We have a similar requirement. In our case we have a legacy system that we do not want to adversely impact performance on the existing transaction table.
Here's my proposal:
A new work table with pk to transaction and insert timestamp
A new audit table that has same columns as transaction table + audit columns
Trigger on transaction table to dump all insert/update/deletes to an audit table
Java process to poll the work table, join to the audit table, publish the event in question and delete from the work table.
Question is: What do you use for polling? Is quartz overkill? How can you scale back the polling frequency based on the current DB load?