I have a scenario where I check for a specific value in the Database every 10 seconds or so. And, if the value is YES, then I execute a bunch of shell scripts from a Java application.
Now, the value in database is only updated to YES once in a while depending on the user submitting a job on a web page. Therefore, running a while loop to check for this value in database seems to be a very bad design and I would like to implement a much cleaner approach using listeners (Observer design pattern).
How would such an implementation look like? Any examples I can follow to do this?
Yes there is much better job. So there is something called binlog reader in mysql. Thats how master and slave sync is done in mysql cluster database.
So either you write your own logic over https://github.com/shyiko/mysql-binlog-connector-java which gets all the chane event on table
or use https://github.com/zendesk/maxwell to read events from particular table and whenver any change in value is there check if it matches your condition and excute the script or java application on basis of that instead of running it as a cron.
The general idea is to use DB triggers, register DB listener from Java side and be notified from DB side when some event has happened.
Pls review proposed solutions
How to implement a db listener in Java
Related
i use the latest Acitiviti 5.22.0 engine (to be more concrete i use Alfresco Process Services 1.6.3) and i have implemented a Spring bean that gets executed every 10 minutes to generate a JSON representation of all my processes (process name, startDate, endDate, current taskName(s) and assignee(s)), to send them to an audit server. The problem is, that i only need to send all changed processes since the last run.
I do not want to send the JSON as soon as a process changes but to do a batch update of my audit system every 10 minutes.
To accomplish this, i've tried different approaches. My latest one:
Create a event listener bean that listens to all PROCESS_STARTED, PROCESS_COMPLETED, PROCESS_CANCELLED, TASK_COMPLETED, ...
Every time the event is triggered, store a process variable "_dirty" and set it to true
Every 10 minutes (wenn my JSON-bean is executed) query for all processes with the "_dirty" variable set to true
After sending the JSON to the audit system, set all "_dirty" process variables to false.
The problem with this approach: I am not able to update the "_dirty" variable after a process is ended. At least i don't know how.
My second approach would be to to store the processInstanceId on every event into a "global" property, but i don't know how to store this "global" property into database in case the server restarts. Is there a way to persist a property or an Entity into DB without creating an extra table, DAO, etc.?
Any ideas on how to solve this task? All tips are very much appreciated!
AFAIK, There's no such option
But you look at this. and see if it can be helpful in your case.
https://www.activiti.org/userguide/#_database_tables
As Linus suggested: This is not possible, so I needed some completely different approach.
I am creating an Ad-Hoc task now and store my properties as a local task variable. The Ad-Hoc task is owned by a system account and not assigned to anybody. This way I can make sure, no one of my real users tries to "complete" the task. Also I've written some code to generate the task if needed, so in case i want to clean it, it is created automatically the next time i want to store data.
Creating an Ad-Hoc task is quite easy by using org.activiti.engine.TaskService autowiring into my class.
Task task = taskService.newTask();
task.setDelegationState(DelegationState.PENDING);
task.setName("Some name goes here");
task.setTenantId("your tenant id (if any)");
task.setOwner("your system accounts ID");
task.setCategory("i use a special category to later query for the task");
taskService.saveTask(task);
After saving the task to the database, I can use the taskService to store and retrieve variables like this:
taskService.setVariableLocal(task.getId(), "variableKey", "variableValue");
Or query for the task like this:
Task task = taskService.createTaskQuery().taskDelegationState(DelegationState.PENDING).taskCategory("your special category").singleResult();
Not a very nice solution (I recommend having the task cached in a bean or something, so you don't need to query it all the time or even cache its values or something), but it works.
I have an Oracle DB with a lot of tables ... How can I write something which listens to the Oracle DB and if any changes are made to the DB, it calls another program which does some processing
It would be asnync
I just need to trigger a java program if there is any kind of DML happening .. I dont want details about which table or what rows or new/old values ...
If there is a DML on any table in a DB call a java program thats it
http://docs.oracle.com/cd/B14117_01/win.101/b10118/o4o00118.htm
OracleDB has some events. Just try to use them; For example, when the db is updated - And to call another java program:
Runtime.getRuntime().exec(/Here the argument/);
I fear you're setting yourself up for failure here. I suggest a related, but slightly different course of action.
Rather than trigging your processing on every data change, consider instead having your processing run every X minutes/hours/whatever using something like Cron or Quartz.
If you're worried about having it run when no changes have been made, you can add triggers to your tables to update a "last updated" table and you can abort if no changes have been made since the last run.
This avoids the biggest concern you would have with an "on-update-processor", namely what do you do if there's an update, which triggers a process, and while that process is running another update happens. Do you trigger another process? What if they conflict? I think you'll find it better to allow there to be a slight delay between the update and the synchronization process.
That's a pretty tall order for a question.
For starters, you are going to need a way to detect that a DML operation (INSERT, UPDATE, DELETE) has occurred.
For auditing on individual tables, we can use TRIGGERS.
CREATE TRIGGER [BEFORE|AFTER] [INSERT][UPDATE][DELETE] ... [FOR EACH ROW]
The normative action is to record information about the information (whatever is needed later) in an "audit log" table. You have to make a decision whether the operations you need to perform are synchronous (should happen before the DML completes) or whether those operations can be asynchronous (can happen anytime after the DML completes).
Normally, synchronous processing is done in PL/SQL, but Oracle does provide a mechanism for calling external procedures (EXTPROC), though you wouldn't want to do that from a TRIGGER.
Oracle also provides "fine grained auditing". The information gathered by FGA is not at a "row by row" level, like we can get with a FOR EACH ROW trigger.
So, it really depends on what you are trying to achieve.
The next step is figuring out how you are going to get this information to your external process. Is that process going to periodically poll table (or set of tables), are you going to use AQ Advanced Queueing, or some other queue mechanism.
I am making a Java gui and web application which will use the same mysql database.
It's a DTh management system where all the information will be stored and retrieved dynamically depending on input.
I believe that views are static by nature and thus would be useless as all my queries will have a different where condition (userid).
Do I need to use triggers? I mean I could code the java to execute multiple statements instead of using a inbuilt trigger (e.g. Insert in customers name and family members name both will have a duplicate copy for head of the family). Is there a performance hit? Am I wrong in some way?
And same thing what is the use of stored procedures? Can't I use methods in java to do everything?
So, I am asking is it possible to shift all the calculation intensive stuff to java and web script instead of the sql. If yes, does this mean I only have to create the backend structure of Database(i.e. all the different tables and FK,PK) and do rest without using any sql stuff on mysql workbench?
Thank you for helping.
There is (as always) one correct answer: It depends.
If you only want to show and query some data, you probably won't need trigger or stored procedures.
Views are a different thing: They are pretty helpful if you want a static viesw to a join-table or something like that. If you don't need this, just don't use it.
Keys are really important. They make your data robust against wrong input.
What you shoud use is PrepearedStatement instead of Statement. If you only use PreparedStatements, you are (nearly ?) safe in the question of SQL-Injection.
We use Views because it just faster than select query and for just showing data (not edit-update) it is faster and preferable.
Trigger are fired at database side so it is faster because it just execute 2 or more queries in single execution.
Same in Stored procedures, because we can execute more than one queries in single database connection. If we execute different queries than it take more time on every execution for database connection (find database server, authenticate, find database,... etc.).
Greetings all
I want to do something like a trigger or a listener (I don't know what) that will listen on a specific database table, and with each new record inserted on this table, do some java code, I mean that it detects that a new record was inserted and get it's data if it's possible
I need some guide about how this process can be accomplished ?
I am using Spring-Hibernate-PostgreSQL
This is what LISTEN/NOTIFY was created for.
The only drawback is that you will need to have some kind of background thread that polls the database on a regular basis to see if any notifications are available.
You can also use the code from the Postgres Wiki to have a starting point
I assume you mean that the DB content is added through your hibernate code.
If so, consult this previous answer of mine for how to set up Hibernate Event Listeners with Spring.
Otherwise, a-horse-with-no-name's answer should be best.
You could add an Interceptor to your Hibernate configuration to detect save events.
There are two different processes developed in Java running independently,
If any of the process modifyies the table, can i get any intimation? As the table is modified. My objective is i want a object always in sync with a table in database, if any modification happens on table i want to modify the object.
If table is modified can i get any intimation regarding this ? Do Database provide any facility like this?
We use SQL Server and have certain triggers that fire when a table is modified and call an external binary. The binary we call sends a Tib rendezvous message to notify other applications that the table has been updated.
However, I'm not a huge fan of this solution - Much better to control writing to your table through one "custodian" process and have other applications delegate to that. To enforce this you could change permissions on your table so that only your custodian process can write to the database.
The other advantage of this approach is being able to provide a caching layer within your custodian process to cater for common access patterns. Granted that a DBMS performs caching anyway, but by offering it at the application layer you will have more control / visibility over it.
No, database doesn't provide these services. You have to query it periodically to check for modification. Or use some JMS solution to send notifications from one app to another.
You could add a timestamp column (last_modified) to the tables and check it periodically for updates or sequence numbers (which are incremented on updates similiar in concept to optimistic locking).
You could use jboss cache which provides update mechanisms.
One way, you can do this is: Just enclose your database statement in a method which should return 'true' when successfully accomplished. Maintain the scope of the flag in your code so that whenever you want to check whether the table has been modified or not. Why not you try like this???
If you're willing to take the hack approach, and your database stores tables as files (eg, mySQL), you could always have something that can check the modification time of the files on disk, and look to see if it's changed.
Of course, databases like Oracle where tables are assigned to tablespaces, and tablespaces are what have storage on disk it won't work.
(yes, I know this is a bad approach, that's why I said it's a hack -- but we don't know all of the requirements, and if he needs something quick, without re-writing the whole application, this would technically work for some databases)