Story: in my java code i have a few ScheduledFuture's that i need to run everyday on specific time (15:00 for example), the only available thing that i have is database, my current application and openshift with multiple pods. I can't move this code out of my application and must run it from there.
Problem: ScheduledFuture works on every pod, but i need to run it only once a day. I have a few ideas, but i don't know how to implement them.
Idea #1:
Set environment variable to specific pod, then i will be able to check if this variable exists (and its value), read it and run schedule task if required. I know that i have a risk of hovered pods, but that's better not to run scheduled task at all than to run it multiple times.
Idea #2:
Determine a leader pod somehow, this seems to be a bad idea in my case since it always have "split-brain" problem.
Idea #3 (a bit offtopic):
Create my own synchronization algorithm thru database. To be fair, it's the simplest way to me since i'm a programmer and not SRE. I understand that this is not the best one tho.
Idea #4 (a bit offtopic):
Just use quartz schedule library. I personally don't really like that and would prefer one of the first two ideas (if i will able to implement them), but at the moment it seems like my only valid choice.
UPD. May be you have some other suggestions or a warning that i shouldn't ever do that?
I would suggest to use a ready-to-use solution. Getting those things right, especially covering all possible corner-cases wrt. reliability, is hard. If you do not want to use quartz, I would at least suggest to use a database-backed solution. Postgres, for example, has SELECT ... FOR UPDATE SKIP LOCKED; (scroll down to the section "The Locking Clause") which may be used to implement one-time only scheduling.
You can create cron job using openshift
https://docs.openshift.com/container-platform/4.7/nodes/jobs/nodes-nodes-jobs.html
and have this job trigger some endpoint in you application that will invoke your logic.
Related
I want to execute testcases in below order.
Example: execution starts with 3rd scenario of 2nd feature then 4th scenario of 6th feature and so on.
Can we do this customization using TestNG/cucumber options/java or any other tool?
Is it possible using hooks or cli options --order ?
You could probably achieve this with use of #tags above the above the line that has the Scenario keyword. Then execute multiple test runs but using the tag options as parameters passed to the build as mentioned here.
However, this approach does seem a little odd. If the reasons for doing this are to force an order due to tests leaving data cached in the running application that require state from the other one then this is not good practice. Each Scenario should be completely runnable in isolation and without any prior one that might set up state.
I have a question:) How does it work karate if it comes to parallel execution?
We use karate in specific way where under the hood we hava a bunch of java code. I'm curious if variables declared as ThreadLocal in our java code is really thread save?
Is each test run in separate thread or there is another way of running test simultaneously?
The problem we faced is that it looks like a least two of running tests have access to ThreadLocal variable which should be isolated from each other.
Could you explain?
If you use a ThreadLocal you are on your own :) The short answer is Karate will create a thread-pool for Scenario execution, and each Scenario can go onto any of these threads. Maybe you need to read this section on how you can force some Scenario-s to run in sequence - but you may still have problems if the ones that ran first do not "clean up".
EDIT: I also think it should NOT be possible for 2 Scenarios to be on the same thread at the same time and if you are seeing this - it is a bug, please help us replicate it (see last line of my answer).
Karate's multi threading is battle-tested and we would not be able to claim that you can do the Gatling integration if this was not the case. Maybe you should just trust Karate for doing all the heavy lifting you need ? For example look at "hooks": https://github.com/intuit/karate#hooks
And of course, if you really do think there's an issue, follow this process please: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I have a Java project with Gradle as build environment.
Simplifying my scenario:
I have a module called Utils and one of its interfaces is TimeProvider which is implemented by either a real time provider (System.currentTimeMillis()) or a logic time provider (incrementing a counter). Another example is instead of sleeping for some time, I can just increment the timestamp. The goal is to run tests independently of the current time. An example use case is to be able to stop at break points without worrying aobut the time to progress and disrupting my tests.
(BTW if you have a betteer way to do do, I would appreciate any insight).
My question is how to force all the parts in the code to use this infrastructure and not the real Java time solutions (Thread.sleep, System.currentTimeMillis, etc.)
I want to prevent a scenario where a programmer accidently writes a code that uses the real Java time instead of my infrastructure.
One option is to use a Security Manager but this is tricky because the error will be catched "some time" during run time.
I prefer a way to catch this during compile time.
I wonder if there is a way, maybe in Gradle to forbid some modules use part of Java code? Maybe there is another way to do so?
Thanks,
I am coding an intricate method in a Spring Controller that takes as input request.getParameterMap(). In developing the method iteratively, each time I make a tweak, I have to Deploy, and then go through the steps on the web form.
That process can take minutes, just to tweak a small code change.
Are there any tricks or methods to speed up this process? All I really need is the input from request.getParameterMap(). Can I serialize that Map data somehow, and re-use it?
I am using Netbeans, if that is relevant.
In my experience the best is to setup a JUnit test, which doesn't use the web server at all, but just instantiates the controller, calls the method and checks the result.
Since your controller wasn't written from the ground up for this kind of approach, it might be quite some work to get this going at this stage. If you post the method in question we might help with this.
The next best thing is setting up an integration test, which starts up the application server, executes the request (possibly through the actual web gui using selenium or something).
Still a lot of work, but the difficulties are less dependent on the current workstyle.
As a final work around you can try to make the roundtrip for a manual test faster. There might be IDE dependent possibilities so you would have to let us know about the IDE in use.
I haven't tested it, but many people praise JRebel for this kind of thing, so you might want to give it a try.
If you don't want to fill the web form again and again try Jmeter(It's a free load testing tool).
Create a test plan with -> set number of threads to 1 --> http request sampler -> set method to post and add post parameters. Once everthing is setup fire the request
please check this link below for reference
http://community.blazemeter.com/knowledgebase/articles/65142-the-new-http-sampler-in-jmeter-2-6
I've been looking for replacements for my companies current batch processing system(java SE + crontab), since there is a lot of java code/shell script duplication, most jobs are ETL and do very similar steps and also i want to provide platform independence instead of relying on crontab, to be more specific with our job role, the current job creation steps are this:
Develop a java program that meets a business requirement.
Test it in a production like enviroment until it meets the business requirement needs.
Pass it to a production server with a shell script that provides file maintenance, java prgram execution and error handling routines(avoid 2 processes of the same name running, mail log to support and developers in case of program error, check output file existence after java program ends if it's relevant for the interface), and specify recurrence data(how often will this program run).
Much of the same logic is being designed and developed into a system that contains generic routines that these programs or "interfaces"(thats how they call it there) do independently(using copy-pasted code usually since most routines are similar), but i am still missing a very important part which i need help with, this concerns the scheduler implementation that i use, and i need it to meet one of these two needs:
-I want to guarantee that whenever i stop the scheduling server for a system update(due to new jobs being added, etc) or whatever other reason, those jobs that could not run due to the system being down(example is 3 jobs that could not run at 3:00 P.M. because the system was down), get to run when the server gets back up, even though their respective scheduling time is gone.
OR in case that the first thing is not possible then:
-I need a way to update the scheduler with new jobs and also update the jars that provide these jobs without restarting the scheduler(sort of like OSGi).
Either of these conditions would satisfy my requirements, and would end my search for the replacement, i've looked into Quartz, Oddjob(theres a scheduler in production with this scheduler, but it needs restarting each time you add new jobs/libraries, does not satisfy my needs) and OSGi using an application server, but i am looking for better suggestions, in case you also know better options, they are also much appreciated.
You might also want to take a look at http://jcrontab.sourceforge.net/
Jcrontab is a scheduler written in Java. The project objective is to provide a fully functional schedules for Java projects.
Alright, found just what i wanted, Quartz does the trick, but i have to develop my own UI Management, FORTUNATELY, there's this project http://code.google.com/p/myschedule/ which contains all that i need(add, remove, resuming jobs), and it is cheap to run the webapp, since you can use tomcat. Now i can focus on designing reusable jobs :), thank god for Quartz!