I have a custom controller type which runs it's own specific test fragments. The important thing to note is that these fragments contain Transaction Controllers, which contain gaussian timers simulating wait times of up to 5 minutes.
The tests I am running are data driven, and should be runnable for a varying length of time. To specify the runtime of a test I have been using the "Duration" option on the Thread Group scheduler.
In the event were a test has ran beyond its duration, I've noticed that when these timer fragments are in use, the test is delayed and cannot end until the transaction (or at least the timer) has been complete. The other timings and samplers recorded seem to be unaffected, however the runtime of the test is impacted.
I'd like to solve this issue without having to rely on the user manually killing a test when it has reached it's duration. Is there any option within JMeter to kill or interrupt any type of running thread when a duration has been reached?
As per my understanding with Jmeter, there is no element which can stop the running test on reaching specific duration.
However, an element named 'Test Action' can be used to Pause/Stop/Stop Now actions on your test during run time and this element can be used under 'If Controller' element so that you can set the condition in this element to stop the thread.
Although JMeter provides various head-on elements to handle different conditions but in rare cases where existing elements could not provide direct solution to the problem then JMeter experts in any software testing company uses multiple elements with child-parent hierarchy to handle the condition [as used above with Test Action & If Controller elements]
I believe this has to do with stop test vs shutdown. When a test reaches its duration, it will issue a stop test, at which point any timer will finish, the request will happen, then the thread stops. This is why manually shutting it down works- shutdown doesn't respect timers, etc.
I don't think there's a way to set duration to use shutdown rather than stop. One thing you might try is multiple, smaller timers, and see if it still waits for all of them.
Related
I have this situation:
Thread Group (n_threads=X,duration Y sec)
Loop
Java Sampler
When the test duration ends, Jmeter does not stop the threads that were making requests and therefore the test does not terminate. How can this be solved?
This can happen in the case when response time of your "Java Sampler" is higher than your test duration. I would recommend introducing reasonable timeout into your Java code so the samplers would fail/exit instead of waiting forever for the response. If you have no idea what's going on there - take a thread dump and see where your thread(s) stuck
As the workaround, the only way to "terminate" the test I can think of would be:
Adding another Thread Group with 1 thread
Adding JSR223 Sampler with the following code:
sleep(5000) // wait for 5 seconds, amend accordingly to your desired test duration
log.info('Exceeded test duration')
System.exit(1) // the process will exit with non-zero exit code (error), change it to 0 if needed
See Apache Groovy - Why and How You Should Use It for more information on Groovy scripting concept in JMeter
Also be aware that this code will terminate the whole JVM so it will probably make sense to add jmeter.save.saveservice.autoflush=true line to user.properties as forcibly terminating the whole JVM might lead to some results loss.
In JMeter,
I'm creating a test plan
In which I need to set a condition
(if the fail count rate is high than success, then I want my test to stop)
Tried :
Auto-stop listener but its useful if we specify error rate %
Tried bean shell post-processor but unsuccessful.
if (!prev.isSuccessful()) {
prev.setStopThread(true);
}
Any ideas much appreciated.! thanks in advance.!
By calling setStopThread you are "asking" JMeter to attempt to stop current thread only, the correct method would be prev.setStopTest(true). Again, if you call this method JMeter will "ask" threads to stop, if you want JMeter terminate in less graceful manner you can go for prev.setStopTestNow(true) method (you can get more failures this way as samplers will be abnormally terminated)
And finally you can call System.exit(1) method which will immediately terminate the whole JVM.
Be aware that since JMeter 3.1 it is recommended to use JSR223 Test Elements and Groovy language for scripting consider migrating from Beanshell to Groovy on next available opportunity.
I think that you can use a test-action sampler, combined with an if-controller and an action stop:
Create an 'If Controller' and set your condition.
Create a 'Test Action' inside where your action is 'stop'.
Errors connected with infinite loops of view events are the topic of this question. Such loops don't throw StackOverflowError, because are executed very frequently, but not recursively. An example of that code error could be invalidate() call inside onDraw of custom view.
By default AndroidJUnit4 waits for app idle after starting a test activity and then performs verifications. But in case of infinite lifecycle loop idle won't be achieved and test will fail with error java.lang.RuntimeException: Could not launch intent Intent within 45 seconds. Perhaps the main thread has not gone idle within a reasonable amount of time?
Such error isn't so declarative and test have to wait for it so long.
For example,
I should test onLayoutChildren method of LayoutManager. By default in case properly implementation this method is executed two or three times after start and that's all. And this test is passed
verify(spy, atMost(3)).onLayoutChildren(any(RecyclerView.Recycler.class), any(RecyclerView.State.class)), where spy is a spy of LayoutManager. A behaviour could be corrupted and onLayoutChildren goes to infinite loop. In case of error under test it stucks.
What i am trying to achive. Possible variant is run test & application for some time (like 500ms or so) then break the app and perform verifications after that. Without any connection to idle
I have 4 separate processes which need to go one after another.
1st process
2nd process
3rd process
4th process
Since, every process is connected to one another, each process should run after process before him finishes.
Each process has its own variable length which will be various as programs data input grows.
But some sketch would be like this
Program Runs
1st process - lasts 10 seconds
2nd process - has 300 HTTP get requests, last 3 minutes
3rd process - has 600 HTTP get requests, lasts 6 minutes
4th process - lasts 1 minute
Program is written in java
Thanks for any answer!
There is no concurrency support in the java API for your use case because what you're asking for is the opposite of concurrent. You have a set of four mutually dependent operations that need to be run in a specific order. You only need, and should probably only use, one thread to correctly handle this case.
It would be reasonable and prudent to put each operation in its own method or class, based on how complex the operations are.
If you insist on using multiple threads, your main thread should maintain a list of runnables. Iterate through the list. Pop the first runnable from the list, create a new thread for that runnable, start the thread, and then invoke join() on the thread. The main thread will block until the runnable is complete. The loop will take you through all the runnables in order. Again, there is no good reason to do this. There may or may not be a bad reason.
This post started out as "What are some common patterns in unit testing multi-threaded code ?", but I found some other discussions on SO that generally agreed that "It is Hard (TM)" and "It Depends (TM)". So I thought that reducing the scope of the question would be more useful.
Background : We are implementing a simple scheduler that gives you a way to register callbacks when starting and stopping jobs and of course configure the frequency of scheduling. Currently, we're making a lightweight wrapper around java.util.Timer.
Aspects:
I haven't found a way to test this scheduler by relying on only public interfaces (something like addJob(jobSchedule, jobArgs,jobListener) , removeJob(jobId)).
How do I time the fact that the the job was called according to the schedule specified ?
you could use a recorder object that record the order, timings and other useful stuff in each unit test of your scheduler. The test is simple:
create a recorder object
configure the schedule
execute a unit test
check that recorder object is "compatible" with the schedule
One thing also to remember is that you don't need to test that Timer works. You can write a mock version of Timer (by extending the class or using EasyMock) that simply checks that you are calling it correctly, possibly even replacing enough that you don't need threads at all. In this case that might be more work than needed if your job listener has enough callbacks to track the scheduler.
The other important thing to remember is that when testing the scheduler, use custom jobs that track how the scheduler is working; when testing scheduled jobs, call the callbacks directly and not through the scheduler. You may have a higher level integration test that checks both together, depending on the system.
There are many failure modes that such a scheduler could exhibit, and each would most likely require its own test case. These test cases are likely to be very different, so "it depends."
For testing concurrent software in Java in general, I recommend this presentation from JavaOne 2007: Testing Concurrent Software.
For testing that a scheduler must execute jobs in accurate accordance to their schedule, I'd create an abstraction of time itself. I've done something similar in one of my projects, where I have a Time or Clock interface. The default implementation will be MillisecondTime, but during testing I will switch it out with a TickTime. This implementation will allow my unit test to control when the time advances and by how much.
This way, you could write a test where a job is scheduled to run once every 10 tick. Then your test just advances the tick counter and checks to make sure that the jobs run at the correct ticks.
A couple of ways to test concurrent code.
run the same code many times under load, some bugs appear only occasionally, but can show up consistently if performed repeatedly.
Store the results of different threads/jobs in a collection such as a BlockingQueue. This will allow you to check the results in the current thread and finish in a timely manner (without ugly arbitrary sleep statements)
If you are finding testing concurrency difficult consider refactoring your objects/components to make them easier to test.
If the scheduler delegates to an Executor or ExecutorService to run the tasks, you could use Dependency Injection to remove a direct dependency on the type of Executor, and use a simple single threaded Executor to test much of the functionality of your scheduler without the complication of truly multi-threaded code. Once you'd got those tests debugged, you could move on the the harder, but now substantially reduced in magnitude, task of testing thread-safety.