I have written tests for my windows based application in TestNG. Sometimes my tests will result in BSOD of windows, so the test will run waiting for the Windows to be up till timeout and fail.
I have written a Listener for BSOD detection and handling, so that whenever the test starts the BSOD detector will start as listener and handle if the BSOD occurs. But still I have no way to notify my test that it has to halt. It still continues to execute till timeout.
How can solve this problem? Is there a way to notify the test to terminate it from Listener?
I think you may be out of luck.
The discussion at https://groups.google.com/forum/#!topic/testng-users/BkIQeI0l0lc clearly indicates that, as of August 2012 there was no way to stop a running test suite.
That combined with a much older post (https://groups.google.com/forum/#!topic/testng-users/FhC3rqs1yDM) from February 2010 suggests it has been a shortcoming of TestNG for a while.
The current TestNG documentation does not describe any methods for stopping the suite.
From what I can see in the source code (although I have not traced through all paths) at https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/TestNG.java, it simply creates new tasks for each test and runs them, providing no way to break early.
Perhaps you could modify your tests to have access to some flag that indicates if subsequent tests should be skipped (triggered by, e.g., the BSOD detector). Then all of your subsequent tests are responsible for indicating themselves as skipped if this flag is set. It might not be ideal, but one way to think about it is: Any test should be skipped if a BSOD (or other terminal event) was detected prior to that test.
You could request this feature on the TestNG Google Group (linked above). If you are so inclined, you could also perhaps customize the TestNG source.
Related
Testing a multithreaded application with JUnit5, the automated tests run just fine.
But in manual debugging, the environment misbehaves, and I would like to know how to get it to act as it should.
IntelliJ (using the Java VM + its debugger) has two general options to suspend on breakpoints: either "All" = all threads, or "Thread" = the current thread only.
(Which thread is "the current thread" may be unclear and causing issues, but that's been discussed separately, see other issues here on stackOverflow.)
Wanted behaviour:
a breakpoint to suspend a specific thread, and the JUnit test environment, which is part of the IDE debugging.
Actual behaviour:
either all threads are suspended, even those which should be running
or only a single thread is suspended, and JUnit #AfterEach/#AfterAll annotations kill the resources currently debugged, terminating the threads that should be running, etc.
How to achieve the Wanted behaviour?
(Environment: AdoptOpenJDK8, AdoptOpenJDK11 in Java 9 mode, IntelliJ 2020.3, JUnit-Jupiter 5.7.0)
You can use two breakpoints. The one you're really interested in and another one in your unit test (somewhere between calling the code under test and cleaning up). Set both to only suspend the current thread.
I am trying to run jenkins job for my automation. Here I am using cucumber Bdd. While running locally in single thread, test cases are working good. But while building the job in jenkins using multiThreading, test cases are failing randomly. Once one is passed , in the successive build it is getting failed. Most of them are having the below error. It seems like there is some server issue that page is not loading in the browser while running the test cases. Can you suggest any way out to run this successfully?
My whole test cases are so large in size that if not run in multiThread it will take 6 to 7 hrs . The server is not supporting such long time. It is getting socket timeout after half of the run(after 3 hrs.)
org.openqa.selenium.TimeoutException: Course wasn't loaded after 30
seconds. Build info:
version:'2.53.1',revision:'a36b8b1cd5757287168e54b817830adce9b0158d',time:
'2016-06-30 19:26:09' System info: host: 'jenkins-50a5c8e3',
ip:'100.64.111.138',os.name:'Linux',os.arch:'amd64',os.version:'2.6.32-5-xen-amd64',java.version:'1.8.0_40'Driver
info:driver.version:unknown
This question is a bit vague so the best I can do is give a vague answer of what you might want to look into.
Make sure your tests can run in isolation You mentioned it works in a single thread. If your tests don't each have isolated data (course, user, whatever) then it's really easy to have one test do something like remove a course while another test is trying to access it if you run them in parallel.
Timing issues If you have timing issues in your tests, you probably need more robust explicit waits.
Make sure you properly dispose of your driver Preferably in a finally block. You need to make sure that no matter what path your tests take, the driver will get closed. You might want to start a strategy as well to clean up stranded driver instances, restart nodes, etc...
We have a weird problem. We are using an automatic test tool. The DSL was implemented in Scala. The system which we test with this tool was written in Java, and the interface between the two components is RMI. Indeed, the interface part of the automatic test tool is also Java (the rest is Scala). We have the full control of the source code of these components.
We already have at the magnitude of thousand test cases. We execute these test cases automatically once every night, using Jenkins on a Linux server. The problem is that we sporadically receive a java.lang.NoClassDefFoundError exception. This typically happens when trying to access a Java artifacts from a Scala code.
If we execute the same test case manually, or check the result of the next nightly run, then typically the problem solves automatically, but sometimes it happens again in a completely different place. In case of some runs no such problem appears at all. The biggest problem is that the error is not reproducible; furthermore, as it happens in case of an automatic run, we have hardly any information about the exact circumstances, just the test case and the log.
Has somebody already encountered with such a problem? Do you have any idea how to proceed? Any hint or piece of information would be helpful, not only the solution of the problem. Thank you!
I found the reason of the error (99% sure). We had the following 2 Jenkins jobs:
Job1: Performs a full clean build of the tested system, written in Java, then performs a full clean build of the DSL, and finally executes the test cases. This is a long running job (~5 hours).
Job2: Performs a full clean build of the tested system, and then executes something else on it. The DSL is not involved. This is a shorter job (~1 hour).
We have one single Maven repository for all the jobs. Furthermore, some parts of the tested component is part of the interface between the two components.
Considering the time stamps the following happened:
Job1 performed the full build of both components, and started a test suite containing several test cases, which execution lasts about half an hour.
The garbage collector might swept out the components not used yet.
Job2 started the build, and it also rebuilt the interface parts, including the one swept out by garbage collector of Job1.
Job1 reached a test case which uses an interface component already swept out.
The solution was the following: we moved Job2 to an earlier time; now it finishes the job before Job1 starts the tests.
I have an issue with Silk Test's Open Agent (Silk Test 2011).
I am using Silk4J for acceptance tests written in Java for a C++ client. It starts the tests properly but does not stop after the tests are completed. I have to do it manually so the job is completed in Jenkins.
Silk Test 2011 with Hotfix3 is installed on my Windows 7 SP1 (64bit).
The Silk Test 2011 release notes mention turning off user access control and I have done so.
I am having a hard time isolating the problem and hope somebody may have some ideas.
Silk Test's Open Agent should not be blocking your tests from completion.
In the past I've seen similar issues by the Java VM leaking sockets to their child processes, which then occasionally causes another processes to think the JVM was still alive.
You could try
Upgrading the involved JVM(s)
Upgrading Silk Test (2011 is ancient)
Take a close look at the network sockets the various processes have open, if the first two steps don't solve your issue, this might help you find out why.
I'm trying to test existing software and I'm running into an issue; if, during a junit test suite, I close a gui window, the whole test suite just ends. No errors, failures, or successes. It definitely ends too, because it closes all other windows as well, and terminates the thread.
I'm not using junit to the test the GUI - that's more of a system test obviously, and I'm using GUI automation software for that. However, in order to test a method in one of the classes in the software, I have to have stuff going on with the gui. At the end of the tests, I need to close the windows or they will remain open and will actually interfere with the other tests (it's networked software). However, as said, sending an onclosing event to any window immediately ends the test suite.
I became curious about the whole thing and slowly came to realize, to my horror, that even in normal execution, if I close a window, it's not handled within the software, it just magically stops executing! The main driver's loop doesn't even have any exit condition whatsoever, just while(true) with no breaks! My thinking is this must be related to why everything just stops when I close a window during the tests.
Anyways, I still don't fully understand what's happening, so if someone could fill me in, that would be great. But my main question is how can I fix/ get around this issue and close windows without killing the whole test suite.
Note: I can't change the behavior of the software. (As you may have guessed, this is for a class)
Sounds like the window close is calling System.exit(). If you have the ability to remove that, you could do so. If you don't, you could install a SecurityManager that prevents the call during testing. Example here: Preventing System.exit() from API