I have a test class written using JUnit 4 that has multiple test methods. Each of the tests prints some important informational output. When I look in IntelliJ's log, however, some of the last output from the previous test is recorded as being part of the first.
Why this this? Is there a way of correcting this behaviour?
I'm just writing to System.out, and calling flush before returning from each test actually made the problem worse.
It is simply a bug. I workaround it by Thread#sleep in #After
https://youtrack.jetbrains.com/issue/IDEA-66683
Test case:
public class LogBug {
#Test
public void see() {
System.out.println("foo");
throw new AssertionError("bar");
}
#Test
public void see2() {
System.out.println("foo2");
throw new AssertionError("bar2");
}
}
Related
I'm testing some methodA which has another one inside and I need to check that that methodB has not been called due to return.
If I run this test with debug, everything works correctly. But test fails as PowerMock runs it somewhere underhood.
My test:
#Test
public void incomingCall_dismissIncoming_incomingDataNull() throws Exception {
mIncomingCallData = null;
Whitebox.setInternalState(SUT, "mLastIncomingCallData", mIncomingCallData);
SUT.dismissIncoming();
verifyPrivate(SUT, times(0)).invoke("onIncomingCallDeclined");
}
The question is why is that and how can I test it? I know I don't need to test private methods, but I need it in this specific case.
I have a test that does a cycle to run the same test but with different inputs. The problem is that when one assert fails, then the test stops, and it is marked as if one test failed.
Here is the code:
#Test
public void test1() throws JsonProcessingException {
this.bookingsTest(Product.ONE);
}
#Test
public void test2() throws JsonProcessingException {
this.bookingsTest(Product.TWO);
}
public <B extends Sale> void bookingsTest(Product product) {
List<Booking> bookings = this.prodConnector.getBookings(product, 100);
bookings.stream().map(Booking::getBookingId).forEach((bookingId) -> {
this.bookingTest(bookingId);
});
}
public <B extends Sale> void bookingTest(String bookingId) {
...
// Do some assert:
Assert.assertEquals("XXX", "XXX");
}
In that case, the methods test1 and test2, execute as two different tests, but inside them, I do a cycle to check some stuff on every item that is returned by the collection. But what I want is to make that test on each item to be treated as a different one, so that if one fails, the others continue executing and I can see which one failed and how many failed over the total.
what you described is parameterized testing. there is plenty of frameworks that may help you. they just have different simplicity to power ratio. just pick the one that fits your needs.
junit's native way is #Parameterized but it's very verbose. other junit plugins that you may find useful are zohhak or junit-dataprovider. if you are not enforced to use junit and/or plain java, you can try spock or testng.
I have a Junit test class with multiple #Test methods in it that I need to run in order. If there is an exception thrown in a method I would like to stop entire test case and error out, but all of the rest of the test methods running.
public class{
#Test{
//Test1 method`enter code here`
}
#Test{
//Test2 method
}
#Test{
//Test3 method
}
}
If Test1 method fails then don't run other Tests
Note: All are independent tests
Unit tests should be designed to run independently of one another. The order of execution cannot be guaranteed. You should redesign your test class so that the order is not important.
Without further information it's hard to advise you specifically. But it may help to have an #before method, which checks for some precondition before running each test. If you included an Assume.assumeTrue(...) method call, then your test could be skipped if the condition fails?
As discribed here, JUnit 4.11 supports the ordered execution with the Annotation #FixMethodOrder, but the others are right, all tests should be independent of each other.
At the end of a test you can set a global success flag. This flag will be tested at the begining of each test. If the flag is not set by the end of one test(because it fails before finishing) all other tests will fail too.
Example:
#FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class ConsecutiveFail{
private boolean success = true;
#Test
public void test1{
//fist two statements in all tests
assertTrue("other test failed first", success);
success = false;
//do your test
//...
//last statement
success = true;
}
#Test
public void test2{
//fist two statements in all tests
assertTrue("other test failed first", success);
success = false;
//do your test
//...
//last statement
success = true;
}
}
I was able to achieve what you are looking with intellij idea IDE, I am using community edition.
Go to Edit Configuration in the class where test methods available.( Run --> Edit Configurations)
Select Test kind as "Class"
as in the below Image.
When you run the Class Test it will execute all #Test Annotated methods inside the class as in below Picture.
If you need the consequence to be kept and failing a test not to fail the whole set, put all such tests in one and test by assume.
Here's example for TESTNG how to specify test running order:
#Test(priority = 1)
public void test1(){}
#Test(priority = 2)
public void test2(){}
#Test(priority = 3)
public void test3(){}
Is it possible in JUnit to add a brief description of the test for the future reader (e.g. what's being tested, some short explanation, expected result, ...)? I mean something like in ScalaTest, where I can write:
test("Testing if true holds") {
assert(true)
}
Ideal approach would be using some annotation, e.g.
#Test
#TestDescription("Testing if true holds")
public void testTrue() {
assert(true);
}
Therefore, if I run such annotated tests using Maven (or some similar tool), I could have similar output to the one I have in SBT when using ScalaTest:
- Testing if entity gets saved correctly
- Testing if saving fails when field Name is not specified
- ...
Currently I can either use terribly long method names or write javadoc comments, which are
not present in the build output.
Thank you.
In JUnit 5, there is #DisplayName annotation:
#DisplayName is used to declare a custom display name for the
annotated test class or test method. Display names are typically used
for test reporting in IDEs and build tools and may contain spaces,
special characters, and even emoji.
Example:
#Test
#DisplayName("Test if true holds")
public void checkTrue() {
assertEquals(true, true);
}
TestNG does it like this, which to me is the neatest solution:
#Test(description="My funky test")
public void testFunk() {
...
}
See http://testng.org/javadocs/org/testng/annotations/Test.html for more information.
Not exactly what you are looking for, but you can provide a description on any assert methods.
Something like:
#Test
public void testTrue() {
assertTrue("Testing if true holds", true);
}
I prefer to follow a standard format when testing in JUnit. The name of the test would be
test[method name]_[condition]_[outcome]
for Example:
#Test
public void testCreateObject_nullField_errorMessage(){}
#Test
public void testCreateObject_validObject_objectCreated(){}
I think this approach is helpful when doing TDD, because you can just start writing all the test names, so you know what you need to test / develop.
Still I would welcome a test description functionality from JUnit.
And this is certainly better than other tests I have seen in the past like:
#Test public void testCreateObject1(){}
#Test public void testCreateObject2(){}
#Test public void testCreateObject3(){}
or
#Test public void testCreateObjectWithNullFirstNameAndSecondNameTooLong(){}
You can name the test method after the test:
public void testThatOnePlusOneEqualsTwo() {
assertEquals(2, 1 + 1);
}
This will show up in Eclipse, Surefire, and most other runners.
The detailed solution would be: You could add a Logger to your test, to log the results to a File. See log4j, for example. Then you can read the results in the File and also print successfull statements, what assertstatemens cannot.
The simple solution: You can add a JDoc description to every test method, this will be outlined, if you generate the JavaDoc.
Also every assertstatement can provide a Message, that will be printed, whenever the assert fails.
/**
* test the List#size() increasement after adding an Object to a List.
*/
public void testAdd(){
List<Object> list = new LinkedList<>();
list.add(new Object());
assertEquals("size should be 1, because of adding an Object", 1, list.size());
}
Do NOT use System.out.println("your message"); because you don't know how the tests will be executed and if the environment does not provide a console, your messages will not be displayed.
The selenium tests I'm gonna be doing are basically based on three main steps, with different parameters. These parameters are passed in from a text file to the test. this allows easy completion of a test such as create three of "X" without writing the code to do the create three times in one test.
Imagine i have a test involving creating two of "X" and one of "Y". CreateX and CreateY are already defined in separate tests. Is there a nice way of calling the code contained in createX and createY from say, Test1?
I tried creating a class with the creates as seperate methods, but got errors on all the selenium.-anything-, ie every damn line. it goes away if i extend seleneseTestCase, but it seems that my other test classes wont import from a class that extends seleneseTestCase. I'm probably doing something idiotic but i might as well ask!
EDIT:
well for example, its gonna be the same setUp method for every test, so id like to only write that once... instead of a few hundred times...
public void ready() throws Exception
{
selenium = new DefaultSelenium("localhost", 4444, "*chrome", "https://localhost:9443/");
selenium.start();
selenium.setSpeed("1000");
selenium.setTimeout("999999");
selenium.windowMaximize();
}
thats gonna be used EVERYWHERE.
its in a class called reuseable. Id like to just call reuseable.ready(); from the tests SetUp... but it wont let me....
public class ExampleTest {
#Before
public void setup() {
System.out.println("setup");
}
public void someSharedFunction() {
System.out.println("shared function");
}
#Test
public void test1() {
System.out.println("test1");
someSharedFunction();
}
#Test
public void test2() {
System.out.println("test2");
someSharedFunction();
}
}
The contents of the function after the #Before annotation is what will be executed before every test. someSharedFunction() is an example of a 'reusable' function. The code above will output the following:
setup
test1
shared function
setup
test2
shared function
I would recommend using JUnit and trying out some of the tutorials on junit.org. The problem you have described can be fixed using the #Before annotation on a method that performs this setup in a super class of your tests