I've only ever used Cucumber in conjunction with Selenium to test websites.
I'm now trying to write a feature file to test a class but I'm not sure how, I have my test step running which calls the main method passing in some arguments. The main method does some things with the arguments and prints them to the console. Is it possible to read the console output to a string so I can assert it matches what I expect?
#Given("I pass the arguments {string} to the generator")
public void i_pass_the_arguments(String string) {
String[] args = string.split(" ");
new Generate();
Generate.main(args);
}
If you are testing a specific class only, I'd recommend using unit tests for that, for example using JUnit. They will have less overhead and be easier to run and maintain.
Related
I ran into some trouble testing a Spring app. The current approach in my team is to write scenarios in Gherkin and have Serenity provide its pretty reports.
A new component in the app will need a lot of test cases. The requirements will be provided in a few 'parsable' excel files so I thought it would be neat to just use them directly, row by row, in a Junit parametrized test. Another option would be to write a bloated Gherkin feature and tediously compose each example manually.
So I thought of something like that:
#RunWith(Parameterized.class)
private static class Tests {
#Parameterized.Parameters(name = "...") // name with the params
public static Collection params() {
// parse excel here or use some other class to do it
}
#Test
public void test() {
/* do the actual test - it involves sending and receiving some JSON objects */
}
}
This works smoothly but I ran into trouble trying to use
#RunWith(SerenityRunner.class)
The problem is that Junit does not support multiple runners. A solution I found is to make a nested class and annotate each with a different runner, but I don't know how to make it work (which runner should be on the outside, where do I actually run the tests, an so on).
Any thoughts?
Actually Serenity provides another runner - SerenityParameterizedRunner which seems to have the same features as JUnit's Parameterized.
I have 2 Java classes which have a symbiotic relationship.
Class 1 produces some output files and Class 2 consumes the output of class 1 and validates it. Both of these classes take input from the commandline. This project is maven based.
Given this symbiotic nature, I am unsure how to "connect them"?
My thinking was, to write another Java class which takes in command line inputs and calls the 2 classes. However there is another uncertainty here, how could I run class 1 (in order to produce the output files) so then I can have class 2 to validate it. Perhaps Junit #Before or some annotation? I really am unsure how to proceed. I hope I am making sense here.
Any help or suggestions would be much appreciated.
Execute the main() method of your class under test from within a JUnit method.
public class Class2 {
#Before
public void produceOutputFiles() {
Class1.main(new String[] { "these", "are", "commandline", "arguments"});
}
#Test
public void validateClass1Output() {
//read in the files and validate the output
}
}
Invoking Class1 via Process.exec() is an option with many downsides. It is much simpler to keep both the test and the tested code within the same JVM.
I have a doubt in TestNG with Java. I am completly new to TestNG. My doubt is, How all the test cases are executing using TestNG in java without having main() method? Please suggest me if you have any ideas. Following code is the example of a sample test case using TestNG in java. But if you notice, you can find one thing that there is no main() method in the code. Then, how does the testcases are executing?
I have another doubt. Is main() method needed for selenium Webdriver and TestNG combination to execute a script? Or can we execute testcases without main() method? If we can execute testcases without main(), then how does it is possible?
package com.first.example;
import org.testng.annotations.Test;
public class demoOne {
#Test
public void firstTestCase()
{
System.out.println("im in first test case from demoOne Class");
}
#Test
public void secondTestCase()
{
System.out.println("im in second test case from demoOne Class");
}
}
This is a valid doubt many testers have. Because the main() method is needed to run the Java program and while writing tests in TestNg we don't use main() method, and we use Annotations instead.
Annotations in TestNG are lines of code that can control how the method below them will be executed. So, in short you don't need to write main() method, TestNg do that by itself. Refer the code at the end in Annotations documentation to get the idea how it happens.
As rightly pointed out in this answer: https://stackoverflow.com/a/1918154/3619412
Annotations are meta-meta-objects which can be used to describe other
meta-objects. Meta-objects are classes, fields and methods. Asking an
object for its meta-object (e.g. anObj.getClass() ) is called
introspection. The introspection can go further and we can ask a
meta-object what are its annotations (e.g. aClass.getAnnotations).
Introspection and annotations belong to what is called reflection and
meta-programming.
Also, it's not necessary to have main() method in your tests, but you can use main() method to run the TestNg tests if you want. Refer this.
to run script from cmd prompt we use below statement,
java org.testng.TestNG testng1.xml
main method in TestNG.java class how accept the command line argument,
public static void main(String[] argv) {
TestNG testng = privateMain(argv, null);
System.exit(testng.getStatus());
}
You saw it right. Test-cases get executed through testng, the testing framework which was inspired from junit without having the main() method but extensively uses annotations.
Annotations
As per the documentation in Annotations majority of the APIs require a huge amount of boilerplate code. To write a web service you need to provide a paired interface and implementation. This boilerplate could be automatically generated by a tool if the program can be decorated with annotations indicating which methods were remotely accessible. Annotations doesn't affects the program semantics directly but they do affect the way programs are treated by tools and libraries, which can in turn affect the semantics of the running program.
TestNG
TestNG is a simple annotation-based test framework which uses a marker annotation type to indicate that a method is a test method and should be run by the testing tool. As an example:
import org.testng.annotations.Test;
#Test
public void foo() {
System.out.println("With in foo test");
}
The testing tool which is being used is as follows:
import java.lang.reflect.*;
public class RunTests {
public static void main(String[] args) throws Exception {
int passed = 0, failed = 0;
for (Method m : Class.forName(args[0]).getMethods()) {
if (m.isAnnotationPresent(Test.class)) {
try {
m.invoke(null);
passed++;
} catch (Throwable ex) {
System.out.printf("Test %s failed: %s %n", m, ex.getCause());
failed++;
}
}
}
System.out.printf("Passed: %d, Failed %d%n", passed, failed);
}
}
I have a junit test testArchive(). The Junit test tests the archive() method that archives a file and returns the url to it as a String. The URL is defined as an instance variable inside the Junit Test class.
class Prepare {
private String url = "";
#Test
public void testArchive() {
String url = getService().archive();
}
#Test
public void testSendEmail() {
getService().sendEmail(url) // Url is turning out to be null
}
} // end of class
I am writing another Junit test for sendEmail() which emails the URL. But the URL is turning out to be null, though its defined as a class variable
Can you please let me know how I need to correct my Junit test for send email?
Thank you
Short answer:
You should really not do that.
Detailed answer:
Unit tests (and therefore JUnit tests as well) are intended to run separately and independently from each other. Each test should check only one method, regardless of result of another method or another test. So in your case, method testSendEmail() should use some hard coded URL, or better few different URLs.
Keep in mind that:
Test cases should not have side effects: the .archive() looks like will produce side effects
Test cases should not assume an execution order of other test cases: your testSendEmail seems to assume testArchive is executed first, which is wrong
Test cases should not depend on external factors: the getService calls looks like an external (uncontrolled) factor
Test cases should be independent and self-contained
Instead of one test cases depending on the outcome of another,
you could use a private helper method that both test cases can call.
I removed the 2nd JUnit test and consolidated the tests into 1. Both archive and email will happen in one test.
I am writing tests for an interpreter from some programming language in Java using JUnit framework. To this end I've created a large number of test cases most of them containing code snippets in a language under testing. Since these snippets are normally small it is convenient to embed them in the Java code. However, Java doesn't support multiline string literals which makes the code snippets a bit obscure due to escape sequences and the necessity to split longer string literals, for example:
String output = run("let a := 21;\n" +
"let b := 21;\n" +
"print a + b;");
assertEquals(output, "42");
Ideally I would like something like:
String output = run("""
let a := 21;
let b := 21;
print a + b;
""");
assertEquals(output, "42");
One possible solution is to move the code snippets to the external files and refer each file from corresponding test case. However this adds significant maintenance burden.
Another solution is to use a different JVM language, such as Scala or Jython which support multiline string literals, to write the tests. This will add a new dependency to the project and will require to port existing tests.
Is there any other way to keep the clarity of the test code snippets while not adding too much maintenance?
Moving the test cases to a file worked for me in the past, it was an interpreter as well:
created an XML file containg the snippets to be interpreted as well as the expected result. It was a fairly simple XML definition, a list of test elements mainly containing testID, value, expected result, type, and a description.
implemented exactly one JUnit test that read the file and looped through its contents, in case of failure we used the testID and description to log failing tests.
It mainly worked because we had one generic well-defined interface to the interpreter like your run method, so refactoring was still possible. In our case this did not increase maintenance effort, in fact we could easily create new tests by just adding more elements to the XML file.
Maybe this is not the optimal way in which Unit tests should be used, but it worked well for us.
Since you are talking about other JVM languages, have you considered Groovy? You would have to add an external dependency, but only at compile/test time (you don't have to put it in your production package), and it provides multiline strings. And one major advantage in your case : its syntax is backwards compatible with Java (meaning you won't have to rewrite your tests)!
I have done this in the past. I've done something similar to what was suggested by home, I used external file(s) containing the tests and their expected results, but using the #Parameterized test runner.
#RunWith(Parameterized.class)
public class ParameterTest {
#Parameters
public static List<Object[]> data() {
List<Object[]> list = new LinkedList<Object[]>();
for (File file : new File("/temp").listFiles()) {
list.add(new Object[]{file.getAbsolutePath(), readFile(file)});
}
return list;
}
private static String readFile(File file) {
// read file
return "file contents";
}
private String filename;
private String contents;
public ParameterTest(String filename, String contents) {
this.filename = filename;
this.contents = contents;
}
#Test
public void test1() {
// here we test something
}
#Test
public void test2() {
// here we test something
}
}
Here we are running test1() & test2() once for each file in /temp, with the parameters of the filename and the contents of the file. The Test Class is instantiated and called for each item that you add into the list in the method annotated with #Parameters.
Using this test runner, you can rerun a particular file if it fails; most IDEs support rerunning a single failed test. The disadvantage of #Parameterized is that there isn't any way to sensibly identify the tests so that the names appear in the Eclipse JUnit plugin. All you get is 0, 1, 2, etc. But at least you can rerun the failed tests.
As home says, good logging is important to identify the failing tests correctly and to aid debugging especially when running outside the IDE.