The JUnit view in Eclipse seems to order the tests randomly. How can I order them by class name?
As Gary said in the comments:
it would be nice if Unit Runner could
be told to go ahead and order them by
class name. Hmm, maybe I should look
into the source code...
I did look but there's no hint of a functionality to sort these names. I would suggest a change request to the JUnit plugin, but I don't think, that there are lot of people using this thing, so: DIY.
I would like to see the solution if you modify the plugin code.
One thing that one might do is using the schema of JUnit 3.x. We used a test suite that was called AllTests where you add the tests to it in a specific order. And for every package we got another AllTests. Giving those test suites a name being the same as the package enables one to easily build a hierarchy that should be valued by the junit plugin.
I really dislike how it is even presenting the test methods inside the Junit viewer. It should be in the very same order as they are specified in the TestCase class. I order those methods in the way of importance and features. So the upmost failing method is to correct first and then the more special one in the later part of the test case.
That is really annoying that the test runner is scrambling those. I will take a look at it myself and if I find a solution I will update this answer.
Update:
My problem with the ordering of method names within a TestCase is related to this one:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7023180 (Thanks Oracle!).
So in the end oracle changed the ordering of the methods within a class.getMethods or class.getDeclaredMethods call. Now the methods are random and can change between different runs of the JVM. It seams to be related to optimizations of compare or even is an attempt to compress method name - who knows... .
So whats left. First one can use: #FixMethodOrder (from javacodegeeks.com):
#FixMethodOrder(MethodSorters.DEFAULT) – deterministic order based on an internal comparator
#FixMethodOrder(MethodSorters.NAME_ASCENDING) – ascending order of method names
#FixMethodOrder(MethodSorters.JVM) – pre 4.11 way of depending on reflection based order
Well that is stupid but explains why people start using test1TestName schema.
Update2:
I use ASM since Javassist also produces random sorted methods on getMethods(). They use Maps internally. With ASM I just use a Visitor.
package org.junit.runners.model;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;
import com.flirtbox.ioc.OrderTest;
/**
* #author Martin Kersten
*/
public class TestClassUtil {
public static class MyClassVisitor extends ClassVisitor {
private final List<String> names;
public MyClassVisitor(List<String> names) {
super(Opcodes.ASM4);
this.names = names;
}
#Override
public MethodVisitor visitMethod(int access, String name, String desc,
String signature, String[] exceptions) {
names.add(name);
return super.visitMethod(access, name, desc, signature, exceptions);
}
}
private static List<String> getMethodNamesInCorrectOrder(Class<?> clazz) throws IOException {
InputStream in = OrderTest.class.getResourceAsStream("/" + clazz.getName().replace('.', '/') + ".class");
ClassReader classReader=new ClassReader(in);
List<String> methodNames = new ArrayList<>();
classReader.accept(new MyClassVisitor(methodNames), 0);
return methodNames;
}
public static void sort(Class<?> fClass, List<FrameworkMethod> list) {
try {
final List<String> names = getMethodNamesInCorrectOrder(fClass);
Collections.sort(list, new Comparator<FrameworkMethod>() {
#Override
public int compare(FrameworkMethod methodA, FrameworkMethod methodB) {
int indexA = names.indexOf(methodA.getName());
int indexB = names.indexOf(methodB.getName());
if(indexA == -1)
indexA = names.size();
if(indexB == -1)
indexB = names.size();
return indexA - indexB;
}
});
} catch (IOException e) {
throw new RuntimeException("Could not optain the method names of " + fClass.getName() + " in correct order", e);
}
}
}
Just put this in your src/test/java folder in the package org.junit.runners.model. Now copy the org.junit.runners.model.TestClass of the junit 4.5 lib to the same package and alter its constructor by adding the sorting routine.
public TestClass(Class<?> klass) {
fClass= klass;
if (klass != null && klass.getConstructors().length > 1)
throw new IllegalArgumentException(
"Test class can only have one constructor");
for (Class<?> eachClass : getSuperClasses(fClass))
for (Method eachMethod : eachClass.getDeclaredMethods())
addToAnnotationLists(new FrameworkMethod(eachMethod));
//New Part
for(List<FrameworkMethod> list : fMethodsForAnnotations.values()) {
TestClassUtil.sort(fClass, list);
}
//Remove once you have verified the class is really picked up
System.out.println("New TestClass for " + klass.getName());
}
Here you go. Now you have nicely sorted methods in the order they are declared within the java file. If you wonder the class path is usually set that way that everything in your src (target or bin) folder is considered first by the classloader. So while defining the very same package and the same class you can 'override' every class / interface in any library you use. Thats the trick!
Update3
I was able to get a tree view of every package and every class in the right order to.
The idea is to subclass ParentRunner and then add all classes to it that you identify as being public and having methods annotated with test.
Add a getName() method returning only the package name of the class your suite runner is representing (so you see the tree as a package tree without the suite's class name).
Inspect subdirectories if you find a certain suite class (I use AllTests for all suite classes).
If you do not find a suite class in a subdirectory check all of its subdirectories, this way you dont miss a package containing tests if the parent directory is not containing a suite.
That was it. The suite class I add everywhere is:
#RunWith(MySuiteRunner.class)
public class AllTests {
}
That's it. It should you give enough to start and extend on this one. The suite runner is only using reflection but I sort the test classes and suits of the subdirectories alphabetically and suits of subdirectories (which represent the packages they are in) are sorted upmost.
If you really need hard dependency between your JUnit test, try JExample extension
JExample introduces producer-consumer relationships to unit-testing.
A producer is a test method that yields its unit under test as return value.
A consumer is a test method that depends on one or more producers and their return values.
You can install it in Eclipse, for Junit4.4 or 4.5.
import jexample.Depends;
#Test
#Depends("#testEmpty")
public Stack<Integer> testPush(Stack<Integer> $) {
$.push(42);
assertFalse($.isEmpty());
return $;
}
As mentioned in this IBM article "In pursuit of code quality: JUnit 4 vs. TestNG":
One thing the JUnit framework tries to achieve is test isolation.
On the downside, this makes it very difficult to specify an order for test-case execution, which is essential to any kind of dependent testing.
Developers have used different techniques to get around this, like specifying test cases in alphabetical order or relying heavily on fixtures (#Before #After) to properly set things up.
These workarounds are fine for tests that succeed, but for tests that fail, they have an inconvenient consequence: every subsequent dependent test also fails. In some situations, this can lead to large test suites reporting unnecessary failures
So beware: if you retain any solution for ordering your JUnit tests the way you want... you need to think if that solution support a "skip" feature in order to allow other tests to proceed even if one of them fails.
mark wrote:
it orders them base on execution time,
maybe you should sort your methods?
source/sort members
mark is right. But you cannot sort your unit test. It's not allowed to speculate about the order of execution.
Unit tests have to be built independently and it's random, how they are called by the UnitRunner.
In most cases, the test methods are sorted alphabetically. The classes are random. Try to use a TestSuite to order your tests.
Ordering tests in JUnit view has been filed as bug #386453 in Eclipse Bugzilla. Commenting and/or voting there may help to get more visibility to this problem.
11 years later, the JUnit view does have name ordering, and just got "Execution time" ordering as well.
See Eclipse 4.17 (2020-09)
Sort test results by execution time
JUnit view now provides the ability to sort results by execution time.
By default, results will be sorted by execution order.
Choosing Sort By > Execution Time from the JUnit View menu will reorder the results once all tests are complete.
While tests are still running, they will be shown in execution order.
Sorting by execution order results in:
I was also searching for a solution for this, and I found a kind of crack from the below URL. I don't know whether it works for you or not, but it worked for me in Spring Tool Suite 2.5.2.
http://osdir.com/ml/java.junit.user/2002-10/msg00077.html
Related
I am running into trouble with JUnit 5 (5.0 or 5.1) and custom extension.
We are using service loader to load all implementations which then modify how our extension is bootstrapped. These implementations can be loaded just once, so I was thinking of using ExtensionContext.Store and placing it there. Every subsequent test instance would then just load it from Store instead of via service loader.
Now, I am even aware of the hierarchical context structure and I know that there is some "root" context which you can get through ExtensionContext.getRoot(). But this "root" context (instance of JupiterEngineExtensionContext) isn't really root - there is different one for every test instance.
Say you have FooTest and BarTest, then printing out getRoot() for each of them yields:
org.junit.jupiter.engine.descriptor.JupiterEngineExtensionContext#1f9e9475
org.junit.jupiter.engine.descriptor.JupiterEngineExtensionContext#6c3708b3
And hence trying to retrieve previously stored information from Store fails.
Is having this limitation intended? It makes the borderline between ClassExtensionContext and JupiterEngineExtensionContext pretty blurred.
Is there another way to globally store some information via extension?
Here is a (very) simplified version of how I tried working with the store (cutting out all other information basically). I also added some System.out.print() calls to underline what I am seeing. Executing this extension on two test classes results in what I described above:
public class MyExtension implements BeforeAllCallback {
#Override
public void beforeAll(ExtensionContext context) throws Exception {
System.out.println(context.getRoot());
if (context.getRoot().getStore(Namespace.create(MyExtension.class)).get("someIdentifier", String.class) == null) {
context.getRoot().getStore(Namespace.create(MyExtension.class)).put("someIdentifier", "SomeFooString");
} else {
// this is never executed
System.out.println("Found it, no need to store anything again!");
}
}
}
EDIT: Here is a minimal project on GH(link), run by mvn clean install, which displays the behaviour I see.
I just copied your MyExtension verbatim (i.e., with zero changes) and ran both FooTest and BarTest.
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
#ExtendWith(MyExtension.class)
class FooTest {
#Test
void test() {
}
}
and
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
#ExtendWith(MyExtension.class)
class BarTest {
#Test
void test() {
}
}
And the result is:
org.junit.jupiter.engine.descriptor.JupiterEngineExtensionContext#2280cdac
org.junit.jupiter.engine.descriptor.JupiterEngineExtensionContext#2280cdac
Found it, no need to store anything again!
Thus, getRoot() works as documented.
The only explanation for why you see two different roots is that you must be executing the tests in different processes.
Please keep in mind that the root ExtensionContext instance is bound to the current execution of your test suite.
So if you run FooTest and BarTest one after the other in an IDE, that will actually result in two "test suites" with different roots. The same is true if you configure your build tool to fork between test classes.
Whereas, if you execute both test classes together in a single "test suite" (e.g., by telling your IDE to run all tests in the same package or same source tree) you will then see that there is one root like in the output I provided above.
Note, however, that there was an issue with the junit-platform-surefire-provider prior to version 1.0.3, whereby the provider launched the JUnit Platform for each test class. This would give the appearance of forking even though Surefire did not actually start a new JVM process. For details, see https://github.com/junit-team/junit5/pull/1137.
I need to make JUnit tests using Mockito or PowerMock or smth else but I don't know what to start with. I created testing folder, set mockito, but what should I do next? I couldn't find any examples so Im stucked with it. Can you show me how to write this JUnit test or at least give some idea.
public void deleteAuthor(ActionRequest actionRequest, ActionResponse actionResponse)
throws SystemException, PortalException {
long authorId = ParamUtil.getLong(actionRequest, "authorId");
AuthorLocalServiceUtil.deleteAuthor(authorId);
SessionMessages.add(actionRequest, "deleted-author");
log.info(DELETE_SUCCESS);
}
Or this:
public void addAuthor(ActionRequest actionRequest, ActionResponse actionResponse)
throws IOException, PortletException, SystemException {
String authorName=ParamUtil.getString(actionRequest,"authorName");
Author author=AuthorLocalServiceUtil.createAuthor(CounterLocalServiceUtil.increment());
author.setAuthorName(authorName);
author=AuthorLocalServiceUtil.addAuthor(author);
}
P.S. Im very newbie and made only 1 JUnit test in my life, so Im really intrested in good advice. Thanks in advance!
UPD:
I try do to smth like this:
private BookAndAuthor portlet;
#Before
public void setUp() {
portlet = new BookAndAuthor();
}
#Test
public void testDeleteBookOk() throws Exception {
PowerMockito.mockStatic(BookLocalServiceUtil.class);
long id = 1;
Book book = BookLocalServiceUtil.createBook(id);
ActionRequest actionRequest = mock(ActionRequest.class);
ActionResponse actionResponse = mock(ActionResponse.class);
when(BookLocalServiceUtil.deleteBook(book)).thenReturn(null);
Book result = BookLocalServiceUtil.deleteBook(book);
assertEquals(result, null);
}
...but with no success.
We are running JUnit test using following set-up:
i. Create test folder beside docroot in your portlet.
ii. Add unit folder to test and create your package in it.
iii. Create portal-ext.properties file in your test folder with following configuration:
jdbc.default.driverClassName=com.mysql.jdbc.Driver
jdbc.default.url=jdbc:mysql://localhost:3309/db_name?useUnicode=true&characterEncoding=UTF-8&useFastDateParsing=false
jdbc.default.username=your_username
jdbc.default.password=your_password
jdbc.default.automaticTestTable=C3P0TestTable
jdbc.default.idleConnectionTestPeriod=36000
jdbc.default.maxIdleTime=1200
iv. Create a suite class (say AbcSuite.java) as following:
package x.x.x;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import com.liferay.portal.util.InitUtil;
#RunWith(Suite.class)
#Suite.SuiteClasses({
// Where AxTest.class would be your test class name
A1Test.class, A2Test.class, AxTest.class
})
public class AbcSuite {
#BeforeClass
public static void setUp() throws Exception {
// Loading properties and establishing connection with database
InitUtil.initWithSpring();
System.out.println("X Portlet's Test Suite Execution : Started.");
}
#AfterClass
public static void tearDown() {
System.out.println("X Portlet's Test Suite Execution : Completed.");
}
}
v. Create a test class (say A1Test.java) as following:
package x.x.x;
import java.util.ArrayList;
import org.junit.Assert;
import org.junit.BeforeClass;
import org.junit.Test;
public class A1Test {
#BeforeClass
public static void setUp() throws Exception {
System.out.println("Test Running : A1Test");
}
#Test
public void testAddAuthor() {
Author author = AuthorLocalServiceUtil.createAuthor(
CounterLocalServiceUtil.increment());
author.setAuthorName("Testcase Author");
author = AuthorLocalServiceUtil.addAuthor(author);
Assert.assertNotNull(author);
Assert.assertTrue(author.getAuthorId() > 0);
}
}
That it! You can execute all test cases together using following command:
ant test -Dtest.class=AbcSuite*
or separately as:
ant test -Dtest.class=A1Test*
This will be an unpopular answer, but...
I have found that JUnit tests with a lot of mocking objects are not particularly useful. The balance comes in when looking at the size of the setUp() method of your test: The longer it is, the less value the test has. In the portlet world you'd have to use a lot of mocks, and you'll be more busy mirroring the runtime environment (and correcting the assumptions you made about it) than you are fixing issues that you only found during the creation of this kind of tests.
That being said, here's my prescription
Build your portlets with one thing in mind: Portlets are a UI technology. UI is inherently hard to test automatically. You're stuck between the JSR-286 standard and your business layer - two layers that probably don't lend themselves particularly well for connecting them in tests.
Keep your UI layer code so ridiculously simple, that you can go with just a bit of code review. You'll learn more from it than from humongous setUp() routines of your JUnit tests.
Factor out meaningful UI-layer code. Extract it into its own utility class or method. Test that - notice that you probably don't even need a full PortletRequest object for it, use just the actual data that it needs
Create Integration tests on top of all this. These will utilize the full stack, your application deployed in a test environment. They will provide a smoke test to see if your code is actually working. But make sure that testing correct wiring doesn't slow you down: Code of the complexity object.setStreet(request.getParameter("street")); should not be tested, rather code reviewed - and it should be either obviously right or obviously wrong.
Use proper coding standards to make reviews easier. E.g. name your input field "street" if that's the data it holds, not "input42"
With these in mind: Whenever you write a portlet with code that you believe should be tested: Extract it. Eliminate the need to mock the portlet objects or your business layer. Test the extracted code. A second { code block } within a portlet's method might be enough code smell to justify extraction to a separate class/method that can typically be tested trivially - and these tests will be totally independent of Liferay, teach you a lot about your code if they fail, and are far easier to understand than those that set up a lot of mock objects.
I'd rather err on the side of triviality of tests than on the side of too complex tests: Too complex tests will slow you down, rather than provide meaningful insight. They typically only fail because an assumption about the runtime environment was false and needs to be corrected.
I've managed to get my Android project transitioned over to JUnit4, and of course the main reason I wanted to do it isn't working. Would love any help if anyone's got ideas here.
The problem I'm trying to solve is that I want to automatically skip certain tests if the build is not pointed at the staging server. I've got this set up with a BUILD_TYPE which is using gradle to inject the base URL.
I set up an assumeThat clause in my setup which correctly identifies when the build is not staging, but instead of halting and ignoring the rest of the test, it throws an exception and fails.
Here's my base class for my live API tests - I've annotated descending from this with #RunWith(AndroidJUnit4.class), so in theory this should always be run with the JUnit4 runner:
package com.[my package].nonuitests.liveservertests;
import android.support.test.runner.AndroidJUnit4;
import com.[my package].nonuitests.BaseAndroidTest;
import org.junit.Test;
import org.junit.runner.RunWith;
/**
* Tests against the live API. All tests descending from this class will
* be ignored if the BUILD_TYPE is not staging.
*/
#RunWith(AndroidJUnit4.class)
public class BaseLiveServerTests extends BaseAndroidTest {
private static final String STAGE = "staging";
/******************
* SETUP/TEARDOWN *
******************/
#Override
public void setUp() throws Exception {
super.setUp();
//TODO: Y U NO WORK?!
//This should cause the rest of the test to be skipped if it fails,
//but is instead throwing an AssumptionViolatedException.
assumeTrue(STAGE.equals(BuildConfig.BUILD_TYPE));
}
}
So, my questions:
Is there a better way to do this? In theory this could be done with flavors, but I was trying that earlier and it made everything else way more complicated.
My research indicates there's some kind of thing that Google's not implementing in their runner that's causing this to bomb out, but I'm having a hell of a time figuring out what I can/should do to fix this. Any suggestions of things I should subclass to get this to work as expected?
Any other thoughts would be most appreciated. Thanks!
Edit (1/26/15 11:40am CST): Per Grzesuav's suggestion, I took a stab at implementing this as an #Rule, but at the moment it's still not working. This seems a promising path, but it ain't working at the moment.
Edit 2 (1/26/15 12:15pm CST): OK, now it's working.
https://github.com/junit-team/junit/wiki/Assumptions-with-assume
ad 2) Custom runners could differently treat assume statement. To fix it you should write own version of Android runner and implement a way of dealing with assumes as native JUnit runner does or make a bug for android test runner.
ad 1) Suggested by me : try use JUnit Rules :
http://www.codeaffine.com/2013/11/18/a-junit-rule-to-conditionally-ignore-tests/
http://cwd.dhemery.com/2010/12/junit-rules/
OK, finally got it working with #Rules per Grzesuav's suggestion, although with significant changes since MethodRule has been deprecated. Here's a gist of what it turned out to be - I'll try to keep that updated as I refine it.
Some important notes:
You have to instantiate your #Rule in your test class, or you'll never actually hit any of your checks.
As of right now, this will not mark the test as ignored on Android, it'll just pass it without actually testing anything.
In Junit 4.12 it cannot handle tearDown
If you have tearDown you have to add an if statement with your condition rather than assumeTrue. I think the owners of Junit say it isn't supposed to work with #After
#After
override fun tearDown() {
if (junit == worksAgain()) {
I've got a utility class that I've created:
package com.g2.quizification.utils;
import com.g2.quizification.domain.Question;
public class ParsingUtils {
public static Question parse(String raw) {
Question question = new Question();
//TODO: parse some stuff
return question;
}
}
...that lives here:
I've also followed the tutorials and created a testing app, that looks like this:
And here's my test code, just waiting for some good 'ole TDD:
package com.g2.quizification.utils.test;
import com.g2.quizification.domain.Question;
import com.g2.quizification.utils.ParsingUtils;
public class ParsingUtilsTest {
public void testParse() {
String raw = "Q:Question? A:Answer.";
Question question = ParsingUtils.parse(raw);
//assertEquals("Question?", question.getQuestion());
//assertEquals("Answer.", question.getAnswer());
}
}
The test class is obviously missing the extension, but all the examples seem to only show extending something like ActivityUnitTestCase. I'm not testing an activity; I just want to test a static method in a utility class. Is that possible?
It seems like creating a utility test class should be simple, but I'm not sure what the next step is and/or what I'm missing.
The best approach for test project is to add the test project so that its root directory tests/ is at the same level as the src/ directory of the main application's project. If you are using junit4 and eclipse, you can just right-click on the util class you want to test and choose New -> JUnit Test Case.
Basically I would expect a new test class named ParsingUtilTest under the source folder tests/ and within the package com.g2.quizification.utils.test. The test class should extend TestCase and each method you want to test in that util class should have a new method in the test class with the name preceded with "test". I mean to say, suppose you have a method name in ParsingUtils called parseXml. The test method name in ParsingUtilsTest (which Extend 'TestCase') should be named testParseXml
The test class is obviously missing the extension, but all the examples seem to only show extending something like ActivityUnitTestCase. I'm not testing an activity; I just want to test a static method in a utility class. Is that possible?
Yes, as long as the class your are testing has nothing to do with android apis. And if you do need to test code with android api dependencies, for example, testing a view or an activity, you might want to have a try with robolectric. It's faster than the ones that extend ActivityUnitTestCase.
I have been playing with robolectric a lot (to do TDD on android), and so far, I prefer version 1.1 or 1.2 to 2.x, more stable and run fast.
Besides the tools mentioned above, there are many practices for writing good test cases, naming conventions, code refactoring and such.
It seems like creating a utility test class should be simple, but I'm not sure what the next step is and/or what I'm missing.
Its good to begin with small steps, xUnit Test Patterns: Refactoring Test Code and Extreme Programming Explained are some good books for your reference.
I've stripped the problem from all unnecessary complexity and attached two files for clarity's sake. In actuality, I want to load the required input for testing from a database. In the example I have the suites map in the Suites class, instead of the result from the query. I also have a rather complex comparison instead of the simple one in the run method of TestOverride. Basically that's how I solved creating test suites with their tests from the database (suites map) in a dynamic way. In addition, it is important that I can see the test name when I run it with eclipse.
If you run Suites (just right click on it and JUnit-run on eclipse) it works fine. The only test that passes is test4. However, I would like to be able to run a single test with this type of construct (a single suite would be nice as well, but I would be happy with a single test). In other words, after running all suites, I would like to go to the JUnit window, right click on a single test and run it. If I do it it doesn't work. I somehow thought the tests were stored somewhere after the first run and that I could use them later.
I am using eclipse 3.6 and JUnit 4.0
Any ideas? I don't use annotations for parametrized classes because everything has to be known before compile time (and I take the input from a database). I've also seen in the forums that it's quite a problem renaming the test cases with that approach.
import java.util.HashMap;
import java.util.Map;
import junit.framework.Test;
import junit.framework.TestSuite;
public class Suites {
public static Test suite() {
Map<String, String[]> suites = new HashMap<String, String[]>();
suites.put("suite1", new String[]{"test1", "test2"});
suites.put("suite2", new String[]{"test3", "test4"});
TestSuite all = new TestSuite("All Suites");
for(Map.Entry<String, String[]> entry : suites.entrySet()) {
TestSuite suite = new TestSuite(entry.getKey());
for(String testName : entry.getValue()) {
suite.addTest(
new TestOverride(
testName
)
);
}
all.addTest(suite);
}
return all;
}
}
import junit.framework.AssertionFailedError;
import junit.framework.TestCase;
import junit.framework.TestResult;
public class TestOverride extends TestCase {
private String name;
public TestOverride(
String name)
{
this.name = name;
}
#Override
public void run(TestResult result) {
result.startTest(this);
if (this.name.equals("test4")) {
result.endTest(this);
} else {
result.addFailure(this, new AssertionFailedError("Not test4"));
}
}
#Override
public String getName() {
return name;
}
}
I don't think it is possible to achieve what you'd like. As far as I know (and experienced) only "real" junit-methods (that are actual methods in existing classes) can be executed from the junit window (this is easily reproduced when using parameterized tests. The specific Tests can't be run here again either).
Perhaps you should try to generate the java code for the tests(and compile it).
It's much easier if you override runTest() and create the TestSuite from your TestCase class.
Here is an example that works:
http://mrlalonde.blogspot.ca/2012/08/data-driven-tests-with-junit.html