Running a subset of tests in Gradle - java

I have a multiproject gradle build which contains a mix of Java with JUnit and Scala with Specs tests, plus a few other technologies.
I have pretty much everything building fine but I am hitting an issue with the tests.
My projects are all organised with the following structure
<project>/src/main/... - containing the production source code
<project>/src/tests-unit/... - containing the JUnit or Specs test code
and then optionally a combination of
<project>/src/tests-util/... - containing utility classes for setting up tests
<project>/src/tests-functional/... - containing functional level test code
<project>/src/tests-integration/... - containing integration level test code
My problem is that running grade tests on this project will actually run the tests in all the test directories and I want to be able to control what tests are being run. I had the idea of setting up a separate task for each level of test, but how do I specify to just run tests in the /src/tests-unit hierarchy and ignore the others?
I have tried several things including setting the runtimeClasspath of the sourceset for a target but I just get a deprecation warning for this and it has no effect.
Many thanks

You'd typically declare separate source sets and separate Test tasks. See samples/java/withIntegrationTests in the full Gradle distribution for details.

Related

How to exclude test classes form Coverage Analysis using Jacoco

I have some test to check my code. I have generated my report in sonarcloud but I have a problem: The coverage percentage takes into account also the test classes, that are obviously uncovered by other test. Is there any option just to take into account all the classes but the test ones??
First of all i would not adapt Jacoco, but you can exclude files from coverage report within sonarqube/sonarcloud with the sonar.coverage.exclusions property, where you can eg. specify a pattern like **/*Test.java to exlude all java files ending with Test.
Additionally you could also set this up within the UI, represented on the following screenshot:
Sidenote: i would inspect the sonar configuration, for me it looks like that test code is provided as normal sources to sonar. This would create such an topic, but normally sonar has an own property to configure test code. see analysis parameters documentation for further details or if you use gradle or maven plugins, check the respective documentation on how to organize the source code.

SonarQube not displaying classes under coverage

I have been working on setting up code coverage metric on SonarQube for a Java project using jacoco plugin.
The project builds successfully (using ant), jacocoreport.xml and jacocoreport.csv files get generated with the right information on covered/missed counter values for the covered/uncovered classes and methods. However, the dashboard on SonarQube reports "0.0%" as the metric value even when there is a single class file which has a covered method, and it provides absolutely NO information on "what classes have what coverage". In fact, upon clicking "0.0%" coverage link, I do not see (under "List" or "Tree") the particular folder which contains the classes I have written jUnit Tests for.
I checked with another developer from a different project who has worked on this before. She tells me that her project has around 2000 Unit Test methods and her jacoco.exec file size is 6MB. My project, on the other hand, has 1 to 2 Unit Test method(s) and the file-size is 6KB. Not sure if the jacoco.exec is generated properly.
How do I make sure the dashboard gives information on the covered/uncovered classes? Do you think it's not reporting because the the project has just one covered class file. I don't think this should be a problem. But I am not sure what's wrong.
NOTE: While running the Sonar Scanner locally, I noticed warnings that said some of the class files couldn't be accessed through the ClassLoader. This issue didn't get resolved even after me adding Sonar.Libraries and Sonar.Binaries properties (please ignore uppercase in this text).

How to organize different java source folders in netbeans (for testing)?

I am developing a project in Netbeans (8.0.2) that uses JUnit (4.x) for testing application code (in a Netbeans module). In the same module I also have some integration tests. I found a way to separate application code, unit tests and integration tests by putting the latter into a separate folder under MODULE_PATH\test\qa-functional\src. By that, the Netbeans IDE puts those three semantical different kinds of classes into individual folders:
The problem is, my functional tests tend to grow complex, so I would prefer to have them, well, unit tested. However if create a unit test (in the Unit Test Package) for a class of the functional test package, the test does not compile. Seems to be some class path issues.
I know I can put the unit test of the functional test in the functional test package as well (and I will do so if this question does not provide me with some solution), but anyway, is there a way to solve this in Netbeans?
I found a bad solution, changing the order of ant targets in common.xml (in line 610 of NB 8.0.2). But that is obiously not portable, changes with a new version of Netbeans,.. So not really a solution. See details in Netbeans Forum: Classpath problem when unit testing functional tests
In Eclipse, I would simply add a new source folder on the Java build path. But that mechanism does not exist in Netbeans?
For clearification, I added a screenshot.
Thanks in advance for your help.
You are are trying to do a wrong thing:
unit tests should verify the smallest possible functionality, e.g. that a method is producing the expected result
functional tests in NetBeans represent integration tests, tests when you treat the application as a black box
It does not make sense to "unit test" the functionality in the "functional tests". NetBeans treats the tests folders independent so that they are not on each other's classpath, that's why it does not work in NetBeans.

Automatically create testcode from class (jars

We have thousands of classes without any true unit test coverage. As special project we aim to add reasonable coverage to all. One option is to create tests from sourced code and use GenerateTestCases plugin with IJ.
I am exploring other options to create tests massively..
Is it possible to create tests (okay, just test-templates) from class files? TestGen4J does this, but the project seems to be inactive. Is there any tool that could help in creating the tests for thousands of classes.
Once such tests are generated, I hope we have more momentum to add more unit tests for existing classes.
Google CodePro Analytix eclipse plugin can generate the unit tests automatically for you and it is actively maintained.
you can try netbeans tools too, select the class and right click tools/create Junit Test but its manual and ugly task to thousands files

Performing code coverage using Clover on a Play! Framework Application using Ant

I'm writing an Ant script to do some additional checks on my Play! Framework Application.
Currently, I am executing my tests from my Ant script by simply making an exec call to "play auto-test".
<exec executable="${play.dir}/play.bat">
<arg line="auto-test"/>
</exec>
Would anyone know how to integrate Clover into a Play test suite? Obviously I am not tied to having to run my tests using the above.
I also tried writing the Ant script using the traditional way of executing JUnit tests (i.e. using Ant's junit target) and I got two problems:
When executing all my tests, only the first would execute successfully while the others would fail for strange reasons
If I just expose one test in my suite and have the test run successfully, it would say I have code coverage of 0%. I then thought I set up clover incorrectly, however, I created a simple class that tested a production class that did nothing and the coverage went up as I would expect.
So if I were to go down the junit route, I would need to know how to execute all my tests so that they can run one after another successfully (it works when using the Play way of running play auto-test) and I would need to know why Clover does not seem to pick up lines of code touched by Play tests.
(I know there is a Cobertura module for Play, however, I find that Clover does a better job telling me an accurate coverage figure)
Update: Unfortunately I am unable to replicate the exact error I was getting before as I have run into compilation issues when I try and compile things manually. I've started to use the Secure module and it only contains Java source files. So in my Ant script, I call play precompile which produces the byte code for the Secure module (as well as everything else in the system including my code). So now when I try and compile my app code using Clover, I think the compiler gets into a bit of a tangle as I have two versions of my classes - one produced by the precompile command (non-clover) and one produced by my own ant compilation (with clover):
[javac] C:\projects\testproject\out\clover\classes\clover8583527041716431332.tmp\model\HouseTest.java:45: incompatible types
[javac] found : play.db.jpa.JPABase
[javac] required: models.House
[javac] __CLR2_5_115y15ygoxiz3dv.R.inc(1527);House found = House.findById(id);
So I essentially have two problems now:
I need to be able to compile my source code that also depends on Play provided modules (e.g. CRUD, Secure) which do not have compiled versions hence my attempt at getting around it by calling play precompile myself in my Ant script
Once I get compilation working, I will undoubtedly have my original issue again of not being able to execute the tests using the junit target.
Update #2: It turns out that the error I got was due to the findById call required a cast from JPABase to House (not that the IDE or play seemed to care about it). So after I went in and put a cast for all of play's "find*" methods, I actually got JUnit and Clover reports! However... I am now getting two kinds of errors:
Entity classes created in Play can be created by extending the Model class which provides magic methods such as those find methods mentioned before as well as the count method. The Model superclass actually extends GenericModel which implements those methods by throwing an UnsupportedOperationException. So obviously Play! does some more magic behind the scenes to provide actual implementations of these methods. Unfortunately, my tests (and production code) rely on methods such as count but they are throwing the exception in my ant/junit scenario (note: everything works fine when running play auto-test.
The other error I am getting is due to the fact that I use the Spring module. In one of my classes (the root class), I call Spring.getBeanOfType(Some.class). Now I use auto-scanning, but in the ant/junit testing environment, the Spring module has yet to set up my spring container and therefore the call just returns null.
I have a feeling there is one magic fix that will resolve both of my issues however I am unsure what that magic fix is.
Clover does source level instrumentation, so it needs source code available. Everything you do before activating clover that generates bytecode will not be "clovered".
Clover for ant intercepts ant-compiler calls, so if you do a <clover-setup/> before any other compilation tasks in your ant script, everything sould be instrumented by clover.
You can execute the resulting compiled code in any way you want, e.g. executing from script or from junit, it does not matter, as long as the code is instrumented (and of course clover.jar is available in the classpath).
Clover hard-codes the location of the clover-database into the instrumented code, so you do not have to specify anything when executing.
It would really help, if you could outline how you are using clover, and you could also do a recheck at clover documentation at http://confluence.atlassian.com/display/CLOVER/1.+QuickStart+Guide.

Categories

Resources