I've been trying for quite some time to implement my own custom Java rule(s) on SonarQube. However, it seems like no matter what I try, I can't get the new rule to show up on the SonarQube UI.
I only have one rule at the moment, a security rule that checks to see if text output is sanitized. The rule extends BaseTreeVisitor and implements JavaFileScanner. It overrides visitMethodInvocation to do some checks on String arguments for the relevant methods. Here is the rule definition annotation:
#Rule(key = "Sanitize_HTML",
name = "HTML Responses Should be Sanitized",
tags = {"security", "owasp-a3"},
priority = Priority.CRITICAL)
#ActivatedByDefault
#SqaleSubCharacteristic(RulesDefinition.SubCharacteristics.SECURITY_FEATURES)
#SqaleConstantRemediation("10min")
public class SanitizeHTMLCheck extends BaseTreeVisitor implements JavaFileScanner{...}
After writing the rule, I wanted to test it, but quickly realized I had to wrap it in a plugin in order to do so. I wrote three additional classes for this, based entirely on the provided example plugin. Here's the base class:
public class SecurityPlugin extends SonarPlugin{
public List getExtensions(){
return Arrays.asList(
JavaClasspath.class,
JavaTestClasspath.class,
Java.class,
SecurityRulesDefinition.class,
SonarComponents.class,
DefaultJavaResourceLocator.class);
}
}
The classes in the list are all irrelevant (added in desperation) except for SecurityRulesDefinition. It mirrors the structure of the MyJavaRulesDefinition class from the example:
public class SecurityRulesDefinition implements RulesDefinition{
public void define(Context context){
NewRepository repository = context
.createRepository(RulesList.REPOSITORY_KEY, Java.KEY)
.setName("Security Rules");
AnnotationBasedRulesDefinition.load(repository, Java.KEY, RulesList.getChecks());
for(NewRule rule : repository.rules()){
rule.setInternalKey(rule.key());
}
repository.done();
}
}
Finally, just like the example, here's RulesList, where all of my rule classes are supposed to go:
public class RulesList {
public static final String REPOSITORY_KEY = "security_java";
private RulesList(){}
public static List<Class> getChecks(){
return ImmutableList.<Class>builder().addAll(getJavaChecks()).addAll(getJavaTestChecks()).build();
}
//Add all checks to here...
public static List<Class<? extends JavaCheck>> getJavaChecks(){
return ImmutableList.<Class<? extends JavaCheck>>builder()
.add(SanitizeHTMLCheck.class)
.build();
}
//Put all test checks here
public static List<Class<? extends JavaCheck>> getJavaTestChecks(){
return ImmutableList.<Class<? extends JavaCheck>>builder()
.build();
}
}
Like I said, these are all pretty much ripped from the example plugin, so I have no idea what could be wrong with them.
I'm using Eclipse with M2E to try and build the plugin. As suggested by the documentation's Coding A Plugin page, I've added the following plugin tag to my POM.xml:
<build>
<plugins>
<plugin>
<groupId>org.codehaus.sonar</groupId>
<artifactId>sonar-packaging-maven-plugin</artifactId>
<version>1.13</version>
<extensions>true</extensions>
<configuration>
<pluginKey>securityrules</pluginKey>
<pluginClass>org.myOrg.sonar_analysis.security_rules_java.SecurityPlugin</pluginClass>
<pluginName>Sonar Java Custom Security Rules</pluginName>
<pluginDescription>Implements several checks against OWASP-Top-10 vulnerabilities.</pluginDescription>
</configuration>
</plugin>
</plugins>
</build>
Now, according to everything I've read, I should be able to build the project (right-click on the project > Run As > Maven Build (with goal "package") and drop the resulting .jar into SONAR_HOME/extensions/plugins, and when I restart the server, the rule (and repository) should be there. However, no matter what I try, it's never there. I've spent hours combing the internet and trying anything I find, but the rule never shows up in the UI.
Am I missing something? Have I done something wrong? Is my code incorrect or missing anything?
Thank you for reading this monster post. Any advice you have is valuable, as I'm out of ideas.
The structure of the code seems right for me (more or less).
In the SecurityPlugin class, you return many classes (JavaClasspath.class, JavaTestClasspath.class and so on)... What are they for? What do they implement/extend?
In my expirience you need to return there:
- a "RulesDefinition" (to see the rule in SonarQube) and
- a CheckRegistrar (to let the checks being used).
Maybe my small rules project will give you some ideas (https://github.com/arxes-tolina/sonar-plugins ; one rule with two checks).
If you are still struggling with the rules try to set the sonar.log.level-property (./conf/sonar.properties) to DEBUG and watch the start-up of SonarQube.
Related
I know that I am gonna have to use // CLOVER: OFF to turn off clover test coverage. I have read https://confluence.atlassian.com/display/CLOVER/Using+Source+Directives
I have added that line before my class declaration like:
// CLOVER: OFF
public class SampleClass{
/*
* Some definitions
*/
}
This thing worked for me yesterday and failing today. I am scratching my head trying to figure out a reason for failure.
But, my maven build failed because it did not meet the coverage percentage. I am using maven 3.3.9 and eclipse Neon for my project.
It's the space between CLOVER: and OFF which causes problems. You should use the directives exactly as it's described in the docs
// CLOVER:ON
// CLOVER:OFF
Are you trying to exclude whole file from instrumentation? If so you can simply exclude classes from instrumentation on a Maven or Eclipse level. Documentation links:
https://confluence.atlassian.com/display/CLOVER/Configuring+instrumentation
https://confluence.atlassian.com/display/CLOVER/4.+Scope+of+instrumentation+in+Eclipse#id-4.ScopeofinstrumentationinEclipse-Excludingandincludingpackages
If you want to exclude one complete class from clover, you can do this in the pom configuration.
<configuration>
<excludes>
<exclude>**/SampleClass.java</exclude>
</excludes>
</configuration>
If you want to exclude just one or more methods, use CLOVER:OFF/CLOVER:ON around them.
public class SampleClass {
private int num;
public SampleClass() {
// intentionally left blank
}
// CLOVER:OFF
public void setNum(int num) {
this.num = num;
}
// CLOVER:ON
}
Beware of the fact that clover will still scan the excluded code if you have nothing left to scan in the class, hence the empty constructor.
I got a Generic GuiComponent class:
public abstract class GuiComponent<THIS extends GuiComponent<THIS>> extends Gui {
//...
}
(If you ask youself, what this type parameter is doing, look at Calling a Generic Interface Method does not work)
This class got a bunch of component subclasses like this:
public class ConcreteComponent extends GuiComponent<ConcreteComponent> {
//...
}
They all got renderers:
public interface ComponentRenderer<T extends GuiComponent<T>> {
//...
}
wich have concrete implementations:
public class FlatConcreteComponentRenderer implements ComponentRenderer<ConcreteComponent> {
//...
}
But now I got the following class:
public class GuiListBox<U> extends GuiComponent<GuiListBox<U>> {
//...
}
which is generic itself. This leads to the following Renderer Implementation:
public class FlatListBoxRenderer implements ComponentRenderer<GuiListBox<?>> {
//...
}
Because the renderer do not need the type of the listbox AND shall be used for ALL types of listboxes, I use the wildcard, so I do not have to care about the type. Inside the draw method of the renderer, the list elements just shall be treated as objects and toString() is called. But this implementation does not work:
Error:(21, 73) java: type argument [...]components.GuiListBox is not within bounds of type-variable T
I need to add a type to the renderer, just to use it for GuiListBox, then it compiles:
public class FlatListBoxRenderer<T> implements ComponentRenderer<GuiListBox<T>>
But this is not very useful, because the same renderer instance shall be handled to all ListBoxes by default. Why does this error occure, though my IDE (IntelliJ IDEA) does not mark this, but fails building?
EDIT #1: I use maven to compile the project, but neither my IDE nor maven are able to compile the class. Anyways, here is my pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>de.cydhra</groupId>
<artifactId>Utility</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.2</version>
</dependency>
</dependencies>
</project>
Maven outputs the exact same error as my IDE build. To clarify: The Code works, if I add a type to the Renderer, but this wont work for me, because the renderer shall be added to all List, despite the type of list. That is why I want to use the wildcard.
EDIT #2: I changed the grammar flow of my description and added a code snippet to clearify when the error occurs, and when the code compiles.
EDIT #3: Since the comments and first answer tried to reproduce and do research on my "bug", here are the results so far:
http://mail.openjdk.java.net/pipermail/compiler-dev/2015-June/009604.html
Someone was reporting a bug in Java Compiler, where the following Statement, which is basically my problematic statement:
class C1<T extends C1<T>> {}
class C2<U> extends C1<C2<U>> {}
C1<? extends C2<String>>
C1<? extends C2<?>>
Did NOT throw a compiler error, though the bug reporter was mentioning some parts of the JLS (Java Language Specification), which are violated of those constructions. The mentioned parts of the specification were dealing with Bbund intersections, so generics with multiple bounds:
class D<T extends T1 & T2 & T3...>
(if I understood that correctly). Maurizius, the guy who has his hands on Java generics, replied, that indeed, the specification where unclear at that point and so the construct was made to result in a compile time error.
I personally do not understand this, because I cannot see any violation of the type bounds here, but I resigned and made my ListBoxRenderer look like this:
public class FlatListBoxRenderer implements ComponentRenderer<GuiListBox<Object>> {
public void draw(final GuiListBox<Object> component) {
//...
}
}
I thought, since the renderer doesn't care about the list content at all, I could just give a unspecific type argument and use the Renderer for any purpose later. But now I come to the following error:
I got a method somewhere else in my project:
public <T extends GuiComponent<T>> void setComponentRenderer(final Class<T> componentClass,
final ComponentRenderer<T> renderer)
this method assignes a Renderer to a class of GuiElements, so all instances of the GuiElements get a renderer by default. The method call
this.setComponentRenderer(GuiListBox.class, new FlatListBoxRenderer());
fails, because:
Inferred Type java.lang.Object for type parameter T is not within bounds; should extend [...]GuiComponent<java.lang.Object>
This error message does not make sense to me either, and slowly I get really confused (I btw know, that Java Generics aren't a nice feature of this language and many other languages offer far better Generics, which are runtime features and not only code style features. But that's another story)
My FlatListBoxRender DOES indeed have its type parameter within bounds, because the type is GuiListBox, which in my understanding extends GuiComponent (which isn't right in this place either, because GuiComponent is indeed bounded, but this bound is fullfilled by GuiListBox' declaration):
public class GuiListBox<U> extends GuiComponent<GuiListBox<U>> {}
I know my constructions are very complex, but Java isn't designed to just accept ArrayList, it should handle more than one Generic as well.
So, if anyone got solutions to my problem, please let me know.
In Addition: Here is my full project code by the way:
https://github.com/Cydhra/Util/tree/master/src/main/java/de/cydhra/customgui
You can find all the GuiComponents in the package components, all the renderer implementations in renderer.defaults, and the instances of the renderers in renderer.theme
OK, I found it I think. So both eclipse and intellij idea got the generics wrong. (Which is not a surprise IMHO since it's very easy to get confused)
So there is/was a bug intellij idea which prevents it from showing any error:
Java code that does not compile with JDK 8, but IntelliJ does not report an error
And there is/was a similar error in eclipse:
Discrepancy between Eclipse compiler and javac - Enums, interfaces, and generics
I hope it will help those people who will scratch their head in the future.
Consider this:
#Nullable Object obj = null;
Optional<Object> optional = Optional.ofNullable(obj);
This fails because checker-framework assumes ofNullable cannot accept null values (after all, its parameter is not marked as #Nullable).
Is there a good way to tell checker-framework that this method (or other methods in legacy code that I cannot change), accepts #Nullable types everywhere without having to change code everywhere?
EDIT: this answer was based on #mernst help in the comments and in the Checker Framework's Issue tracker
If you, like me, do not want or cannot use the annotated JDK, you will run into this issue.
Note: In most Java shops I've worked, we simply cannot switch which compiler we use or provide a "custom" JDK (that's really unthinkable). For that to be portable, I would have to add the custom JDK to my source repository, for starters, or distribute it to every machine, including CI servers, where the code compiles, and make sure they are in the exact same path across different OS's. Just not cool.
The solution is to provide stub classes and pass them as an argument to the javac process.
This can be done quite easily with whatever tool you use to compile.
For example, with Maven (using the standard compiler plugin):
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<annotationProcessors>
<annotationProcessor>org.checkerframework.checker.nullness.NullnessChecker</annotationProcessor>
</annotationProcessors>
<compilerArgs>
<arg>-Astubs=checkerframework/stubs</arg>
<arg>-AstubWarnIfNotFound</arg>
</compilerArgs>
</configuration>
</plugin>
You also need to add these dependencies to your project:
<dependency>
<groupId>org.checkerframework</groupId>
<artifactId>checker-qual</artifactId>
<version>1.9.2</version>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.checkerframework</groupId>
<artifactId>checker</artifactId>
<version>1.9.2</version>
<optional>true</optional>
</dependency>
Here, checkerframework/stubs is a directory (relative to the location of the pom), containing the stubs. For Optional, my stub looks like this (strangely, stubs must be named *.astub, so this file is called Optional.astub):
package java.util;
import org.checkerframework.checker.interning.qual.*;
import javax.annotation.Nullable;
class Optional<T> {
static <T> Optional<T> ofNullable(#Nullable T value);
#Nullable T orElse(#Nullable T other);
}
This approach is simple, requires little work, does not mess with which compiler I use or the Java libraries at all, makes sure these definitions are only used with the checkerframework (so I can, for example, add this to a Maven profile and enable it only if I want to by simply passing a Maven argument), will work across machines and OS's without previous setup in the true Java way of doing things.
I'm not sure why you say "its parameter is not marked as #Nullable".
When I look at file
checker-framework/checker/jdk/nullness/src/java/util/Optional.java,
I see the following annotated method:
public static <T> Optional<#NonNull T> ofNullable(#Nullable T value) {
return value == null ? empty() : of(value);
}
Furthermore, when I run the Checker Framework on the following code, it issues no warning.
// run like this:
// javacheck -g TestOptional.java -processor nullness
import java.util.*;
import org.checkerframework.checker.nullness.qual.Nullable;
import org.checkerframework.checker.nullness.qual.NonNull;
public class TestOptional {
void m() {
#Nullable Object obj = null;
Optional<Object> optional1 = Optional.ofNullable(obj);
}
}
I'm not sure what is going on in your case because you didn't provide a complete test case, you didn't say what command you ran, and you didn't give an actual error message. (You did provide a diagnosis, but I'm not sure it is accurate.)
Maybe providing more details would enable better understanding of your problem.
Is it possible to make creation of (Java) file(s) in Eclipse easier/quicker.
I know there is that "Create new class wizard", but it is slow for my "special" needs...
I have a specific project in which I'm creating new classes often, but the structure for those classes is the same. Let say I want to create class A, so I want file A.java to be created as
class A {
public static void main(String[] args) {
}
static int solve() {
}
}
and it would be perfect that also ATest.java is created for this class, for example
class ATest {
#Test
int test1() {
Assert.assertEquals(0, A.solve());
}
}
or is there such plugin for Eclipse?
Yes, you can try using FastCode Plugin, where you can create new templates as per your requirement. As FastCode plugin supports custom templates, here is an example how to create the above class and test class together:
<template name="CREATE_NEW_CLASS">
<description>Used to Create class in the specified package.</description>
<allowed-file-names>*.*</allowed-file-names>
<first-template-item>package</first-template-item>
<second-template-item>none</second-template-item>
<additional-parameters>className</additional-parameters>
<template-body>
<![CDATA[
<fc:class type="class" name="${className}" package="${package.name}" project="${package.javaProject}">
public class ${className} {
public static void main(String[] args) {
}
static int solve() {
}
}
</fc:class>
<fc:class type="test" name="${className}Test" package="${package.name}" project="${package.javaProject}">
public class ${className}Test {
#Test
int test1(){
Assert.assertEquals(0, ${className}.solve());
}
}
</fc:class>
]]>
</template-body>
</template>
Once you add the template, you need to do import using import option in template preference page as explained in the document.
Yes, this is relatively simple to set up. Open the Project Properties and navigate to the Java Code Style > Code Templates section. Once there, check the box to enable project-specific settings. The generation template you want is under the Code part of the tree; you want Class Body. It is probably empty, but click the Edit... button to modify it.
Whatever you enter in the Edit dialog will be inserted between the class' brackets when using the New Class wizard.
There's no way I know of to automatically create another class (the Test in your case). But Eclipse has a JUnit wizard that makes doing so very easy. Just right-click on a class and choose New > Other... and then find Junit Test Case in the list. That wizard will guide you through creating the test class, including selecting the method(s) you want to test.
Note: these instructions set up the template for just the project or projects you select. You could also set up the same thing for your entire workspace Preferences, but doing so provides no way to share that configuration so that the same project checked out into another workspace will use it. I usually recommend making these kinds of settings on a per-project basis.
I'm not sure you need an IDE-specific template for that. When I was going through Project Euler, I had a setup like this:
public interface Problem {
public Object solve();
}
public class MyProblem implements Problem {
#Override
public Object solve() {
// do some stuff
return result;
}
}
Then in your (JUnit?) tests, you could use
Assert.assertEquals(expected, myProblemInstance.solve());
You can see my implementation here
If you do use an IDE template, you won't be able to use a generic solve() method, since it is not guaranteed that that class has that method. I would highly recommend using interfaces.
I've got a utility class that I've created:
package com.g2.quizification.utils;
import com.g2.quizification.domain.Question;
public class ParsingUtils {
public static Question parse(String raw) {
Question question = new Question();
//TODO: parse some stuff
return question;
}
}
...that lives here:
I've also followed the tutorials and created a testing app, that looks like this:
And here's my test code, just waiting for some good 'ole TDD:
package com.g2.quizification.utils.test;
import com.g2.quizification.domain.Question;
import com.g2.quizification.utils.ParsingUtils;
public class ParsingUtilsTest {
public void testParse() {
String raw = "Q:Question? A:Answer.";
Question question = ParsingUtils.parse(raw);
//assertEquals("Question?", question.getQuestion());
//assertEquals("Answer.", question.getAnswer());
}
}
The test class is obviously missing the extension, but all the examples seem to only show extending something like ActivityUnitTestCase. I'm not testing an activity; I just want to test a static method in a utility class. Is that possible?
It seems like creating a utility test class should be simple, but I'm not sure what the next step is and/or what I'm missing.
The best approach for test project is to add the test project so that its root directory tests/ is at the same level as the src/ directory of the main application's project. If you are using junit4 and eclipse, you can just right-click on the util class you want to test and choose New -> JUnit Test Case.
Basically I would expect a new test class named ParsingUtilTest under the source folder tests/ and within the package com.g2.quizification.utils.test. The test class should extend TestCase and each method you want to test in that util class should have a new method in the test class with the name preceded with "test". I mean to say, suppose you have a method name in ParsingUtils called parseXml. The test method name in ParsingUtilsTest (which Extend 'TestCase') should be named testParseXml
The test class is obviously missing the extension, but all the examples seem to only show extending something like ActivityUnitTestCase. I'm not testing an activity; I just want to test a static method in a utility class. Is that possible?
Yes, as long as the class your are testing has nothing to do with android apis. And if you do need to test code with android api dependencies, for example, testing a view or an activity, you might want to have a try with robolectric. It's faster than the ones that extend ActivityUnitTestCase.
I have been playing with robolectric a lot (to do TDD on android), and so far, I prefer version 1.1 or 1.2 to 2.x, more stable and run fast.
Besides the tools mentioned above, there are many practices for writing good test cases, naming conventions, code refactoring and such.
It seems like creating a utility test class should be simple, but I'm not sure what the next step is and/or what I'm missing.
Its good to begin with small steps, xUnit Test Patterns: Refactoring Test Code and Extreme Programming Explained are some good books for your reference.