Develop Reference

JSON files formatting in Spring boot based projects

I have spring boot projects with lots of files in .json apart from .java files.
For java formatting, we are using pre-commit hooks with google-java-format . However, for formatting .json files I am a bit struggling.
I have used a maven spotless plugin
<plugin>
<groupId>com.diffplug.spotless</groupId>
<artifactId>spotless-maven-plugin</artifactId>
<version>${spotless.version}</version>
<configuration>
<formats>
<format>
<includes>
<include>*.json</include>
</includes>
<prettier>
<!-- Specify at most one of the following 3 configs: either 'prettierVersion' (2.0.5 is default) , 'devDependencies' or 'devDependencyProperties' -->
<prettierVersion>2.0.5</prettierVersion>
<!-- Specify config file and/or inline config, the inline always trumps file -->
<config>
<useTabs>true</useTabs>
</config>
</prettier>
</format>
</formats>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
The problem with this approach is, that it requires npm in the machine to be installed else mvn clean install will fail. Many machines on the jenkins server don't have npm pre-installed so it fails during the build.
Is there an easy way to solve this?
PS: This project uses GIT as version control here.

Introduction
Let's consider the limitation: the Spotless formatter steps must be Java-based, i.e. avoid using external programs (npm, etc.).
Let's consider Spotless Maven plugin 2.23.0 as the current version.
Let's call Java-based JSON document formatting and validation the feature.
Analysis
According to the feature matrix table, the following formatter steps are available for Spotless Gradle plugin, but not available for Spotless Maven plugin:
json.gson.GsonStep.
json.JsonSimpleStep.
These feature steps are Java-based.
Summary
Currently, there are no such Spotless formatter steps available for Spotless Maven plugin — the feature is absent.
Possible solutions
Implement feature
Implement the feature.
Create a pull request to propose and collaborate on changes to the GitHub repository.
Request feature
Request the feature by creating an issue in the GitHub repository.
It looks like there was a quick attempt to request it. Please, see the comment:
GitHub issue: Add JVM-based JSON formatter by jamietanna · Pull Request #853 · diffplug/spotless.
Comment.
Find and use additional formatter Maven plugin
Find and use additional formatter Maven plugin that has the feature.
For example, it seems that the following formatter Maven plugin supports JSON document formatting and validation:
<dependency>
<groupId>net.revelc.code.formatter</groupId>
<artifactId>formatter-maven-plugin</artifactId>
<version>2.19.0</version>
</dependency>
Some related references:
formatter-maven-plugin – Introduction.
formatter-maven-plugin – formatter:format - configJsonFile parameter.
formatter-maven-plugin – formatter:validate - configJsonFile parameter.
formatter-maven-plugin/JsonFormatter.java at formatter-maven-plugin-2.19.0 · revelc/formatter-maven-plugin.
formatter-maven-plugin/JsonFormatterTest.java at formatter-maven-plugin-2.19.0 · revelc/formatter-maven-plugin.

How to aggregate maven subproject javadoc output without regenerating javadoc

I have a largish multimodule Maven build. I need to generate the javadoc for all of the modules and produce an "aggregated" javadoc result that I can deploy to a box for consumption by users.
I did have this working perfectly fine for quite a while, until I tried implementing a custom taglet with specific features and requirements, which makes this more complicated to produce.
All of the submodules inherit a parent pom that is not the aggregator pom. In that parent pom I define the maven-javadoc-plugin. This is what it looked like before I added the custom taglet:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.10.4</version>
<configuration>
<additionalparam>-Xdoclint:none</additionalparam>
<bottom>Unified Service Layer - bottom</bottom>
<doctitle>Unified Service Layer - title</doctitle>
<footer>Unified Service Layer - footer</footer>
<groups></groups>
<header>Unified Service Layer - header</header>
<level>public</level>
<packagesheader>Unified Service Layer - packagesheader</packagesheader>
<top>Unified Server Layer - top</top>
<windowtitle>Unified Service Layer - windowtitle</windowtitle>
</configuration>
<executions>
<execution>
<id>module-javadoc-jar</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
<show>protected</show>
<detectLinks>false</detectLinks>
</configuration>
</execution>
<execution>
<id>aggregated-documentation</id>
<phase>package</phase>
<inherited>false</inherited>
<goals>
<goal>aggregate-jar</goal>
</goals>
<configuration>
<show>protected</show>
<detectLinks>false</detectLinks>
</configuration>
</execution>
</executions>
</plugin>
With this, I could build all all of the modules, which will generate their own javadoc (which I now know is just a validation step, as aggregate-jar doesn't use this output). I have a separate step I call from jenkins that runs "javadoc:aggregate-jar" in the root project, which produces the aggregated javadoc jar that I deploy.
Again, this has been working fine until now.
I implemented a custom javadoc taglet which requires getting access to the Class object associated with the source file it is contained within. I got this to work, at least in the individual module builds by adding the following to the configuration above:
<taglets>
<taglet>
<tagletClass>com.att.det.taglet.ValidationConstraintsTaglet</tagletClass>
</taglet>
<taglet>
<tagletClass>com.att.det.taglet.ValidationConstraintsCombinedTaglet</tagletClass>
</taglet>
</taglets>
<tagletArtifacts>
<tagletArtifact>
<groupId>com.att.detsusl.taglets</groupId>
<artifactId>validationJavadocTaglet</artifactId>
<version>0.0.1-SNAPSHOT</version>
</tagletArtifact>
</tagletArtifacts>
In order to have the taglet get access to the class file, I had to add a minimal plugin configuration to each subproject pom.xml, which looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<configuration>
<tagletArtifacts combine.children="append">
<tagletArtifact>
<groupId>com.att.detsusl</groupId>
<artifactId>artifact-name</artifactId>
<version>${current.pom.version}</version>
</tagletArtifact>
</tagletArtifacts>
</configuration>
</plugin>
With these minimal changes, I could run the build in each module, generating the javadoc, and examining the generated javadoc output in each module, verifying that it all worked.
However, the problem is, when I run "javadoc:aggregate-jar" in the root project, all of that already built output is ignored. It reruns the javadoc generation for all of the subprojects, also ignoring the appended tagletArtifacts list in each subproject pom.xml file. As a result, I get ClassNotFound errors when it tries to get the class file.
I could "fix" this by putting all of the subproject GAVs into the top-level "tagletArtifacts" list, but I definitely do not want to do that. I liked the ability to specify this in the subproject pom.xml (with combine.children="append") to make it work.
What I need is an overall javadoc package for all of the subprojects, with the taglet able to get access to the class file, without forcing the parent pom to know about all of its subprojects. How can I do this?

I'm facing the same problem with all aggregate goals. I checked the source code to maven-javadoc-plugin and it turns out that aggregate work by traversing submodules and collecting source files and nothing more, thus completely ignoring any form configurations specified in the submodules.
During execution every submodule is completely ignored:
source
if ( isAggregator() && !project.isExecutionRoot() ) {
return;
}
And during collection of source files submodules are traversed: source
if ( isAggregator() && project.isExecutionRoot() ) {
for ( MavenProject subProject : reactorProjects ) {
if ( subProject != project ) {
List<String> sourceRoots = getProjectSourceRoots( subProject );
So at the moment, there is no way to do this.
This is not easy to fix either since the whole plugin works by composing a single call to the actual javadoc tool. If you would like to respect settings in the submodules as well, you'll have to merge the configuration blocks of them. While this would work in your case with tagletArtifacts, it does not work for all the settings you can specify, e.g. any form of filter, and can therefore not be done in a generic way.

jacoco only shows coverage for classes in the same module

I have a somewhat large multi-module Maven project. I have the unit tests in each module being processed by Jacoco. I have a separate child module doing "merge" and "report-aggregate", and this appears to be generating data. I'm even using the generated data in SonarQube. Most of my tests are using PowerMock, and I'm using offline instrumentation.
However, after looking closer at the coverage data, I see that it is leaving out coverage data for classes and methods that I know are being executed during tests. The pattern I see in every module is that it only reports coverage for a single class in each module, which is a class actually in the current module. Almost all of the tests also call out to other classes in other modules in the build, and coverage for those classes are never reported.
The following plugin configurations are in the parent pom used by each child module:
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<executions>
<execution>
<id>default-instrument</id>
<goals>
<goal>instrument</goal>
</goals>
</execution>
<execution>
<id>default-restore-instrumented-classes</id>
<goals>
<goal>restore-instrumented-classes</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<argLine>-Xmx1024m</argLine>
<includes>
<include>**/*Test.java</include>
</includes>
<systemPropertyVariables>
<jacoco-agent.destfile>${project.build.directory}/jacoco.exec</jacoco-agent.destfile>
<running-unit-test>true</running-unit-test>
</systemPropertyVariables>
</configuration>
</plugin>
When I inspect the generated HTML results for each module, I find that it only reports results for the single class in the current module, and not the data for classes in other modules. From this, I would assume that how I do "merge" and "report-aggregate" in the separate child module is probably irrelevant to this problem.
The generated "jacoco.exec" file is binary, but I tried "catting" out one from one module just to see what ascii text was visible, and it showed only one occurrence of anything that looked like a file name, and it was the only file name reported in the HTML coverage report for that module.
I'm not sure what other information I can report.
Update:
I guess I can see pretty clearly now that when surefire runs unit tests, it uses the instrumented classes from the current module, but the uninstrumented classes from the maven artifacts. This is why I only see coverage for classes in the current module.
So it seems like I need a way to specify that the "target/generated-classes/jacoco" folder for each module the current module depends on, is prepended to the classpath that surefire uses. I don't see a way to do that.
Alternatively, I see that the "instrument" goal has an "includes" configuration element. Should I be specifying paths to all of the "target/classes" directories for each of the modules that the current module depends on?

Recording of code coverage for some class requires its instrumentation. Goal instrument performs instrumentation of classes of current module.
all of the tests also call out to other classes in other modules
so the ones that are not instrumented. And if I correctly understood, then exactly those for which you are missing coverage.
If you don't use PowerMock for classes that come from other modules, but only for classes in current module, then you can combine offline instrumentation with on-the-fly using agent. But in this case make sure that classes instrumented offline are explicitly excluded from instrumentation by agent, otherwise agent will be throwing IllegalStateException: Class ... is already instrumented.
If you use PowerMock for classes that come from other modules, then this becomes more complex due to strictness of Maven in regards of manipulations with classpaths and dependencies. And I doubt that this can be easily achieved using one mvn comand, however seems possible using more:
instrument and run tests, but don't use restore-instrumented-classes
restore classes and generate report(s)
Unfortunately you haven't provided complete example (https://stackoverflow.com/help/mcve) and I don't have time to prepare full example to test this approach right now.
As a side note: inability to simply use agent comes from the fact that PowerMock bypasses any agent and reads class files from disk.

Specify pom properties via a properties file?

Due to the way my build system is designed (RTC Build Engine), I would like to provide maven with property values via a properties file, instead of specifying -Dkey=value for every property.
I found a couple of questions on S.O. (How to set build properties from a file in Maven POM? and How to read an external properties file in Maven) that relate precisely to this question, but they are relatively old, and both require custom plugins to work (in alpha state).
I realize that passing parameters to Maven like this is probably not the best solution, but the other option is specifying everything on the command line via -D settings which is not ideal either.
Furthermore, given that this properties file is only really used by the build engine (and not by the individual developer), I don't truly believe it belongs in the pom. But I cannot find any other mechanism that would allow me to specify a plugin to use - settings.xml does not permit specifying plugins.
Is my only choice in this case to use a plugin and specify it in the project pom?

in the pom you can place...
<properties>
<core-version>1234</core-version>
<lib-version>1234</lib-version>
<build-version>9999</lib-version>
<build-date>20150101</build-date>
</properties>
with all the properties you require.
Or you can use...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>dev.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
and the file dev.properties will contain the properties
core-version=1234
lib-version=1234
build-version=9999
build-date=20150101
...
Or... you can inject the properties using a settings.xml file as shown here
You may also find the Maven build number plugin useful... here

The best in such cases is to upgrade to at least Maven 3.2.1 which supports defining such properties on the command line like the following:
mvn -Drevision=1234 -Dchangelist=WhatEver -Dsha1=XXXX clean package
But you can only use the above names.
Excerpt from release notes:
A simple change to prevent Maven from emitting warnings about versions
with property expressions. Allowed property expressions in versions
include ${revision}, ${changelist}, and ${sha1}. These properties can
be set externally, but eventually a mechanism will be created in Maven
where these properties can be injected in a standard way. For example
you may want to glean the current Git revision and inject that value
into ${sha1}. This is by no means a complete solution for continuous
delivery but is a step in the right direction.

Ant: Weave aspects both from library and user aspects

I have a library created by myself with simple logging annotations and logger aspect.
I have several AspectJ projects where I want to reuse this library compiled into the jar.
Library is very similar to http://www.jcabi.com/ library with it's #Loggable annotation, but has some differences, main difference is that my library aspect tries to retrieve a Username if class of method marked as #Loggable implements HasUser interface (my own interface). Other differences are not so important.
I want an example of Ant script to compile both from library and project aspects.
AspectJ Ant Documentation is too complex for me because I have little experience with Ant.
Partially similar questions are hear:
aspectj: How to weave aspects from a library into user code?
iajc fails to weave aspects from a jar but succeedes from class files

I would recommend to use jcabi-maven-plugin, which weaves your .class files using AspectJ aspects from your current Maven module (in src/main/aspect) and from all its dependencies:
<plugin>
<groupId>com.jcabi</groupId>
<artifactId>jcabi-maven-plugin</artifactId>
<version>0.7.19</version>
<executions>
<execution>
<goals>
<goal>ajc</goal>
</goals>
</execution>
</executions>
</plugin>