JSON files formatting in Spring boot based projects - java

I have spring boot projects with lots of files in .json apart from .java files.
For java formatting, we are using pre-commit hooks with google-java-format . However, for formatting .json files I am a bit struggling.
I have used a maven spotless plugin
<plugin>
<groupId>com.diffplug.spotless</groupId>
<artifactId>spotless-maven-plugin</artifactId>
<version>${spotless.version}</version>
<configuration>
<formats>
<format>
<includes>
<include>*.json</include>
</includes>
<prettier>
<!-- Specify at most one of the following 3 configs: either 'prettierVersion' (2.0.5 is default) , 'devDependencies' or 'devDependencyProperties' -->
<prettierVersion>2.0.5</prettierVersion>
<!-- Specify config file and/or inline config, the inline always trumps file -->
<config>
<useTabs>true</useTabs>
</config>
</prettier>
</format>
</formats>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
The problem with this approach is, that it requires npm in the machine to be installed else mvn clean install will fail. Many machines on the jenkins server don't have npm pre-installed so it fails during the build.
Is there an easy way to solve this?
PS: This project uses GIT as version control here.

Introduction
Let's consider the limitation: the Spotless formatter steps must be Java-based, i.e. avoid using external programs (npm, etc.).
Let's consider Spotless Maven plugin 2.23.0 as the current version.
Let's call Java-based JSON document formatting and validation the feature.
Analysis
According to the feature matrix table, the following formatter steps are available for Spotless Gradle plugin, but not available for Spotless Maven plugin:
json.gson.GsonStep.
json.JsonSimpleStep.
These feature steps are Java-based.
Summary
Currently, there are no such Spotless formatter steps available for Spotless Maven plugin — the feature is absent.
Possible solutions
Implement feature
Implement the feature.
Create a pull request to propose and collaborate on changes to the GitHub repository.
Request feature
Request the feature by creating an issue in the GitHub repository.
It looks like there was a quick attempt to request it. Please, see the comment:
GitHub issue: Add JVM-based JSON formatter by jamietanna · Pull Request #853 · diffplug/spotless.
Comment.
Find and use additional formatter Maven plugin
Find and use additional formatter Maven plugin that has the feature.
For example, it seems that the following formatter Maven plugin supports JSON document formatting and validation:
<dependency>
<groupId>net.revelc.code.formatter</groupId>
<artifactId>formatter-maven-plugin</artifactId>
<version>2.19.0</version>
</dependency>
Some related references:
formatter-maven-plugin – Introduction.
formatter-maven-plugin – formatter:format - configJsonFile parameter.
formatter-maven-plugin – formatter:validate - configJsonFile parameter.
formatter-maven-plugin/JsonFormatter.java at formatter-maven-plugin-2.19.0 · revelc/formatter-maven-plugin.
formatter-maven-plugin/JsonFormatterTest.java at formatter-maven-plugin-2.19.0 · revelc/formatter-maven-plugin.

Related

Which lifecycle for a Java annotation that generates documentation?

I would like to be able to document the dependencies of my services directly in my code, using Java annotations. Those annotations could bear the information about the target system, whether the connection is incoming/outgoing/2-ways, and the type of connection (REST, RabbitMQ...).
It could look like this:
#Dependency(target = "Twitter API", type = "Outgoing", medium = "REST")
The idea would be to generate a DOT file from all the annotations inside the project.
I have a fair idea on how to create my own annotation, with the required attributes. However, i am not sure at which part of the lifecycle of the compilation/processing i should handle those annotations.
I understand that the annotation processors generate source files, but in my case the generated files are not at all required by the compiler nor the application itself.
Ideally i would like to be able to trigger the annotation processing, and DOT file generation, by a dedicated Maven or Gradle task.
Is it something that is easily doable when writing my own annotations?
If you want to create documentation via maven than you need add the bellow two dependencies as plugins and then execute site maven goal.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
If you just want to document REST endpoints than you can use swagger.
If your project uses spring than the integration of swagger is pretty easy. You can use this tutorial.
If you want to save in a file the dependency graph of your project you can simply execute the following maven command.
mvn dependency:tree -Doutput=/path/to/file
Annotations are processed by the Java compiler directly.
For Maven, one can use the maven-compiler-plugin. It is also possible to perform the compilation in 2 steps, as explained here.
For Gradle, one can add the processors in the dependencies block under the annotationProcessor configuration, as explained here.
Annotation retention can be specified as SOURCE, so they won't be kept after compilation.

Maven plugin instead of javaagent for Byte Buddy?

can there be a maven plugin instead of javaagent to reduce startup time of an application? Many ORM tools have both javaagent and maven plugin, so it should be possible - is it? Or at least something like "CompiledClassFilesBuilder" similar in functionality to AgentBuilder :-)
Regards,
Pavel
It is perfectly possible to use Byte Buddy from within a Maven plugin. You can implement a ClassFileLocator for the Maven target after the compile phase and redefine the classes within this folder using a ByteBuddy instance. The API is identical to that of the AgentBuilder inside a transformer.
You would need to implement such a plugin, though. There is none currently existing.
Do however note that this static transformation has its limitations. Using a plugin, it is not possible to redefine bootstrap classes of the VM. It would also be difficult to change library classes as they are external artifacts. Finally, Byte Buddy can register live callbacks that need to be set wt runtime.
UPDATE: I just added the plugin in question and will release a Maven Byte Buddy plugin in version 1.4.21. A transformation can be added as follows:
<plugin>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>transform</goal>
</goals>
</execution>
</executions>
<configuration>
<transformations>
<transformation>
<plugin>net.bytebuddy.test.SimplePlugin</plugin>
<!-- Optionally, specify groupId, artifactId, version of the class -->
</transformation>
</transformations>
<!-- Optionally, add 'initialization' block with EntryPoint class -->
</configuration>
</plugin>
The two relevant interfaces Plugin and EntryPoint are added to the library directly and are straight-forward to implement.

Specify pom properties via a properties file?

Due to the way my build system is designed (RTC Build Engine), I would like to provide maven with property values via a properties file, instead of specifying -Dkey=value for every property.
I found a couple of questions on S.O. (How to set build properties from a file in Maven POM? and How to read an external properties file in Maven) that relate precisely to this question, but they are relatively old, and both require custom plugins to work (in alpha state).
I realize that passing parameters to Maven like this is probably not the best solution, but the other option is specifying everything on the command line via -D settings which is not ideal either.
Furthermore, given that this properties file is only really used by the build engine (and not by the individual developer), I don't truly believe it belongs in the pom. But I cannot find any other mechanism that would allow me to specify a plugin to use - settings.xml does not permit specifying plugins.
Is my only choice in this case to use a plugin and specify it in the project pom?
in the pom you can place...
<properties>
<core-version>1234</core-version>
<lib-version>1234</lib-version>
<build-version>9999</lib-version>
<build-date>20150101</build-date>
</properties>
with all the properties you require.
Or you can use...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>dev.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
and the file dev.properties will contain the properties
core-version=1234
lib-version=1234
build-version=9999
build-date=20150101
...
Or... you can inject the properties using a settings.xml file as shown here
You may also find the Maven build number plugin useful... here
The best in such cases is to upgrade to at least Maven 3.2.1 which supports defining such properties on the command line like the following:
mvn -Drevision=1234 -Dchangelist=WhatEver -Dsha1=XXXX clean package
But you can only use the above names.
Excerpt from release notes:
A simple change to prevent Maven from emitting warnings about versions
with property expressions. Allowed property expressions in versions
include ${revision}, ${changelist}, and ${sha1}. These properties can
be set externally, but eventually a mechanism will be created in Maven
where these properties can be injected in a standard way. For example
you may want to glean the current Git revision and inject that value
into ${sha1}. This is by no means a complete solution for continuous
delivery but is a step in the right direction.

PMD coulnd't find ruleset

I'm on creating a maven based java project, which contains the PMD maven plugin. I use my own rule set XML and it works like a charm, except two rule sets: the emptycode and the unnecessary: when I run the build, maven says: "can't find resource". The role definitions look like:
<role ref="rulesets/emptycode" />
and
<role ref="rulesets/unnecessary" />
In every other cases, this kind of definition works. What I found out is that: there is a rule set with the name "unnecessary" under ecmasrcipt category, so maybe this definition needs some suggestion to use java version. I tried multiple thinks, like set language attribute to the ruleset xml node ("JAVA", based on PMD JavaDoc), and some pre-postfix in ref, but it doesn't work and I found no working solution over the web. Does someone has an idea, what I forgot to set, or what I fail? Thanks for any help!
PMD seems to be a fiddly beastie to use from Maven. I've just figured this out with version 3.0 of the plugin - there are two solutions:
The quick-and-dirty solution: put rulesets in your project:
download the PMD jar (http://sourceforge.net/projects/pmd/files/latest/download)
extract lib/pmd-x.x.x.jar
extract from that PMD jar file the rulesets/<type>/<ruleset>.xml files you want to use
place them in a folder under your project - something like ${basedir}/pmd/...
reference them as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<configuration>
<rulesets>
<ruleset>${basedir}/pmd/<ruleset>.xml</ruleset>
</rulesets>
</configuration>
</plugin>
The advantage is this is easy, the disadvantage is if you update the PMD version in future you'll need to remember to update these files.
The nice solution: reference rulesets in pmd-x.x.x.jar.
create a custom ruleset such as: ${basedir}/pmd/custom.xml (see http://pmd.sourceforge.net/pmd-5.0.2/howtomakearuleset.html)
reference the PMD rulesets in the following way: <rule ref="rulesets/java/imports.xml"/>
NB: the path is the path inside pmd-x.x.x.jar (see quick-and-dirty above) with no leading slash
reference your custom ruleset as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<configuration>
<rulesets>
<ruleset>${basedir}/pmd/custom.xml</ruleset>
</rulesets>
</configuration>
</plugin>
The advantage is this will always reference the current PMD rulesets from the PMD jar, the disadvantage is it's a bit fiddly to get right.
To experiment with this until it was working (maven-pmd-plugin version 3.0) I kept running mvn pmd:pmd (<linkXref>false</linkXref> in pom.xml) and tweaked the paths until I stopped getting errors.

Retrieve version from maven pom.xml in code

What is the simplest way to retrieve version number from maven's pom.xml in code, i.e., programatically?
Assuming you're using Java, you can:
Create a .properties file in (most commonly) your src/main/resources directory (but in step 4 you could tell it to look elsewhere).
Set the value of some property in your .properties file using the standard Maven property for project version:
foo.bar=${project.version}
In your Java code, load the value from the properties file as a resource from the classpath (google for copious examples of how to do this, but here's an example for starters).
In Maven, enable resource filtering. This will cause Maven to copy that file into your output classes and translate the resource during that copy, interpreting the property. You can find some info here but you mostly just do this in your pom:
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
You can also get to other standard properties like project.name, project.description, or even arbitrary properties you put in your pom <properties>, etc. Resource filtering, combined with Maven profiles, can give you variable build behavior at build time. When you specify a profile at runtime with -PmyProfile, that can enable properties that then can show up in your build.
The accepted answer may be the best and most stable way to get a version number into an application statically, but does not actually answer the original question: How to retrieve the artifact's version number from pom.xml? Thus, I want to offer an alternative showing how to do it dynamically during runtime:
You can use Maven itself. To be more exact, you can use a Maven library.
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-model</artifactId>
<version>3.3.9</version>
</dependency>
And then do something like this in Java:
package de.scrum_master.app;
import org.apache.maven.model.Model;
import org.apache.maven.model.io.xpp3.MavenXpp3Reader;
import org.codehaus.plexus.util.xml.pull.XmlPullParserException;
import java.io.FileReader;
import java.io.IOException;
public class Application {
public static void main(String[] args) throws IOException, XmlPullParserException {
MavenXpp3Reader reader = new MavenXpp3Reader();
Model model = reader.read(new FileReader("pom.xml"));
System.out.println(model.getId());
System.out.println(model.getGroupId());
System.out.println(model.getArtifactId());
System.out.println(model.getVersion());
}
}
The console log is as follows:
de.scrum-master.stackoverflow:my-artifact:jar:1.0-SNAPSHOT
de.scrum-master.stackoverflow
my-artifact
1.0-SNAPSHOT
Update 2017-10-31: In order to answer Simon Sobisch's follow-up question I modified the example like this:
package de.scrum_master.app;
import org.apache.maven.model.Model;
import org.apache.maven.model.io.xpp3.MavenXpp3Reader;
import org.codehaus.plexus.util.xml.pull.XmlPullParserException;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStreamReader;
public class Application {
public static void main(String[] args) throws IOException, XmlPullParserException {
MavenXpp3Reader reader = new MavenXpp3Reader();
Model model;
if ((new File("pom.xml")).exists())
model = reader.read(new FileReader("pom.xml"));
else
model = reader.read(
new InputStreamReader(
Application.class.getResourceAsStream(
"/META-INF/maven/de.scrum-master.stackoverflow/aspectj-introduce-method/pom.xml"
)
)
);
System.out.println(model.getId());
System.out.println(model.getGroupId());
System.out.println(model.getArtifactId());
System.out.println(model.getVersion());
}
}
Packaged artifacts contain a META-INF/maven/${groupId}/${artifactId}/pom.properties file which content looks like:
#Generated by Maven
#Sun Feb 21 23:38:24 GMT 2010
version=2.5
groupId=commons-lang
artifactId=commons-lang
Many applications use this file to read the application/jar version at runtime, there is zero setup required.
The only problem with the above approach is that this file is (currently) generated during the package phase and will thus not be present during tests, etc (there is a Jira issue to change this, see MJAR-76). If this is an issue for you, then the approach described by Alex is the way to go.
There is also the method described in Easy way to display your apps version number using Maven:
Add this to pom.xml
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>test.App</mainClass>
<addDefaultImplementationEntries>
true
</addDefaultImplementationEntries>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
Then use this:
App.class.getPackage().getImplementationVersion()
I have found this method to be simpler.
If you use mvn packaging such as jar or war, use:
getClass().getPackage().getImplementationVersion()
It reads a property "Implementation-Version" of the generated META-INF/MANIFEST.MF (that is set to the pom.xml's version) in the archive.
To complement what #kieste has posted, which I think is the best way to have Maven build informations available in your code if you're using Spring-boot: the documentation at http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#production-ready-application-info is very useful.
You just need to activate actuators, and add the properties you need in your application.properties or application.yml
Automatic property expansion using Maven
You can automatically expand info properties from the Maven project using resource filtering. If you use the spring-boot-starter-parent you can then refer to your Maven ‘project properties’ via #..# placeholders, e.g.
project.artifactId=myproject
project.name=Demo
project.version=X.X.X.X
project.description=Demo project for info endpoint
info.build.artifact=#project.artifactId#
info.build.name=#project.name#
info.build.description=#project.description#
info.build.version=#project.version#
When using spring boot, this link might be useful: https://docs.spring.io/spring-boot/docs/2.3.x/reference/html/howto.html#howto-properties-and-configuration
With spring-boot-starter-parent you just need to add the following to your application config file:
# get values from pom.xml
pom.version=#project.version#
After that the value is available like this:
#Value("${pom.version}")
private String pomVersion;
Sometimes the Maven command line is sufficient when scripting something related to the project version, e.g. for artifact retrieval via URL from a repository:
mvn help:evaluate -Dexpression=project.version -q -DforceStdout
Usage example:
VERSION=$( mvn help:evaluate -Dexpression=project.version -q -DforceStdout )
ARTIFACT_ID=$( mvn help:evaluate -Dexpression=project.artifactId -q -DforceStdout )
GROUP_ID_URL=$( mvn help:evaluate -Dexpression=project.groupId -q -DforceStdout | sed -e 's#\.#/#g' )
curl -f -S -O http://REPO-URL/mvn-repos/${GROUP_ID_URL}/${ARTIFACT_ID}/${VERSION}/${ARTIFACT_ID}-${VERSION}.jar
Use this Library for the ease of a simple solution. Add to the manifest whatever you need and then query by string.
System.out.println("JAR was created by " + Manifests.read("Created-By"));
http://manifests.jcabi.com/index.html
<build>
<finalName>${project.artifactId}-${project.version}</finalName>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>3.2.2</version>
<configuration>
<failOnMissingWebXml>false</failOnMissingWebXml>
<archive>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
Get Version using this.getClass().getPackage().getImplementationVersion()
PS Don't forget to add:
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
</manifest>
Step 1: If you are using Spring Boot, your pom.xml should already contain spring-boot-maven-plugin. You just need to add the following configuration.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<id>build-info</id>
<goals>
<goal>build-info</goal>
</goals>
</execution>
</executions>
</plugin>
It instructs the plugin to execute also build-info goal, which is not run by default. This generates build meta-data about your application, which includes artifact version, build time and more.
Step2: Accessing Build Properties with buildProperties bean. In our case we create a restResource to access to this build info in our webapp
#RestController
#RequestMapping("/api")
public class BuildInfoResource {
#Autowired
private BuildProperties buildProperties;
#GetMapping("/build-info")
public ResponseEntity<Map<String, Object>> getBuildInfo() {
Map<String, String> buildInfo = new HashMap();
buildInfo.put("appName", buildProperties.getName());
buildInfo.put("appArtifactId", buildProperties.getArtifact());
buildInfo.put("appVersion", buildProperties.getVersion());
buildInfo.put("appBuildDateTime", buildProperties.getTime());
return ResponseEntity.ok().body(buldInfo);
}
}
I hope this will help
I had the same problem in my daytime job. Even though many of the answers will help to find the version for a specific artifact, we needed to get the version for modules/jars that are not a direct dependency of the application. The classpath is assembled from multiple modules when the application starts, the main application module has no knowledge of how many jars are added later.
That's why I came up with a different solution, which may be a little more elegant than having to read XML or properties from jar files.
The idea
use a Java service loader approach to be able to add as many components/artifacts later, which can contribute their own versions at runtime. Create a very lightweight library with just a few lines of code to read, find, filter and sort all of the artifact versions on the classpath.
Create a maven source code generator plugin that generates the service implementation for each of the modules at compile time, package a very simple service in each of the jars.
The solution
Part one of the solution is the artifact-version-service library, which can be found on github and MavenCentral now. It covers the service definition and a few ways to get the artifact versions at runtime.
Part two is the artifact-version-maven-plugin, which can also be found on github and MavenCentral. It is used to have a hassle-free generator implementing the service definition for each of the artifacts.
Examples
Fetching all modules with coordinates
No more reading jar manifests, just a simple method call:
// iterate list of artifact dependencies
for (Artifact artifact : ArtifactVersionCollector.collectArtifacts()) {
// print simple artifact string example
System.out.println("artifact = " + artifact);
}
A sorted set of artifacts is returned. To modify the sorting order, provide a custom comparator:
new ArtifactVersionCollector(Comparator.comparing(Artifact::getVersion)).collect();
This way the list of artifacts is returned sorted by version numbers.
Find a specific artifact
ArtifactVersionCollector.findArtifact("de.westemeyer", "artifact-version-service");
Fetches the version details for a specific artifact.
Find artifacts with matching groupId(s)
Find all artifacts with groupId de.westemeyer (exact match):
ArtifactVersionCollector.findArtifactsByGroupId("de.westemeyer", true);
Find all artifacts where groupId starts with de.westemeyer:
ArtifactVersionCollector.findArtifactsByGroupId("de.westemeyer", false);
Sort result by version number:
new ArtifactVersionCollector(Comparator.comparing(Artifact::getVersion)).artifactsByGroupId("de.", false);
Implement custom actions on list of artifacts
By supplying a lambda, the very first example could be implemented like this:
ArtifactVersionCollector.iterateArtifacts(a -> {
System.out.println(a);
return false;
});
Installation
Add these two tags to all pom.xml files, or maybe to a company master pom somewhere:
<build>
<plugins>
<plugin>
<groupId>de.westemeyer</groupId>
<artifactId>artifact-version-maven-plugin</artifactId>
<version>1.1.0</version>
<executions>
<execution>
<goals>
<goal>generate-service</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>de.westemeyer</groupId>
<artifactId>artifact-version-service</artifactId>
<version>1.1.0</version>
</dependency>
</dependencies>
Feedback
It would be great if maybe some people could give the solution a try. Getting feedback about whether you think the solution fits your needs would be even better. So please don't hesitate to add a new issue on any of the github projects if you have any suggestions, feature requests, problems, whatsoever.
Licence
All of the source code is open source, free to use even for commercial products (MIT licence).
It's very easy and no configuration is needed if you use Spring with Maven.
According to the “Automatic Property Expansion Using Maven” official documentation you can automatically expand properties from the Maven project by using resource filtering. If you use the spring-boot-starter-parent, you can then refer to your Maven ‘project properties’ with #..# placeholders, as shown in the following example:
project.version=#project.version#
project.artifactId=#project.artifactId#
And you can retrieve it with #Value annotation in any class:
#Value("${project.artifactId}#${project.version}")
private String RELEASE;
I hope this helps!
With reference to ketankk's answer:
Unfortunately, adding this messed with how my application dealt with resources:
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
But using this inside maven-assemble-plugin's < manifest > tag did the trick:
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
<addDefaultSpecificationEntries>true</addDefaultSpecificationEntries>
So I was able to get version using
String version = getClass().getPackage().getImplementationVersion();
Preface: Because I remember this often referred-to question after having answered it a few years ago, showing a dynamic version actually accessing Maven POM infos dynamically (e.g. also during tests), today I found a similar question which involved accessing module A's Maven info from another module B.
I thought about it for a moment and spontaneously had the idea to use a special annotation, applying it to a package declaration in package-info.java. I also created a multi-module example project on GitHub. I do not want to repeat the whole answer, so please see solution B in this answer. The Maven setup involves Templating Maven Plugin, but could also be solved in a more verbose way using a combination of resource filtering and adding generated sources directory to the build via Build Helper Maven. I wanted to avoid that, so I simply used Templating Maven.
Accepted answer worked for me once in the step #2 I changed ${project.version} to ${pom.version}

Categories

Resources