Develop Reference

How to fix "org.apache.velocity.exception.ResourceNotFoundException" during schema generating

I am trying to integrate avro maven plugin into my application. I was forced to use custom templates because of Avro limitations.
When I include that plugin into build it fails (on windows, not unix) with exception:
[ERROR] Failed to execute goal org.apache.avro:avro-maven-plugin:1.9.1:schema (schemas) on project cloud-poc: Execution schemas of goal org.apache.avro:avro-maven-plugin:1.9.1:schema failed: org.apache.velocity
.exception.ResourceNotFoundException: Unable to find resource 'C:\..........\cloud-microservices\cloud-poc/src/main/resources/avro/templates/record.vm'
But when I do cat C:\..........\cloud-microservices\cloud-poc/src/main/resources/avro/templates/record.vm from PowerShell then it prints the file - it can find it.
The configuration works on unix systems without any issues. Here is pom:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.....</groupId>
<artifactId>cloud-poc</artifactId>
<version>0.8.0</version>
<packaging>jar</packaging>
<name>cloud-poc</name>
<description>Proof of concept microservice</description>
<properties>
<kafka.version>2.2.8.RELEASE</kafka.version>
</properties>
<dependencies>
.... avro + kafka
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>schemas</id>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
<goal>protocol</goal>
<goal>idl-protocol</goal>
</goals>
<configuration>
<sourceDirectory>${project.basedir}/src/main/resources/avro</sourceDirectory>
<outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
<fieldVisibility>PRIVATE</fieldVisibility>
<stringType>String</stringType>
<createSetters>false</createSetters>
<templateDirectory>${project.basedir}/src/main/resources/avro/templates/</templateDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
I also tried to use ablsoute path, different maven variables and relative path. I tried it on few projects, without luck.
I would expect the classes to be generated instead of.

The root of problem is about how velocity looking for a templates using a file path. In another hand, classpath lookup have issues with maven plugin classpath. At the moment I was able to fix this in a two ways: using classpath resource and using file resource.
Solution 1: Using a classpath.
Since you already have you templates in resourources, it will be copied to classpath and you may refers to it once it copied to target/classes.
Change templateDirectory to refer classpath resource, and change phase of avro-plugin execution to be after resources copied:
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>schemas</id>
<phase>process-resources</phase>
<goals>
<goal>schema</goal>
<goal>protocol</goal>
<goal>idl-protocol</goal>
</goals>
<configuration>
<sourceDirectory>${project.basedir}/src/main/resources/avro</sourceDirectory>
<outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
<fieldVisibility>PRIVATE</fieldVisibility>
<stringType>String</stringType>
<createSetters>false</createSetters>
<templateDirectory>/avro/templates/</templateDirectory>
</configuration>
</execution>
</executions>
</plugin>
Solutuion 2: Using a relative path.
You must refers to a template directory, using the relative path from project root. At the moment, I'm not sure would it work properly for multimodular maven project or not.
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.9.1</version>
<executions>
<execution>
<id>schemas</id>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
<goal>protocol</goal>
<goal>idl-protocol</goal>
</goals>
<configuration>
<sourceDirectory>${project.basedir}/src/main/resources/avro</sourceDirectory>
<outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
<fieldVisibility>PRIVATE</fieldVisibility>
<stringType>String</stringType>
<createSetters>false</createSetters>
<templateDirectory>/src/main/resources/avro/templates/</templateDirectory>
</configuration>
</execution>
</executions>
</plugin>

You can use profiles by OS family.
Add to your pom:
<profiles>
<profile>
<id>Windows</id>
<activation>
<os>
<family>Windows</family>
</os>
</activation>
<properties>
<avro.template.dir>src/main/resources /avro/templates/</avro.template.dir>
</properties>
</profile>
<profile>
<id>unix</id>
<activation>
<os>
<family>unix</family>
</os>
</activation>
<properties>
<avro.template.dir>${basedir}/src/main/resources /avro/templates/</avro.template.dir>
</properties>
</profile>
</profiles>
And then set templateDirectory to ${avro.template.dir}

Maven ignoring build segment in settings.xml?

I have the following profile in my settings.xml:
<profile>
<id>ossrh</id>
<properties>
<gpg.passphrase>secretKey</gpg.passphrase>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-gpg-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<id>sign-artifacts</id>
<phase>verify</phase>
<goals>
<goal>sign</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
The gpg.passphrase property is accessible just fine from my pom.xml, but when I try to do mvn release:prepare, the GPG-task doesn't get executed.
Using Maven 3.6.0.
The settings.xml path is set correctly, because I can access the property and other properties inside the settings.xml work just fine. And of course, the profile is under my active profiles, too:
<activeProfiles>
<activeProfile>ossrh</activeProfile>
</activeProfiles>

Settting up line separator in Maven

We are working on a multi module Maven project where in one sub project we have some resources file (free marker template files). When we create EAR out of the project depending on Operating System Maven is updating line separator in template files.
If we run mvn install on windows it keeps line separator as and when we run it on linux it changes line separator to .
Template are created on Windows machine and have default separator as , we don't want Maven to changes to even if we build project on windows / linux. As code is always deployed on Windows machine and fails when encounters linux line separator.
Is there any way to tell Maven not to mess up with line separators?
Project POM looks like
<project>
<parent>
<groupId>com.xyz</groupId>
<artifactId>Foo</artifactId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>FooBatch</artifactId>
<packaging>jar</packaging>
<properties>
<spring-version>4.1.4.RELEASE</spring-version>
</properties>
<dependencies>
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>2.3.21</version>
</dependency>
// and more dependencies
</dependencies>
</project>
and here is the Parent POM
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>com.xyz</groupId>
<artifactId>Foo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>FooBatch</module>
<module>FooEJB</module>
<module>FooEAR</module>
</modules>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<!-- http://maven.apache.org/plugins/maven-compiler-plugin/ -->
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<!-- skips surefire tests without skipping failsafe tests. Property
value seems to magically default to false -->
<skipTests>${skipUnitTests}</skipTests>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
<configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<argLine>${jacoco.agent.arg}</argLine>
</configuration>
</execution>
</executions>
<configuration>
<!-- skips failsafe tests without skipping surefire tests. Property
value seems to magically default to false -->
<skipITs>${skipITTests}</skipITs>
</configuration>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.2.201409121644</version>
<executions>
<execution>
<id>jacoco-initialize</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>jacoco-site</id>
<phase>package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>

Try to add the following to your parent pom.
<project>
...
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>

Maven multi module project with separate tests module - Code Coverage?

I have a maven multi module project.
root:
moduleA/ # no unit tests
moduleB/ # no unit tests
moduleC/ # no unit tests
tests/ # All unit tests, since depends on modules A, B and C
All tests are in single module called tests/ and all code is in separate modules.
Is there a way I can get code coverage?

There is a way to accomplish this. The magic is to create a combined jacoco.exec file and to do it in two steps. My pom:
<properties>
...
<jacoco.overall.exec>${maven.multiModuleProjectDirectory}/target/jacoco_analysis/jacoco.exec</jacoco.overall.exec>
</properties>
<build>
<pluginManagement>
<plugins>
...
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.7.8</version>
<configuration>
<destFile>${jacoco.overall.exec}</destFile>
<dataFile>${jacoco.overall.exec}</dataFile>
</configuration>
</plugin>
...
</plugins>
</pluginManagement>
</build>
<profiles>
<profile>
<id>runTestWithJacoco</id>
<activation>
<property>
<name>runTestWithJacoco</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>default-prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
<configuration>
<append>true</append>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>createJacocoReport</id>
<activation>
<property>
<name>createJacocoReport</name>
<value>true</value>
</property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<executions>
<execution>
<id>default-report</id>
<phase>validate</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
Add this to your parent pom and execute mvn clean install -DrunTestWithJacoco and than mvn validate -DcreateJacocoReport. Now you have the complete coverage of a class and it doesn't matter which test covered it. The magic is to use maven.multiModuleProjectDirectory to create a combined jacoco.exec file. This property is available since maven 3.3.1 and points to the folder where you started your maven build.

I don't think either of jacoco or cobertura is capable of reporting code coverage across modules. You may want to try instrumenting the compiled classes before running the test coverage report rather than relying on on-the-fly instrumentation.
See this jacoco maven goal to perform the offline instrumentation.

Since Jacoco version: 0.7.7, you can use report-aggregate.
Root pom.xml :
<project>
[...]
<build>
<plugins>
<!-- refer：https://prismoskills.appspot.com/lessons/Maven/Chapter_06_-_Jacoco_report_aggregation.jsp -->
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.6</version>
<executions>
<execution>
<id>prepare-agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>report</id>
<phase>test</phase>
<goals>
<goal>report</goal>
<goal>report-aggregate</goal>
</goals>
</execution>
</executions>
</plugin>
<plugins>
</build>
[...]
<pluginManagement>
<plugins>
<!-- unit test plugin -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>3.0.0-M5</version>
</dependency>
</dependencies>
<configuration>
<argLine>${argLine} -Dfile.encoding=UTF-8</argLine>
</configuration>
</plugin>
</plugins>
</pluginManagement>
[...]
</project>
Sub-modules pom.xml:
<project>
[...]
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>[path]</include>
</includes>
</configuration>
</plugin>
</plugins>
[...]
</project>
If you use Jenkin, you can just use jacoco plugin and <goal>report</goal> without other new things.

How to skip the maven antrun copy/shade:shade goal dynamically?

I am using maven for configuration of an application consisting of multiple small services. Most of the services developed in java share the same maven configuration, as in the same build lifecycle, some shared resources (like spring AMQP).
So I have organized the shared resources in a SuperPom.
While the shade plugin doesn't really seem to disturb the install process, the antrun plugin of course won't find any of the files it should copy, due to there not being created any jar files by the shade plugin.
As I'd like the configuration of the shade/antrun plugin to be abstracted in the SuperPom, I need to skip the shade/copy goal.
I have tried mvn clean install -Dmaven.shade.skip=true, mvn clean install -Dmaven.copy.skip=true, mvn clean install -Dmaven.shade.shade.skip=true
Here is a small sample for you to play with:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Test</groupId>
<artifactId>SuperTest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<properties>
<log4j.version>1.2.17</log4j.version>
<destination>pleasedeleteme</destination>
<mainpackage>com.uk.cc.it.info.gov.test.xxx</mainpackage>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>${mainpackage}.Main</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>copy</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>${groupId}</groupId>
<artifactId>${artifactId}</artifactId>
<version>${version}</version>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${destination}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
</project>

Did you try setting the phase of maven-shade-plugin to none in the super-pom and then overriding this in the client poms?
So in the parent pom:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.3</version>
<executions>
<execution>
<id>shade</id>
<phase>none</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<!-- ... -->
</configuration>
</execution>
</executions>
</plugin>
And in the child poms that need it:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<!-- no need to specify version -->
<executions>
<execution>
<id>shade</id>
<phase>package</phase>
<!-- no need to specify configuration -->
</execution>
</executions>
</plugin>

The maven-shade-plugin doesn't have a parameter to skip. Often the shade-plugin isn't there just for fun, so you might wonder if you really want to skip this. If you think it is still valid, you have to create a profile with activation like this:
<activation>
<property>
<name>skipShade</name>
<value>!true</value>
</property>
</activation>
This way it is activated by default, unless you add -DskipShade or -DskipShade=true.

Maven 3.6.1 gives you a new approach.
In the superPom you can define a profile for your shading configuration:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Test</groupId>
<artifactId>SuperTest</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<properties>
<log4j.version>1.2.17</log4j.version>
<destination>pleasedeleteme</destination>
<mainpackage>com.uk.cc.it.info.gov.test.xxx</mainpackage>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.8</version>
<executions>
<execution>
<id>copy</id>
<phase>package</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>${groupId}</groupId>
<artifactId>${artifactId}</artifactId>
<version>${version}</version>
<type>jar</type>
<overWrite>true</overWrite>
<outputDirectory>${destination}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>${log4j.version}</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>shade</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>${mainpackage}.Main</mainClass>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
In the user's settings.xml under .m2 you can add a profile of the same id to enable the shade profile configuration of your superPom. This gives you the option to simple toggling the shading from inside your IDE like Intellij IDEA (only tested in Intellij).
<settings xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
https://maven.apache.org/xsd/settings-1.0.0.xsd">
...
<!-- toggle shading from inside Intellij IDEA -->
<profiles>
<profile>
<id>shade</id>
</profile>
</profiles>
<!-- Shade Profile has to be activeProfile to be
able to explicitly disable shading -->
<activeProfiles>
<activeProfile>shade</activeProfile>
</activeProfiles>
</settings>
In the child project you can add a .mvn/maven.config file to your child project template to predefine shading for the project by default. (Requires a CVS template that is used to predefine a company standard.)
The approach using a maven.config is useful if some of your team members do not have the profile in their settings.xml file and you have to take care that shading will be done most of the time.
.mvn/maven.config:
-Pshading
The profile can also be activated by default using jenkinsfile for Jenkins by passing -Pshade. It will overwrite the maven.config setting. To disable use -P!shade
Please note if you are using maven.config file in Intellij (2020.2.2): The .mvn/maven.config file must exists in the subdirectory of the root aggregator pom folder. Building a subproject form the IDE does not respect a .mvn/maven.config file on the subproject level at the moment. Running a mvn command from the command line in the subproject folder will repespect both, the child project .mvn/maven.config and the parent .mvn/maven.config.

Disabling the maven shade plugin worked for me. The build was stock trying to produce the dependency reduced pom file before I disabled the Maven shade plugin.

The skip option was introduced in version 3.3.0 of the shade plugin, so now skipping can be done dynamically using, for example, properties:
<properties>
....
<skipShaded>true</skipShaded>
</properties>
...
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.3.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<skip>${skipShaded}</skip>
...
</configuration>
</execution>
</executions>
</plugin>
In the above the default is to skip, and this can be overridden with passing -DskipShaded=false to mvn.