Develop Reference

How to make a plugin-like architecture runnable?

We have a couple of legacy Java projects, which we converted to Maven projects / modules. Previously, all projects were NetBeans projects and had no real dependency management. External dependencies existed on the companies network drive and were directly included as JARs in the NetBeans projects of each module. For the internal dependencies, simple project references were used. It was a pain to build everything because the programmer had to build everything in the right order.
Now, we are in the position that we can open all the Maven modules in IntelliJ IDEA and NetBeans. However, I am having trouble figuring out the best way to combine the different modules and external dependencies in a specific way, which conforms to in-house plugin-like structure. Especially with NetBeans (developing with both IDEs must be possible).
Here is how the git repositories / project structure roughly looks like. The folder structure of the modules is the default Maven structure for each module. The list feature of this site was too clumsy, so I included it as screenshot...
We have an internal maven repository for the stuff and building with maven etc. is working. For Intellij IDEA i can run and debug the end product for customer1 via a custom run configuration, which copies the needed files in the needed structure:
With IntelliJ IDEA, I can debug the software, but I think that the approach (custom IntelliJ run config I created, pointing to all needed JARs and files directly) is rather ugly, and for NetBeans I could not find a similar "run configuration" mechanism.
So I tried to achieve this build process by creating a new "Customer1Runnable" Maven project as a sort of build description, which points to all needed Maven modules. Based on this, I believed I could achieve and automatism to create the needed software structure. Ergo copy all modules into a plugin folder and all dependencies of the modules into a lib folder inside the Customer1Runnable project, using the maven-assembly-plugin.
First off, is my assumption correct that this is a possible use case for the maven-assembly-plugin?
The project itself does not have any source files, it is only a pom.xml and the assembly-config.xml descriptor. I attached the assembly-plugin to the package phase. When running the mvn package command all connected modules are built, but for the execution of the assembly-plugin I get the following output:
For starters, I only tried to include one module in the assembly descriptor. This is the XML (opicom-assembly.xml) for it:
<assembly xmlns="http://maven.apache.org/ASSEMBLY/2.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/ASSEMBLY/2.1.0 http://maven.apache.org/xsd/assembly-2.1.0.xsd">
<id>opicom-assembly</id>
<formats>
<format>dir</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<moduleSets>
<moduleSet>
<useAllReactorProjects>true</useAllReactorProjects>
<includes>
<include>my.company.reporting:module1</include>
</includes>
</moduleSet>
</moduleSets>
</assembly>
pom.xml of Customer1Runnable project
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<version>1.6</version>
<groupId>my.company.customer1</groupId>
<artifactId>OpicomRunnable</artifactId>
<packaging>pom</packaging>
<name>OpicomRunnable</name>
<repositories>
<repository>
<id>Company-Maven-Repo</id>
<url>file:\\\\MyCompany\TFSDrop\MavenRepo</url>
</repository>
</repositories>
<modules>
<module>../my.company.customer1.module1</module>
<module>../my.company.customer1.module2</module>
.
.
.
<module>../../MyCompany_Common/Report/my.company.reporting.module1</module>
</modules>
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.3.0</version>
<inherited>true</inherited>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<descriptors>
<descriptor>opicom-assembly.xml</descriptor>
</descriptors>
</configuration>
</plugin>
</plugins>
</build>
</project>
The pom of a module looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>my.company</groupId>
<artifactId>reporting</artifactId>
<version>1.3</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>module1</artifactId>
<version>1.3</version>
<packaging>jar</packaging>
<dependencies>
<!-- external dependencies -->
<dependency>
<groupId>commons-pool</groupId>
<artifactId>commons-pool</artifactId>
<version>1.6</version>
</dependency>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc8</artifactId>
<version>21.1.0.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.3.0</version>
<configuration>
<finalName>my-company-${project.artifactId}</finalName>
<appendAssemblyId>false</appendAssemblyId>
<outputDirectory>../build</outputDirectory>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
</project>
Thanks for any input on what I am doing wrong here / how to achieve this with Maven.
EDIT:
As requested, here an example project as ZIP-File.
https://drive.google.com/drive/folders/1ilJeTrOPgYgUTdOP0J4BQcBnPT5fls0k?usp=sharing
The parent directories ModuleGroupCustomer and ModuleGroupCommon do represent git repositories in the real scenario. The relative module path is caused, because the maven project which should be my "run config" points to maven projects in both repositories.
Maybe I am misunderstanding Maven in general? I thought of it in terms of use cases for dependency management similar to .Net nuget packages, but also as "project configuration" like ordinary NetBeans/Intellij projects.
Is it better to simply stick to the existing NetBeans projects for day to day development?

After a long and tedious process of trial and error, I have found a solution which is working for me. So I decided to share the solution online, in case someone else runs into a similar problem. Here is a link to the final zip archive containing working example projects => File CustomerRunnable_RunningAssemblyPluginStackoverflowExample.zip https://drive.google.com/drive/u/0/folders/1ilJeTrOPgYgUTdOP0J4BQcBnPT5fls0k
My error was that I misunderstood how the assembly-plugin works. The approach that I executed the plugin inside my aggregator pom (CustommerRunnable) is wrong, as this maven project only exists as parent pom.
The CustommerRunnable pom.xml references all customer plugins as modules. Those modules have not the CustommerRunnable as parent, but a different pom. Then I created a separate maven project "distribution". The pom.xml of the distribution defines all the plugins (needed customer maven modules) as dependencies. It also has the CustommerRunnable pom.xml as parent. Hence when I run the project in NetBeans, all connected modules are also build(if necessary).
It also configures the assembly plugin. The assembly plugin is attached to the maven package-phase and thus executed with it. It also uses a custom assembly descriptor, which copies all the previously defined plugins into the right folders. This is done by using dependencySets with include and exclude patterns.
See https://maven.apache.org/plugins/maven-assembly-plugin/advanced-descriptor-topics.html for details on this.
So one dependencySet copies all jar files of all plugins to a /plugin folder by using an include pattern. Then this approach is inversed to copy the jar files of all external dependencies to a /lib folder.
The descriptor also defines some extra files to copy to a particular location. exec-maven-plugin, so I can comfortably start the customer software out of NetBeans. I didn't yet manage to configure the execute plugin correctly regarding the needed classpath arguments.
Endresult looks like this:
It is also worth noting that the configurations of the "Build project", "Run project" and "Debug project" inside NetBeans need a tiny bit of modification. (Right Click Module "distribution" -> "Properties" -> point "Actions"

Import Maven local .jar as a lib

I made a simple Maven project for my class. According to the teachers tutorial we cannot upload it to our school repo due to some server issues, so we have to store it locally using altDeploymentRepository. I have the following pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.dpp</groupId>
<artifactId>simple_lib</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
</properties>
<build>
<plugins>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
<configuration>
<altDeploymentRepository>internal.repo::default::file://${project.basedir}/../${project.name}-mvn-repo</altDeploymentRepository>
</configuration>
</plugin>
</plugins>
</build>
</project>
So in the directory with my Maven project I have two directories:
sample_lib
sample_lib-mvn-repo
In the second one, deep down in : sample_lib-mvn-repo\com\dpp\sample_lib\1.0-SNAPSHOT I have a .jar file which I want to import (but not using just .jar file like passing the path to it - I need to do this "Maven way", import it as Maven lib). Can I do it if the file is not stored on any remote repository, but on my hard drive?

Running simply mvn install will install the file in your local repository. The local repository, by default, is in your home directory, under .m2\repository.
Using your pom above, after running mvn install, you would have jar (and some other files) in .m2\repository\com\dpp\sample_lib\1.0-SNAPSHOT.
To import this subsequently in another project, you would create a dependency in that project's pom like:
<dependency>
<groupId>com.dpp</groupId>
<artifactId>simple_lib</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
This all takes place only on your machine, and will not use any remote repository.

Now you have a local simulation of a repository.
You can import it using the repository tag as described in https://maven.apache.org/pom.html#Repositories. To specify a file make it a file url like file:

Yes, you can add maven repository and point it to a local directory:
<repository>
<id>local</id>
<name>local</name>
<url>file:${user.dir}/sample_lib-mvn-repo</url>
</repository>
https://maven.apache.org/guides/introduction/introduction-to-repositories.html
Given that your jar file is here sample_lib-mvn-repo\com\dpp\sample_lib\1.0-SNAPSHOT.jar, you then can add it as a dependency:
<dependency>
<groupId>com.dpp</groupId>
<artifactId>sample_lib</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>

It's not exactly what you're asking. But a quick and dirty solution is to make a POM project (no source code).
Inside the POM project you have the main and external projects.
You can simply make a dependency on the other project.

Can't run a Spring-Boot jar file due to "Unable to open nested entry"

I know there are a dozen questions like this out there but I didn't find any of their solutions suitable for my case!
So, I'm trying to pack a jar file for a Spring-Boot project using Maven. I've created the whole project using IntelliJ IDEA and it runs under IDE. In order to build the package, I use mvn package from the command line.
The generated .jar file can be run on my development machine too but when I copy the .jar file to some raw docker container, it gives out the following error:
Exception in thread "main" java.lang.IllegalStateException: Unable to open nested entry 'BOOT-INF/lib/lucene-highlighter-5.4.1.jar'. It has been compressed and nested jar files must be stored without compression. Please check the mechanism used to create your executable jar file
And here's my pom.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>project</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>demo</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.3.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>2.2.0</version>
</dependency>
</dependencies>
</project>

As it turned out, it had nothing to do with Maven and/or Spring-Boot. In case someone else might face the same issue, here's what I've done that led to this problem:
As a part of my deployment process, I would unzip the jar file, revise the config.properties and zip it again. It seems for some reason (unknown to myself), the generated .jar file is not executable and leads to the mentioned error.
My workaround was not to uncompress the .jar file, instead just replace the config.properties with a single command, like zip ./project.jar ./config.properties, while the project.jar is already in the current path. This one works!

I also faced this issue and tried the following way which worked well for me.
NOTE : Dont' extract and build new jar again. Simply extract and update existing jar with modified file.
For example, there is a jar namely myjar.jar in lab_directory
i) move to lab_directory and extract myjar.jar ( which has dir1/config.properties file )
ii) copy dir1 in extracted folder to lab_directory folder
iii) modify lab_directory/dir1/config.properties
iv) Run following command in lab_directory
jar -uf myjar.jar dir1/config.properties
That's it. Existing myjar.jar has been updated with modified config.properties file now.

It has been compressed and nested jar files must be stored without compression. Please check the mechanism used to create your executable jar file
It just means that we should use no compression by a command like this:
jar -uvf0 example_app.war DIR_TO_ADD
Where "0" means that no compression will be used.
See "0" options in:
https://docs.oracle.com/javase/7/docs/technotes/tools/windows/jar.html#options
Such a way helped me to add the directory with files and run the java without errors like in the subject.

installing a jar with maven-install-plugin to local repo creates pom without dependencies [duplicate]

I have a maven plugin, which I have not uploaded to the central repository, but which I want to use in my projects.
What works
I can install the maven plugin like this:
git clone https://github.com/RudolfVonKrugstein/jinja-maven-plugin.git
cd jinja-maven-plugin
mvn install
Then I can use the plugin like this pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project>
<modelVersion>4.0.0</modelVersion>
<groupId>de.wintercloud</groupId>
<artifactId>sample</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>de.wintercloud</groupId>
<artifactId>jinja-maven-plugin</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>de.wintercloud</groupId>
<artifactId>jinja-maven-plugin</artifactId>
<version>1.0</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>renderjinja</goal>
</goals>
</execution>
</executions>
<configuration>
<outputFile>out.txt</outputFile>
<templateFile>templ.jinja</templateFile>
<varFile>vars.yaml</varFile>
</configuration>
</plugin>
</plugins>
</build>
</project>
Here are the other files relevant to the compile:
templ.jinja:
{{ Name }}
vars.yaml:
Name: MyWonderfullName
This works:
> mvn compile
> cat out.txt
MyName
Nice!
What does not work
Now I am trying to give the plugin as a jar to my colleagues so that they can simple install the jar. The Idea is to do it like this:
git clone https://github.com/RudolfVonKrugstein/jinja-maven-plugin.git
cd jinja-maven-plugin
mvn package
mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=target/jinja-maven-plugin-1.0.jar
When I now do (in the sample project dir)
mvn compile
I get this error:
[ERROR] Failed to execute goal de.wintercloud:jinja-maven-plugin:1.0:renderjinja (default) on project sample: Execution default of goal de.wintercloud:jinja-maven-plugin:1.0:renderjinja failed: A required class was missing while executing de.wintercloud:jinja-maven-plugin:1.0:renderjinja: org/yaml/snakeyaml/Yaml
How can I install the jar so that I can use it as a plugin?
It looks to me as if dependencies are missing. Why?

You just hit MINSTALL-110, which is going to be fixed in the next 3.0.0 release of the Maven Install Plugin. The core issue here is that you're installing manually a JAR file with the file parameter, the plugin detects that there is a POM inside, but only keeps the coordinate information (group id, artifact id, packaging and version), not the whole POM. As such, when the POM is installed, the dependencies aren't kept.
Indeed, if you take a look at the installed POM, you will see
<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<modelVersion>4.0.0</modelVersion>
<groupId>de.wintercloud</groupId>
<artifactId>jinja-maven-plugin</artifactId>
<version>1.0</version>
<packaging>maven-plugin</packaging>
<description>POM was created from install:install-file</description>
</project>
which shows that the dependencies weren't retained. And if you take a look at the Maven logs in debug mode when running the install-file command, it will show
[DEBUG] Using META-INF/maven/de.wintercloud/jinja-maven-plugin/pom.xml for groupId, artifactId, packaging and version
A work-around waiting for version 3.0.0 is to specify the path to the POM file to install, along with the main artifact, by specifying the pomFile parameter. Run the following instead:
mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=target/jinja-maven-plugin-1.0.jar -DpomFile=pom.xml
Then the full POM will be installed, not a generic stub.
With this change, the error
[ERROR] Failed to execute goal de.wintercloud:jinja-maven-plugin:1.0:renderjinja (default) on project sample: Execution default of goal de.wintercloud:jinja-maven-plugin:1.0:renderjinja failed: A required class was missing while executing de.wintercloud:jinja-maven-plugin:1.0:renderjinja: org/yaml/snakeyaml/Yaml
will not happen anymore. Maven will correctly download the dependencies and use them for the plugin.

Issues with Maven In Eclipse

Am newbie in Maven and trying to build my first app with Maven using the latest version of Eclipse. After right-clicking the pom.xml file choosing Run-As and Maven Build, i get a window with the title "Edit Configuration and Launch". This window has 3 text areas "Goals","Profiles","User settings".
My question is what should i enter into these text areas to run my application successfully?
Thank you
EDIT
My application is a simple console application in Eclipse so i just want the output from the Main class to appear on the console.

Depending on your project, goals can be one of the life cycle phases phases to achieve.
To start with you can use "install" or "compile". The other two can be left empty for now.

We can attach maven-antrun-plugin:run goal to test phase. This will allow us to echo text messages for different profiles. We will be using pom.xml to define different profiles and will activate profile at command console using maven command.
Assume, we've created following pom.xml in C:\MVN\project folder.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.companyname.projectgroup</groupId>
<artifactId>project</artifactId>
<version>1.0</version>
<profiles>
<profile>
<id>test</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.1</version>
<executions>
<execution>
<phase>test</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<tasks>
<echo>Using env.test.properties</echo>
<copy file="src/main/resources/env.test.properties" tofile
="${project.build.outputDirectory}/env.properties"/>
</tasks>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
And assume, we've created following properties file in C:\MVN\project\src\resources folder.
env.properties
environment=debug env.test.properties
environment=test env.prod.properties
environment=prod
Now open command console, go to the folder containing pom.xml and execute the following mvn command. Pass the profile name as argument using -P option.
C:\MVN\project>mvn test -Ptest
For better clarification: http://www.tutorialspoint.com/maven/maven_build_profiles.htm

To expand on the previous answer, I would most often use 2 goals: clean install or clean compile
The clean remove all files generated by the previous build.
The install will run phases 1-7 below which copies the resulting jar into your local maven repository (usually a .m2 directory in your home directory) so that it is available and can be referenced as a dependency to other maven projects.
The compile will only run phase 1-2 below which compiles your code, but stops before running any configured tests.
Other options are copied from the Maven site below for convenience.
A Build Lifecycle is Made Up of Phases
Each of these build lifecycles is defined by a different list of build
phases, wherein a build phase represents a stage in the lifecycle.
For example, the default lifecycle comprises of the following phases
(for a complete list of the lifecycle phases, refer to the Lifecycle
Reference):
validate - validate the project is correct and all necessary
information is available
compile - compile the source code of the
project
test - test the compiled source code using a suitable unit
testing framework. These tests should not require the code be
packaged or deployed
package - take the compiled code and package it
in its distributable format, such as a JAR.
integration-test - process and deploy the package if necessary into an environment where integration tests can be run
verify - run any checks to verify
the package is valid and meets quality criteria
install - install
the package into the local repository, for use as a dependency in
other projects locally
deploy - done in an integration or release
environment, copies the final package to the remote repository for
sharing with other developers and projects.