Develop Reference

Create maven archetype with custom params using requiredProps and fileSets

I need to create a new maven archetype where project files contains some custom params that I would like to provide for mvn archetype:generate. Only -DgroupId, -DartifactId and version works fine. But I cannot provide e.g. -Dparam1=value1 .
I tried using requiredProperties and fileSets, but it doesn't work. How can I achieve that ?
archetype.xml
<archetype xmlns="http://maven.apache.org/plugins/maven-archetype-plugin/archetype/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-archetype-plugin/archetype/1.0.0 http://maven.apache.org/xsd/archetype-1.0.0.xsd">
<id>my-archetype</id>
<requiredProperties>
<requiredProperty key="param1">
<defaultValue>lol</defaultValue>
</requiredProperty>
</requiredProperties>
<fileSets>
<fileSet filtered="true" encoding="UTF-8" >
<directory>src/main/</directory>
</fileSet>
</fileSets>
<resources>
<resource>src/main/someFile.txt</resource>
</resources>
</archetype>
src/main/someFile.txt
I am just a txt file, but I need ${param1} value to be here !
I also need the pom.xml of my project (the one that is generated when mvn archetype:generate) to have custom properties replaced.
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>${groupId}</groupId>
<artifactId>${artifactId}-server</artifactId>
<version>${version}</version>
<packaging>pom</packaging>
<name>This name has to contains ${param1} !</name>
// ... all the rest stuff
</project>

Per the archetype descriptor documentation the archetype descriptor needs to be named archetype-metadata.xml. The example shows archetype.xml.
Also note, the initial XML element is archetype-descriptor, not just archetype.
Files must also be in a well-defined location for the plugin to find them.
<yourArchetypeProject>
src
main
resources
archetype-resources
<files and directories for project>
META-INF
maven
archetype-metadata.xml
The Maven docs could definitely be clearer on this. I found this tutorial helpful when I was learning.

including submodule assembly artifacts in another module's assembly?

I have a multimodule project such as:
top
|----sub1/
|----sub2/
|----packager/
sub1 and sub2 have multiple assemblies attached. Let's say sub1's assemblies creates target/dist/artifactA.jar and target/dist/artifactB.jar
If I create another assembly in packager, how can I pull in artifactA and artifactB from sub1, etc?
If I use this including groupId:artifactId:type:classifier as below, I get the error: "You must set at least one file".
Even if this did work, this isn't exactly what I want, anyway. I want the versions of the files in sub1/target/dist, which don't have version or artifactIds on them (as per sub1's assembly configuration).
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.2 http://maven.apache.org/xsd/assembly-1.1.2.xsd">
<id>my-distribution</id>
<formats>
<format>zip</format>
</formats>
<moduleSets>
<moduleSet>
<useAllReactorProjects>true</useAllReactorProjects>
<binaries>
<unpack>false</unpack>
<includes>
<include>com.example:sub1:jar:artifactA</include>
<include>com.example:sub1:jar:artifactB</include>
</includes>
</binaries>
</moduleSet>
</moduleSets>
edit: looks like <moduleset> works only if the sub modules are directly below the module doing the assembling. <dependencyset> is recommended in my case... but I still have the problem of rounding up attached jars that are not the primary jar.

Why does the maven-assembly-plugin descriptor say <!-- TODO: a jarjar format would be better --> in its example site?

Something that has always made me wonder with maven-assembly-plugin is that the jar-with-dependencies example file on the apache website has a TODO comment about a jarjar format.
Is this just an artifact of code that keeps getting copied through indefinitely or is there some actual reason behind that TODO on the sample file?

The documentation states the following
Use jar-with-dependencies as the descriptorRef of your assembly-plugin
configuration in order to create a JAR which contains the binary
output of your project, along its the unpacked dependencies.
So jar-with-dependencies creates a single JAR will all the class files of your project and also all the unpacked class files of your dependencies.
I'm guessing, "jarjar" refers to something like the Spring Boot Maven plugin does: it also creates a single JAR file, but the dependencies are included in their original JAR files, not unpacked. In contrast to the assembly plugin approach, this requires a custom class loader.
However, you should probably ask this question on the Maven mailing list. I just did so and will update this answer if I get any new information.
Updates
In found out that this particular part of the web page (including the TODO) was added over 10 years ago. Apparently the TODO hasn't been changed since.

I suspect this is referring to a jarjar format, which repackages Java libraries and embeds them into your own distribution, to allow for no dependencies.
see: jarjar
In the context of the example file, which contains:
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.3 http://maven.apache.org/xsd/assembly-1.1.3.xsd">
<!-- TODO: a jarjar format would be better -->
<id>jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<unpack>true</unpack>
<scope>runtime</scope>
</dependencySet>
</dependencySets>
</assembly>
I think the todo is to indicate that this is how this certain method is done, but that a jarjar would be better. That is, "here's how you do it, but you should do it this other way".

How to use Maven and various application.properties for artifact generation of different testing regions in a Spring configured app

I want to use Maven to handle artifact generation for the different local and testing regions. I believe I can use different profiles but I am not certain.
In Maven can I select different directories to select files used on packaging (such as application.properties)? How would I set that up?
An idea of what I want is to have a the following folders for resources in my project
local
build server
dev
sys
prod
Each folder should contain a different version of application.resources which is a file in Spring that can be used to handle hard-coded strings for use in variables. For local builds- our developers also work on different operating systems. Should I require I want to make it seamless on different OS' also.
Key outcomes would be:
Control Maven lifecycle phases from inside the IDE (IntelliJ)
Not complicate phases and team processes
Keep things as consistent for each developer
Make the different configurations per developer/region appear invisible when running a phase e.g. install
Ideally I would have my project set up according to best practices (Duvall, Matyas, Glover).

We provide different properties currently but not by way of different folders. We do this
via a mix of
Spring's PropertyPlaceholderConfigurer
Maven profiles (something we use to build our Dev environment),
Build Server (TeamCity in our case)
Maven phases to produce the correct artifact
start-up and build arguments
My understanding of what we do is limited, but hopefully this serves as a useful example for others and maybe myself to consider.
We provide parameters, as you'll see below, to point to different property files.
Each property file has configuration for a region/environment. I'll explain the current use
as best I can in-case it provides some use to others.
To use Maven profiles we have created a profile in our pom identified as development which includes a region configuration property called env. I don't yet know entirely how that is being used yet in our project however you'll see below our POM includes the a Maven Compiler plugin and a Maven Tomcat plugin.
Day to day, as developers we run our our applications locally on Tomcat from within IntelliJ
and provide the env property. On start-up the env property is provided as an argument to
set to classpath*:dev-common.properties.
This file is a properties configuration file - setting placeholder values for our different
regions.
The value of env is made available to our PropertyPlaceholderConfigurer
Example 1 - Implementation of Maven profile in pom.xml:
The implementation of a profile in our pom is:
<profile>
<id>development</id>
<activation>
<property>
<name>env</name>
<value>development</value>
</property>
</activation>
<build>
<pluginManagement>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.tomcat.maven</groupId>
<artifactId>tomcat7-maven-plugin</artifactId>
<version>2.0-SNAPSHOT</version>
...
</plugin>
</plugins>
</pluginManagement>
</build>
</profile>
Example 2 - Property placeholder configurer for normal build:
We also make use ofa Spring component, a PropertyPlaceholderConfigurer. We use this in collaboration with a build argument to set up a classpath pointer to resource files.
<bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>
${env}
</value>
</list>
</property>
Example 3 - Property placeholder configurer for test:
We have Spring Contexts specifically set up for integration testing which also use the PropertyPlaceholderConfigurer. These are picked up by a integration testing class using a combination of #ContextConfiguration(locations = {"classpath:test-dataexchange-application-context.xml"}) and #RunWith(SpringJUnit4ClassRunner.class)).
In the testing context we configure the PropertyPlaceholderConfigurer as follows to pick up the properties of an integration testing region:
<bean id="testpropertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath*:dev-local-common.properties</value>
</list>
</property>
Other notes:
We make use of Team City for build management but I have not seen how these settings are
used there, if they are at all. I can conceive there's an ability to combine the above approaches together to aid Continuous Integration and Delivery.
I do not see where the profile identified as development is being used. It is something I
must follow up with my fellow team members.
Resources:
Building for different environments at the Maven Project site.
Maven 3 does not allow configuration of a profile outside of a pom or settings.xml (the Maven configuration file) and says that users who used these external settings should now put them inside of settings.xml

If you are using Spring boot, there is an easy way of doing this.
Create two profiles in maven, and set a property in each profile with the name of the Spring profile you want to execute.
<profile>
<id>dev</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<!-- Default spring profile to use -->
<spring.profiles.active>dev</spring.profiles.active>
<!-- Default environment -->
<environment>develop</environment>
</properties>
</profile>
Inside your application.properties, add this property:
spring.profiles.active=${spring.profiles.active}
Create an application.property for each profile, using this pattern application-profile.properties. For example:
application-dev.properties
application-prod.properties
Be sure to active filtering in the resource plugin:
...
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
...
Another way is to create a file during the execution of maven called activeprofile.properties. Spring boot looks this file to load the active profile. You can create this file as follows:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<executions>
<execution>
<phase>prepare-package</phase>
<configuration>
<target>
<echo message="spring.profiles.active=${spring.profiles.active}" file="target/classes/config/activeprofile.properties" />
</target>
</configuration>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
<configuration>
</configuration>
</plugin>

Aim to generate an artifact for each environment at one time on the central server (CI/Build server), aim to generate an artifact and start/test the application with one click locally, provide a consistent easy to learn way to check out and run your build, and check in and configure your CI.
You can use profiles in Maven and utilize Maven targets to achieve the right build using a build server which in our case is TeamCity.
Use property placeholder configurer in Spring context with an application.resources file for each region and a filename mask e.g. application-resources-${region}.

How to make a monolithic jar.file?

I need to create a hadoop job jar file that uses mahout and a bunch of other libraries. I need ti be able to run the job without needing additional jar.files such that all referenced classes are packaged with the resultant jar file. How can this be done?

Configure your build file to copy all the referenced classes to the build directory. For example, in ant:
<path id="classpathunjar">
<fileset dir="${lib.dir}" includes="*.jar" excludes="sqljdbc4.jar"/>
</path>
<target name="compile" depends="clean">
...
<unjar dest="${build.dir}">
<path refid="classpathunjar" />
</unjar>
...
</target>
But it is better if you can manage without doing this. Use the libjars feature to load the jars into all nodes if you are doing this for running mapreduce jobs on a hadoop cluster

Hadoop has the ability to read jars-in-jar. Amend you Ant script to include all the dependency jars in a folder called lib, and add this lib folder to your output Jar. This is sometimes a better choice if you have number of larger jars as it decreases your jar build time.
See this article on a number of options you have when using 3rd party libs with hadoop
http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/

Jar is just a Zip container.
You can manually unzip and modify your Jar file with the classes needed, or you can make use of e.g., the jar-with-dependencies descriptor of the Maven build system.

In the generic sense, it is sometimes impossible, as JAR files have resources that must be in particular locations, and two conflicting but necessary resources might prevent the combination (Think META-INF/MANIFEST.MF)
However, in many cases it is very easy. Basically you unzip the JAR file to be added (it is a zip file format) and "add" the classes and what-not to the existing JAR file.
A better choice if you are making an executable JAR file is to add a ClassPath entry into your launching MANIFEST.MF and ship both JAR files in a directory structure compatible with your added ClassPath entry.

Note that the additional jars have to be put under a lib/ subdirectory (Yes, jars within a jar). I use the following maven assembly, which I found somewhere else.
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>job</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<unpack>false</unpack>
<scope>runtime</scope>
<outputDirectory>lib</outputDirectory>
<excludes>
<exclude>org.apache.hadoop:hadoop-core</exclude>
<exclude>${artifact.groupId}:${artifact.artifactId}</exclude>
</excludes>
</dependencySet>
<dependencySet>
<unpack>false</unpack>
<scope>system</scope>
<outputDirectory>lib</outputDirectory>
<excludes>
<exclude>${artifact.groupId}:${artifact.artifactId}</exclude>
</excludes>
</dependencySet>
</dependencySets>
<fileSets>
<fileSet>
<directory>${basedir}/target/classes</directory>
<outputDirectory>/</outputDirectory>
<excludes>
<exclude>*.jar</exclude>
</excludes>
</fileSet>
</fileSets>
</assembly>