Develop Reference

Maven Deploying Project as Jar - Missing Class definition

I am trying to create a runnable jar from a maven project created in Intellij Idea.
I tried building the Artifact through Intellij, but that did not work out. It could not find the main file.
After that I tried it through maven with:
- mvn compile
- mvn package
This creates a runnable jar which executes, but later when parsing a csv it throws an Exception:
Exception in thread "JavaFX Application Thread"
java.lang.NoClassDefFoundError: org/apache/commons/csv/CSVParser
But it is added in my pom.xml... I downloaded everything even the docs. I can see the org.apache.commons:commons-csv package in the external libraries, but it seems to be missing when creating the jar.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<packaging>jar</packaging>
<groupId>sceosa</groupId>
<artifactId>CEO_SA_ReportingLine</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<finalName>CEOSA</finalName>
<plugins>
<!-- download source code in Eclipse, best practice -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<version>2.9</version>
<configuration>
<downloadSources>true</downloadSources>
<downloadJavadocs>false</downloadJavadocs>
</configuration>
</plugin>
<!-- Set a compiler level -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<!-- Make this jar executable -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<archive>
<manifest>
<mainClass>de.Main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.hsqldb/hsqldb -->
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>25.1-jre</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-csv</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Can anybody see what is wrong with the pom file or intellij?

By default maven does not include dependencies when building jars. The jar will only work if you have the dependency in some other way.
You can use the maven-assembly-plugin to build a jar-with-dependencies.

I have used the maven-shade-plugin and it works great if you want to bundle all your dependencies into one jar. All you need is the plugin and provide the Main class from your project and it should have a main method. It will look something like this:
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <shadedArtifactAttached>true</shadedArtifactAttached>
                <transformers>
                    <transformer implementation=
                      "org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                        <mainClass>com.test.MainClass</mainClass>
                </transformer>
            </transformers>
        </configuration>
        </execution>
    </executions>
</plugin>
And just build your project using mvn clean install and you should find the the jar in your target directory and it will be suffixed with -shaded.jar. Hope this helps

unable to find logger class while running a maven build jar file

I am trying to write a maven integrated Java API. I have included log4j for logging purpose. Which works well when running through eclipse, but when maven package is done and the jar is run it is unable to run from cmd line using java -jar jar_name.jar throwing an error
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/Logger
Now the log4j.properties file is placed under src/main/resources folder. And the pom.xml is mentioned. Have tried searching for answers but none worked for me.
Any help available
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>Weather_Simulator</groupId>
<artifactId>weather_simulator</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>weather_simulator</name>
<url>http://maven.apache.org</url>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>com.test.weather.simulator.MainClass</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<!-- NOTE: We don't need a groupId specification because the group is
org.apache.maven.plugins ...which is assumed by default.
-->
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-configuration2</artifactId>
<version>2.1.1</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.6.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>

Your <scope> in your pom.xml appears to be wrong. Try this (notice I've changed "test" to "compile").
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.6.1</version>
<scope>compile</scope> <!-- look here -->
</dependency>
The way you currently have your pom.xml configured (with "test"), maven will only provide the log4j jar when doing "mvn test". If you need the jar at both compile time and run time (which is the scenario that appears to be causing problems for you), the scope needs to be "compile".
Note that "compile" is the default scope, so if you leave the <scope> element off, the scope will be "compile".
From the maven docs: "This [compile] is the default scope, used if none is specified. Compile dependencies are available in all classpaths of a project. Furthermore, those dependencies are propagated to dependent projects"
For more info about maven "scopes" look here.

This is a classpath problem, the jar generated by Maven contains only your classes. To fix this you can pack all the dependencies inside your project jar: How can I create an executable JAR with dependencies using Maven?

There are two things you should be aware if you like to create an executable jar which contains all dependent jars you have to use maven-assembly-plugin as you already did but you forgot to bind it to the life cycle...which looks like this:
<project>
[...]
<build>
[...]
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
[...]
</project>
Furthermore having plugins as dependencies is simply wrong..
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
</dependency>
This means remove this entry from your pom file.
Defining a dependency with a scope test means it will be available only during the unit tests which means also it will never being packaged into a resulting jar file. This means you have to change the following:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.6.1</version>
<scope>test</scope>
</dependency>
into the following:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.6.1</version>
</dependency>
After you have fixed those issues you should be able to build your app via:
mvn clean package
and find the resulting jar file which contains the dependencies in the target directory named like weather_simulator-1.0.0-SNAPSHOT-jar-with-dependencies.jar which you should use to call your app.

Not adding JARs to staging folder on Google Cloud when using Maven

I'm running integration tests on the cloud for the Google Cloud Dataflows that I have written; checking that they read from Pub/Sub and write to BigQuery correctly, but when using Maven (mvn clean install), the staging folder is not populated with the required JARs. The only JAR that appears is a surefirebooter.jar. As a result, I get a NoClassDefFoundError for PipelineOptions (most likely because it is the first class from a dependency that's trying to be referenced) in the Stackdriver logs, and consequently the tests fail. Since they're running on the cloud I am indeed using a DataflowRunner as opposed to a DirectRunner.
When I run the integration tests from my IDE they work fine; the staging folder is populated with all the JARs and all is well. Also, when I run the tests using Maven but with a DirectRunner the tests run successfully, thus my problem only occurs when using Maven and a DataflowRunner. I assume that problem therefore lies with the pom.xml file, which I have given below:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>group</groupId>
<artifactId>artifact</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.0.0-beta3</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.0.0-M3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>2.0.2-beta</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Anyone know why this is happening and how I may resolve it?

When staging files, the Dataflow runner will automatically stage the the classes available to the current class loader. I believe that surefire plays some tricks with the classloader to make the tests easier to run.
One option would be to specify filesToStage on the pipeline options, which will override the normal "detect JARs to stage from the class loader". Alternatively, look at how surefire is managing the classpath, and make sure the SDK JARs are available in the classloader the test is running in.

Maven Not Copying Dependencies to WEB-INF/lib in Eclipse

I realize anywhere from very similar to identical questions have been asked numerous times on StackOverflow, but most if not all seem to end up having different solutions.
I have tried almost everything that I have found from similar questions posted here and elsewhere, yet I still can't get Maven to put the dependency JARs in WEB-INF/lib.
The oddest thing is that every time I run Maven -> Update Project, Maven seems to reconfigure the class output folder to be /target/classes instead of /war/WEB-INF/classes which is required by the Google web app plugin.
Everything shows up correctly under Maven Dependencies in the build path, but nothing shows up in WEB-INF/lib, so the app fails with ClassNotFoundExceptions at runtime.
Here is the relevant part of the pom.xml:
<packaging>war</packaging>
<build>
<sourceDirectory>src</sourceDirectory>
<resources>
<resource>
<directory>src</directory>
<excludes>
<exclude>**/*.java</exclude>
</excludes>
</resource>
</resources>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.5</version>
<configuration>
<webXml>war/WEB-INF/web.xml</webXml>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.google.appengine.tools</groupId>
<artifactId>appengine-gcs-client</artifactId>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.appengine.tools</groupId>
<artifactId>appengine-gcs-client</artifactId>
<version>RELEASE</version>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.3.1</version>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Is there something I have to configure in the M2E settings? Do I need to use a maven terminal command? I am really at a loss for what to do other than manually manage all dependencies (please... no).

<build>
<outputDirectory>war/WEB-INF/classes</outputDirectory>
</build>
Add that to your pom. It will stop the output directory changing everytime you do a Maven Update.

Log4j2 custom plugins - annotation processing with Maven Assembly Plugin

I am not very familiar with Maven, I started using it just yesterday, but I like it. In my project I use Log4j2 library for logging and because of insufficiecy of advanced plugins (like appenders, converters) I need to use custom plugins. log4j-api and log4j-core (also with a bunch of other libraries) are added as dependencies in pom.xml associated with my project. Actually I am using version 2.0 of Log4j.
Log4j uses annotation processing to pre-load classes marked as #Plugin. As far as I know, in older releases of log4j, additional plugin entry had to be specified in pom.xml to trigger plugin processing, or the packages with custom plugins had to be typed into the packages attribute in the configuration file (https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax). But this is not supported since 2.0-rc2.
In v2.0 this should be done automatically, as long as log4j-core is available to the building engine. There is a file Log4j2Plugins.dat in myproject-0.0.1-SNAPSHOT.jar/META-INF/org/apache/logging/log4j/core/config/plugins/ that contains mappings of my custom plugins - that's OK.
For building with Maven I use also Maven Assembly Plugin. Its goal single is binded to package phase. After packaging the project I naturally have one additional jar in the target directory - myproject-0.0.1-SNAPSHOT-jar-with-dependencies.jar. However, Log4j2Plugins.dat file in this jar contains mappings of original plugins, the same file as in the log4j-core library. And that's the problem, since it doesn't hold any references to my custom plugins. It seems that the file from myproject-0.0.1-SNAPSHOT.jar is being overwritten with the original file from the log4j library, but I am not sure what's the case.
So when I run myproject-0.0.1-SNAPSHOT-jar-with-dependencies.jar, log4j can't find the plugin classes from my project. I think that myproject-0.0.1-SNAPSHOT.jar would run ok, but I can't run it without the dependencies.
The packages attribute in configuration should be re-enabled in 2.0.1 release, but if I don't want to wait for the release, I have to use the annotation processing method.
Do you have an idea how to fix it?
I tried to run it with release 2.0-rc1 of log4j, where the packages attribute of the configuration element was usable. The result is: log4j successfully loaded the class of my custom plugin. However, there were so many other errors (that arised in this specific release) that make the program even more unusable.
This is one good point, that ensures me that if the packages attribute will be enabled in next release 2.0.1, my plugin will work. It should be reinstated according to this issue tracking: https://issues.apache.org/jira/browse/LOG4J2-741
Added my pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.jjurm</groupId>
<artifactId>twbot</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>twbot</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.jjurm.twbot.system.Run</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.jjurm.twbot.system.Run</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
<version>1.1.7</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.3</version>
</dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
<dependency>
<groupId>net.snaq</groupId>
<artifactId>dbpool</artifactId>
<version>6.0</version>
</dependency>
<dependency>
<groupId>org.fusesource.jansi</groupId>
<artifactId>jansi</artifactId>
<version>1.11</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>commons-jxpath</groupId>
<artifactId>commons-jxpath</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit</artifactId>
<version>2.15</version>
</dependency>
</dependencies>
</project>

I think the problem stems from packaging the dependencies into the jar. Doing a quick dive into the code, it looks like the plugin processor overwrites the plugin dat file for each set of plugins it handles. My guess is that during the packaging process, your custom plugins are processed and written to the dat file, and then overwritten when your log4j dependency is processed for inclusion in the package. There may be better solutions, but off the top of my head I would suggest that you do one of the following:
Do not package the dependencies into your jar. Just package your project and then include the dependencies on your classpath when you execute. Even if you want to package everything in a single portable jar, doing this would allow you to at least confirm if your plugins are being overwritten, or if there is something else wrong.
Create a separate project for your custom plugins, package it separately from your main project, and then include the resulting jar as a dependency. As with option 1, make sure that you do not include the log4j jars in this package. Once you have created your custom plugin jar, you can package it along with the other dependencies in your main jar and it should work fine since your custom plugin jar will have its own plugin dat file.
Good luck!

Another solution is mentioned in Log4j 2 issue 673. Use maven shade plugin with specific transformer instead of maven assembly plugin.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<transformers>
<transformer implementation="com.github.edwgiz.mavenShadePlugin.log4j2CacheTransformer.PluginsCacheFileTransformer" />
</transformers>
</configuration>
<dependencies>
<dependency>
<groupId>com.github.edwgiz</groupId>
<artifactId>maven-shade-plugin.log4j2-cachefile-transformer</artifactId>
<version>2.6.1</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
If I understand it accurately, transformer creates Log4j2Plugins.dat file by correctly merging Log4j2Plugins.dat from all dependencies and the main jar, i.e. all plugins will be included.