Develop Reference

Maven running .JAR file claims missing dependencies

I have a Maven project which uses a few external dependencies, mainly Jackson.
When I run mvn clean -e install, it creates a jar file as specified in the target directory. However, when I run this jar file with java in command line I get the error message:
Error: Unable to initialize main class com.test.package.Main
Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonProcessingException
The specific element of Jackson which it's getting confused over does not matter, the error message will point out whatever portion of Jackson the code uses first. The strange part is when I run the project from within IntelliJ, it works fine. It seems that the JAR file does not have access to the proper dependencies.
This is the relevant portion of the pom.xml:
<dependencies>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>com.test.package.Main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.0</version>
<inherited>true</inherited>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
<compilerArgument>-Xlint:unchecked</compilerArgument>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
Is there something I should be doing for the built JAR file to run properly anywhere?

Solved:
This solution from another thread fixed my problem: https://stackoverflow.com/a/42231035/8402030
I had to add a plugin called Maven Shade which creates an uber jar containing all dependencies. Additionally had to remove the plugin from the tag.
Not entirely sure why it matters that the plugin is outside of . If anyone knows, feedback would be appreciated.

Running maven project in cmd

I'm have to run maven project in cmd. This is my pom.xml file:
http://maven.apache.org/xsd/maven-4.0.0.xsd">
4.0.0
<groupId>com.mycompany.app</groupId>
<artifactId>my-app</artifactId>
<version>1.0-SNAPSHOT</version>
<name>my-app</name>
<!-- FIXME change it to the project's website -->
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
<maven.compiler.release>11</maven.compiler.release>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>8.0.16</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.4.2.Final</version>
</dependency>
</dependencies>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
</plugin>
</plugins>
</pluginManagement>
</build>
The project is running perfectly fine in IDE (Eclipse), but when I try to run it run it from command line I get the following error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/hibernate/cfg/Configuration
at com.mycompany.app.App.main(App.java:15)
Caused by: java.lang.ClassNotFoundException: org.hibernate.cfg.Configuration
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 1 more

The problem in your case is that you dont have the dependencies in the classpath. You can use a maven pluging to obtain such.
<build>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>fully.qualified.MainClass</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Source How can I create an executable JAR with dependencies using Maven?

It really depends on how you setup your project, but what your stack trace is saying literally that JVM could not locate mentioned class. Now, you can imagine (this is just only one of many possible issues) - you have your code referencing 'Configuration' class somewhere, but when the project runs from a command line - JVM can not find it.
I would recommend taking a look at this topic to understand received error. Many thoughts were given and it was explained well.
Essentially once you get all your dependencies in place, that error will goaway and if you're using TestNG or JUnit, you won't need a main class. You will run your tests with just mvn clean test Or even shorter mvn if you add <defaultGoal>clean test</defaultGoal> to POM.

Maven Deploying Project as Jar - Missing Class definition

I am trying to create a runnable jar from a maven project created in Intellij Idea.
I tried building the Artifact through Intellij, but that did not work out. It could not find the main file.
After that I tried it through maven with:
- mvn compile
- mvn package
This creates a runnable jar which executes, but later when parsing a csv it throws an Exception:
Exception in thread "JavaFX Application Thread"
java.lang.NoClassDefFoundError: org/apache/commons/csv/CSVParser
But it is added in my pom.xml... I downloaded everything even the docs. I can see the org.apache.commons:commons-csv package in the external libraries, but it seems to be missing when creating the jar.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<packaging>jar</packaging>
<groupId>sceosa</groupId>
<artifactId>CEO_SA_ReportingLine</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<finalName>CEOSA</finalName>
<plugins>
<!-- download source code in Eclipse, best practice -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<version>2.9</version>
<configuration>
<downloadSources>true</downloadSources>
<downloadJavadocs>false</downloadJavadocs>
</configuration>
</plugin>
<!-- Set a compiler level -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<!-- Make this jar executable -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<archive>
<manifest>
<mainClass>de.Main</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<!-- https://mvnrepository.com/artifact/org.hsqldb/hsqldb -->
<!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-compiler-plugin -->
<dependency>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.google.guava/guava -->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>25.1-jre</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.0</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-csv</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Can anybody see what is wrong with the pom file or intellij?

By default maven does not include dependencies when building jars. The jar will only work if you have the dependency in some other way.
You can use the maven-assembly-plugin to build a jar-with-dependencies.

I have used the maven-shade-plugin and it works great if you want to bundle all your dependencies into one jar. All you need is the plugin and provide the Main class from your project and it should have a main method. It will look something like this:
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <executions>
        <execution>
            <goals>
                <goal>shade</goal>
            </goals>
            <configuration>
                <shadedArtifactAttached>true</shadedArtifactAttached>
                <transformers>
                    <transformer implementation=
                      "org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                        <mainClass>com.test.MainClass</mainClass>
                </transformer>
            </transformers>
        </configuration>
        </execution>
    </executions>
</plugin>
And just build your project using mvn clean install and you should find the the jar in your target directory and it will be suffixed with -shaded.jar. Hope this helps

Not adding JARs to staging folder on Google Cloud when using Maven

I'm running integration tests on the cloud for the Google Cloud Dataflows that I have written; checking that they read from Pub/Sub and write to BigQuery correctly, but when using Maven (mvn clean install), the staging folder is not populated with the required JARs. The only JAR that appears is a surefirebooter.jar. As a result, I get a NoClassDefFoundError for PipelineOptions (most likely because it is the first class from a dependency that's trying to be referenced) in the Stackdriver logs, and consequently the tests fail. Since they're running on the cloud I am indeed using a DataflowRunner as opposed to a DirectRunner.
When I run the integration tests from my IDE they work fine; the staging folder is populated with all the JARs and all is well. Also, when I run the tests using Maven but with a DirectRunner the tests run successfully, thus my problem only occurs when using Maven and a DataflowRunner. I assume that problem therefore lies with the pom.xml file, which I have given below:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>group</groupId>
<artifactId>artifact</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.0.0-beta3</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.0.0-M3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>2.0.2-beta</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Anyone know why this is happening and how I may resolve it?

When staging files, the Dataflow runner will automatically stage the the classes available to the current class loader. I believe that surefire plays some tricks with the classloader to make the tests easier to run.
One option would be to specify filesToStage on the pipeline options, which will override the normal "detect JARs to stage from the class loader". Alternatively, look at how surefire is managing the classpath, and make sure the SDK JARs are available in the classloader the test is running in.

Log4j2 custom plugins - annotation processing with Maven Assembly Plugin

I am not very familiar with Maven, I started using it just yesterday, but I like it. In my project I use Log4j2 library for logging and because of insufficiecy of advanced plugins (like appenders, converters) I need to use custom plugins. log4j-api and log4j-core (also with a bunch of other libraries) are added as dependencies in pom.xml associated with my project. Actually I am using version 2.0 of Log4j.
Log4j uses annotation processing to pre-load classes marked as #Plugin. As far as I know, in older releases of log4j, additional plugin entry had to be specified in pom.xml to trigger plugin processing, or the packages with custom plugins had to be typed into the packages attribute in the configuration file (https://logging.apache.org/log4j/2.x/manual/configuration.html#ConfigurationSyntax). But this is not supported since 2.0-rc2.
In v2.0 this should be done automatically, as long as log4j-core is available to the building engine. There is a file Log4j2Plugins.dat in myproject-0.0.1-SNAPSHOT.jar/META-INF/org/apache/logging/log4j/core/config/plugins/ that contains mappings of my custom plugins - that's OK.
For building with Maven I use also Maven Assembly Plugin. Its goal single is binded to package phase. After packaging the project I naturally have one additional jar in the target directory - myproject-0.0.1-SNAPSHOT-jar-with-dependencies.jar. However, Log4j2Plugins.dat file in this jar contains mappings of original plugins, the same file as in the log4j-core library. And that's the problem, since it doesn't hold any references to my custom plugins. It seems that the file from myproject-0.0.1-SNAPSHOT.jar is being overwritten with the original file from the log4j library, but I am not sure what's the case.
So when I run myproject-0.0.1-SNAPSHOT-jar-with-dependencies.jar, log4j can't find the plugin classes from my project. I think that myproject-0.0.1-SNAPSHOT.jar would run ok, but I can't run it without the dependencies.
The packages attribute in configuration should be re-enabled in 2.0.1 release, but if I don't want to wait for the release, I have to use the annotation processing method.
Do you have an idea how to fix it?
I tried to run it with release 2.0-rc1 of log4j, where the packages attribute of the configuration element was usable. The result is: log4j successfully loaded the class of my custom plugin. However, there were so many other errors (that arised in this specific release) that make the program even more unusable.
This is one good point, that ensures me that if the packages attribute will be enabled in next release 2.0.1, my plugin will work. It should be reinstated according to this issue tracking: https://issues.apache.org/jira/browse/LOG4J2-741
Added my pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.jjurm</groupId>
<artifactId>twbot</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>twbot</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.4</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.jjurm.twbot.system.Run</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<id>make-assembly</id> <!-- this is used for inheritance merges -->
<phase>package</phase> <!-- bind to the packaging phase -->
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>com.jjurm.twbot.system.Run</mainClass>
</manifest>
</archive>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
<version>1.1.7</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.3</version>
</dependency>
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
<dependency>
<groupId>net.snaq</groupId>
<artifactId>dbpool</artifactId>
<version>6.0</version>
</dependency>
<dependency>
<groupId>org.fusesource.jansi</groupId>
<artifactId>jansi</artifactId>
<version>1.11</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.7</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>commons-jxpath</groupId>
<artifactId>commons-jxpath</artifactId>
<version>1.3</version>
</dependency>
<dependency>
<groupId>net.sourceforge.htmlunit</groupId>
<artifactId>htmlunit</artifactId>
<version>2.15</version>
</dependency>
</dependencies>
</project>

I think the problem stems from packaging the dependencies into the jar. Doing a quick dive into the code, it looks like the plugin processor overwrites the plugin dat file for each set of plugins it handles. My guess is that during the packaging process, your custom plugins are processed and written to the dat file, and then overwritten when your log4j dependency is processed for inclusion in the package. There may be better solutions, but off the top of my head I would suggest that you do one of the following:
Do not package the dependencies into your jar. Just package your project and then include the dependencies on your classpath when you execute. Even if you want to package everything in a single portable jar, doing this would allow you to at least confirm if your plugins are being overwritten, or if there is something else wrong.
Create a separate project for your custom plugins, package it separately from your main project, and then include the resulting jar as a dependency. As with option 1, make sure that you do not include the log4j jars in this package. Once you have created your custom plugin jar, you can package it along with the other dependencies in your main jar and it should work fine since your custom plugin jar will have its own plugin dat file.
Good luck!

Another solution is mentioned in Log4j 2 issue 673. Use maven shade plugin with specific transformer instead of maven assembly plugin.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.3</version>
<configuration>
<transformers>
<transformer implementation="com.github.edwgiz.mavenShadePlugin.log4j2CacheTransformer.PluginsCacheFileTransformer" />
</transformers>
</configuration>
<dependencies>
<dependency>
<groupId>com.github.edwgiz</groupId>
<artifactId>maven-shade-plugin.log4j2-cachefile-transformer</artifactId>
<version>2.6.1</version>
</dependency>
</dependencies>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
If I understand it accurately, transformer creates Log4j2Plugins.dat file by correctly merging Log4j2Plugins.dat from all dependencies and the main jar, i.e. all plugins will be included.