Develop Reference

Setting up JVM debugging in Maven project

I'm trying to debug my application that's running on a remote JVM, but I can't manage to get my dependencies there too.
Here are a few things I tried:
Remote debugger:
I've set up a remote debugger, and it corrects correctly to the JVM but when I then use my application I get a bunch of NoClassDefFoundError errors coming from external libraries that are in my maven dependencies (which makes sense because I building the app with the Intellij building tools and not Maven itself. I have some relocation rules set up so maybe that's the issue?
The JVM itself is running on my local machine.
Maven debugger
There is a maven debugger integrated inside Intellij but it seems like that is just using the remote JVM during the jar packaging but nothing else. Maybe I'm going something wrong there?
I also have a few yaml files in my resources folder that should be correctly included in the debugger.
Here is my project structure:
Here is my POM file for reference:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>me.playbosswar.com</groupId>
<artifactId>commandtimer</artifactId>
<name>CommandTimer</name>
<version>6.0</version>
<description>Plugin</description>
<build>
<finalName>${project.name}_v${project.version}</finalName>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.3.1</version>
<configuration>
<outputDirectory>/Users/tristan/Documents/minecraft/servers/1.16.5/plugins</outputDirectory>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.4</version>
<configuration>
<relocations>
<relocation>
<pattern>com.cryptomorin.xseries</pattern>
<shadedPattern>me.playbosswar.com</shadedPattern>
</relocation>
</relocations>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
</build>
<repositories>
<repository>
<id>papermc</id>
<url>https://papermc.io/repo/repository/maven-public/</url>
</repository>
<repository>
<id>placeholderapi</id>
<url>http://repo.extendedclip.com/content/repositories/placeholderapi/</url>
</repository>
<repository>
<id>codemc-snapshots</id>
<url>https://repo.codemc.io/repository/maven-snapshots/</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>com.destroystokyo.paper</groupId>
<artifactId>paper-api</artifactId>
<version>1.16.5-R0.1-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>me.clip</groupId>
<artifactId>placeholderapi</artifactId>
<version>2.10.9</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<dependency>
<groupId>com.github.cryptomorin</groupId>
<artifactId>XSeries</artifactId>
<version>7.2.1.2</version>
</dependency>
<dependency>
<groupId>fr.minuskube.inv</groupId>
<artifactId>smart-invs</artifactId>
<version>1.2.7</version>
</dependency>
<dependency>
<groupId>net.wesjd</groupId>
<artifactId>anvilgui</artifactId>
<version>1.5.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.8.1</version>
</dependency>
</dependencies>
</project>
Here is the actual error:

Spark fails with NoClassDefFoundError for org.apache.kafka.common.serialization.StringDeserializer

I am developing a generic Spark application that listens to a Kafka stream using Spark and Java.
I am using kafka_2.11-0.10.2.2, spark-2.3.2-bin-hadoop2.7 - I also tried several other kafka/spark combinations before posting this question.
The code fails at loading StringDeserializer class:
SparkConf sparkConf = new SparkConf().setAppName("JavaDirectKafkaWordCount");
JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, Durations.seconds(2));
Set<String> topicsSet = new HashSet<>();
topicsSet.add(topics);
Map<String, Object> kafkaParams = new HashMap<>();
kafkaParams.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, brokers);
kafkaParams.put(ConsumerConfig.GROUP_ID_CONFIG, groupId);
kafkaParams.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
kafkaParams.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
The error I get is:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/common/serialization/StringDeserializer
From Why does Spark application fail with "Exception in thread "main" java.lang.NoClassDefFoundError: ...StringDeserializer"? it seems that this could be a scala version mismatch issue, but my pom.xml doesn't have that issue:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>yyy.iot.ckc</groupId>
<artifactId>sparkpoc</artifactId>
<version>1.0-SNAPSHOT</version>
<name>sparkpoc</name>
<!-- FIXME change it to the project's website -->
<url>http://www.example.com</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<java.version>1.8</java.version>
<spark.scala.version>2.11</spark.scala.version>
<spark.version>2.3.2</spark.version>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${spark.scala.version}</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_${spark.scala.version}</artifactId>
<version>${spark.version}</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-10_${spark.scala.version}</artifactId>
<version>${spark.version}</version>
</dependency>
</dependencies>
<build>
<pluginManagement><!-- lock down plugins versions to avoid using Maven defaults (may be moved to parent pom) -->
<plugins>
<!-- clean lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#clean_Lifecycle -->
<plugin>
<artifactId>maven-clean-plugin</artifactId>
<version>3.1.0</version>
</plugin>
<!-- default lifecycle, jar packaging: see https://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_jar_packaging -->
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<version>3.0.2</version>
</plugin>
<plugin>
<artifactId>maven-install-plugin</artifactId>
<version>2.5.2</version>
</plugin>
<plugin>
<artifactId>maven-deploy-plugin</artifactId>
<version>2.8.2</version>
</plugin>
<!-- site lifecycle, see https://maven.apache.org/ref/current/maven-core/lifecycles.html#site_Lifecycle -->
<plugin>
<artifactId>maven-site-plugin</artifactId>
<version>3.7.1</version>
</plugin>
<plugin>
<artifactId>maven-project-info-reports-plugin</artifactId>
<version>3.0.0</version>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
The submission script I use is:
./bin/spark-submit \
--class "yyy.iot.ckc.KafkaDataModeler" \
--master local[2] \
../sparkpoc/target/sparkpoc-1.0-SNAPSHOT.jar
Can anyone please point me in the right direction as to where I am going wrong?

Spark runs the program as by running an instance of a JVM. So if the libraries (JARs) are not in the classpath of that JVM we run into this runtime exception. The solution is to package all the dependent JARs along with main JAR. The following build script will work for that.
Also, as mentioned in https://stackoverflow.com/a/54583941/1224075 the scope of the spark-core and spark-streaming libraries need to be declared as provided. This is because some of the libraries are implicitly provided by the Spark JVM.
The build section of the POM which worked for me -
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2.1</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

You need to use the Maven Shade Plugin to package the Kafka clients along with your Spark application, then you can submit the shaded Jar, and the Kafka serializers should be found on the classpath.
Also, make sure you set the provided Spark packages
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${spark.scala.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_${spark.scala.version}</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>

Allure reporting does not generate with mvn test -Darguments

Problem: I am using allure reporting to generate some reports after running some tests, when I run the tests manually from within eclipse the allure-results directory is successfully created inside the /target/allure-results directory.
However, when I pass in some arguments after calling maven from the command line, with something like this:
call mvn clean
call mvn test -Dbrowser=Chrome -DseleniumEnvironment=local -Dreporttogenerate=censoredconfigsetting -Dcucumber.options="--tags #censoredTag
Note: mvn clean test does generate allure-reports from the command line, why are my arguments breaking this?
This instead, is creating a /cucumber/results/ directory in /target, I am very confused as to why this differs, doesn't seem like these tags should have much problems there, I suspect its possibly my pom.xml causing issues.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.kore</groupId>
<artifactId>kore-automation</artifactId>
<version>0.0.1</version>
<packaging>jar</packaging>
<name>kore-automation</name>
<properties>
<maven.build.timestamp.format>yyyy-MM-dd</maven.build.timestamp.format>
<build.number>${maven.build.timestamp}/${platform}/${platformVersion}/${browser}/${browserVersion}</build.number>
<reports.directory>${project.build.directory}/../reports/${build.number}</reports.directory>
<cucumber.tags>~#ignore</cucumber.tags>
<!-- Needed for allure-maven-plugin to specify report version -->
<aspectj.version>1.8.4</aspectj.version>
</properties>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>3.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.8.0-alpha2</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12 -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.8.0-alpha2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-junit</artifactId>
<version>2.0.0.0</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-server</artifactId>
<version>3.0.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>ru.yandex.qatools.allure</groupId>
<artifactId>allure-cucumber-jvm-adaptor</artifactId>
<version>1.6.3</version>
</dependency>
<dependency>
<groupId>info.cukes</groupId>
<artifactId>cucumber-java</artifactId>
<version>1.2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>16.0.1</version>
</dependency>
<dependency>
<groupId>net.masterthought</groupId>
<artifactId>cucumber-reporting</artifactId>
<version>3.7.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>6.1.0.jre8</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-core-asl</artifactId>
<version>1.9.13</version>
</dependency>
<dependency>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version>2.1.0</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>read-properties</id>
<activation>
<file>
<exists>src/test/resources/config.properties</exists>
</file>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>src\test\resources\config.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<build>
<plugins>
<plugin>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<version>4.1.0</version>
<executions>
<execution>
<id>generateRunners</id>
<phase>generate-test-sources</phase>
<goals>
<goal>generateRunners</goal>
</goals>
<configuration>
<!-- Mandatory -->
<!-- comma separated list of package names to scan for glue code -->
<glue>
<package>webDriver</package>
<package>censored</package>
<package>censored</package>
<package>testRunner</package>
</glue>
<outputDirectory>${project.build.directory}/generated-test-sources/cucumber</outputDirectory>
<!-- The directory, which must be in the root of the runtime classpath, containing your feature files. -->
<featuresDirectory>src/test/java/</featuresDirectory>
<!-- Directory where the cucumber report files shall be written -->
<!-- cucumberOutputDir>target/cucumber-parallel</cucumberOutputDir-->
<!-- comma separated list of output formats json,html,rerun.txt -->
<!-- format>json</format-->
<!-- CucumberOptions.strict property -->
<!-- strict>true</strict>-->
<!-- CucumberOptions.monochrome property -->
<monochrome>true</monochrome>
<!-- The tags to run, maps to CucumberOptions.tags property you can pass ANDed tags like "#tag1","#tag2" and ORed tags like "#tag1,#tag2,#tag3" -->
<tags />
<!-- If set to true, only feature files containing the required tags shall be generated. -->
<filterFeaturesByTags>false</filterFeaturesByTags>
<!-- Generate TestNG runners instead of default JUnit ones. -->
<useTestNG>false</useTestNG>
<!-- The naming scheme to use for the generated test classes. One of 'simple' or 'feature-title' -->
<namingScheme>simple</namingScheme>
<!-- The class naming pattern to use. Only required/used if naming scheme is 'pattern'.-->
<namingPattern>Parallel{c}IT</namingPattern>
<!-- One of [SCENARIO, FEATURE]. SCENARIO generates one runner per scenario. FEATURE generates a runner per feature. -->
<parallelScheme>FEATURE</parallelScheme>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.2</version>
<configuration>
<encoding>UTF-8</encoding>
<source>1.7</source>
<target>1.7</target>
<compilerArgument>-Werror</compilerArgument>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>2.18.1</version>
</plugin>
<plugin>
<groupId>ru.yandex.qatools.allure</groupId>
<artifactId>allure-maven-plugin</artifactId>
<version>2.6</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<testFailureIgnore>false</testFailureIgnore>
<argLine>-javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
-Dcucumber.options="--plugin ru.yandex.qatools.allure.cucumberjvm.AllureReporter"</argLine>
<properties>
<property>
<name>listener2</name>
<value>/core-automation/src/test/java/testRunner/CustomAllureListener</value>
</property>
</properties>
<forkCount>10</forkCount>
<reuseForks>true</reuseForks>
<includes>
<include>**/*IT.class</include>
</includes>
</configuration>
<dependencies>
<dependency>
<groupId>org.aspectj</groupId>
<artifactId>aspectjweaver</artifactId>
<version>${aspectj.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
<pluginManagement>
<plugins>
<!--This plugin's configuration is used to store Eclipse m2e settings only. It has no influence on the Maven build itself.-->
<plugin>
<groupId>org.eclipse.m2e</groupId>
<artifactId>lifecycle-mapping</artifactId>
<version>1.0.0</version>
<configuration>
<lifecycleMappingMetadata>
<pluginExecutions>
<pluginExecution>
<pluginExecutionFilter>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<versionRange>[1.0-alpha-2,)</versionRange>
<goals>
<goal>read-project-properties</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore />
</action>
</pluginExecution>
<pluginExecution>
<pluginExecutionFilter>
<groupId>com.github.temyers</groupId>
<artifactId>cucumber-jvm-parallel-plugin</artifactId>
<versionRange>[4.1.0,)</versionRange>
<goals>
<goal>generateRunners</goal>
</goals>
</pluginExecutionFilter>
<action>
<ignore />
</action>
</pluginExecution>
</pluginExecutions>
</lifecycleMappingMetadata>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>
</project>
I noticed that when generate parallel .IT files to run my tests in parallel, they each reference:
plugin = {"json:C:/Users/sy/git/censor/censor/target/cucumber-parallel/1.json"}
Still a tad confused with that because when running inside eclipse -> Run as: Maven Build... -> clean test I do not have these json files created, only when passing in command like mvn args do I get them
Looking more closely, its specifically when I pass in -Dcucumber.Options for tags in the mvn call, seems to be overwriting the -Dcucumber.options="--plugin ru.yandex.qatools.allure.cucumberjvm.AllureReporter" that is inside my surefire plugin in the pom.xml
Thanks for your time, any help is greatly appreciated.

After a long morning I've figured out the problem, it was simply my passing of -Dcucumber.options with tags was overwriting the maven surefire argLine for the allurereporter.
Fixed by running with the following:
call mvn clean test -Dbrowser=Chrome -DseleniumEnvironment=local -Dcucumber.Options="--plugin ru.yandex.qatools.allure.cucumberjvm.AllureReporter --tags #censoredtag"
call mvn allure:report
and removing the -Dcucumber.Options argLine from my maven surefire plugin.

Not adding JARs to staging folder on Google Cloud when using Maven

I'm running integration tests on the cloud for the Google Cloud Dataflows that I have written; checking that they read from Pub/Sub and write to BigQuery correctly, but when using Maven (mvn clean install), the staging folder is not populated with the required JARs. The only JAR that appears is a surefirebooter.jar. As a result, I get a NoClassDefFoundError for PipelineOptions (most likely because it is the first class from a dependency that's trying to be referenced) in the Stackdriver logs, and consequently the tests fail. Since they're running on the cloud I am indeed using a DataflowRunner as opposed to a DirectRunner.
When I run the integration tests from my IDE they work fine; the staging folder is populated with all the JARs and all is well. Also, when I run the tests using Maven but with a DirectRunner the tests run successfully, thus my problem only occurs when using Maven and a DataflowRunner. I assume that problem therefore lies with the pom.xml file, which I have given below:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>group</groupId>
<artifactId>artifact</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>com.google.cloud.dataflow</groupId>
<artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
<version>2.0.0-beta3</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.0.0-M3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>2.0.2-beta</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Anyone know why this is happening and how I may resolve it?

When staging files, the Dataflow runner will automatically stage the the classes available to the current class loader. I believe that surefire plays some tricks with the classloader to make the tests easier to run.
One option would be to specify filesToStage on the pipeline options, which will override the normal "detect JARs to stage from the class loader". Alternatively, look at how surefire is managing the classpath, and make sure the SDK JARs are available in the classloader the test is running in.

Maven running tests annotated with junit #Ignore when building Java project

I've annotated a bad test with #Ignore, both on the method-level and on the class level. When running the test through the command line (I've tried "mvn clean install", "mvn test", "mvn clean install -DskipTests; mvn test"), however, the #Ignore annotation is ignored, the test is run, and- as it is a bad test- it fails.
Here is the test:
public class UserTest extends PersistentTestBase {
#Test
#Ignore
public void testPersistence() {
...
}
}
And here is my pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
...
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
...
<!-- Testing -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jmock</groupId>
<artifactId>jmock-junit4</artifactId>
<version>2.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jmock</groupId>
<artifactId>jmock-legacy</artifactId>
<version>2.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.easymock</groupId>
<artifactId>easymock</artifactId>
<version>3.4</version>
<scope>test</scope>
</dependency>
...
<!-- For dep management, see Mykong.com's "How to create a jar file with Maven" -->
<build>
<plugins>
<!-- Necessary to force language level of Java 8 -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.2</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.18.1</version>
<configuration>
<junitArtifactName>junit:junit-dep</junitArtifactName>
<useFile>false</useFile>
<trimStackTrace>false</trimStackTrace>
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
</configuration>
</plugin>
<!-- Packages into a jar, looking for deps in target/dependency-jars -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.2</version>
<configuration>
<excludes>
<exclude>**/log4j.properties</exclude>
</excludes>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>...Application</mainClass>
<classpathPrefix>dependency-jars</classpathPrefix>
</manifest>
</archive>
</configuration>
</plugin>
<!-- Moves all compiled deps to target/dependency-jars -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.5.1</version>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<includeScope>runtime</includeScope>
<outputDirectory>${project.build.directory}/dependency-jars</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<configuration>
<mainClass>...Application</mainClass>
</configuration>
</plugin>
</plugins>
</build>
Worth noting that the test is skipping when the class is run in Intellij IDEA 14, and that there are some other issues with tests failing on mvn test that are passing in Intellij (but are out of the scope of this question). Thank you for the help!

For those who are working with Junit 5.
#Ignore did not work formaven surefire neither Intellij tests even if I had already included junit-vintage-engine
I used #Disabled and it worked for maven at least!

Wemu posted the solution:
I think you can simply remove:
junit:junit-dep from the
surefire config – wemu Nov 20 at 7:10
Removed the element and all tests passed!

I had a same problem with the JUNIT library. After so many tries, it worked with the Pom.xml below:
<repositories>
<repository>
<id>junit</id>
<url>http://junit.org</url>
</repository>
</repositories>
<dependencies>
<dependency><groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version></dependency>
</dependencies>

For me the solution was to simply update the JUnit version to 4.13:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>
<scope>test</scope>
</dependency>