Develop Reference

How to add a scala dependency to a Java project in intelliJ that use maven

I want to use a scala dependency in a pure Java project. There is no need to write scala in my project but of course I will have to use classes/etc defined in the scala project that I add as a dependency. Want to do this in a intelliJ maven project.
My pom looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.xxx</groupId>
<artifactId>xxxx</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>group ID</groupId>
<artifactId>scala dependancy</artifactId>
</dependency>
</dependencies>
</project>
Although I dont see any errors in pom, intellij is unable to import packages defined in the scala library. Essentially it cant find them.
What am I doing wrong here?

You can do so by adding the following dependency to your pom.xml file:
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.11.7</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- This plugin compiles Scala files -->
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- This plugin compiles Java files -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<!-- This plugin adds all dependencies to JAR file during 'package' command.
Pay EXTRA attention to the 'mainClass' tag.
You have to set name of class with entry point to program ('main' method) -->
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.5.3</version>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>ScalaRunner</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
Take a look at this for more information.

Maven can't retrieve buildNumber while package the jar

I am using a buildnumber-maven-plugin to generate a sequence number for my jar. I will use it during my CICD process.
I followed all the examples available online. However, I can get the ${buildNumber} to print it, but while packaging the jar, I can't get the number, and I got the ${buildNumber} text. I searched for Maven LifeCycle, and I found I need to add it in the validation before any other plugin. But still, I can't solve the issue.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>${group.id}</groupId>
<artifactId>${artifact.id}</artifactId>
<version>${model.version}</version>
<profiles>
<profile>
<id>scala-2.11</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<dependencies>
......
</dependencies>
</profile>
</profiles>
<scm>
<connection>scm:svn:http://127.0.0.1/dummy</connection>
<developerConnection>scm:svn:https://127.0.0.1/dummy</developerConnection>
<tag>HEAD</tag>
<url>http://127.0.0.1/dummy</url>
</scm>
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<finalName>${artifact.id}.${model.version}</finalName>
<plugins>
<!-- buildnumber-maven-plugin for automatic increment the version number -->
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>buildnumber-maven-plugin</artifactId>
<version>1.4</version>
<configuration>
<buildNumberPropertiesFileLocation>${build.number.dir}</buildNumberPropertiesFileLocation>
<revisionOnScmFailure>no.scm.config.in.pom</revisionOnScmFailure>
<doCheck>true</doCheck>
<doUpdate>true</doUpdate>
<format>{0,number}</format>
<items>
<item>buildNumber</item>
</items>
</configuration>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>create</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>${build.number.dir}</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.6</version>
<executions>
<execution>
<phase>validate</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<target>
<echo>current build number is "${buildNumber}"</echo>
</target>
</configuration>
</execution>
</executions>
</plugin>
<!-- ================== ;-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${java.source.version}</source>
<target>${java.source.version}</target>
<skipMain>true</skipMain> <!-- skip compile -->
<skip>true</skip> <!-- skip testCompile -->
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>${maven.assembly.plugin.version}</version>
<executions>
<execution>
<id>jar-with-dependencies</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<descriptors>
<descriptor>
src/main/assembly/assembly-jar.xml
</descriptor>
</descriptors>
<finalName>${artifact.id}</finalName>
</configuration>
</execution>
<execution>
<id>bin</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<appendAssemblyId>false</appendAssemblyId>
<descriptors>
<descriptor>
src/main/assembly/assembly-bin.xml
</descriptor>
</descriptors>
<finalName>${tar.name}</finalName>
</configuration>
</execution>
</executions>
</plugin>
<properties>
<argLine>-Dfile.encoding=UTF-8 -Dlog4j.skipJansi=false -DmodelLogLevel=${modelLogLevel}</argLine>
<modelLogLevel>DEBUG</modelLogLevel>
<build.number.dir>${project.basedir}/buildNumber.properties</build.number.dir>
<java.source.version>1.8</java.source.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<!-- Plugin Versions -->
<maven.compiler.plugin.version>3.6.2</maven.compiler.plugin.version>
<maven.reports.plugin.version>2.9</maven.reports.plugin.version>
<maven.assembly.plugin.version>3.1.0</maven.assembly.plugin.version>
<surefire.plugin.version>2.7</surefire.plugin.version>
<group.id>com.test.model_name</group.id>
<artifact.id>proj_abc</artifact.id>
<build.type>snapshot</build.type>
<major.minor.version>0.1</major.minor.version>
<!-- here I am trying to retrieve the actual number-->
<build.number>${buildNumber}</build.number>
<model.version>${major.minor.version}.${build.number}.${build.type}</model.version>
</properties>
</project>
Note: If I run validate I could found the echo message below which means I can get the number correctly from the properties file. If I run package or run I got ${artifact.id}_source-.0.1.${buildNumber}.snapshot without the number.
main:
[echo] current build number is "90"
I also checked this similar question here & enter link description here and other links but can't figure out the problem.

I think what's happening here is that Maven resolves properties near the beginning of the build process.
<build.number>${buildNumber}</build.number>
<model.version>${major.minor.version}.${build.number}.${build.type}</model.version>
When it resolves build.number, ${buildNumber} does not have a value. So, it leaves the variable name unchanged. And I suspect you'd have the same problem if you modified model.version to this:
<model.version>${major.minor.version}.${buildNumber}.${build.type}</model.version>
for the same reason.
You can try using the build-helper-maven-plugin as described in this answer. However, this still may not work since model.version is used as the value of the project.version, which is resolved quite early as it's part of the GAV coordinates. It's worth a try though.

Why is my Maven project packaging test files?

I have a Maven project which I'm trying to package, but I've noticed that all my Java test classes (but none of my Scala test classes) and generated Avro test classes are ending up in the jar.
Folder structure looks fine:
I also noticed that if I add junit as a dependency with <scope>test</scope>, my tests won't compile as it can't find the junit classes, so it looks like Maven is treating all my code including tests as being part of main.
Pom:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.me</groupId>
<artifactId>proj</artifactId>
<version>1.0.0</version>
<properties>
<log4j.version>2.8.1</log4j.version>
</properties>
<dependencies>
...
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.scalatest</groupId>
<artifactId>scalatest-maven-plugin</artifactId>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.2.1</version>
<executions>
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>add-source</goal>
<goal>compile</goal>
</goals>
</execution>
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
<configuration>
<scalaVersion>2.11.8</scalaVersion>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.avro</groupId>
<artifactId>avro-maven-plugin</artifactId>
<version>1.8.1</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>schema</goal>
</goals>
<configuration>
<sourceDirectory>${project.basedir}/src/test/resources/</sourceDirectory>
<outputDirectory>${project.basedir}/src/test/java</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.6.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
<executions>
<execution>
<id>attach-sources</id>
<goals>
<goal>jar</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

Figured it out.
The issue was with the avro-maven-plugin.
It's configuration had sourceDirectory and outputDirectory, both of which had test source paths.
<configuration>
<sourceDirectory>${project.basedir}/src/test/resources/</sourceDirectory>
<outputDirectory>${project.basedir}/src/test/java</outputDirectory>
</configuration>
Apparently these were interfering with the compiler plugin which thought that these directories were main source directories and compiled them along with the rest of the main classes. This also explains why my tests were failing to compile when the junit dependency was given a test scope.
The solution was to use testSourceDirectory and testOutputDirectory instead:
<configuration>
<testSourceDirectory>${project.basedir}/src/test/resources/</testSourceDirectory>
<testOutputDirectory>${project.basedir}/src/test/java</testOutputDirectory>
</configuration>

mvn clean install -Dmaven.test.skip=true -X

Error reading assemblies in Maven building

Here is my pom.xml file and the error i got when building
Error:
Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single (default-cli) on project load-xml-to-s3: Error reading assemblies: No assembly descriptors found.
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.tte.s3.load</groupId>
<artifactId>load-xml-to-s3</artifactId>
<version>1.0</version>
<name>load-xml-to-s3</name>
<build>
<plugins>
<!-- TOBE USED LATER - DO NOT DELETE. - KC <plugin> <groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId> <version>1.5.3</version> <executions> <execution>
<phase>prepare-package</phase> <goals> <goal>replace</goal> </goals> </execution>
</executions> <configuration> <file>target/classes/somefile.txt</file> <replacements>
<replacement> <token>SOME TOKEN</token> <value>SOME VALUE</value> </replacement>
</replacements> </configuration> </plugin> -->
<plugin>
<groupId>com.jolira</groupId>
<artifactId>onejar-maven-plugin</artifactId>
<version>1.4.4</version>
<executions>
<execution>
<configuration>
<mainClass>com.tte.s3.load.driver.Driver</mainClass>
</configuration>
<goals>
<goal>one-jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>make-assembly</id>
<phase>assembly</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>distribution.xml</descriptor>
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>

The descriptors attribute of the maven-assembly-plugin expects a path starting from the base directory of your project. For example, if the file is located inside src/assembly/distribution.xml, this is the path you should specify.
There is also a problem in the phase you bound the plugin to. Phase assembly does not exist, you should use <phase>package</phase> instead.
As a side-node, you are using an old version of the plugin (2.2-beta-5). Consider using the latest version instead, which is 2.6.
Sample configuration:
<plugin>
<!-- no need to specify the groupId, it defaults to "org.apache.maven.plugins" -->
<artifactId>maven-assembly-plugin</artifactId>
<version>2.6</version> <!-- explicit version is safer for build consistency than implicit -->
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase> <!-- bind the execution to the "package" phase -->
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/assembly/distribution.xml</descriptor> <!-- use the path to the file starting from base directory -->
</descriptors>
</configuration>
</execution>
</executions>
</plugin>
Running Maven with mvn clean install will invoke the plugin correctly in the package phase.

Scala Eclipse IDE compiler giving errors until "clean" is run

I am writing a mixed Java/Scala project and my Eclipse compiler is giving errors for all of the Scala code. It seems to try to compile it as Java instead of Scala. However, when I run a clean, all of the errors go away. Also, building this using maven works completely fine.
I am building this project in Maven. Originally this project was only Java, so I added the necessary plugins/dependencies into my pom.xml.
Any idea why Eclipse is treating my Scala files as Java?
Here's my pom.xml:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.antlrfun</groupId>
<artifactId>antlrfun</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>Antlr4 Spark/Solr</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<antlr4.visitor>true</antlr4.visitor>
<antlr4.listener>true</antlr4.listener>
</properties>
<dependencies>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
<version>4.5</version>
</dependency>
<dependency>
<groupId>junit</groupId> <!-- NOT org.junit here -->
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.10.5</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.10</artifactId>
<version>1.3.1</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<goals>
<goal>java</goal>
</goals>
</execution>
</executions>
<configuration>
<mainClass>org.antlrfun.Hello</mainClass>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>1.7</version>
<executions>
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/generated-sources</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
<showWarnings>true</showWarnings>
<showDeprecation>true</showDeprecation>
<compilerArguments>
<Xlint/>
</compilerArguments>
</configuration>
</plugin>
<plugin>
<groupId>org.antlr</groupId>
<artifactId>antlr4-maven-plugin</artifactId>
<version>4.1</version>
<configuration>
<sourceDirectory>src/main/java</sourceDirectory>
<outputDirectory>src/main/generated-sources</outputDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>antlr4</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.scala-tools</groupId>
<artifactId>maven-scala-plugin</artifactId>
<version>2.15.2</version>
<executions>
<!-- Run scala compiler in the process-resources phase, so that dependencies on
scala classes can be resolved later in the (Java) compile phase -->
<execution>
<id>scala-compile-first</id>
<phase>process-resources</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
<!-- Run scala compiler in the process-test-resources phase, so that dependencies on
scala classes can be resolved later in the (Java) test-compile phase -->
<execution>
<id>scala-test-compile</id>
<phase>process-test-resources</phase>
<goals>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<executions>
<!-- Add src/main/scala to source path of Eclipse -->
<execution>
<id>add-source</id>
<phase>generate-sources</phase>
<goals>
<goal>add-source</goal>
</goals>
<configuration>
<sources>
<source>src/main/scala</source>
</sources>
</configuration>
</execution>
<!-- Add src/test/scala to test source path of Eclipse -->
<execution>
<id>add-test-source</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>src/test/scala</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
<!-- to generate Eclipse artifacts for projects mixing Scala and Java -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-eclipse-plugin</artifactId>
<version>2.8</version>
<configuration>
<downloadSources>true</downloadSources>
<downloadJavadocs>true</downloadJavadocs>
<projectnatures>
<projectnature>org.scala-ide.sdt.core.scalanature</projectnature>
<projectnature>org.eclipse.jdt.core.javanature</projectnature>
</projectnatures>
<buildcommands>
<buildcommand>org.scala-ide.sdt.core.scalabuilder</buildcommand>
</buildcommands>
<classpathContainers>
<classpathContainer>org.scala-ide.sdt.launching.SCALA_CONTAINER</classpathContainer>
<classpathContainer>org.eclipse.jdt.launching.JRE_CONTAINER</classpathContainer>
</classpathContainers>
<excludes>
<!-- in Eclipse, use scala-library, scala-compiler from the SCALA_CONTAINER rather than POM <dependency> -->
<exclude>org.scala-lang:scala-library</exclude>
<exclude>org.scala-lang:scala-compiler</exclude>
</excludes>
<sourceIncludes>
<sourceInclude>**/*.scala</sourceInclude>
<sourceInclude>**/*.java</sourceInclude>
</sourceIncludes>
</configuration>
</plugin>
<!-- When run tests in the test phase, include .java and .scala source files -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.8.1</version>
<configuration>
<includes>
<include>**/*.java</include>
<include>**/*.scala</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</project>

Clear the Scala IDE project settings by removing and adding back Scala Nature