I created a project that includes multiple Java projects, each one has its own dependencies.
One project is the main project and uses all other projects, but not directly. For example :
Assume that I have three project's A, B and C, A is the main one (the application). A uses B as dependency while B uses C as dependency, project C has its own external dependencies.
Each person of my team has a permission to download only one project, and will use the others as dependencies as needed.
The problem appears on the team that developing main project, Maven did not download B & C dependencies to their local machine.
These are the pom files for each one:
A(main project)
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.a.c</groupId>
<artifactId>D</artifactId>
<version>1.0.0</version>
<classifier>jar-with-dependencies</classifier>
<exclusions>
<exclusion>
<artifactId>commons-io</artifactId>
<groupId>commons-io</groupId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.a.c</groupId>
<artifactId>B</artifactId>
<version>0.0.1</version>
<classifier>jar-with-dependencies</classifier>
</dependency>
</dependencies>
</dependencyManagement>
B:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.a.c</groupId>
<artifactId>C</artifactId>
<version>0.0.1</version>
<classifier>jar-with-dependencies</classifier>
</dependency>
</dependencies>
</dependencyManagement>
C:
<dependencies>
<dependency>
<groupId>org.jsystemtest</groupId>
<artifactId>jsystemApp</artifactId>
<version>6.1.01</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.50.0</version>
</dependency>
</dependencies>
Which configuration should I set in order to get the dependencies of B and C without downloading the source code for each machine?
You seem to confuse <dependencyManagement> with <dependencies>. <dependencyManagement> in A and B only sets the version of dependencies without including them. This is why Maven will not download the dependencies of B and thus not of C (except if there is an additional <dependency> section in both A and B which you didn't show).
Related
I need to import a dependency in my project; The problem is that that dependency is specified by a pom in this way:
<dependency>
<groupId>it.xxxx.yyyyy.be.esb</groupId>
<artifactId>CR_XXXXX_BE_PRODO_YYYYYY_V1</artifactId>
<version>1.0.0-SNAPSHOT</version>
<type>pom</type>
</dependency>
When I add this dependency in my project it compile well, and in my local repository (.m2) I find the folder at the path it.xxxx.yyyyy.be.esb;
But not find any jar inside the folder, so How I can use that dependency?
That dependency should be a client to make soap call to a server!
In other project the dependency was:
<dependency>
<groupId>it.xxxx.yyyyy.be.esb</groupId>
<artifactId>CR_XXXXX_BE_PRODO_YYYYYY_V1</artifactId>
<version>1.0.0-SNAPSHOT</version>
<type>jar</type>
</dependency>
And when I compile, the jar file is downloaded and i'm able to find it in my local repository (.m2), also I can decompile and see all class inside the package, and I can import in my class;
So my question is, is there a way to use the first dependency (the one with pom as type: pom)?
thanks
In dependencyManagement you can import a "bill of materials" (BOM) that is provided in pom format. Here all the supplied components, their requirements and their versions are included so that you know that you are using all the versions that the BOM provider used.
For example, Spring boot (I know you can do it as a parent pom):
<dependencyManagement>
<dependencies>
<!-- Import dependency management from Spring Boot -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>3.0.2</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
.
.
<dependencies>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
</dependency>
</dependencies>
In their BOM, all the versions of the Spring Boot dependencies, including the versions of their plugins, are defined in one place.
Another good example is junit:
<!-- use dependency management to import pom -->
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit</groupId>
<artifactId>junit-bom</artifactId>
<version>5.9.2</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
.
.
<dependencies>
<!-- Version is defined in the parent pom -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
I am trying to access the Hive server present in GCP - Dataproc from my local machine(eclipse) using java and spark. But I am getting the below error while starting the application. I tried to find the problem but unable to solve it.
Exception in thread "main" java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:870)
at com.hadoop.Application.main(Application.java:22)
Pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.5.1</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.hadoop</groupId>
<artifactId>hadoop</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>hadoop</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>libraries-bom</artifactId>
<version>20.6.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-dataproc</artifactId>
<version>1.5.2</version>
</dependency>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>google-cloud-storage</artifactId>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>2.4.7</version>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.11</artifactId>
<version>2.4.7</version>
<scope>provided</scope>
<exclusions>
<exclusion>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.47.Final</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-hdfs</artifactId>
<version>2.10.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>2.4.7</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-client</artifactId>
<version>1.9</version>
</dependency>
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>2.5.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
The problem is with the scope of the following dependency:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-hive_2.11</artifactId>
<version>2.4.7</version>
<scope>provided</scope>
</dependency>
According to the Maven doc:
compile: This is the default scope, used if none is specified. Compile dependencies are available in all classpaths of a project. Furthermore, those dependencies are propagated to dependent projects.
provided: This is much like compile, but indicates you expect the JDK or a container to provide the dependency at runtime. For example, when building a web application for the Java Enterprise Edition, you would set the dependency on the Servlet API and related Java EE APIs to scope provided because the web container provides those classes. A dependency with this scope is added to the classpath used for compilation and test, but not the runtime classpath. It is not transitive.
You might want to change it to compile or remove the line. Or download the jar and add it to the classpath.
Also see this doc on how to create a Spark application uber jar which includes its dependencies.
I'm trying to implement my own jar (library) in another project I'm working on. But I'm getting the error NoClassDefFoundError when I run the program and try to instantiate a class from the library I built.
I try to add the dependency in different ways, with maven local repo or adding the jar file directly as an external jar in Build Path. I don't get any warning/error in eclipse and I can build and install the app with maven successfully. But when I execute the part of the code that use a class from that library I get NoClassDefFoundError.
My library groupId is com.mycompany.myapp, and the classes I'm using are in the package com.mycompany.myapp.business. I'm try to move the class to the parent package just in case but I'm always getting the same error.
1 - Do I need to define which classes need to be available when importing the jar?
2- Do I need to compile the library in an specific way apart from mvn clan package/install?
3 - Do I need to add the same dependencies I have in my library in my parent project to have everything working?
This is my children pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.mylibrary</groupId>
<artifactId>mylibrary</artifactId>
<version>0.0.1-SNAPSHOT</version>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.1</version>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>commons-discovery</groupId>
<artifactId>commons-discovery</artifactId>
<version>0.5</version>
</dependency>
<dependency>
<groupId>wsdl4j</groupId>
<artifactId>wsdl4j</artifactId>
<version>1.6.3</version>
</dependency>
<dependency>
<groupId>jaxen</groupId>
<artifactId>jaxen</artifactId>
<version>1.1.6</version>
</dependency>
<dependency>
<groupId>javax.xml.rpc</groupId>
<artifactId>javax.xml.rpc-api</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>dom4j</groupId>
<artifactId>dom4j</artifactId>
<version>1.6.1</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<version>1.47</version>
</dependency>
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpkix-jdk15on</artifactId>
<version>1.47</version>
</dependency>
<dependency>
<groupId>org.apache.axis</groupId>
<artifactId>axis</artifactId>
<version>1.4</version>
</dependency>
<dependency>
<groupId>javax.xml.rpc</groupId>
<artifactId>javax.xml.rpc-api</artifactId>
<version>1.1.1</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.5.0-M1</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>1.4</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.5.0-M1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
<version>1.5.0-M1</version>
</dependency>
<dependency>
<groupId>org.junit</groupId>
<artifactId>junit-bom</artifactId>
<version>5.3.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</project>
And then in my main project I instantiate the class in the following way:
Init oInit = new Init();
Where Init is a public class defined in com.mycompany.mylibrary.business package.
java.lang.NoClassDefFoundError: com/mycompany/mylibrary/business/Init
pom.xml from the main proyect:
<dependency>
<groupId>com.mycompany.mylibrary</groupId>
<artifactId>mylibrary</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>
In the parent project, in the libraries imported by maven I see that my library has a folder icon instead of the one the other have.
You can use the two ways to include a library into your project:
Including the library directly into your project classpath
Through Maven
When using the Maven option, you must be sure that the library is well exported to the Maven repository, local or remote (such as Nexus). The first place Maven will search when looking for a library defined in the pom.xml is on your local repository (located at ~/.m2). The it will look into the Central Maven Repository (located at https://mvnrepository.com/repos/central). In between, you can declare your own repositories into your pom.xml as
<repositories>
<repository>
<id>my-repo</id>
<name>My Maven Repository</name>
<url>${nexus.repositories.url}</url>
</repository>
<repositories>
How to build the library to be included into your own local or remote repository? Just be sure to include the install goal when compiling with Maven, and also the repository definition into its pom.xml
As shown in following screenshot - the project SDK and language level are 1.8:
And I went through all of the project modules: they are all likewise defined as 1.8. One of them is shown below:
Yet somehow IJ attached a language level of 1.4 to one of the modules
But actually .. it is not even that straightforward. E.g. the intellisense thinks 1.4:
But when I click on the java.lang.String underlined in red, it DOES go to 1.8:
The pom.xml for the culprit module 'icloud' makes no mention of any java level - so it is not to blame:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-parent</artifactId>
<version>1</version>
<relativePath>../../parent</relativePath>
</parent>
<artifactId>ignite-cloud</artifactId>
<version>1.1.3-SNAPSHOT</version>
<properties>
<jcloud.version>1.9.0</jcloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-core</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.jclouds</groupId>
<artifactId>jclouds-allcompute</artifactId>
<version>${jcloud.version}</version>
</dependency>
<dependency>
<groupId>org.apache.jclouds.labs</groupId>
<artifactId>google-compute-engine</artifactId>
<version>${jcloud.version}</version>
</dependency>
<dependency>
<groupId>org.apache.jclouds.labs</groupId>
<artifactId>docker</artifactId>
<version>${jcloud.version}</version>
</dependency>
<dependency>
<groupId>org.apache.jclouds.provider</groupId>
<artifactId>cloudsigma-zrh</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>org.apache.ignite</groupId>
<artifactId>ignite-core</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>${spring.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${spring.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
I have done Build|Rebuild twice. Also have done
mvn clean package
from command line twice. This is just a big time sink (/waste). So then: why is IJ sabotaging this module - and is there a workaround?
Add a the compiler plugin to pom.xml and reimport the pom will solve this issue
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
The problem was that intellij could not handle the examples subproject being
underneath a parent directory containing the main parent project
but the directory is its own self contained project with no relation to the parent
The workaround is to create an entirely independent project for the examples module.
The other suggested answer - about adding the compiler plugin - made no difference.
I created a Google App Engine maven project according to this documentation and it works successfully. Then after i import the same project into eclipse by using the option Existing Maven Project . But in eclipse its only just maven project without having java facet.Then i manually added facet by right clicking Project -> Properties -> Project Facet -> Tick on java option. Now eclipse shows java errors on my created application (for some app engine related classes). But this project works fine by using maven command like mvn clean install and mvn appengine:devserver. Here follows the pom for my maven project
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.google.appengine.demos</groupId>
<artifactId>guestbook</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<appengine.target.version>1.9.0</appengine.target.version>
</properties>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-maven-plugin</artifactId>
<version>${appengine.target.version}</version>
</plugin>
</plugins>
</pluginManagement>
</build>
<modules>
<module>guestbook-war</module>
<module>guestbook-ear</module>
</modules>
</project>
You do not appear to have any dependencies in you pom. These define the libraries your application depend on. Minimally, for an appengine war, you'll need something like this:
<dependencies>
<!-- Servlet dependencies -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
<version>1.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>javax.el</groupId>
<artifactId>javax.el-api</artifactId>
<version>2.2.4</version>
<scope>provided</scope>
</dependency>
<!-- GAE Dependencies -->
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-api-1.0-sdk</artifactId>
<version>${gae.version}</version>
<scope>compile</scope>
</dependency>
</dependencies>
If you also want to do some unit testing, you'll need to add the following to the above block:
<!-- Test Dependencies -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<version>1.9.5</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-testing</artifactId>
<version>${gae.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.google.appengine</groupId>
<artifactId>appengine-api-stubs</artifactId>
<version>${gae.version}</version>
<scope>test</scope>
</dependency>
If you want to work in eclipse, make sure you have the wtp-m2e eclipse plugin installed, and the Google Plugin for Eclipse plugin installed. They'll make your life much easier.