Develop Reference

Is there any way to speed up or optimize the resolving of Maven dependencies in Spring Boot projects using IntelliJ? [duplicate]

Unfortunately the maven initial build is very slow due to artifacts downloading
f.e. I've tried to download the same jar using curl - it is 3 times faster!!!
Why? And how I can speed up it? Maybe maven has some config keys related to artifacts downloading speed?
Thank you.

I know its an old question, but stumbled here from Google. I already had proxy in place, just needed to speed up with concurrent downloads. You can using mvn option:
-Dmaven.artifact.threads=30
Source:
https://maven.apache.org/guides/mini/guide-configuring-maven.html
Configuring Parallel Artifact Resolution
By default, Maven 2.1.0+ will download up to 5 artifacts (from different groups) at once. To change the size of the thread pool, start Maven using -Dmaven.artifact.threads. For example, to only download single artifacts at a time:
mvn -Dmaven.artifact.threads=1 verify
You may wish to set this option permanently, in which case you can use the MAVEN_OPTS environment variable. For example:
export MAVEN_OPTS=-Dmaven.artifact.threads=3

Use a local repository manager/mirror/proxy. All downloads will then go against this instead against the public repositories on the internet. The most popular ones are:
Archiva: http://archiva.apache.org/
Artifactory: http://www.jfrog.org/
Nexus: http://www.sonatype.org/nexus/
They are fairly easy to install and set up and provide a lot of value. Most of them have free versions as well. Just use an old development box to get started and move to a real server once you want to broaden the scope and make it available to more people.

The key point of the question was missed in the answers above:
I've tried to download the same jar using curl - it is 3 times faster!!!
This means it is a software issue, mitigation by installing a local proxy or altering the snapshot policy in the settings.xml both come with extra work and potential side effects, such as snapshot dependencies not being updated.
The issue described by the question is maven not utilizing the available bandwidth, thus being slow. This issue was identified in https://issues.apache.org/jira/browse/WAGON-537 and is resolved since maven 3.6.1, see https://maven.apache.org/docs/3.6.1/release-notes.html and https://issues.apache.org/jira/browse/MNG-6591, respectively. There is thus no need to do anything else but update to the latest maven version.

The best optimization is to avoid downloading. Have a look to your settings.xml maven configuration and check if the updatePolicy flag is set to "daily" on releases and snapshots. This should be the default but sometimes it may be set to 'always' - e.g. in repository manager configurations.
Caution: In this case (daily) you have to be cautious on snapshot changes that you might not get immediatly.
I know that this is not a direct answer to your question but the best maven download optimization I know.

You can download artifact using curl (if you think that is faster) and install it to your maven repository using following command:
mvn install:install-file -Dfile=<path-to-file> -DgroupId=<group-id> \
-DartifactId=<artifact-id> -Dversion=<version> -Dpackaging=<packaging>
http://maven.apache.org/guides/mini/guide-3rd-party-jars-local.html
Once you install it in local repository, next time maven will pick it up from there and will not try to download again.
Additionally, if the central maven repository site is slower to you, please consider using Maven Repository Mirrors.
Guide to Mirror Settings - http://maven.apache.org/guides/mini/guide-mirror-settings.html

Maven downloading from central repository

I am on Netbeans and don't know Maven much. Whenever I import, open some Maven project, it starts donwloading something from some central repository, sometimes huge. It downloads things in .m2\repository.cache\m2e. I have limited bandwidth and don't want this. How to stop this?
I have set Options>Java>Maven>Dependency Download Strategy to never. Also tried mvn -o install and mvn -o for offline. Not solved.

The Maven way is to get you what the project says it needs, but you have not already downloaded to your local repository.
The huge file is the list of what is actually available in Maven Central, and for some reason unknown to me it is downloaded on a regular basis. If you do it once, it should be kept for future sessions.

Maven will download all the dependency only once to the local repository and not again and again.
Weather you have limited or unlimited bandwidth you have to download it to execute your project.

Maven has a very modular architecture. That means the the thing you get when you download the Maven distribution is in reality small core functionality.
The rest is downloaded from a Maven artifact repository, like Maven Central (which is the default repo).
Note that this applies not only for dependencies (the library your project uses), but also your plugins (i.e. the stuff that compiles, packages, and otherwise builds the projects). Hence the large number of downloads.
Like the other answers said, if you don't delete your local repository it should eventually contain all the artifacts (dependencies and plugins) you need without re-downloading. The only exception are SNAPHSOT dependencies which can get re-downloaded periodically, depending what's in your POM and settings.
Ultimately, you have two possibilities:
If you have access to a higher-bandwith connection somewhere, you can build the projects while using it, and your local repo will still store the needed artifacts.
If you have several computers/configurations behind a local network, you can set up a Maven repository manager, like Nexus or Artifactory, and use it as a local mirror. Note that those still need to download the artifacts at first as well.
But there isn't much else you can do. "Maven downloading the Internet" is, unfortunately in your case, by design.

Create and deploy a maven site for the most recent SNAPSHOT?

For the longest time I used the default site I specified in the distributionManagement section of my POM to host the latest SNAPSHOT release's site. However now that I have my first version out the door, I can't use that anymore as its holding the release site.
I've gone ahead and setup a new place to store the site SNAPSHOT release, but I can't figure out how to automate the site plugin deploying there. mvn site:deploy and even mvn site:stage-deploy simply deploys to specified release site in distributionManagement, which isn't what I want. It seems the only way is to provide the URL over command line arguments, which isn't exactly an ideal situation.
Is there any way to specify a release repository and a SNAPSHOT repository for sites?

You can setup a profile for your SNAPSHOT builds that publishes to another location. Profiles let you alter numerous pom settings including distributionManagement and repositories. You would still need a way to tell maven to use the profile. There are several options to do that, some of which are automatic, and the best way is going to be dependent on your build process.

How to build without using locally installed artifacts

Is there any way to force Maven to use remote artifacts and not those installed on your machine? since I worry about runtime errors and not compilation errors build server is not valid option.
P.S. I know I could delete or rename the .m2 folder, but I bet there is some more clever way of doing this. Maybe some plugin or special command param?

Having no local repository would mean your classpath consisting almost entirely of URLs on remote servers. I can't see why this would be supported as execution would be awful, and any dropped connection would result in classloader issues. Having a local repository ensures the jars are available before compilation/execution begins.
Also consider that WAR and EAR projects (and many using the dependency plugin) rely on downloading the jars to complete their packaging. There would be a huge overhead if these had to be retrieved from a remote repository on every build. I'm pretty sure the managers of central would not be keen on dealing with that load.
Some alternatives for you to consider:
If you want to force a clean local repository on each build, you can use the purge goal of the dependency plugin.
If you want to keep builds isolated, you can use separate Maven settings by passing -Dorg.apache.maven.global-settings=/path/to/global/settings.xml
Alternatively you can override the local repository on a per build basis by passing -Dmaven.repo.local=/some/repo/path
If you want to avoid hitting remote repositories on each build, add <updatePolicy>never</updatePolicy> to your remote repository configurations. This means Maven will only check for updates if you force it to with a "-U" switch on the command line
If you want to take the latest version of a dependency, you can use the LATEST keyword in the version declaration (instead of the version number), though this can be risky if the dependency is incompatible.
If you want to take the current release version of a dependency, you can use the RELEASE keyword in the version declaration (instead of the version number). This is like LATEST, but tends to be the newest stable build, rather than the newest.
If you want to take the latest version of a dependency within a range, use Maven's version range notation, for example [1.0.0,2.0.0) means any version from 1.0.0 inclusive to 2.0.0 exclusive
For more details on LATEST and RELEASE, see section 9.3.1.3 of the Maven book.
If you use an internal repository manager (obligatory Nexus and Artifactory references here), the overhead of purging the local repository is greatly reduced - you'll just have an increased local network traffic load.

I don't think there's really a way to do what you are asking for. You could look into depending on SNAPSHOT releases (but that means changing your version string of the upstream projects to be SNAPSHOT versions).
Incidentally, this was discussed at length in a recent Java Posse episode (#268). I don't think they ended up with a solution, but you may get some good ideas there.
I also like some of Rich Seller's ideas, which I'll be looking into myself.

How to keep updated libraries in MAVEN?

On the development shop I work for, we have an internal MAVEN repository, to keep our libraries (proprietary & open-souce). A common problem that we face is that, sometimes, the open-source libraries in our local MAVEN repository gets obsolete. Is there an automatic way to keep all the open-source libraries I use in my MAVEN repository always updated? What do you suggest to resolve this issue?

Archiva has been mentioned, but nexus seems more popular. Both have been designed to solve problems like the one you're having

Assuming you:
Don't want to download everything
Don't want to run another server
process
Only want to track a limited number
of projects
You might want to create a separate pom.xml file with dependencies like this:
<dependency>
<groupId>org.openfoo</groupId>
<artifactId>jfoo</artifactId>
<version>[1.0.0,2.0.0)</version>
</dependency>
This will tell maven to use jfoo 1.0.0 up to jfoo 2.0.0 so when jfoo releases version 1.2.17, you'll be fetching that in the next build assuming your settings are set to check versions each time.
This pom doesn't have to actually build anything. Just list those things you want to track.
Running:
cd the-path-to-the-project; mvn -q -B -U package
Via cron once a day will update all the dependencies in that pom and only report when there is a problem
BTW, this is a hack. If the number of developers is > 3 and you have the resources to run nexus, don't bother with the hack.

Take a look at Apache Archiva, a repository manager for Maven.