Maven: configure parallel build in pom.xml - java

Maven has a capability to perform parallel builds:
https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3
mvn -T 4 clean install # Builds with 4 threads
mvn -T 1C clean install # 1 thread per cpu core
mvn -T 1.5C clean install # 1.5 thread per cpu core
Is it possible to specify this arguments in pom.xml or settings.xml? Repeating this options could be annoying.

This solution is a bit of a hack but worked for me. It involves specifying a new environment variable, assigning the value -T3 to it and adding this variable to the Maven launch script.
For Windows (Linux in parens):
Open the Environment Variables window: Computer -> Properties -> Advanced System settings -> Environment Variables
Add the property MAVEN_CMD_LINE_OPTS with your desired value. In my case -T 3 as I want Maven to use 3 threads to build in parallel.
Edit the mvn.cmd file (In Linux: the mvn file). Find the part where the Java command is actually executed, the line starting with %MAVEN_JAVA_EXE% (In Linux: generally after the line defining the main class: org.codehaus.plexus.classworlds.launcher.Launcher)
Add %MAVEN_CMD_LINE_OPTS% to the end of the line (In Linux: $MAVEN_CMD_LINE_OPTS)
When you run mvn compile on a Maven project you will now see the following line:
Using the MultiThreadedBuilder implementation with a thread count of 3
This has the advantage of the user being able to 'override' this value. So if the user executes mvn -T4 compile, then 4 threads are used instead of the default 3.
Note:
I tried this on Maven 3.3.9 but the concept should work on any Maven
version.
Multi-threaded builds can suffer from issues where plugins
especially custom plugins are not thread safe. So use with care and
consider disabling this as a fix in case of issues.

I could not find a way to configure this in the pom.xml or settings.xml There is a good solution on unix systems, edit your .bashrc and add an alias.
alias mvnp='mvn -T 4'
now from the terminal run maven using mvnp
mvnp clean install

You are able to specify the option in the MAVEN_OPTS environment variable (see http://maven.apache.org/guides/mini/guide-configuring-maven.html). Once this is done, you don't have to repeat it. Configuring the environment variable depends on your system. However this will affect all maven runs within your environment. Maybe it's possible for you to enable different environments, so that only the project you actually want to build in parallel is running in such an environment.

Related

How to get the JDK path in Gradle within Android Studio?

In order to build some Android projects, it's necessary to set the environment variable JAVA_HOME. (See this Stack Exchange question and flutter bug report.)
This is the case for a project I'm working on. I would like to change the Gradle file so it is not necessary to set JAVA_HOME.
Is there any way for Gradle to get the path to the JDK used by the enclosing Android Studio process (or for the process to pass in the JDK without user intervention)? This should work when JAVA_HOME had not been set.
you can define a task in your_project_path/app/build.gradle
task javaHome {
println "JAVA_HOME:" + System.getProperty("java.home")
}
run task:
./gradlew -q app:javaHome
RESULT:
JAVA_HOME:/usr/lib/jvm/java-8-openjdk-amd64/jre
There are generally two options available:
a) Edit the gradle.properties file and define which JDK you want to use:
org.gradle.java.home=(path to JDK home)
There it cannot be set dynamically, because it's an egg/hen problem.
And it might also fail on other computers, because the path may vary.
But one can pass it as command-line option:
./gradlew -Dorg.gradle.java.home=$JAVA_PATH
For reference: Build Environment.
b) However, one can also add an export JDK_HOME statement on top of file gradlew. Came up with a shell script, which can at least detect the default JDK install on Linux (Android Studio runs on JRE):
tmp=`which java`
export JAVA_HOME=${tmp::-9}
echo $JAVA_HOME
/usr/java/jdk1.8.0_172
Of course, one also would have to consider no JDK being present at all:
/usr/bin/which: no java in ...
Generally, this assumes a default JDK had been set with alternatives, as a package manger usually would do; eg yum install jdk1.8.0_102.x86_64. The problem here is, that there is no easy way to identify which JDK path to use on Windows, because Windows has no which command and one would likely have to read from the registry. A helper PS script or executable could look up the value and truncate as required, called from file gradlew.bat. There still may be other ways to get the path.

How create maven alias for set of goals?

I'm configuring a Maven project and want to be able to alias, like
mvn server - to execute mvn clean package tomcat:run
The Grunt task runner does it very well, but I haven't found a way to do the same in Maven. Is it possible?
You can define a <defaultGoal>...</defaultGoal> in your pom if you like. So you can define something like this:
<project>
<build>
<defaultGoal>clean package tomcat:run</defaultGoal>
.
</build>
</project>
will be activated if you simply call mvn...not really an alias, cause usually you don't need one...
Out of the box I don't know of any solution that doesn't imply using a plugin. A simple solution may be adding aliases to your .bashrc file in your home directory (for Linux) or .bash_profile (on OS X) for your desired instructions.
E.g: Adding a line alias my-alias="mvn clean install" will allow you to execute the command my-aliasin the terminal, obtaining the same result as running the mvn clean install instruction itself. Add another line alias my-alias-port="mvn clean install -Dcrx.port=9200"for a second instruction, and so on.
Optionally, you can execute alias to see a list of all your aliases and their respective instructions.
The best solution I have found to this is to use a combination of:
New Maven 3.3 command line config support: project/.mvn/maven.config
GNU Make (which then calls maven)
Bash scripts
Blaze
Roughly in that order of preference.
GNU Make is especially nice because it offers bash completion.
An example Makefile for your specific example would be:
.PHONEY: server
server:
[tab]mvn clean package tomcat:run
Replace [tab] with a real tab! See make documentation.
Then you can run:
make server
For windows environments you will need to install cygwin or something equivalent. For Mac you don't have to but you should probably install homebrew.
Finally the Maven Bash completion albeit doesn't do aliases but will greatly facilitate typing maven commands (press tab). Many package managers have this as a package (ie homebrew has it as maven-completion).
You can also add the following function to your .bashrc file:
function mvn() {
if [ "$1" = "i" ]; then
command mvn install
else
command mvn $#
fi
}
And so you can invoke the mvn install with the mvn i alias.
Everything else that is not mvn i will call the original mvn command instead.
Alias-maven-plugin is what you are looking for.
Following the site:
Whenever you type a command in a shell, for instance
mvn clean install
you could spare time in simply using an alias like this
i
It has also more advantages - you could configure plugin by XML file.

Docker and Java IDE integration

I'll start by saying that I'm not a Java developer and also I'm not a Docker expert.
In order to minimize the gap between frontend and backend (in this specific case, Java) developers I started to put some docker images in place with java and maven and after the build I start a docker container with a volume pointing to the java project (and frontend developers don't have to worry about dependencies or how to run backend services).
Already here I have a question.
I've seen other people building an image with the actual code inside instead of attaching it later, so what's the best case (if there's one)?
I've done this way since I can reuse that image for "every" project and avoid building different images.
For starting/stopping/restarting docker containers I created a script that does all of that, so I can make some changes to the code, bring it down and up again.
It kinda works, and what I mean is, I'm well aware this is not a normal workflow of a Java developer to do that kind of stuff from a console.
So now, to the most important question, how do you integrate docker with a Java IDE?
I know that you can create custom build/run commands but I also read that things like logs are not displayed on the IDE's.
Can someone explain me how are you using Docker + Java IDE's?
Note: Maven is also used for compiling java code, like mvn clean install (if this helps)
I do not use Docker with a Java IDE. I use the IDE (Eclipse) to write and test the code, with Maven to manage the build. Then I have a Dockerfile like this:
FROM java:8
RUN apt-get update || apt-get update
RUN apt-get install -y maven
# Maven installs Java 7, which set itself as the default...
RUN update-alternatives --remove java /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
CMD java -jar target/main.jar
# Pull down dependencies here to allow Docker to cache more
ADD pom.xml /opt/app/pom.xml
WORKDIR /opt/app
RUN mvn dependency:go-offline -X
# I use the maven-shade-plugin to build a single jar
ADD src /opt/app/src
RUN mvn package
If you build all your images on one machine, then Docker will cache intelligently and you don't need to do anything more. If you want to run across more machines, or you just want to make it explicit, you could do something like this:
base/Dockerfile:
FROM java:8
RUN apt-get update || apt-get update
RUN apt-get install -y maven
RUN update-alternatives --remove java /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
CMD java -jar target/main.jar
$ docker build -t yourorg/java-base:8 ./base/
myapp/Dockerfile:
FROM yourorg/java-base:8
ADD pom.xml /opt/app/pom.xml
WORKDIR /opt/app
RUN mvn dependency:go-offline -X
ADD src /opt/app/src
RUN mvn package
You don't get as big an effect from Docker with Java, because JARs are already pretty portable and well-contained. I suppose it makes it easy to run different Java versions side-by-side. I use it because it allows me to run applications in different languages without needing to know what's inside the container. I have some in Java, some in Python, some in JavaScript, some in Erlang, but they all get started as docker run -d <flags> myorg/myimage:someversion.

JAVA_HOME is not defined correctly (only from jenkins)

Trying to setup jenkins but my builds fails with:
$ ant test
Error: JAVA_HOME is not defined correctly.
We cannot execute /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
Build step 'Invoke Ant' marked build as failure
If I manually run "ant test" on the machine it works just fine and the JAVA_HOME is set to the exact same value. Any idea why it fails when jenkins try to run it ? Are there any more environment variabled involved ( I could not see any though ) ?
For others, I had to add the PATH /usr/bin/ to my PATH variable within Jenkins. (Find your correct path using which java).
Jenkins > Manage Jenkins > Configure System
Add an Environment Variable >>
e.g:
Name: PATH
Value: /usr/local/bin/:/usr/bin/
Screenshot
The problem was this, I had forgot to check the box "Restrict where this project can be run" in the project configuration. Thus the testing tried to execute on "master" where JAVA_HOME was not the same as expected on the build executor I intended to run it on. Thus where I tested and where it actually ran was different machines.
Set JAVA_HOME in your Jenkins system config in Jenkins 2.107.1.
open your jenkins, e.g. http://192.168.1.12:9090, then add /configure to the url,
that is http://192.168.1.12:9090/configure, then you can find like next:
For me the options above did not help, solved by creating a link to what's asked:
sudo ln -s ${actual_java_location} /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java
actual_java_location can be read by this:
readlink -f $(which java)

Debugging in Maven?

Is it possible to launch a debugger such as jdb from Maven? I have a pom.xml file that compiles the project successfully. However, the program hangs somewhere and I would really like to launch jdb or an equivalent debugger to see what's happening.
I compile using mvn compile and launch using:
mvn exec:java -Dexec.mainClass="com.mycompany.app.App"
I was expecting something like:
mvn exec:jdb -Dexec.mainClass="com.mycompany.app.App"
to launch the debugger but, as usual, my expectations are incongruent with maven's philosophy.
Also, I couldn't find any documentation (on Maven's website or google) to describe how debugging works. I suspect that I have to use some plugin.
If you are using Maven 2.0.8+, run the mvnDebug command in place of mvn and attach a debugger on port 8000.
For Maven <2.0.8, uncomment the following line in your %M2_HOME%/bin/mvn.bat (and maybe save the modified version as mvnDebug.bat):
#REM set MAVEN_OPTS=-Xdebug -Xnoagent -Djava.compiler=NONE -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000
More details in MNG-2105 and Dealing with Eclipse-based IDE.
Just as Brian said, you can use remote debugging:
mvn exec:exec -Dexec.executable="java" -Dexec.args="-classpath %classpath -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=1044 com.mycompany.app.App"
Then in your eclipse, you can use remote debugging and attach the debugger to localhost:1044.
I thought I would expand on these answers for OSX and Linux folks (not that they need it):
I prefer to use mvnDebug too. But after OSX maverick destroyed my Java dev environment, I am starting from scratch and stubbled upon this post, and thought I would add to it.
$ mvnDebug vertx:runMod
-bash: mvnDebug: command not found
DOH! I have not set it up on this box after the new SSD drive and/or the reset of everything Java when I installed Maverick.
I use a package manager for OSX and Linux so I have no idea where mvn really lives.
(I know for brief periods of time.. thanks brew.. I like that I don't know this.)
Let's see:
$ which mvn
/usr/local/bin/mvn
There you are... you little b#stard.
Now where did you get installed to:
$ ls -l /usr/local/bin/mvn
lrwxr-xr-x 1 root wheel 39 Oct 31 13:00 /
/usr/local/bin/mvn -> /usr/local/Cellar/maven30/3.0.5/bin/mvn
Aha! So you got installed in /usr/local/Cellar/maven30/3.0.5/bin/mvn.
You cheeky little build tool. No doubt by homebrew...
Do you have your little buddy mvnDebug with you?
$ ls /usr/local/Cellar/maven30/3.0.5/bin/mvnDebug
/usr/local/Cellar/maven30/3.0.5/bin/mvnDebug
Good. Good. Very good. All going as planned.
Now move that little b#stard where I can remember him more easily.
$ ln -s /usr/local/Cellar/maven30/3.0.5/bin/mvnDebug /usr/local/bin/mvnDebug
ln: /usr/local/bin/mvnDebug: Permission denied
Darn you computer... You will submit to my will. Do you know who I am? I am SUDO! BOW!
$ sudo ln -s /usr/local/Cellar/maven30/3.0.5/bin/mvnDebug /usr/local/bin/mvnDebug
Now I can use it from Eclipse (but why would I do that when I have IntelliJ!!!!)
$ mvnDebug vertx:runMod
Preparing to Execute Maven in Debug Mode
Listening for transport dt_socket at address: 8000
Internally mvnDebug uses this:
MAVEN_DEBUG_OPTS="-Xdebug -Xnoagent -Djava.compiler=NONE \
-Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"
So you could modify it (I usually debug on port 9090).
This blog explains how to setup Eclipse remote debugging (shudder)
http://javarevisited.blogspot.com/2011/02/how-to-setup-remote-debugging-in.html
Ditto Netbeans
https://blogs.oracle.com/atishay/entry/use_netbeans_to_debug_a
Ditto IntelliJ
http://www.jetbrains.com/idea/webhelp/run-debug-configuration-remote.html
Here is some good docs on the -Xdebug command in general.
http://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/jrdocs/refman/optionX.html
"-Xdebug enables debugging capabilities in the JVM which are used by the Java Virtual Machine Tools Interface (JVMTI). JVMTI is a low-level debugging interface used by debuggers and profiling tools. With it, you can inspect the state and control the execution of applications running in the JVM."
"The subset of JVMTI that is most typically used by profilers is always available. However, the functionality used by debuggers to be able to step through the code and set breakpoints has some overhead associated with it and is not always available. To enable this functionality you must use the -Xdebug option."
-Xrunjdwp:transport=dt_socket,server=y,suspend=n myApp
Check out the docs on -Xrunjdwp too. You can enable it only when a certain exception is thrown for example. You can start it up suspended or running. Anyway.. I digress.
I found an easy way to do this -
Just enter a command like this -
>mvn -Dtest=TestClassName#methodname -Dmaven.surefire.debug test
It will start listening to 5005 port. Now just create a remote debugging in Eclipse through Debug Configurations for localhost(any host) and port 5005.
Source - https://doc.nuxeo.com/display/CORG/How+to+Debug+a+Test+Run+with+Maven
If you are using Netbeans, there is a nice shortcut to this.
Just define a goal exec:java and add the property jpda.listen=maven
Tested on Netbeans 7.3
If you don't want to be IDE dependent and want to work directly with the command line, you can use 'jdb' (Java Debugger)
As mentioned by Samuel with small modification (set suspend=y instead of suspend=n, y means yes which suspends the program and not run it so you that can set breakpoints to debug it, if suspend=n means it may run the program to completion before you can even debug it)
On the directory which contains your pom.xml, execute:
mvn exec:exec -Dexec.executable="java" -Dexec.args="-classpath %classpath -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1044 com.mycompany.app.App"
Then, open up a new terminal and execute:
jdb -attach 1044
You can then use jdb to debug your program!=)
Sources:
Java jdb remote debugging command line tool
Why not use the JPDA and attach to the launched process from a separate debugger process ? You should be able to specify the appropriate options in Maven to launch your process with the debugging hooks enabled. This article has more information.
I use the MAVEN_OPTS option, and find it useful to set suspend to "suspend=y" as my exec:java programs tend to be small generators which are finished before I have manage to attach a debugger.... :) With suspend on it will wait for a debugger to attach before proceding.
I couldn't make any of the other solutions work and then I tried starting maven with the following.
mvn exec:exec -Dexec.executable="java" -Dexec.args="-classpath %classpath -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000 com.example.Main"
This worked for me and then I searched some more and found this other answer. IntelliJ can run Maven project in debug mode but ignores breakpoints
The fact is that my application is a spring-boot application. Now, the link referred to above may be right and the problem is that the spring-boot plugin creates a new fork that somehow invalidates the interaction between debugger and target app. I'm saying this because I got some info on how this may work from here
In any case, I don't know how the two solutions are related to each other. Is the first solution also preventing spring-boot from creating a fork??
Sadly I have no clue at the moment.
Here is how I do it.
For example:
mvn clean azure-functions:run -DenableDebug
Then, after it starts up, you'll see a link in the Terminal that says "Attach Debugger".
To me, that is the easiest way to do it.

Categories

Resources