Develop Reference

Surefire Maven plugin: "Corrupted STDOUT by directly writing to native stream in forked JVM"

My JUnit tests are failing when running them through Maven and the Surefire plugin (version information below). I see the error message:
Corrupted STDOUT by directly writing to native stream in forked JVM 4. See FAQ web page and the dump file C:\(...)\target\surefire-reports\2019-03-20T18-57-17_082-jvmRun4.dumpstream
The FAQ page points out some possible reasons but I don't see how I can use this information to start solving this problem:
Corrupted STDOUT by directly writing to native stream in forked JVM
If your tests use native library which prints to STDOUT this warning message appears because the library corrupted the channel used by the plugin in order to transmit events with test status back to Maven process. It would be even worse if you override the Java stream by System.setOut because the stream is also supposed to be corrupted but the Maven will never see the tests finished and build may hang.
This warning message appears if you use FileDescriptor.out or JVM prints GC summary.
In that case the warning is printed "Corrupted STDOUT by directly writing to native stream in forked JVM", and a dump file can be found in Reports directory.
If debug level is enabled then messages of corrupted stream appear in the console.
It refers to some native library printing out to STDOUT directly but how can I figure out which one, and even if I do, how do I deal with this issue if I need the library for my project?
It mentions "debug level" but it is unclear if this means Maven's debug level or Surefire plugin's debug level. I enabled Maven's debug but I don't see the console outputs as mentioned by the FAQ. And Surefire's debug option seems to be about pausing tests and waiting for a debugger to be connected to the process, not simply showing more information on the console.
The dump files also don't seem very helpful:
# Created on 2019-03-20T18:42:58.323
Corrupted STDOUT by directly writing to native stream in forked JVM 2. Stream 'FATAL ERROR in native method: processing of -javaagent failed'.
java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'FATAL ERROR in native method: processing of -javaagent failed'.
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:511)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:209)
at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:176)
at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:88)
at java.base/java.lang.Thread.run(Thread.java:834)
So, how can I solve this problem?
Update: requested configuration information below.
I'm using OpenJDK 11 (Zulu distribution) on Windows 10, Maven 3.5.3, and Surefire 2.21.0 (full configuration below).
I'm running Maven from Eclipse using the "Run As..." context menu option on the pom.xml file, but obtain the same results when running it on the console.
I had never heard of JaCoco before the first comment to this question, but I see several error messages mentioning it:
[ERROR] ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was cmd.exe /X /C ""C:\Program Files\Zulu\zulu-11\bin\java" -javaagent:C:\\Users\\E26638\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.8.0\\org.jacoco.agent-0.8.0-runtime.jar=destfile=C:\\Users\\E26638\\git\\aic-expresso\\target\\jacoco.exec -Xms256m -Xmx1028m -jar C:\Users\E26638\AppData\Local\Temp\surefire10089630030045878403\surefirebooter8801585361488929382.jar C:\Users\E26638\AppData\Local\Temp\surefire10089630030045878403 2019-03-21T21-26-04_829-jvmRun12 surefire10858509118810158083tmp surefire_115439010304069944813tmp"
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
This is the Surefire Maven plugin configuration:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.21.0</version>
<configuration>
<skipTests>${skipUnitTests}</skipTests>
<testFailureIgnore>false</testFailureIgnore>
<forkCount>1.5C</forkCount>
<reuseForks>true</reuseForks>
<parallel>methods</parallel>
<threadCount>4</threadCount>
<perCoreThreadCount>true</perCoreThreadCount>
<reportFormat>plain</reportFormat>
<trimStackTrace>false</trimStackTrace>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
</configuration>
</plugin>

Run in the same problem while migrating project from JAVA 8 to JAVA 11, upgrading jacoco-plugin from 0.8.1 to 0.8.4 did the job.
Analysing maven dependencies, seeing from where jacoco is pulled and then fixing the version should solve the issue.

I was running into this issue when running my Junit tests using a custom Runner. If I made any output to System.out or System.err in my custom runner or in my test class, this exact warning would show up. In my case the problem was not caused by some older Jacoco version. Updating the surefire plugin to version 2.22.2 or the more recent 3.0.0-M4 did not solve the issue.
According to the Jira issue SUREFIRE-1614, the problem will be fixed in the 3.0.0-M5 release of the maven-surefire-plugin (not released as of May 21st 2020).
Update
The Maven Surefire plugin version 3.0.0-M5 has now been released. In your pom.xml you can do the following:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<!-- Activate the use of TCP to transmit events to the plugin -->
<forkNode implementation="org.apache.maven.plugin.surefire.extensions.SurefireForkNodeFactory"/>
</configuration>
</plugin>
Original answer
If you cannot wait for the release of the 3.0.0-M5 plugin, you can use the "SNAPSHOT" version of the plugin. It did fix the issue for me. You have to enable some specific setting in the plugin so that the plugin uses TCP instead of the standard output/error to obtain the events raised in your tests. Configuration changes below:
In my pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<!-- Add the repository to download the "SNAPSHOT" of maven-surefire-plugin -->
<pluginRepositories>
<pluginRepository>
<id>apache.snapshots</id>
<url>https://repository.apache.org/snapshots/</url>
</pluginRepository>
</pluginRepositories>
<build>
<pluginManagement>
<plugins>
...
<artifactId>maven-surefire-plugin</artifactId>
<!-- Use the SNAPSHOT version -->
<version>3.0.0-SNAPSHOT</version>
<configuration>
<!-- Activate the use of TCP to transmit events to the plugin -->
<forkNode implementation="org.apache.maven.plugin.surefire.extensions.SurefireForkNodeFactory"/>
</configuration>
</plugin>

For me it was updating the failsafe plugin from 2.22.0 to 2.22.2

If you are unable to upgrade to the latest JaCoCo version, I was also able to fix this for my project by setting forkCount to 0:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.0</version>
<configuration>
<forkCount>0</forkCount>
</configuration>
</plugin>

We are using log4j backend and could also fix it using follow set to true.
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="${messagePattern}" />
<follow>true</follow>
</Console>
</Appenders>
(We had the same error message but without jacoco being involved. But I can confirm that setting the forkNode in maven-surefire-plugin to TCP did also work.)

None of the listed answers helped in our case. The issue began after we've upgraded from Java 8 to Java 11.
Note: updating Surefire plugin was not possible in our case since it has broken some mechanisms the tests in our projects rely on - inspecting the issue took too long so we started identifying the root cause in Jacoco for that behaviour.
After some debugging and JVM dumps we found the cause: in our case we had JavaFX dependencies on the classpath which were loaded automatically by a resolver util. Loading these classes with jacoco enabled led to the JVM crash (without jacoco - encapsulated in a profile "coverage" in our case - did run fine). Excluding the loading of classes from JavaFX libraries (were not needed in our case) fixed the issue. Tests are running fine now without JVM crashs.
The exact class that led to the JVM crash (or at least the last that were loaded before) was in our case: com.sun.javafx.logging.jfr.JFRPulsePhaseEvent
Jar: javafx-base-12-win.jar
Hint: in many IDEs you can debug the Maven build with specific profile and check what is going on exactly.
Using Jacoco 0.8.6 and Surefire plugin 2.22.2

What solved it for me is upgrading maven surefire plugin to 2.22.2

For me it was upgrading org.testng to the latest version (7.3.0)

This issue happens to me too at random
I'm using
IntelliJ IDEA 2020 (Community Edition)
Surefire plugin (3.0.0-M5)
Maven 3.3.9
AdoptOpenJDK 11
And when it happens usually Windows 10 after several minutes shows
a beautiful blue screen of death.
And then after a restart everything goes back to normal

In my case I moved my development to a new PC and didn't have all our company library-dependencies in my Maven-repo yet. So when Maven ran there was such a library missing and I had to install it with mvn install:install-file ....
Therefore, it's important to read the latest surefire-logs as it suggests those things.
No idea, why the surefire-plugin doesn't just print that conflicting line on STDOUT to the console, so it would be obvious in less than a second.

We faced the same problem (OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (11.0.17+8, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)). We already had a jacoco-plugin version of 0.8.5. A downgrade to 0.8.4 did not help. We dug deepter into the dump file and it (i.e., the gitlab runner) spit out:
Corrupted STDOUT by directly writing to native stream in forked JVM 1. Stream '# /builds/mygroup/myproject/hs_err_pid114.log'.
In that file we discovered
Internal Error (sharedRuntime.cpp:1262), pid=114, tid=115
guarantee((retry_count++ < 100)) failed: Could not resolve to latest version of redefined method
... which lead to this bug issue: https://bugs.openjdk.org/browse/JDK-6776659
The issue should be fixed in Java 16. Upgrading at this point was not possible. Luckily, the thread creator provided a workaround:
CUSTOMER SUBMITTED WORKAROUND :
-Xint or -server
As we were running a maven docker image in the gitlab runner, we had to set the forkCount to 0 in the maven surefire plugin configuration:
<configuration>
<forkCount>0</forkCount>
</configuration>
That solved the VM crash. Unfortunately, we also had a testcoverage check (JaCoCo) and that was no longer working, as the surefire plugin created a separate VM for e.g., test classes instrumentatlization.
target/site/jacoco/jacoco.csv: No such file or directory
Finally, we had to use the argLine configuration of surefire (2, 3)
<configuration>
<argLine>-Xint ${argLine}</argLine>
</configuration>
.. and it worked.

I was getting this error when running Maven build in Intelij Idea. I had a couple of projects open in separate windows and had other strange errors in a different project.
Solved for me by closing all the Intellij Idea windows and re-opening the project. No dependencies versions were changed.

The newer Surefire plugin versions are completely buggy and broken.
for me (tested all the way up to Java 12) the only solution was to stick with 2.20.
Don't use 2.20.1 either, that failed with a NPE, although maybe it is specific to particular tests, but I don't have time to investigate that.

How to execute maven plugins from gradle without having maven and java installed

I need to execute a maven plugin on a system that does not have both maven and java installed and installing both of them on the system is not an option. While searching for the ways, I found out that by using gradle we can build executable binaries that does not even require gradle to get executed, which perfectly fits my situation :) . Is there any way to execute maven plugins by using gradle. Thanks in advance

Java is required for Maven and for Gradle too.
You can use Maven Wrapper exactly like with Gradle.
Execute this command in project directory (this creates executable script like in gradle)
mvn -N io.takari:maven:wrapper
To invoke this project without maven installed use:
./mvnw GOAL

Gradle is a build system. You can build anything with it, including native binaries.
And yes, you can call Maven Plugins from Gradle, as Maven Plugins are written in Java and Gradle is based on Groovy and thus on Java.
But both facts have nothing to do with each other.
You can of course also use Gradle (or any other build system including manually stuffing things together) to build a distributable that includes a Java runtime environment. But then Java is, as said, shipped with your result. You cannot run Java code without having Java around, besides porting the Java code to something else like C++ that compiles to a native binary.
Yes, you can also call Java code from native code if you do some glueing, but no, also this will not work without having Java around, because that's the way Java works.

Both Maven and Gradle require java to be installed. See https://docs.gradle.org/current/userguide/installation.html

Find Java options used by Maven

How can I find which Java options (Xmx, Xms, Xss, etc) are being used by Maven?
I've found out that a way to set them is via the environment MAVEN_OPTS. Now I want a way to be assured it's getting the right settings.
EDIT: I believe it´s different to this question as I don´t want to see the value of an environment variable. I rather want to see which settings are actually being used by Maven, whether it comes from a env var, settings.xml or other means.
EDIT2: I'm also interested in other ways of setting Java options for a Maven build

You can set Java options for Maven in different points and level (globally or via plugins configuration):
Plugin configuration: just for Compilation
Using the Maven Compiler Plugin configuration for compiling application code and test code, you can set the required Xmx, Xms, Xss options via the compileArgs configuration entry, available for both compile and testCompile goals. An official example is available here and on other SO answers like this one.
An example is also shown below on the third point.
Plugin configuration: just for Tests execution
Using the Maven Surefire Plugin configuration for tests executions, you can set the required Java options to be used at runtime via the argLine configuration entry of the test goal. An official example is available here.
An example is also shown below on the third point.
Plugin configuration: via Properties (and profiles)
You can combine the two options above (in case of common Java options) as a property value to pass to both compileArgs and argLine configuration entry or have different properties per configuration (according to your needs).
<property>
<jvm.options>-Xmx256M</jvm.options>
</property>
[...]
<build>
[...]
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.3</version>
<compilerArgs>
<arg>${jvm.options}</arg>
</compilerArgs>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<configuration>
<argLine>${jvm.options}</argLine>
</configuration>
</plugin>
</plugins>
[...]
</build>
[...]
Using properties gives you also an two extra advantages (on top of centralization): you can use profiles then to personalize it based on different desired behaviours (and example in this SO answer) and you can override them via command line as well, like:
mvn clean install -Djvm.options=-Xmx512
Global/Project configuration: Options file
Since Maven 3.3.1, you can specify your options in a .mvn/jvm.config file per project. It's an alternative to MAVEN_OPTS and a more narrowed scope (per project). However, since it sis a new Maven feature, people should be aware of it, otherwise troubleshooting and maintenance may be impacted.
Global/Env configuration: MAVEN_OPTS
Maven well-known environment variable to set global execution options, however applied to all Maven builds sharing that environment (i.e. per logged user).
When running Maven using the -X option (debug enabled), you will have the following output as part of your build:
[DEBUG] properties used {java.vendor=Oracle Corporation, ... , env.MAVEN_OPTS=-Xmx256M, ...
Update
After all, the executed mvn command is an OS script. Having a look at it in Windows, I found the possibility of using the MAVEN_BATCH_ECHO option which, if enabled (value set to on), will echo any command executed by the script and as such also the invocation of the java command, where you can see if your options (the MAVEN_OPTS) are picked up correctly together with the full list of parameters passed to it.
Here is an execution I tested on Windows:
set MAVEN_BATCH_ECHO=on
set MAVEN_OPTS=-Xmx256M
mvn compile > output.txt
NOTE: the output.txt will contain quite a lot of text, providing build output and additional echos executions. As part of it, it provided:
>"path_to_\jdk1.7\bin\java.exe" -Xmx256M -classpath "path_to\apache-maven-3.1.1\bin\..\boot\plexus-classworlds-2.5.1.jar" "-Dclassworlds.conf=path_to\apache-maven-3.1.1\bin\..\bin\m2.conf" "-Dmaven.home=path_to\apache-maven-3.1.1\bin\.." org.codehaus.plexus.classworlds.launcher.Launcher compile
As you can see, the -Xmx256M option was picked up correctly. If the maven script for other OS doesn't provide this option, then you can simply add it to the script (a simple echo before the java execution, showing the command, would also be enough).
You can find the maven script in your maven installation folder, under the bin subfolder.
Update2
Furthermore, since Maven is a Java tool after all and as you can see from its script it invokes the java command, you could see all the available options as suggested in this SO answer by slightly changing the Maven script and use the jinfo command which should really give you the answer according to this other SO answer.

Maybe it helps to start Maven with verbose debug output (-debug, I think?). Otherwise just do a ps aux | grep java and check the arguments of the process (assuming *nix).

GAE ClassFormatError: Incompatible magic value 4022320623 EFBFBDEF

I have a simple webapp in an mavenized Eclipse project.
My pom.xml contains some lines that add Google's appengine-maven-plugin to the build plugins.
When I run mvn appengine:devserver I eventually get an error telling me that my class could not be loaded:
[INFO] WARNING: Error starting handlers
[INFO] java.lang.ClassFormatError: Incompatible magic value 4022320623 in class file com/teamlazerbeez/http/di/StartupListener
That number is EFBFBDEF in hexadecimal notation, something that is clearly not CAFEBABE, the byte sequence Java class files should start with.
I only found this and this on the subject matter, which leads me to believe that the encoding went wrong during either writing or reading the class file.
Is this the problem? How do I force maven to read/write classes with e.g. UTF-8 encoding?
And what is a good encoding?
My java files are all encoded the same way: Eclipse says ISO-8859-1, Notepad++ says ANSI.
PS: I'm on a windows machine.

As Joakim Erdfeld's answer suggests, one of the plugins is fiddling with the build.
The culprit is the maven-war-plugin v2.3, but only in the following situation:
Having the GAE nature enabled on the mavenized Ecipse project (where the appengine-maven-plugin v1.7.5 is listed in the pom.xml) will break running mvn appengine:devserver iff the following conditions apply:
'default output folder' (in the Eclipse project's 'Build Path' settings) is set to src/main/webapp/WEB-INF/classes/ (required for the GAE Eclipse plugin to work)
The GAE plugin decided to put GAE-jars in src/main/webapp/WEB-INF/lib/, even though these jars are already in the classpath.
In conclusion, the maven-war-plugin improperly copies .class and .jar files from your WEB-INF/ to your resulting .war file. Somewhere in the process, it mis-encodes these files.

Check your src/main/webapp/WEB-INF folder, it should not contain a classes folder.
If it does, it means you probably misconfigured your eclipse project.
One case we encountered is that the developer imports a maven project and runs the project "as a web application". The first time he does this, Eclipse asks him where the webapp folder is. If you answer src/main/webapp then Eclipse will put its classes there, and it will get messed up by maven after.
To solve this, delete the project from Eclipse and remove the src/main/webapp/WEB-INF/classes folder, then re-import the project.
And when Eclipse asks where the webapp folder is, it is in target/yourproject-yourversion.

Something in your maven build is messing with the class files after the compile and before the package phases.
Start by commenting out all of your <plugins> and then adding them back 1 at a time till the build breaks again.
Be sure you run > mvn help:effective-pom to check on what your pom looks like when it has been fully resolved to parent poms and whatnot.
(Experimental) You can possibly even get the build itself to tell you if the classes are bad by simply using the project-info-reports command line for the dependencies report.
> mvn clean org.apache.maven.plugins:maven-project-info-reports-plugin:2.6:dependencies

I found this question which also mentioned the invalid magic number EFBFBDEF. It also seems to suggest that you set the maven encodings properly. Use the following property:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
But I think that this only helps if you get an encoding warning during the maven build process.

Using maven-release-plugin in Eclipse

I recently tried to use maven-release-plugin since it is apparently the recommended way of building and packages releases in the Maven universe.
However I wanted to use this within Eclipse, as the rest of my development workflow is Eclipse based. I normally run Maven commands via the m2eclipse plugin provided as part of Eclipse Juno (4.2)
I noticed a few oddities when I tried to run "release:prepare" within Eclipse:
Some extra files were created in the root project directory - "pom.xml.releaseBackup" and "release.properties". Do they really belong there? Have I got the release directories set up correctly? I wouldn't really consider these temporary artifacts as part of my source code tree......
The pom.xml gets manually overwritten with the updated release number. Eclipse warns you and is happy to reload the updated version - but is this generally safe?
The prepare ultimately fails giving the error [ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.0:prepare (default-cli) on project clisk: Failed to invoke Maven build. Error configuring command-line. Reason: Maven executable not found at: C:\Users\Mike\git\clisk\EMBEDDED\bin\mvn.bat -> [Help 1] - presumably because I am using the built-in Maven excetable provided by m2eclipse rather than the command line. I guess I could install command line maven as well.... but is that sensible or will it just cause more problems?
Given these kind of issues, Is there a way to get maven-release-plugin to work smoothly within Eclipse, or should I just give up and continue to do releases manually?

I have been using release plugin, but only from command line.
Re. 1. The backup files that release plugin creates are needed if something goes wrong in time of preparing the release. You can always rollback the prepared release using release:rollback command. When you do release:perform they will be deleted.
Re. 2. The plugin changes the version number from snapshot version for example: 0.0.1-SNAPSHOT: to release version: 0.0.1. Then after release:perform release version is moved to the maven repository and release plugin changes version again to 0.0.2-SNAPSHOT. Now you can use you full released (tested) version in your testing or production enviornment and snapshot version for developping purposes.
Re. 3. I don't know what is causing the problem, but I don't see the problem by using release plugin from command line.

Also mvn 3.3.3 installs a mvn.cmd file, instead of a mvn.bat file in Windows.
You should copy mvn.cmd to mvn.bat

Well i know this link is OLD , but to help some on who reffers this link for the issue 3.
Install maven separately on to local Box and give the path of the installation under Windows->Preferences--> maven--> Installation. Also you have to define the same in the run configuration within the Eclipse.
Attached is a link that explains the same.
http://maven.40175.n5.nabble.com/Build-Failure-prepare-release-td510949.html

You might be able to overcome the error you mention by installing command line Maven and configure Eclipse to use that rather than the embedded one by choosing Window -> Preferences -> Maven -> Installations, but I agree with the advice of making your releases outside Eclipse.