Can't run created -jar file [duplicate] - java

My java program is packaged in a jar file and makes use of an external jar library, bouncy castle. My code compiles fine, but running the jar leads to the following error:
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
I've googled for over an hour searching for an explanation and found very little of value. If anyone has seen this error before and could offer some help, I would be obliged.

For those who got this error when trying to create a shaded uber-jar with maven-shade-plugin, the solution is to exclude manifest signature files by adding the following lines to the plugin configuration:
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<!-- Additional configuration. -->
</configuration>

For those using gradle and trying to create and use a fat jar, the following syntax might help.
jar {
doFirst {
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
}
exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
}

Please use the following command
zip -d yourjar.jar 'META-INF/*.SF' 'META-INF/*.RSA' 'META-INF/*.DSA'

Some of your dependencies are likely signed jarfiles. When you combine them all into one big jarfile, the corresponding signature files are still present, and no longer match the "big combined" jarfile, so the runtime halts thinking the jar file has been tampered with (which it...has so to speak).
Assuming you're using ant, you can solve the problem by eliminating the signature files from your jarfile dependencies. Unfortunately, it's not possible to do this in one step in ant.
However, I was able to get this working with Ant in two steps, without specifically naming each jarfile dependency, by using:
<target name="jar" depends="compile" description="Create one big jarfile.">
<jar jarfile="${output.dir}/deps.jar">
<zipgroupfileset dir="jars">
<include name="**/*.jar" />
</zipgroupfileset>
</jar>
<sleep seconds="1" />
<jar jarfile="${output.dir}/myjar.jar" basedir="${classes.dir}">
<zipfileset src="${output.dir}/deps.jar" excludes="META-INF/*.SF" />
<manifest>
<attribute name="Main-Class" value="com.mycompany.MyMain" />
</manifest>
</jar>
</target>
The sleep element is supposed to prevent errors about files with modification dates in the future.
Other variations I found in the linked threads didn't work for me.

The solution listed here might provide a pointer.
Invalid signature file digest for Manifest main attributes
Bottom line :
It's probably best to keep the official jar as
is and just add it as a dependency in the manifest file for your
application jar file.

Security is already a tough topic, but I'm disappointed to see the most popular solution is to delete the security signatures. JCE requires these signatures. Maven shade explodes the BouncyCastle jar file which puts the signatures into META-INF, but the BouncyCastle signatures aren't valid for a new, uber-jar (only for the BC jar), and that's what causes the Invalid signature error in this thread.
Yes, excluding or deleting the signatures as suggested by #ruhsuzbaykus does indeed make the original error go away, but it can also lead to new, cryptic errors:
java.security.NoSuchAlgorithmException: PBEWithSHA256And256BitAES-CBC-BC SecretKeyFactory not available
By explicitly specifying where to find the algorithm like this:
SecretKeyFactory.getInstance("PBEWithSHA256And256BitAES-CBC-BC","BC");
I was able to get a different error:
java.security.NoSuchProviderException: JCE cannot authenticate the provider BC
JCE can't authenticate the provider because we've deleted the cryptographic signatures by following the suggestion elsewhere in this same thread.
The solution I found was the executable packer plugin that uses a jar-in-jar approach to preserve the BouncyCastle signature in a single, executable jar.
UPDATE:
Another way to do this (the correct way?) is to use Maven Jar signer. This allows you to keep using Maven shade without getting security errors. HOWEVER, you must have a code signing certificate (Oracle suggests searching for "Java Code Signing Certificate"). The POM config looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>org.bouncycastle:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>your.class.here</mainClass>
</transformer>
</transformers>
<shadedArtifactAttached>true</shadedArtifactAttached>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jarsigner-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<id>sign</id>
<goals>
<goal>sign</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<keystore>/path/to/myKeystore</keystore>
<alias>myfirstkey</alias>
<storepass>111111</storepass>
<keypass>111111</keypass>
</configuration>
</plugin>
No, there's no way to get JCE to recognize a self-signed cert, so if you need to preserve the BouncyCastle certs, you have to either use the jar-in-jar plugin or get a JCE cert.

I had this problem when using IntelliJ IDEA 14.01.
I was able to fix it by:
File->Project Structure->Add New (Artifacts)->jar->From Modules With Dependencies on the Create Jar From Module Window:
Select you main class
JAR File from Libraries
Select copy to the output directory and link via manifest

I faced the same issue, after reference somewhere, it worked as below changing:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>

Assuming you build your jar file with ant, you can just instruct ant to leave out the META-INF dir. This is a simplified version of my ant target:
<jar destfile="app.jar" basedir="${classes.dir}">
<zipfileset excludes="META-INF/**/*" src="${lib.dir}/bcprov-jdk16-145.jar"></zipfileset>
<manifest>
<attribute name="Main-Class" value="app.Main"/>
</manifest>
</jar>

I've recently started using IntelliJ on my projects. However, some of my colleagues still use Eclipse on the same projects. Today, I've got the very same error after executing the jar-file created by my IntelliJ. While all the solutions in here talking about almost the same thing, none of them worked for me easily (possibly because I don't use ANT, maven build gave me other errors which referred me to http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException, and also I couldn't figure out what are the signed jars by myself!)
Finally, this helped me
zip -d demoSampler.jar 'META-INF/*.SF' 'META-INF/*.RSA' 'META-INF/*SF'
Guess what's been removed from my jar file?!
deleting: META-INF/ECLIPSE_.SF
deleting: META-INF/ECLIPSE_.RSA
It seems that the issue was relevant to some eclipse-relevant files.

I had the same issue in gradle when creating a fat Jar; updating the build.gradle file with an exclude line corrected the problem.
jar {
from {
configurations.compile.collect {
it.isDirectory() ? it : zipTree(it)
}
}
exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
manifest {
attributes 'Main-Class': 'com.test.Main'
}
}

In case you're using gradle, here is a full farJar task:
version = '1.0'
//create a single Jar with all dependencies
task fatJar(type: Jar) {
manifest {
attributes 'Implementation-Title': 'Gradle Jar File Example',
'Implementation-Version': version,
'Main-Class': 'com.example.main'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
with jar
}

Compare the folder META-INF in new jar with old jar (before you added new libraries). It is possibility that there will be new files. If yes, you can remove them. It should helps.
Regards,
999michal

A strategy would consist in using ANT to simplify the removal of the signature from each Jar file. It would proceed with the following steps:
Copying the MANIFEST.MF in a temporary file
Removing the Name and SHA entries from the temporary file
Creating a temporary Jar file with the temporary manifest
Removing the temporary manifest
Swapping the original Jar file with the temporary one
Here is an ANT macrodef doing the work:
<macrodef name="unsignjar" description="To unsign a specific Jar file">
<attribute name="jarfile"
description="The jar file to unsign" />
<sequential>
<!-- Copying to the temporary manifest file -->
<copy toFile="#{jarFile}_MANIFEST.tmp">
<resources>
<zipentry zipfile="#{jarFile}" name="META-INF/MANIFEST.MF"/>
</resources>
</copy>
<!-- Removing the Name and SHA entries from the temporary file -->
<replaceregexp file="#{jarFile}_MANIFEST.tmp" match="\nName:(.+?)\nSH" replace="SH" flags="gis" byline="false"/>
<replaceregexp file="#{jarFile}_MANIFEST.tmp" match="SHA(.*)" replace="" flags="gis" byline="false"/>
<!-- Creating a temporary Jar file with the temporary manifest -->
<jar jarfile="#{jarFile}.tmp"
manifest="#{jarFile}_MANIFEST.tmp">
<zipfileset src="#{jarFile}">
<include name="**"/>
<exclude name="META-INF/*.SF"/>
<exclude name="META-INF/*.DSA"/>
<exclude name="META-INF/*.RSA"/>
</zipfileset>
</jar>
<!-- Removing the temporary manifest -->
<delete file="#{jarFile}_MANIFEST.tmp" />
<!-- Swapping the original Jar file with the temporary one -->
<move file="#{jarFile}.tmp"
tofile="#{jarFile}"
overwrite="true" />
</sequential>
`
The definition can then be called this way in an ANT task:
<target name="unsignJar">
<unsignjar jarFile="org.test.myjartounsign.jar" />
</target>

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:314)
at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:268)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:316)
at java.util.jar.JarVerifier.update(JarVerifier.java:228)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
at java.util.jar.JarFile.getInputStream(JarFile.java:450)
at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:977)
at sun.misc.Resource.cachedInputStream(Resource.java:77)
at sun.misc.Resource.getByteBuffer(Resource.java:160)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:454)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
What helped me (IntelliJ IDEA 2016.3):
File -> Project Structure -> Artifacts -> Add JAR -> Select Main Class -> Choose "copy to the output directory and link via manifest" -> OK -> Apply -> Build -> Build Artifacts... -> Build

It's possible that two different signers mess up java mind.
Try removing META-INF folder from jar, adding manifest and signing JAR again, it helped me: http://jehy.ru/articles/2013/12/13/invalid-signature-file-digest-for-manifest-main-attributes/

If you are looking for a Fat JAR solution without unpacking or tampering with the original libraries but with a special JAR classloader, take a look at my project here.
Disclaimer: I did not write the code, just package it and publish it on Maven Central and describe in my read-me how to use it.
I personally use it for creating runnable uber JARs containing BouncyCastle dependencies. Maybe it is useful for you, too.

You can use Shadow to generate a jar.
Shadow is a Gradle plugin for combining a project's dependency classes and resources into a single output Jar. The combined Jar is often referred to a fat-jar or uber-jar.
Modify build.gradle
plugins {
...
// ① Add the shadow plugin
id "com.github.johnrengelman.shadow" version "5.2.0"
}
...
// ② Config the shadow jar, its name is baseName-1.0-classifier.jar
shadowJar {
archiveBaseName.set('baseName')
archiveClassifier.set('classifier')
archiveVersion.set('1.0')
manifest {
attributes 'Main-Class': 'Main'
}
}
// ③ Disable the default jar task
jar.enabled = false
// ④ Execute the shadowJar task when compiling
build.dependsOn(shadowJar)
Execute the command gradle build, the jar file will be generated:
<Project Directory>/build/libs/baseName-1.0-classifier.jar

For those who have trouble with the accepted solution, there is another way to exclude resource from shaded jar with DontIncludeResourceTransformer:
https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#DontIncludeResourceTransformer
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">
<resource>BC1024KE.DSA</resource>
</transformer>
</transformers>
From Shade 3.0, this transformer accepts a list of resources. Before that you just need to use multiple transformer each with one resource.

This happened to me in Intellij when I clicked "Add as a Maven Project" on bottom line when Intellij said "non-managed pom files found.". Meanwhile out folder was already generated. So it did not get recent changes.
Deleting out folder and running the program solved the issue for me. out folder was then recreated.
See the answer of Little Fox as well. The error I received was very similar to his.

For someone who using kotlin script(kts) as building script:
task("fatJar", type = Jar::class) {
baseName = "${project.name}-fat"
manifest {
attributes["Implementation-Title"] = "Watcher Jar File"
attributes["Implementation-Version"] = version
attributes["Main-Class"] = "MainKt"
}
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) }) {
exclude(listOf("META-INF/*.RSA", "META-INF/*.SF", "META-INF/*.DSA"))
}
with(tasks.jar.get() as CopySpec)
}

I had a similar problem. The reason was that I was compiling using a JDK with a different JRE than the default one in my Windows box.
Using the correct java.exe solved my problem.

If you're getting this when trying to bind JAR files for a Xamarin.Android bindings project like so:
JARTOXML : warning J2XA006: missing class error was raised while reflecting com.your.class : Invalid signature file digest for Manifest main attributes
Just open the JAR files using Winzip and delete the meta-inf directories. Rebuild - job done

Related

Seperate spring.factories into multiple files

In Spring Boot, you can do the following:
src/main/resources/META-INF/spring.factories
# Auto Configure
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.boot.autoconfigure.admin.AConfiguration,\
org.springframework.boot.autoconfigure.admin.BConfiguration,\
org.springframework.boot.autoconfigure.admin.CConfiguration,\
org.springframework.boot.autoconfigure.admin.DConfiguration,\
org.springframework.boot.autoconfigure.admin.EConfiguration,\
org.springframework.boot.autoconfigure.admin.FConfiguration,\
Which is very nice. However after a year of development the list of auto configuration is now > 15 lines, which makes it hard to manage.
Would like to know if it is possible to separate the spring.factories into multiple files? Preferably would like to keep the whole project in one JAR.
Or maybe there is another ways to help organize the EnableAutoConfiguration that I am not aware of?
Thanks in advance!
While using spring-boot we use multiple "starters", each with an auto-configuration and spring.factories file.
So, one way could be to split your project into modules - one for each auto-configuration, define a dedicated spring.factories file in the module, and import all the modules as a runtime dependency in the main application module.
You can use maven or gradle to manage the multi-module project and the dependencies among them:
Gradle: https://guides.gradle.org/creating-multi-project-builds/
Maven: https://www.baeldung.com/maven-multi-module
Example:
root
moduleA
src/main/resources/META-INF/spring.factories
moduleB
src/main/resources/META-INF/spring.factories
and so on...
I have found a solution for this question.
Note: This exact solution assume that you only used EnableAutoConfiguration in your spring.factiores, it would crash if you use more than one type of config inside spring.factories.
One can do:
src/main/resources/META-INF/spring.factories
src/main/resources/META-INF/spring-2.factories
src/main/resources/META-INF/spring-3.factories
src/main/resources/META-INF/spring-4.factories
and merge this into one file.
Note, I am using Maven Antrun but I suspect Gradle would also have a similar feature.
In your pom.xml, add the following:
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.8</version>
<executions>
<execution>
<id>default-ci</id>
<goals>
<goal>run</goal>
</goals>
<phase>process-resources</phase>
<configuration>
<target>
<replace token='org.springframework.boot.autoconfigure.EnableAutoConfiguration=' value=','
dir="${project.build.directory}/classes/META-INF">
<include name="spring-*.factories"/>
</replace>
<concat destfile="${project.build.directory}/classes/META-INF/spring.factories" overwrite="yes" append="yes">
<fileset dir="${project.build.directory}/classes/META-INF" includes="spring-*.factories" />
</concat>
</target>
</configuration>
</execution>
</executions>
</plugin>
And in spring.factories is the normal config:
# Auto Configure
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.boot.autoconfigure.admin.AConfiguration,\
org.springframework.boot.autoconfigure.admin.BConfiguration
But in spring-2.factories and others, you start with ,\ instead of the default statement:
spring-2.factories:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.boot.autoconfigure.admin.CConfiguration
spring-3.factories:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.boot.autoconfigure.admin.DConfiguration
After all that, the outcome spring.factories in your output class directories will be a very nice:
# Auto Configure
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
org.springframework.boot.autoconfigure.admin.AConfiguration,\
org.springframework.boot.autoconfigure.admin.BConfiguration,\
org.springframework.boot.autoconfigure.admin.CConfiguration,\
org.springframework.boot.autoconfigure.admin.DConfiguration

What's the best practice to put freemarker template files

Please refer to this thread about my current practice.
It worked well for a period of time and I thought all issues had been figured out. However, when I built the jar in different folder, "Template index.ftl not found" was thrown. I use jar xf xxx.jar to extract target jar and found *.ftl under templates folder has been compressed into that jar.
I tried solution here to add below configuration to pom.xml but it didn't work.
<plugin>
<!-- Build an executable JAR -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.1.0</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>libs/</classpathPrefix>
<mainClass>com.gearon.app.App</mainClass>
</manifest>
</archive>
<includes>
<include>**/*.class</include>
<include>**/*.jdo</include>
<include>**/*.properties</include>
<include>**/*.xml</include>
<include>**/*.ftl</include>
</includes>
</configuration>
</plugin>
The OP also said:
Better yet, I removed the configuration tag entirely, and it's still
working. I think it was a remnant from before I figured out that the
.properties files and other things I needed on the classpath needed to
be in src/main/resources and not src/main/java
I'd like to give it a try to put the templates/xxx.ftl to src/main/resources but not src/main/java/com/gearon/app/templates/*.ftl.
But the way to load template should be changed right? Currently, it's
cfg.setClassForTemplateLoading(Config.class, "/templates");
So here comes the question, how to change it? Or if my understanding above is totally wrong, what's the best practice to put templates into a jar and make sure the class into that jar can find those templates?
Maven committer here...
What you are doing is plain wrong, you are going against the conventions. It says that all resources which have to be available in the runtime classpath must reside in src/main/resouces. There is no further configuration necessary. I'd highly advise to do that. In your case: src/main/resources/templates and you are done. No <includes /> in the JAR Plugin necessary.

How to make a monolithic jar.file?

I need to create a hadoop job jar file that uses mahout and a bunch of other libraries. I need ti be able to run the job without needing additional jar.files such that all referenced classes are packaged with the resultant jar file. How can this be done?
Configure your build file to copy all the referenced classes to the build directory. For example, in ant:
<path id="classpathunjar">
<fileset dir="${lib.dir}" includes="*.jar" excludes="sqljdbc4.jar"/>
</path>
<target name="compile" depends="clean">
...
<unjar dest="${build.dir}">
<path refid="classpathunjar" />
</unjar>
...
</target>
But it is better if you can manage without doing this. Use the libjars feature to load the jars into all nodes if you are doing this for running mapreduce jobs on a hadoop cluster
Hadoop has the ability to read jars-in-jar. Amend you Ant script to include all the dependency jars in a folder called lib, and add this lib folder to your output Jar. This is sometimes a better choice if you have number of larger jars as it decreases your jar build time.
See this article on a number of options you have when using 3rd party libs with hadoop
http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/
Jar is just a Zip container.
You can manually unzip and modify your Jar file with the classes needed, or you can make use of e.g., the jar-with-dependencies descriptor of the Maven build system.
In the generic sense, it is sometimes impossible, as JAR files have resources that must be in particular locations, and two conflicting but necessary resources might prevent the combination (Think META-INF/MANIFEST.MF)
However, in many cases it is very easy. Basically you unzip the JAR file to be added (it is a zip file format) and "add" the classes and what-not to the existing JAR file.
A better choice if you are making an executable JAR file is to add a ClassPath entry into your launching MANIFEST.MF and ship both JAR files in a directory structure compatible with your added ClassPath entry.
Note that the additional jars have to be put under a lib/ subdirectory (Yes, jars within a jar). I use the following maven assembly, which I found somewhere else.
<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
<id>job</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<unpack>false</unpack>
<scope>runtime</scope>
<outputDirectory>lib</outputDirectory>
<excludes>
<exclude>org.apache.hadoop:hadoop-core</exclude>
<exclude>${artifact.groupId}:${artifact.artifactId}</exclude>
</excludes>
</dependencySet>
<dependencySet>
<unpack>false</unpack>
<scope>system</scope>
<outputDirectory>lib</outputDirectory>
<excludes>
<exclude>${artifact.groupId}:${artifact.artifactId}</exclude>
</excludes>
</dependencySet>
</dependencySets>
<fileSets>
<fileSet>
<directory>${basedir}/target/classes</directory>
<outputDirectory>/</outputDirectory>
<excludes>
<exclude>*.jar</exclude>
</excludes>
</fileSet>
</fileSets>
</assembly>

Use Maven just to fetch some library jars

We are using ANT for our build process and don't have plans to change this in the near future.
Is it possible to use Maven to just fetch common Open Source jar files (e.g. Log4J, SWT, JFace) and put them in the right location of our project, so we don't have to store them in our version control — preferable without creating the typical Maven-cache in the home directory?
NO NO NO Everyone!
If you're using Ant, the best way to use Maven repositories to download jar dependencies is to use Ivy with Ant. That's exactly what Ivy is for.
Installing Ivy and getting to work with current Ant projects is simple to do. It works with Nexus and Artifactory if you use those as your local Maven repositories.
Take a look at Ivy. It is probably exactly what you want.
In variation of org.life.java's answer, I would not do mvn install.
Instead, in the pom.xml I would add the following bit:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<version>2.1</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Now you just need to do mvn generate-sources, which is a lot faster than the full mvn install, and all dependencies will be copied to the specified directory.
Oh btw, isn't that what Apache Ivy is about? Extending Ant to understand Maven's dependency management?
It's possible, you should use maven-ant-tasks.
In particular its dependencies ant task. With this setup no Maven install is required.
<?xml version="1.0" encoding="UTF-8"?>
<project
name="download-dependency"
basedir="."
default="download-dependency"
xmlns:artifact="antlib:org.apache.maven.artifact.ant"
>
<target name="download-dependency">
... define properties ...
<taskdef
resource="org/apache/maven/artifact/ant/antlib.xml"
uri="antlib:org.apache.maven.artifact.ant"
/>
<artifact:dependencies>
<localRepository path="${local-repo.dir}"/>
<remoteRepository id="central" url="${repository-uri}"/>
<dependency
groupId="${groupId}"
artifactId="${artifactId}"
version="${version}"
type="${type}"
classifier="${classifier}"
scope="runtime"
/>
</artifact:dependencies>
</target>
</project>
The only binary you should check into your project is maven-ant-tasks.jar.
Actually in our project I used Sonatype Nexus ( documentation ) Maven repository manager to centralize access to different repositories and even maintain some binaries unique to our environment. With Nexus' help I just fetch maven-ant-tasks.jar with ant's <get> task from a known URL. You don't have to use Nexus, but it greatly speeds up builds, because it caches binaries close to your developer's machines.
Ivy does just this,
when it bootstraps itself:
http://ant.apache.org/ivy/history/latest-milestone/samples/build.xml
<property name="ivy.install.version" value="2.0.0-beta1"/>
<property name="ivy.jar.dir" value="lib"/>
<property name="ivy.jar.file" value="${ivy.jar.dir}/ivy.jar"/>
<target name="resolve" unless="skip.download">
<mkdir dir="${ivy.jar.dir}"/>
<echo message="installing ivy..."/>
<get src="http://repo1.maven.org/maven2/org/apache/ivy/ivy/${ivy.install.version}/ivy-${ivy.install.version}.jar" dest="${ivy.jar.file}" usetimestamp="true"/>
</target>

"Invalid signature file" when attempting to run a .jar

My java program is packaged in a jar file and makes use of an external jar library, bouncy castle. My code compiles fine, but running the jar leads to the following error:
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
I've googled for over an hour searching for an explanation and found very little of value. If anyone has seen this error before and could offer some help, I would be obliged.
For those who got this error when trying to create a shaded uber-jar with maven-shade-plugin, the solution is to exclude manifest signature files by adding the following lines to the plugin configuration:
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<!-- Additional configuration. -->
</configuration>
For those using gradle and trying to create and use a fat jar, the following syntax might help.
jar {
doFirst {
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
}
exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
}
Please use the following command
zip -d yourjar.jar 'META-INF/*.SF' 'META-INF/*.RSA' 'META-INF/*.DSA'
Some of your dependencies are likely signed jarfiles. When you combine them all into one big jarfile, the corresponding signature files are still present, and no longer match the "big combined" jarfile, so the runtime halts thinking the jar file has been tampered with (which it...has so to speak).
Assuming you're using ant, you can solve the problem by eliminating the signature files from your jarfile dependencies. Unfortunately, it's not possible to do this in one step in ant.
However, I was able to get this working with Ant in two steps, without specifically naming each jarfile dependency, by using:
<target name="jar" depends="compile" description="Create one big jarfile.">
<jar jarfile="${output.dir}/deps.jar">
<zipgroupfileset dir="jars">
<include name="**/*.jar" />
</zipgroupfileset>
</jar>
<sleep seconds="1" />
<jar jarfile="${output.dir}/myjar.jar" basedir="${classes.dir}">
<zipfileset src="${output.dir}/deps.jar" excludes="META-INF/*.SF" />
<manifest>
<attribute name="Main-Class" value="com.mycompany.MyMain" />
</manifest>
</jar>
</target>
The sleep element is supposed to prevent errors about files with modification dates in the future.
Other variations I found in the linked threads didn't work for me.
The solution listed here might provide a pointer.
Invalid signature file digest for Manifest main attributes
Bottom line :
It's probably best to keep the official jar as
is and just add it as a dependency in the manifest file for your
application jar file.
Security is already a tough topic, but I'm disappointed to see the most popular solution is to delete the security signatures. JCE requires these signatures. Maven shade explodes the BouncyCastle jar file which puts the signatures into META-INF, but the BouncyCastle signatures aren't valid for a new, uber-jar (only for the BC jar), and that's what causes the Invalid signature error in this thread.
Yes, excluding or deleting the signatures as suggested by #ruhsuzbaykus does indeed make the original error go away, but it can also lead to new, cryptic errors:
java.security.NoSuchAlgorithmException: PBEWithSHA256And256BitAES-CBC-BC SecretKeyFactory not available
By explicitly specifying where to find the algorithm like this:
SecretKeyFactory.getInstance("PBEWithSHA256And256BitAES-CBC-BC","BC");
I was able to get a different error:
java.security.NoSuchProviderException: JCE cannot authenticate the provider BC
JCE can't authenticate the provider because we've deleted the cryptographic signatures by following the suggestion elsewhere in this same thread.
The solution I found was the executable packer plugin that uses a jar-in-jar approach to preserve the BouncyCastle signature in a single, executable jar.
UPDATE:
Another way to do this (the correct way?) is to use Maven Jar signer. This allows you to keep using Maven shade without getting security errors. HOWEVER, you must have a code signing certificate (Oracle suggests searching for "Java Code Signing Certificate"). The POM config looks like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>org.bouncycastle:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>your.class.here</mainClass>
</transformer>
</transformers>
<shadedArtifactAttached>true</shadedArtifactAttached>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jarsigner-plugin</artifactId>
<version>1.4</version>
<executions>
<execution>
<id>sign</id>
<goals>
<goal>sign</goal>
</goals>
</execution>
<execution>
<id>verify</id>
<goals>
<goal>verify</goal>
</goals>
</execution>
</executions>
<configuration>
<keystore>/path/to/myKeystore</keystore>
<alias>myfirstkey</alias>
<storepass>111111</storepass>
<keypass>111111</keypass>
</configuration>
</plugin>
No, there's no way to get JCE to recognize a self-signed cert, so if you need to preserve the BouncyCastle certs, you have to either use the jar-in-jar plugin or get a JCE cert.
I had this problem when using IntelliJ IDEA 14.01.
I was able to fix it by:
File->Project Structure->Add New (Artifacts)->jar->From Modules With Dependencies on the Create Jar From Module Window:
Select you main class
JAR File from Libraries
Select copy to the output directory and link via manifest
I faced the same issue, after reference somewhere, it worked as below changing:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<configuration>
<createDependencyReducedPom>false</createDependencyReducedPom>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
Assuming you build your jar file with ant, you can just instruct ant to leave out the META-INF dir. This is a simplified version of my ant target:
<jar destfile="app.jar" basedir="${classes.dir}">
<zipfileset excludes="META-INF/**/*" src="${lib.dir}/bcprov-jdk16-145.jar"></zipfileset>
<manifest>
<attribute name="Main-Class" value="app.Main"/>
</manifest>
</jar>
I've recently started using IntelliJ on my projects. However, some of my colleagues still use Eclipse on the same projects. Today, I've got the very same error after executing the jar-file created by my IntelliJ. While all the solutions in here talking about almost the same thing, none of them worked for me easily (possibly because I don't use ANT, maven build gave me other errors which referred me to http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException, and also I couldn't figure out what are the signed jars by myself!)
Finally, this helped me
zip -d demoSampler.jar 'META-INF/*.SF' 'META-INF/*.RSA' 'META-INF/*SF'
Guess what's been removed from my jar file?!
deleting: META-INF/ECLIPSE_.SF
deleting: META-INF/ECLIPSE_.RSA
It seems that the issue was relevant to some eclipse-relevant files.
I had the same issue in gradle when creating a fat Jar; updating the build.gradle file with an exclude line corrected the problem.
jar {
from {
configurations.compile.collect {
it.isDirectory() ? it : zipTree(it)
}
}
exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
manifest {
attributes 'Main-Class': 'com.test.Main'
}
}
In case you're using gradle, here is a full farJar task:
version = '1.0'
//create a single Jar with all dependencies
task fatJar(type: Jar) {
manifest {
attributes 'Implementation-Title': 'Gradle Jar File Example',
'Implementation-Version': version,
'Main-Class': 'com.example.main'
}
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
exclude 'META-INF/*.RSA', 'META-INF/*.SF','META-INF/*.DSA'
with jar
}
Compare the folder META-INF in new jar with old jar (before you added new libraries). It is possibility that there will be new files. If yes, you can remove them. It should helps.
Regards,
999michal
A strategy would consist in using ANT to simplify the removal of the signature from each Jar file. It would proceed with the following steps:
Copying the MANIFEST.MF in a temporary file
Removing the Name and SHA entries from the temporary file
Creating a temporary Jar file with the temporary manifest
Removing the temporary manifest
Swapping the original Jar file with the temporary one
Here is an ANT macrodef doing the work:
<macrodef name="unsignjar" description="To unsign a specific Jar file">
<attribute name="jarfile"
description="The jar file to unsign" />
<sequential>
<!-- Copying to the temporary manifest file -->
<copy toFile="#{jarFile}_MANIFEST.tmp">
<resources>
<zipentry zipfile="#{jarFile}" name="META-INF/MANIFEST.MF"/>
</resources>
</copy>
<!-- Removing the Name and SHA entries from the temporary file -->
<replaceregexp file="#{jarFile}_MANIFEST.tmp" match="\nName:(.+?)\nSH" replace="SH" flags="gis" byline="false"/>
<replaceregexp file="#{jarFile}_MANIFEST.tmp" match="SHA(.*)" replace="" flags="gis" byline="false"/>
<!-- Creating a temporary Jar file with the temporary manifest -->
<jar jarfile="#{jarFile}.tmp"
manifest="#{jarFile}_MANIFEST.tmp">
<zipfileset src="#{jarFile}">
<include name="**"/>
<exclude name="META-INF/*.SF"/>
<exclude name="META-INF/*.DSA"/>
<exclude name="META-INF/*.RSA"/>
</zipfileset>
</jar>
<!-- Removing the temporary manifest -->
<delete file="#{jarFile}_MANIFEST.tmp" />
<!-- Swapping the original Jar file with the temporary one -->
<move file="#{jarFile}.tmp"
tofile="#{jarFile}"
overwrite="true" />
</sequential>
`
The definition can then be called this way in an ANT task:
<target name="unsignJar">
<unsignjar jarFile="org.test.myjartounsign.jar" />
</target>
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:314)
at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:268)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:316)
at java.util.jar.JarVerifier.update(JarVerifier.java:228)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
at java.util.jar.JarFile.getInputStream(JarFile.java:450)
at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:977)
at sun.misc.Resource.cachedInputStream(Resource.java:77)
at sun.misc.Resource.getByteBuffer(Resource.java:160)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:454)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
What helped me (IntelliJ IDEA 2016.3):
File -> Project Structure -> Artifacts -> Add JAR -> Select Main Class -> Choose "copy to the output directory and link via manifest" -> OK -> Apply -> Build -> Build Artifacts... -> Build
It's possible that two different signers mess up java mind.
Try removing META-INF folder from jar, adding manifest and signing JAR again, it helped me: http://jehy.ru/articles/2013/12/13/invalid-signature-file-digest-for-manifest-main-attributes/
If you are looking for a Fat JAR solution without unpacking or tampering with the original libraries but with a special JAR classloader, take a look at my project here.
Disclaimer: I did not write the code, just package it and publish it on Maven Central and describe in my read-me how to use it.
I personally use it for creating runnable uber JARs containing BouncyCastle dependencies. Maybe it is useful for you, too.
You can use Shadow to generate a jar.
Shadow is a Gradle plugin for combining a project's dependency classes and resources into a single output Jar. The combined Jar is often referred to a fat-jar or uber-jar.
Modify build.gradle
plugins {
...
// ① Add the shadow plugin
id "com.github.johnrengelman.shadow" version "5.2.0"
}
...
// ② Config the shadow jar, its name is baseName-1.0-classifier.jar
shadowJar {
archiveBaseName.set('baseName')
archiveClassifier.set('classifier')
archiveVersion.set('1.0')
manifest {
attributes 'Main-Class': 'Main'
}
}
// ③ Disable the default jar task
jar.enabled = false
// ④ Execute the shadowJar task when compiling
build.dependsOn(shadowJar)
Execute the command gradle build, the jar file will be generated:
<Project Directory>/build/libs/baseName-1.0-classifier.jar
For those who have trouble with the accepted solution, there is another way to exclude resource from shaded jar with DontIncludeResourceTransformer:
https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html#DontIncludeResourceTransformer
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.DontIncludeResourceTransformer">
<resource>BC1024KE.DSA</resource>
</transformer>
</transformers>
From Shade 3.0, this transformer accepts a list of resources. Before that you just need to use multiple transformer each with one resource.
This happened to me in Intellij when I clicked "Add as a Maven Project" on bottom line when Intellij said "non-managed pom files found.". Meanwhile out folder was already generated. So it did not get recent changes.
Deleting out folder and running the program solved the issue for me. out folder was then recreated.
See the answer of Little Fox as well. The error I received was very similar to his.
For someone who using kotlin script(kts) as building script:
task("fatJar", type = Jar::class) {
baseName = "${project.name}-fat"
manifest {
attributes["Implementation-Title"] = "Watcher Jar File"
attributes["Implementation-Version"] = version
attributes["Main-Class"] = "MainKt"
}
from(configurations.runtimeClasspath.get().map { if (it.isDirectory) it else zipTree(it) }) {
exclude(listOf("META-INF/*.RSA", "META-INF/*.SF", "META-INF/*.DSA"))
}
with(tasks.jar.get() as CopySpec)
}
I had a similar problem. The reason was that I was compiling using a JDK with a different JRE than the default one in my Windows box.
Using the correct java.exe solved my problem.
If you're getting this when trying to bind JAR files for a Xamarin.Android bindings project like so:
JARTOXML : warning J2XA006: missing class error was raised while reflecting com.your.class : Invalid signature file digest for Manifest main attributes
Just open the JAR files using Winzip and delete the meta-inf directories. Rebuild - job done

Categories

Resources