Develop Reference

Custom manifest added multiple times in -post-jar in a common dependency

I have a multi-module native Netbeans Java EE project. In it I have a Java Class Library project that is used by multiple other projects which in turn are packaged into the root .ear project.
I'm adding the "build timestamp" and the "build user" attributes to a custom manifest using the library's build.xml:
<target name="-post-jar">
<jar destfile="${dist.jar}" update="true">
<manifest>
When I "clean and build" the root project, each project that refers the library calls:
<ant antfile="${call.script}" target="jar">
And my -post-jar target is called multiple times. This wouldn't be a problem, but sometimes the second invocation of the <jar> task fails with Unable to rename old file (probably due to Netbeans scanning the files in background, but I can't tell for sure).
There are repeating pairs of Building jar and Updating jar messages in Ant's output. However, if I remove my -post-jar target, the second invocation of the jar target does nothing, because it thinks that the jar is up to date and I see only one Building jar message.
How do I mark the updated jar up to date, so the second invocation of the jar target does nothing?
There's a github repo that demonstrates the problem.

I haven't found a way to not re-generate the manifest every time, but I found a way to make the generated file look the same as the zipped file (and we know that the <jar> task doesn't repack when the contents are the same).
Instead of updating the zipped manifest in -post-jar I now update the source file in -pre-jar. This way the final version of the manifest is zipped and since its contents don't change during build, subsequent <jar> invocations update nothing.
It worth mentioning that before adding the attributes Main-Class, Profile, etc. the build-impl.xml of Netbeans creates an empy manifest template, if the user doesn't provide a valid path in the manifest.file= property. The addition happens after -pre-jar, however the existence of the user-provided manifest is checked much earlier, during the init target and the result is saved to the manifest.available property.
My manifest template is not a static file. It contains the "build timestamp" and the "build user" attributes. Therefore the file doesn't exist during the init target, so I had to add the following line at the beginning of my build.xml:
<property name="manifest.available" value="true"/><!-- It will be available by the time we need it -->
Secondly, manifest.file still has to be set and I set it in project.properties (there's no UI for that setting yet and I wonder how it would behave in the presence of the variable in path)
manifest.file=${build.dir}/manifest.tmp
Next, I overwrite the manifest template in the -pre-jar target:
<tstamp>
<format property="current.time" pattern="HH:mm:ss" locale="de,DE"/>
</tstamp>
<target name="-pre-jar" >
<manifest file="${manifest.file}">
<attribute name="MyApp-built-time" value="${current.time}"/>
<attribute name="MyApp-built-by" value="${user.name}"/>
</manifest>
After that, the new problem became obvious: the timestamp was different for each invocation of
<ant antfile="mylib/build.xml" target="jar">
in the multi-module project and Ant had to repack the jar with the new timestamp in the manifest. I solved this by defining the timestamp property in every project's build.xml. Although the properties are not inherited due to inheritall="false", Netbeans allows for overcoming that:
<property name="transfer.current.time" value="${current.time}"/>
This mechanism is broken in Java EE projects, but the workaround is simple:
<presetdef name="ant">
<!-- workaround transfer.* not working in Java EE projects -->
<ant>
<propertyset>
<propertyref prefix="transfer."/>
<mapper from="transfer.*" to="*" type="glob"/>
</propertyset>
</ant>
</presetdef>

How to include all dependent Jars within a single non-executable jar?

I am stuck in a very common problem.
I am plugging my jar (which has many dependencies on third party vendor) into an application server lib directory. If I just copy my jar along with its dependencies into server lib then server classpath becomes to long and hence server is not able to work. Therefore I want to package this Jar with all its dependencies in a single jar so that server's classpath doesn't become too long. I found on various forums that there is a utility to do this i.e. OneJar. But this utility works on executable jar. In my case, my final jar will not be executable.
Also I tried ZIPFileSetGroup utility provided by ANT but that is causing security issues with Manifest file.
Can you please help me in resolving this issue?
Thanks!

If you use Maven to build, you can use the maven dependency plugin and use the copy-dependency task. It will copy all dependencies into your jar file when it creates it.
If you manually add the jars to your jar file, then you need to make sure your jar file has a Manifest.mf file in it and specify the main class and classpath inside of that.
Manifest-Version: 1.0
Main-Class: com.mypackage.MainClass
Class-Path: my.jar log4j.jar
Another option may be to build an .ear file, that is usually how you see enterprise apps or a .war file for web apps when they package specific jar files with them. It sounds like you are using a server, so one of those formats may be a better fit for you.

Using zipgroupfileset in the jar task in ANT is the easiest approach.
<jar destfile="MyApplication.jar" filesetmanifest="mergewithoutmain">
<zipgroupfileset dir="lib" includes="*.jar" />
<!-- other options -->
<manifest>
<attribute name="Main-Class" value="Main.MainClass" />
</manifest>
</jar>
Note the filesetmanifest flag set to mergewithoutmain merges everything but the Main section of the manifests.
Signed jars are causing the SecurityException which need to be handled manually. If any classes associated with signed jars verify the signature on the jar as a whole then those will fail at runtime. Digest signatures against a particular file will be added to the manifest without a problem. Since problem is your classpath getting too large you may not be able to bundle all the jars into a single jar but merge most of them making the CLASSPATH manageable.
There is also : http://code.google.com/p/jarjar/
Create target directory with all dependent jars. Next move 10 jars into a temp directory and keep moving the jars in batches of 10 and each time try to create the single jar from that group. When you get the security exception you can isolate which one is causing the problem. Try divide-and-conquer approach. If you have 300 jars then only have to do this 30 times.

When you say
child process picks up classpath from server/lib directory
is this a process that is under your control? If the parent process were to specify the classpath just as
server/lib/*
(i.e. a literal *) then the target java process will enumerate the jar files in the lib directory itself - they do not all need to be named on the classpath.
But if the parent process is explicitly enumerating server/lib/*.jar to build the -cp value then you could take advantage of the fact that the Class-Path in a JAR manifest takes effect even if a JAR is not "executable". You could use a stanza like this to create a manifest-only JAR file
<!-- location of your 300 dependency JAR files, file1.jar ... file300.jar -->
<property name="lib.dir" location="lib" />
<fileset id="dependencies" dir="${lib.dir}" includes="*.jar" />
<pathconvert property="manifest.classpath" dirsep="/" pathsep=" "
refid="dependencies">
<map from="${lib.dir}" to="myapp" />
</pathconvert>
<jar destfile="myapp-manifest.jar">
<manifest>
<attribute name="Class-Path" value="${manifest.classpath}" />
</manifest>
</jar>
This will produce a JAR file named myapp-manifest.jar whose manifest contains
Class-Path: myapp/file1.jar myapp/file2.jar ... myapp/file300.jar
You put this file into server/lib and the 300 dependencies into a new directory server/lib/myapp. Now the generated -cp will include just one file (myapp-manifest.jar) but the resulting java process will have all the 300 myapp JAR files available to it.

JAR classpath and external jars

I actualy have 2 problems
I use eclipse -> export project to generate a jar file for my simple desktop (GUI) program
It generates a jar file and an ant script.
first problem:
the generated jar works fine when double-clicked.
When I use the generated ant script to generate the jar
by myself, it doesn't work.
What can be wrong with a target like this (assuming that all dependencies are met)
<target name="create_run_jar">
<jar destfile="G:/dev/myproj/myproj.jar">
<manifest>
<attribute name="Main-Class" value="view.myproj"/>
<attribute name="Class-Path" value=". myproj_lib/grouplayout.jar"/>
</manifest>
<fileset dir="G:/dev/myproj/bin"/>
</jar>
<delete dir="G:/dev/myproj/myproj_lib"/>
<mkdir dir="G:/dev/myproj/myproj_lib"/>
<copy file="G:/dev/.metadata/.plugins/org.dyno.visual.swing/layoutext/grouplayout.jar" todir="G:/dev/myproj/myproj"/>
</target>
//nevemind
//Second problem:
//when I double click on the auto-generated jar file the program launches and works fine.
//when I do java myjar from the command-line I get main class not found exception..
//weird huh?

I suggest that you take the JAR files generated the two ways, use the jar command to expand them into temporary directories, and then use diff in recursive mode to compare them.
However, I suspect that #Pace has put his finger on the problem; i.e. that you are using relative paths in the Class-Path manifest entry and this is liable to cause problems.

java -jar <jar name> is the proper way to execute a jar.

The ant target is creating a manifest with a classpath attribute. If you look at those paths you'll notice that they are relative to the current directory. When you execute java -jar from the command line are you in the...
G:/dev/myproj
...directory?

How to combine two Jar files

Is it possible to combine two jar files such that in an applet tag I can simply do something like
archive="jarjar.jar/jar1.jar"... ...archive="jarjar.jar/jar2.jar"... instead of
archive="jar1.jar"... ...archive="jar2.jar"...
I need to only have one jar file so putting two jar files in a folder will not help me.

Sure, just extract the two jar files and recreate a new one
$ mkdir tmp
$ (cd tmp; unzip -uo ../jar1.jar)
$ (cd tmp; unzip -uo ../jar2.jar)
$ jar -cvf combined.jar -C tmp .
The stuff with tmp ensures that the two existing jars are extracted into a clean directory and then the new one made from that.
Be aware that you may also need to merge any manifest.mf files contained therein, and if there are any also include the '-m' option in that file command.

Use zipgroupfileset with the Ant Zip task
<zip destfile="out.jar">
<zipgroupfileset dir="lib" includes="*.jar"/>
</zip>
Might help you.

If you are using gradle, just add the following to build.gradle. No plugins required. If you need special options, then go with Fatjar plugin, as initialZero suggests.
task superSimpleJar(type: Jar) {
baseName = project.name + '-all'
from { configurations.compile.collect { it.isDirectory() ? it : zipTree(it) } }
with jar
}
For Android project, add this to app/build.gradle and run "gradlew superSimpleJar". Find jar in build/libs/app-all.jar
task superSimpleJar(type: Jar) {
baseName = project.name + '-all'
from {
configurations.compile.findAll {
it.getName() != 'android.jar'
}.collect {
it.isDirectory() ? it : zipTree(it)
}
}
}

<?xml version="1.0" encoding="UTF-8"?>
<project name="zip-test" default="zip" basedir=".">
<target name="zip">
<zip destfile="out.jar">
<zipgroupfileset dir="." includes="*.jar"/>
</zip>
</target>
</project>
save this code in build.xml file and keep it in same folder where all the jar files to be combined are kept. Open cmd, give path of folder and run command : ant zip.
It will generate out.jar which is combination of all jars.

Just unzip both jar files, then zip the results into one zip file, and rename this to jar again.
But as adarshr said: Better use the jar command for that.

Extract both jars and create a new one works. (Use jar commands shown above).
One caveat about manifest file is that you want to extract the jar whose manifest file you want to retain in the last.

I know it's an old question and I just wanted to add my two cents (no permission to comment yet, so creating a new answer).
I do see the value in sumanth.donthula's answer as the problem for all of us merging jars will be how to deal with the manifest files. In my case I wanted to merge some library files (mainly generated web service client code) into the jar of an application written by me. It was OK to replace the manifests with the one of my own jar.
The simplest way of doing this is taking care of the order in which you unzip the original files (as Alnitak and sumanth.donthula noted).
I wanted to use the zip ant task (thank you, ykombinator, for the idea). It turned out that the only way of controlling the order of compressing/packaging is renaming the files. See my ant target below.
The output directory in my example is called codemodule.dir (I created a FileNet code module). The rest of the names are self-explaining. The important step is renaming the application jar to 0_... to be the 1st in order. This way its manifest will be retained as the duplicate attribute of the zip ant task is set to preserve.
<target name="merge_jars">
<delete dir="${codemodule.dir}" quiet="true" />
<mkdir dir="${codemodule.dir}" />
<copy todir="${codemodule.dir}">
<fileset dir="${lib.dir}" includes="*.jar"/>
<fileset dir="${basedir}" includes="${app-name}.jar"/>
</copy>
<move file="${codemodule.dir}/${app-name}.jar" tofile="${codemodule.dir}/0_${app-name}.jar"/>
<zip destfile="${codemodule.dir}/${app-name}-fat.jar" duplicate="preserve">
<zipgroupfileset dir="${codemodule.dir}">
<include name="*.jar"/>
</zipgroupfileset>
</zip>

java.lang.ClassNotFoundException when running java -jar

I'm using ant to build my build.xml file, it compiles ok, but then getting a runtime java.lang.NoClassDefFoundError when running the resulting jar via "java -jar my_jar.jar". It seems like this comes up a lot but none of the related questions' solutions worked for me.
My classpath for javac contains only "/usr/local/lib/libthrift.jar" and the main .java file imports a bunch of thrift packages such as org.apache.thrift.transport.TTransportException.
When I try running the program via:
java -jar MyClass.jar
, I get the error:
Exception in thread "main" **java.lang.NoClassDefFoundError**: org/apache/thrift/transport/TTransportException
Caused by: java.lang.ClassNotFoundException: org.apache.thrift.transport.TTransportException
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
Could not find the main class: **MyClass**. Program will exit.
Here are the things I've tried so far that don't work:
adding a flag on the command line like "java -cp /usr/local/lib/libthrift.jar -jar my_jar.jar", the result is the same error as above
adding <attribute name="Class-Path" value="./:/usr/local/lib/libthrift.jar"/> inside my jar's manifest> tag, the result is the same error as above
adding -Xbootclasspath/a:/usr/local/lib/libthrift.jar:./ to the java command line. it solves the first error but a different error comes up:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/log4j/Logger
at org.apache.thrift.transport.TServerSocket.<clinit>(TServerSocket.java:36)
at MyClass.start(Unknown Source)
at MyClass.main(Unknown Source)
EDIT:
If I comment out the code that instantiates the missing classes but leave the imports, the code executes fine.
EDIT:
I moved my java classes to a server and referenced the MainClass with the server in the manifest attribute, but that didn't fix anything.

Could not find the main class: MyClass
The error seems actually related to your MANIFEST which:
may not have a complete classpath Class-Path: see this HowTo
The best solution when you have a jar is to try to include the required jars into the manifest declaration.
Manifest-Version: 1.0
Class-Path:
customer_client.jar
mailer_client.jar
signon_client.jar
or may not define adequately the MainClass within your 'my_jar.jar'.
See this HowTo:
<target name="jar" depends="compile">
<delete file="hello.jar"/>
<delete file="MANIFEST.MF"/>
<manifest file="MANIFEST.MF">
<attribute name="Built-By" value="${user.name}"/>
<attribute name="Main-Class" value="howto.Hello"/>
</manifest>
<jar destfile="hello.jar"
basedir="."
includes="**/*.class"
manifest="MANIFEST.MF"
/>
</target>
the <attribute name="Main-Class" value="howto.Hello"/> needs to specify the full path (packages) of the MainClass, not just MainClass.
If your main class is in the default package (the unnamed package), I am not sure it can be referenced by the loader (see this SO question)
So move your JarRunner into a package, and declare it appropriately in the <attribute name="Main-Class" value="myPackage.JarRunner"/> element.

You need to specify all the other jars that are required in your classpath in the manifest file before you can execute java -jar my-test.jar, here is a copy of one of my manifest files. With all these entries in the manifest I can specify java -jar db_field_cleaner.jar and all the other jars are inlined into the classpath :
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: James B
Build-Jdk: 1.6.0_01
Package: com.blah.dbfieldcleaner
Specification-Title: db_field_cleaner
Specification-Version: 2.5.7-SNAPSHOT
Implementation-Title: db_field_cleaner
Implementation-Version: 2.5.7-SNAPSHOT
Implementation-Vendor-Id: com.blah.dbfieldcleaner
Implementation-Vendor:
Main-Class: com.blah.dbfieldcleaner.main.Main
mode: development
url: ..\..\db_field_cleaner\target\site
Class-Path: log4j-1.2.14.jar cygna_commons-2.5.7-SNAPSHOT.jar mail-1.4
.jar activation-1.1.jar jdic-0.9.5.jar jdic_native-0.9.5.jar jdic_plu
s-0.2.2.jar jdic_plus_native-0.2.2.jar jtds-1.2.2.jar xstream-1.3.1.j
ar xpp3_min-1.1.4c.jar commons-net-2.0.jar text_processing-2.5.7-SNAP
SHOT.jar
Alternatively, use Maven, it's loads better at this kind of stuff!

You had given answer yourself :-) add all the jars to your runtime classpath.As you said earlier *.jar solved one problem but loggers are not able to find out, so add log4j.jar to the path. Basically the idea is add all the jars required for running in to classpath.

The command line options for java can be found here.
The -jar and -cp/-classpath options are mutually exclusive. The -jar option requires the use of a manifest and the relative paths to dependencies should be listed in this file. But essentially, the manifest is an optional mechanism - you can specify the required information externally at bootstrap time. If the manifest is causing you problems, don't use one.
I would test that you have you have located all your dependencies with a command like this:
java -cp /usr/local/lib/libthrift.jar:my_jar.jar MyClass
Note that the compiler may successfully compile your classes even if all the classes that might be required at runtime are not present. Compilation will succeed if the direct dependencies of your classes are present. The dependencies of your dependencies are not necessary to create the binary and the compiler will not inspect them needlessly.
The message about org/apache/log4j/Logger suggests that you have a missing dependency on log4j. It will be necessary to add this library to the classpath. Check the documentation for the Thrift library to determine its dependencies.

The class path references in the manifest file are relative refs. Just to debug, you might want to copy all the jars into the same location as my_jar.jar and attempt it again.
reference :
http://www.rgagnon.com/javadetails/java-0587.html

You might try adding the jars to the domain of the server. I had a similar problem and this worked for me when I was running it on glassfish. I would get those not found exceptions. Eclipse recognized it and it compiled fine but when ran on the server it couldn't find the file. Try adding it to whatever lib directory the server is installed to.

This is the problem that is occurring,
if the JAR file was loaded from "C:\java\apps\appli.jar", and your manifest file has the Class-Path: reference "lib/other.jar", the class loader will look in "C:\java\apps\lib\" for "other.jar".  It won't look at the JAR file entry "lib/other.jar".
Solution:-
Right click on project, Select Export.
Select Java Folder and in it select Runnable JAR File instead of JAR file.
Select the proper options and in the Library Handling section select the 3rd option i.e. (Copy required libraries into a sub-folder next to the generated JAR).
Click finish and your JAR is created at the specified position along with a folder that contains the JARS mentioned in the manifest file.
open the terminal,give the proper path to your jar and run it using this command java -jar abc.jar
Now what will happen is the class loader will look in the correct folder for the referenced JARS since now they are present in the same folder that contains your app JAR..There is no "java.lang.NoClassDefFoundError" exception thrown now.
This worked for me... Hope it works you too!!!