Develop Reference

Add Native Shared Library (.so or .dll/.lib) to Jar File Using Maven Plugin

There are lots of posts about this already but I have not found one that describes my exact situation. Which is:
I have a preexisting build system that generates a C++ shared library in both .dll and .so format (depends on OS).
The build system also produces a set of java files generated by swig.
I also have a pom.xml file that builds the java source into a package.
What I need is a plugin for maven that simply copies the native shared library into the jar. It would be nice if the native shared library was also loadable from within the jar so clients don't have to manually add it to the library path. This seems like a problem that has been solved before.
I have looked at maven-dependency-plugin. Which seems to be able to copy artifacts that are in a maven repository (not my case). Or if you use an assembly it might be possible. However, if it is possible, it seems overly complicated for what I want to accomplish. I would like to accomplish it with only a pom.xml.
I have also looked at nar-maven-plugin, but this seems like it focuses only on building the native library and adding it to a .nar file. This is not what I need since I can already build the C++ library.
Finally, I looked at one-jar which I got to place the native library inside the jar. However, I was not able to import the classes from within that jar, and the clients of this library will expect to be able to do that.
If what I want is possible and I don't need to install the shared library into my local repository please provide a link or an example of how you accomplished it. If it is not possible, please state why.

Java how to find out required libaries download them and then run the program?

After using eclipse i decided to ask this what bugs me the most. When exporting a project to working .jar you have 3 options. Because using extract or package required libraries into jars gives my program a lot of space to have than 3 options, copy required libraries to directory. Now my project has only about 70 kb than 15 mb before. Actually i didn't recognize that not possessing folder with my library classes cause some issues. Why?
I heard from a friend of mine that some thing were not working for him and i send him only jar, than lib.
Is there a difference between exporting types and machines( i poses jdk my friend not only jre)?
Is there a way to check for required essentials libraries or class that are required, check on system and order to download them? Or just download the external lib folder. The most i am interested in detecting missing classes or libraries.

You would have to learn some things but you may want to try using Maven projects. Maven is a Java building tool. One of Maven's most important features is transitive dependencies.

Trying to remotely compile, using the command line, a Java program with multiple dependencies that I can currently only compile locally in Eclipse

Some weeks ago at work I took over a Java-based back-end web application written using Eclipse. The nature of the application is that it cannot be adequately tested locally, and instead changes need to be tested on our testing network (which involves pushing the changes to an AWS Micro server that we connect to via SSH).
Until now, I pushed changes in the same way as my predecessor: compile the program using Eclipse's Export to Runnable JAR File option, then scp the jar to the remote server and run it. However, this process has a huge problem. While compilation takes only seconds, the jar is well over 30MB, and pushing the entire thing from the office to the remote server over our fairly ordinary internet connection takes well over 10 minutes. If I'm having a particularly bad day and, say, introduce several minor bugs to the code and then discover them one by one, I can easily end up losing an hour or more in total twiddling my thumbs while pushing the whole jar over and over for a series of one-line changes.
Clearly, a saner solution than scping the entire jar for every change would be to simply remotely pull only the changed .java files from source control, and then compile the new version entirely remotely. However, I'm quite new to Java (and indeed programming generally) and all my Java work has been on existing Eclipse projects that I've taken over partway through development. As such, I know very little about compiling Java, and I have found the tutorials about this online are mostly either opaque or completely fail to address the question of how to compile code that uses external libraries.
I will relate here what information about the project's dependencies I can find from Eclipse, and my questions are these: what do I need to copy to the remote server, and where do I need to put it, to be able to compile remotely? What tools, if any, do I need to install on the remote server to be able to compile there? And once I've got everything set up, what do I actually type at the command line to get it to compile?
Anyway, here's what I know about the dependencies and directory structure (I've anonymised our application name by calling it “bunnies”):
The application source code is located in bunnies/src
We compile to bunnies/bin/main.jar
bunnies/dependencies contains three jars of external libraries that we use.
Right-clicking on the project in Eclipse, going to the Java Build Path section, and selecting the Libraries tab, I see
the three libraries above
(appearing in the form, e.g. “json-simple-1.1.1.jar - /home/mark/workspace/bunnies/dependencies”)
a fourth jar file in another location
(“M2_REPO/com/google/guava/guava/r09/guava-r09.jar - /home/mark/.m2/repository/com/google/guava/guava/r09/guava-r09.jar”)
JRE System Library [java-6-openjdk-i386]
But there's more! We also use two libraries, mahout-core and mahout-integration, that are included as separate projects in the same workspace rather than as jar files in the dependencies folder. They appear by name on the Projects tab of the Java Build Path section of the bunnies project, and are located at /home/mark/workspace/mahout-core and /home/mark/workspace/mahout-integration respectively.
Since I am not a Java whiz, perhaps there are also some other hidden dependencies I'm missing, that don't appear in any of the places I've looked so far?
If anyone can walk me through the steps of compiling this huge mess from the command line, without needing to use the Export option in Eclipse, so that I can ultimately compile it all remotely, I would be highly appreciative.

Look into Apache Ant. It's a build-suite for Java, sort of like an XML based Makefile system.
I have a Java system running on a remote server. I have a directory structure separated into /src and /build. I then just scp the .java files from my local machine to the /src folder and build using ant.

Eclipse (java) web services, what files to version and how to properly pull the project to a new computer?

Alright, so I have a web service that was created using an eclipse dynamic web project. It is currently shared on a CVS repository, but the versioning system used is irrelevant. At the moment, I have literally NEVER been able to pull this project out as is and get it working. It leads to countless errors that cannot be fixed. Every time I need to work on this webservice in a new machine I have to create an entirely new dynamic project, copy over the source files, add all the necessary libraries and make the deployment assembly work correctly again. After finally making it run I share the project as the same one, stop after a second, and then synchronize again (in a way tricking eclipse into thinking this was the shared project all along).
I feel like others must have run into this problem and found a way around it. So if you have a web service or any dynamic web project, what files do you share, and how do you successfully pull it from the repository and get it to run on another machine besides what I currently do now?
Your help is much appreciated,
-Asaf
Edit: After reading some of the responses I feel that this question is actually more specific to those who use WTP to create/test their web services. Just wanted to add the clarification.
Edit2: Let me also clarify that the other 20 or so projects not using WTP are shared just fine. I am able to pull and run them with no problem. Only web service projects are an issue.

In general, you want to check in everything that's not "derived" (generated or compiled - that's usually the contents of the bin directory or other place where your code is compiled/built into). For Eclipse Java projects, you want to include the .project, .classpath, .settings, and any other similar files that Web Tools might create for Dynamic Web projects. The Eclipse CVS client will ignore files marked as Derived so you shouldn't have to worry to much about it.
Without more detail about what kind of problems you've run into, it's not possible to guess what was causing them. My only guess is that perhaps you had different versions of Eclipse and/or the WTP (Web Tools Platform) plugins installed on the different machine. That's just a wild guess, but could explain some incompatibility when you check out the project from CVS.
Bottom line, checking in those .* files is the long recommended approach from Eclipse gurus. Maven can kind of change things, but you didn't mention it so I'm assuming you aren't using it.

I am primarily sharing my experience, may be you can find some help.
Conceptually speaking, the files which the IDE can generate itself while creating new project should not be pushed. I.e the IDE specific files should not be pushed. And everything which the IDE cannot generate on its own must be pushed.
Forexample in case of eclipse, following files should not be pushed:
.settings
build
.classpath
.project
For setting the project on new machine, first pull the files from server, and then create a project from IDE using pulled files.
EDIT: If your project has external jars/libraries, then you will have to add to the classpath manually. You could also push .classpath but that might give errors while creating a new project.

I think it's easiest to use a build system and let the IDE generate the project from your build system.
Eclipse, Netbeans, and Intellij are all pretty good at building projects from maven or ant build files. With this solution you have a simple build that is easy to setup in CI (Hudson, Bamboo, whatever) and you don't have any IDE specific files checked in. If my workspace is totally different than yours, with different versions, plugins, whatever, I'm not stuck with your project file and you're not stuck with mine. My IDE creates the project appropriate for my environment and your IDE does the same for yours.

Since you mentioned having to manually add libraries, I assume you are not using any build manager (like, maven or ant) besides ecplise.
For ecplise to handle the project properly you need the source files (*.java) in their respective directories, any resources bundled with the web service (e.g. services.xml), the ".project", ".classpath", ".settings", etc. files for eclipse. This should be enough for eclipse to generate anything else necessary to build the project.
Any files/directories that are generated by eclipse during the build process (e.g. target & bin directory, *.class, *.war) should not be checked in -- they will be generated when needed during the build.
I am thinking that, since you are adding the necessary 3rd-party jars manually, these libraries might reside in a different path between computers (e.g. if the path contains the username, it will not be transferable to another computer for a different user). To fix that you can set up the classpath using an eclipse classpath variable. In Preferences->Java->Build Path->Classpath Variables set up a varable linked to the "root" folder where the 3rd party jars a stored. Then add the libraries to the project using this new variable, not their full path. To make it work on someone else's computer, you would only need to set this classpath variable to have the build path point to the correct libraries.
It might be beneficial if you migrated your project from eclipse only to a build manager (e.g. maven) that takes care of many of these issues for you. Eclipse can build a project from the configuration of the build manager, making it easier to manage the project.

Whats best way to package a Java Application with lots of dependencies?

I'm writing a java app using eclipse which references a few external jars and requires some config files to be user accessable.
What is the best way to package it up for deployment?
My understanding is that you cant put Jars inside another jar file, is this correct?
Can I keep my config files out of the jars and still reference them in the code? Or should the path to the config file be a command line argument?
Are there any third party plugins for eclipse to help make this easier? I'm using an ant build file at the moment but I'm not sure I know what I'm doing.
Is there an equivelent of the deployment projects in Visual studio, that will figure out everything you need and just make an installer? I've used install4j before, and it was powerful if no where near as automated as .Net deployment projects.
Cheers.

There is no one 'best way'. It depends on whether you are deploying a swing application, webstart, applet, library or web application. Each is different.
On point 2, you are correct. Jar files cannot contain other jar files. (Well, technically they can, its just that the inner jar file won't be on your classpath, effectively meaning that jar files do not contain jar files).
On point 3, you certainly can reference config files outside the jar file. You can typically reference a config file as a file or a resource. If you use the resource approach, it typically comes from the classpath (can be in a jar). If you use a file, then you specify the filename (not in a jar).
In general, most Java developers would use Apache Ant to achieve deployment. Its well documented, so take a look.

(1) An alternative to ant that you may wish to consider is maven.
A brief intro to maven can be found here.
For building a JAR, maven has the jar plugin, which can automate the process of ensuring all dependent jars are listed in your jar's manifest.
If you're using eclipse, then download the maven integration as well.
(2) Another alternative is to use OneJar (disclaimer: haven't tried this myself).

The answers vary depending on what kind of thing you're building.
If you're building a library, it's best to distribute your work as a jar file. It's possible to refer to your jar dependencies via the Class-path attribute in your jar manifest, although I generally think that's uncool. That attribute was designed for applets and it's used infrequently enough in libs that when this technique pulls stuff into the classpath (particularly common stuff the user might already be using), you can get unexpected version conflicts. And it's hard to track down why you're seeing them.
Publishing a jar to a Maven repo with pom info to track dependencies is an excellent choice for libraries as well. If you do that, please publish your Maven coordinates in your docs!
If you're building an app, the two popular choices are to distribute a zip/tar/whatever of a deployment structure OR to use an installer program. If the program is a server-ish kind of thing, the former is far more common. The latter is more common for clients. Generally, the installer program is just going to lay out the deployment structure and maybe do some extra tasks like installing in OS-specific locations.
To build your deployment structure (aka "kit") you'll want to create a repeatable process in whatever build system you're using. Ant has copious examples of this and Maven has the assembly plugins that can help. Generally you'll want to include a jar of your code, any dependencies, scripts to start the program, maybe a JRE, and any other resources you might need.
If you want to create an installer, there are many options both free and commercial. Some folks I know have recently had good experiences with the free IzPack but check out your options.

You should try FatJar. It's an Eclipse plugin that with just a right click at the Project can build a JAR file with all you need to run the application, including the necesary third party JAR.
We use it everyday, in conjuction with JSmooth to create the executables, to deploy our software packages to our customers, and works like a charm.

Well, if you are speaking of deployment of a standalone desktop application:
Before we switched to web start we have been creating three deployment archives, one for windows, one for mac and one for other platforms.
On windows we have successfully used the Nullsoft Scriptable Install System (known for it's usage by the older winamp versions) and its ant task, although some drawbacks are:
It is only usable on windows AFAIR
You have to do some work by hand, i.e. customizing the wizard-created script AFAIR
It can create a windows installation with start menu entries on the other hand. There also exists an eclipse plugin for integrated NSIS shell script editing.
On Mac OS X there is an ant task to create an .app file from your java files so that you can start it like a native os x application. But beware of not writing any setting to your home dir and using the the application dir instead.
For others you have to expect they are in a un*x env and deploy your app with a shell script to start the application.
In any case you may have to deploy your custom policy file to get access rights for your application.
If you want to get rid of all the packaging and stuff you should seriously consider using web start. We have saved much time since switching to it, i.e. simplified our deployment process, takes care of updates etc.
Update 2014
Use maven assembly plugin, see section "Creating an executable jar"

Ant. It's not the best thing in the world, but it's standard, it's apache, and it works.
There's some good examples on the web how to make a simple build.xml for any, and it's got some features like the 'war' task that knows how to put all the basic stuff (classes, web.xml etc) in the jar file for you.
You can also tell it to pick up other config files and jars and it will happily do it.
It's also really smart about what to compile. You give it a directory, and it finds all the java files and builds them only if their classfile is out of date, so you get some of the traditional make functionality for free without much effort.

You could look at other java projects (e.g. JMeter, SquirrelSQL, JEdit, Cernunnos, etc.). Each package their applications slightly differently, so consider your goals when you review these.