I have a Java program that is calling C code through JNI that I'm attempting to run on Linux. The external code consists of two .so files: one for the JNI bindings (built with swig) and the other with the actual functions. I have the two libraries in the same directory and LD_LIBRARY_PATH is set correctly. ldd reports no problems when running from the command line, but when I set the LD_LIBRARY_PATH to the same value in the "run configurations" dialog in the Eclipse editor and attempt to execute the program, it gets the following error:
java.lang.UnsatisfiedLinkError: [path to libraries]/[JNI binding library].so: [actual code library].so: cannot open shared object file: No such file or directory
This leads me to believe that the JNI wrapper library is loaded successfully, but there is a failure when that library attempts to load the library containing the actual code. Is there any way to debug this further?
I will further note that this problem is happening in the eclipse editor itself and that I haven't attempted to package the code into a jar and run it within a free-standing jvm instance.
I think the issue is with the call to System.loadLibrary(String) and using LD_LIBRARY_PATH. Using loadLibrary("foo") will look in your java.library.path for something named libfoo.so. If nothing named libfoo.so is found you will get this error.
Now if you just set up the LD_LIBRARY_PATH, the native symbols you want will automatically be picked up by the linker, so you don't need to set up -Djava.library.path.
In my experience with swig in the gdal project, this error is actually harmless and since the LD_LIBRARY_PATH is set up, this will work fine.
I would recommend using -Djava.library.path and calling loadLibrary explitly, the reason being that if you ever decide to deploy your app with webstart, you will explicitly need to call loadLibrary to get your native libs picked up.
When I use eclipse I follow the instructions that Daff gave where you edit the native library under the jar in the Libraries tab in the Build Path. Just to mention again, this just sets java.library.path under the covers.
It may be that you just have to find the right place on the run config dialog to put the -Djava.library.path=... option. Offhand I think you want -D defines in the "vm arguments" on the arguments tab, whereas if you want to define LD_LIBRARY_PATH that goes on the environment tab. Eclipse will merrily let you put things in places where they won't mean what you think they mean. Anyway, I've used libraries this way before and if I get a chance I will look up what I did and edit my answer here.
Another thing to try is to play with LD_DEBUG. You can set the environment variable LD_DEBUG to various things (try ALL), and then the linux loader will divulge all sorts of useful information about what an application is trying to load, where it's looking for things, etc. Of course, this pre-supposes you launch eclipse from a command line, so you can both set the env vars and see the loader diagnostics; but as far as the system is concerned, when you run your app from inside eclipse, your app is just something eclipse is doing, so any library loading behavior can be seen in this way.
You could try -Djava.library.path=actual.so in command line parameters perhaps?
On windows, I had similar problems with a 3rd party library, which used a JNI wrapper DLL for its DLLs. My project had the DLL in the lib directory so I added lib to the PATH (e.g. PATH=%PATH%;./lib environment variable and everything started working.
As far as I know the Eclipse doesn't use the LD_LIBRARY_PATH.
The easiest way to set up the right native library path is to go to
Project properties -> Java Build Path -> Libraries
Then expand either the JRE System Library entry or (if available) the
Jar File that uses you native Library,
choose "Native Library Location" then click "Edit..." and choose the folder your libraries are in. Actually it does set the -Djava.library.path variable so you have to include this in your command line if you start your program from outside eclipse.
Are there any other libraries that your two libraries depend on? If so, you need to make sure they are also accessible to the JVM.
Be aware, manually setting "-Djava.library.path" seems to erase the default library path.
So with the following code:
public class LibTest {
public static void main(String[] args) {
String property = System.getProperty("java.library.path");
StringTokenizer parser = new StringTokenizer(property, ":");
while (parser.hasMoreTokens()) {
System.err.println(parser.nextToken());
}
}
}
Launched from eclipse with Java 1.6.0_14 outputs:
/opt/java/jre/lib/i386/client
/opt/java/jre/lib/i386
/opt/java/jre/../lib/i386
/opt/java/jre/lib/i386/client
/opt/java/jre/lib/i386
/usr/lib/xulrunner-devel-1.9.0.11
/usr/lib/xulrunner-devel-1.9.0.11
/usr/java/packages/lib/i386
/lib
/usr/lib
But when I set the JVM arg "-Djava.library.path=/tmp/" I only get:
/tmp/
If you are manually setting java.library.path this may explain why ldd works from the command line but your .so does not from eclipse/java.
You can try not setting java.library.path and use System.load with the absolute path to your library instead of calling System.loadLibrary. This may allow the JVM to find your .so and still use the default path when searching for its dependencies.
Of course, if this is no use then you can also try turning on jni debug output with "-verbose:jni" on the command line. This may give you some clues to the problem.
Yes the LD_LIBRARY_PATH worked for me
Adding this answer may be it can be useful In AIX Machines we need to setup LIBPATH environment variable instead of LD_LIBRARY_PATH.
Related
The Java application has the JNI module to use.
Where should a user (or an installation script of this application) put the JNI module on Linux (Ubuntu) or on MacOS X so that this JNI module could be loaded without specifying the path to the module in code?
This is a link to a detailed explanation of shared objects and how they are searched for by the OS.
I wish Java people would stop using LD_LIBRARY_PATH and start using the existing directory structures and the ld.so.conf mechanism. Even the OpenJDK libraries are dumped in a place that's not on a standard path and they don't add an ld.so.conf file either ( just how hard is that ? ).
This approach avoids the need to set up your own LD_LIBRARY_PATH and launch via a shell script.
If a required shared object is to be installed, first test for somewhere like /usr/local/lib as an installation choice system wide, and if it exists and an existing file does not already use your file's name, then put your library there. A more systematic approach would be to check all the ld.so.conf files and see if any of the directories match something you know can be used. A shell script can do that at install time.
Put the compiled libraries (.so files on Linux or .dylib on MacOS) into a directory of your choice and include this directory in the library search path LD_LIBRARY_PATH used to start your JVM.
I've got a Java project that uses several native DLLs but only uses System.load() to load one of them, and it's dependent on the others. I'm not allowed to modify that code. System.load() looks in java.library.path, which I can set via a command line parameter, but according to the top answer in Java JNI and dependent libraries on Windows the loading of dependent DLLs is done by Windows, which only cares about PATH, not java.library.path. So I'd need to add the directory that has the DLLs to PATH, and it'd be lovely to do that via command line parameters as well. Can it be done this way?
Edit for clarification: I'm running my code from Eclipse as a jUnit plug-in test, and I'm trying to figure out how to change the run configuration to get this effect. I changed java.library.path by adding the following to the VM arguments box in the Arguments tab:
-Djava.library.path="D:/prototype/resources/nativelib/x64;${system_property:java.library.path}"
D:/prototype/resources/nativelib/x64 is, of course, the directory where the DLLs are stored.
I tried adding
-Dpath "D:/prototype/resources/nativelib/x64;${system_property:path}"
or
-DPATH "D:/prototype/resources/nativelib/x64;${system_property:PATH}"
in the same place, but neither of them got the desired result.
On the command line enter the following command.
set PATH=%PATH%;C:\path_to_ur_dlls
And yes, you can do it inside java code as following.
Process proc = Runtime.getRuntime().exec("cmd set PATH=%PATH%;C:\\path_to_ur_dlls");
proc.waitFor();
I have a Java program that uses vlcj to play videos and that packages the VLC libs in the jar. At runtime, the VLC libs are extracted to the user's home, let's say path A. The normal way to indicate this path to vlcj is through the jna method:
NativeLibrary.addSearchPath(RuntimeUtil.getLibVlcLibraryName(), "A");
This works under Windows and MacOSX but not Linux where it throws a UnsatisfiedLinkError.
After some trial and error, I found that the only to get this to work under linux was by using
export LD_LIBRARY_PATH=A
prior to execution and despite the JNA documentation, none of these worked in JVM settings:
-Djava.library.path=A
-Djna.library.path=A
-Djna.platform.library.path=A
My problem with using LD_LIBRARY_PATH is that it is not something I can set at runtime (can I?) which I need to do. Does anyone know of a way to get around this?
I never found an ideal solution to this myself, but this is what I found during my trials with my own vlcj projects.
If you build VLC yourself on Linux you will see these warnings:
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
None of those things it suggests can you do from within your JVM, at least not without calling native code with privilege escalation.
So, in general what you are left with is: -Djna.library.path=LIBDIR should work; alternatively in code System.setProperty("jna.library.path", "LIBDIR"); should also work.
In fact I just tested that with my own native library that I happen to use in my own vlcj projects and both of those approaches worked just fine.
However, it seems it is not so easy with VLC itself, probably because of the way VLC loads its plugins.
In theory, if you structure your directories correctly, the plugins should get discovered automatically so you only need to point jna.library.path to the directory that contains the libvlc and libvlccore shared objects. In my build of VLC, the directory structure looks like this:
VLCDIR
VLCDIR/libvlc.so
VLCDIR/libvlc.so.5
VLCDIR/libvlc.so.5.4.0
VLCDIR/libvlccore.so
VLCDIR/libvlccore.so.7.0.0
VLCDIR/vlc/plugins
If this still fails then, again in theory, you can set the VLC_PLUGIN_PATH environment variable to point to the directory containing the VLC plugins. The problem is that this must be set for the native process, it will not work if you set it as a system property from inside your Java application.
I can only really suggest you generate a shell-script file that sets up the environment correctly when you install your application, or if you want to do it programmatically inside the JVM you could maybe have a bootstrap application that prepares the native environment and then kicks off a new Java process for your actual application - but it's messy to do things that way.
What I have also seen on Linux is that the library paths seem "baked in" to the ".so" files, and you can't just copy those files anywhere and still expect it to work. That is why you must use e.g. LD_LIBRARY_PATH or libtool or one of the other proposed solutions.
And this does not even touch on what you do with all the other libraries that VLC and its plugins may depend on at run-time - are you going to ship all those too?
My recommendation really is just to have the user install VLC first, or have your installer application install VLC first, by using the OS native package installation commands. Not ideal, but it works.
Background:
I'm doing machine learning research, and want to use the FANN library to construct neural networks. The source code is written in C, but I need it wrapped so that I can use it with a lot of Java classes we've created.
Question:
The website provides a link to an already well received wrapper software called fannj. Its dependencies is the FANN library source code and JNA. I've never done wrapping before so JNA is brand new to me. The github homepage for the code mentions "you must set the jna.library.path system property to the path of the FANN Library". I'm currently in the process of trying to do this in Eclipse on Mac OS X. A friend of mine mentioned to me earlier that it means I have to pass the location of the FANN library as argument to the virtual machine, but he only showed me how to do this via the shell. I never actually ran this command from the shell, because I wasn't sure how it would conflict with whatever configurations I do in Eclipse later.
How do I set the jna.library.path in Eclipse? The JNA directory is huge, and I don't know where the actual executable is. My friend suggested I modify its run configuration through use of the argument tab, but like I said I don't actually know where the executable is. All I have is the large uncompressed jar file. How can I go about getting it set up in Eclipse?
You don't set in an IDE, just insert in some piece of code that is called BEFORE JNA is called, something like this:
System.setProperty("jna.library.path", "path you need");
Where System is java.lang.System. As for what path you need it must be the path that contains compiled dynamically linked FAAN library (a *dll).
Or just forget that alltogether and dump FAAN (*dlls, *so..) into Windows/system32 (or other appropriate folder on system you use) and these dll's will be on default search path.
Two allow Eclipse to use JNA, all you need to do is to put two jar files in Eclipse's Java Build Path, jna.jar and platform.jar. That's it.
To do this for an individual project, right click on the project in the Package Explorer, click Properties (at the bottom), click Java Build Path on the left, then the Add External Jar files. Browse to the directory with your JNA files and add those two files.
Perhaps this is a duplicate question, but I havn't found something by myself.
Basically I have an understanding problem.
I have an application which works fine when I call it as follows:
java -Duser.dir="some path" -Djava.library.path="pathToDLL1;pathToDLL2;pathToMyDir" classToCall par1
With the call above my application will start and run.
My problem in understanding is within the pathToMyDir directory (the content of this directory is mainly some needed dll's).
When I start the same application with following command it will fail.
java -Duser.dir="some path" -Djava.library.path="pathToMyDir;pathToDLL1;pathToDLL2" classToCall par1
I'm not sure if this is because of some dependencies, but I thought the java.library.path is mainly used to tell java where to look for external libraries. Or is this wrong? Is there also some order information hidden?
-Djava.library.path is used for pointing to native system libraries (dll or so files). It points to a directory and calls to native code that use System.loadLibrary look in that directory for the native libs.
The project dependencies (jar files) should be specified on the application's classpath, not in this location.