How to influence search path of System.loadLibrary() through Java code? - java

In a Java project, I am using a third-party library that loads some native library via
System.loadLibrary("libName");
I'd like to be able to influence the search path of this method from within my application, so that the user doesn't need to specify a correct java.library.path value on the command line (this value depends on the current OS and architecture). E.g on Windows I want to set it to "lib/native/windows", on Linux 32bit to "lib/native/linux32" etc.
I tried
System.setProperty("java.library.path", ...)
but this is ignored, apparently because the JVM reads this property only once before my code is run.
I also tried to load the native libray before using the Java library that depends on it with
System.load("fullPath/lib")
This call succeeds, but there will still be an UnsatisfiedLinkError when the native library is loaded again with System.loadLibrary().
The only way I found is the following:
Add interfaces that abstract the whole API of the external library.
Use only these interfaces in the rest of the code.
Add classes that implement the interfaces and delegate to the library.
Write an own ClassLoader, that
overwrites findLibary() so that the native library is found in the correct path
overwrites loadClass() and loads all classes of the external library and the wrapper layer by itself instead of trying to delegate to its parent like the default ClassLoader would do
Ensure that the interfaces are loaded with the normal ClassLoader and the wrapping classes and the external library are loaded with my own ClassLoader.
This works, but I find it very complicated and it is much effort because I need to add all those interfaces. Is there a simpler way?

I needed to change the dll path for my unit tests. I tried the following hack and it worked:
System.setProperty( "java.library.path", "/path/to/libs" );
Field fieldSysPath = ClassLoader.class.getDeclaredField( "sys_paths" );
fieldSysPath.setAccessible( true );
fieldSysPath.set( null, null );
For explanation, see the original link.

There is no approved way to change the library path for a running JVM.
You cannot load a native library more than once ... and you cannot unload a native library so that you can load it again: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4171986
Based on your comments above (particularly, the behavior of the 3rd-party library), I'd say that your best option is to get the library path right when you launch the JVM.
Note that there is a hacky way to change the library path (see https://stackoverflow.com/a/24258955/139985) but it involves nasty reflection, and it reportedly doesn't work for all Java releases. Certainly, it relies on undocumented private implementation details of ClassLoader that could change from one release to the next.

Just recently ran into this issue and using OpenJDK where a NullPointerException is thrown (as #0-0 mentioned in his comment to Samil's answer). The following works in OpenJDK and should work with Oracle JDK as well.
(Option 1) Replace the java.library.path
System.setProperty("java.library.path", newPath);
Field field = ClassLoader.class.getDeclaredField("sys_paths");
field.setAccessible(true);
field.set(ClassLoader.getSystemClassLoader(), new String[]{newPath});
(Option 2) Add to existing java.library.path
String libPath = System.getProperty("java.library.path");
String newPath;
if (libPath == null || libPath.isEmpty()) {
newPath = path;
} else {
newPath = path + File.pathSeparator + libPath;
}
System.setProperty("java.library.path", newPath);
Field field = ClassLoader.class.getDeclaredField("sys_paths");
field.setAccessible(true);
// Create override for sys_paths
ClassLoader classLoader = ClassLoader.getSystemClassLoader();
List<String> newSysPaths = new ArrayList<>();
newSysPaths.add(path);
newSysPaths.addAll(Arrays.asList((String[])field.get(classLoader)));
field.set(classLoader, newSysPaths.toArray(new String[newSysPaths.size()]));

I tried to following to load a native Growl library for a Java application on my Mac where the lib is in the root of the classpath of my application:
System.load(GrowlUtils.class.getResource("/libgrowl.jnilib").getFile().toString());

Is there a simpler way?
Yes, provide batch/script files to start the application. Then you can set the correct path in the batch/shell file or even read the value from an environment variable. Much easier then trying to do it from inside the application.

While technically correct these answers are missleading. Setting the environment variables PATH on Windows, or LD_LIBRARY_PATH on unix will change where the jvm looks for libraries:
What is LD_LIBRARY_PATH and how to use it?
on linux:
export LD_LIBRARY_PATH=/usr/.../
then:
java ....

Related

Java JNA load DLL

I have a problem when I try load a DLL like this:
String a = "C:\\Users\\ElteGps 022\\Documents\\NetBeansProjects\\JavaApplication1\\src\\lib\\EQ2008_Dll.dll";
String strDllFileName = m_strUserPath + "\\res\\EQ2008_Dll.dll";
String strEQ2008_Dll_Set_Path = m_strUserPath + "\\res\\EQ2008_Dll_Set.ini";
m_DllLibrary = (DllLibrary) Native.loadLibrary(a,DllLibrary.class);
I see this:
Exception in thread "main" java.lang.UnsatisfiedLinkError: Unable to load library 'C:\Users\ElteGps 022\Documents\NetBeansProjects\JavaApplication1\src\lib\EQ2008_Dll.dll': Nie mo¿na odnaleæ okrelonego modu³
at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:163)
at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:236)
at com.sun.jna.Library$Handler.<init>(Library.java:140)
at com.sun.jna.Native.loadLibrary(Native.java:379)
at com.sun.jna.Native.loadLibrary(Native.java:364)
at javaapplication1.Fun.main(Fun.java:280)
I read and I did this:
Can not add all the classes files from the JNI folder in Eclipse (JAVA, Windows 7)
giving 'java.library.path' in netbeans for .dll/.so files
From the JNA javadoc
Library Search Paths
A search for a given library will scan the following locations:
jna.library.path User-customizable path
jna.platform.library.path Platform-specific
paths On OSX, ~/Library/Frameworks,
/Library/Frameworks, and
/System/Library/Frameworks will be searched for a
framework with a name corresponding to that requested. Absolute
paths to frameworks are also accepted, either ending at the framework
name (sans ".framework") or the full path to the framework shared
library (e.g. CoreServices.framework/CoreServices).
Context class loader classpath. Deployed native libraries
may be installed on the classpath under
${os-prefix}/LIBRARY_FILENAME, where
${os-prefix} is the OS/Arch prefix returned by Platform.getNativeLibraryResourcePrefix().
If bundled in a jar file, the resource will be extracted to
jna.tmpdir for loading, and later removed (but only if
jna.nounpack is false or not set). You may
set the system property jna.debug_load=true to make JNA
print the steps of its library search to the console.
Native.loadLibrary doesn't work with a full path, try instead System.load
If you can't use that you could also specify the directory of the dll before the loading by setting the enviroment variable of java like this
System.setProperty("jna.library.path", "C:\\Users\\ElteGps 022\\Documents\\NetBeansProjects\\JavaApplication1\\src\\lib");
But this is higly not recommended since it will works only on your computer

portable statement to load JNI library from a different directory using relative pathname?

Is there a platform-independent Java statement to load a native library from a different directory than the Java source code is in? I would like to use something like this:
public class HelloWorld {
static {
System.loadLibrary("../some_project/HelloWorld");
}
public static native void print();
}
The problem is that System.loadLibrary() doesn't support directory separators in the pathname argument. Also, System.load() unfortunately requires an absolute pathname, which not only means I can't specify a relative directory as above (which I would like to do), but it also requires the argument to include, for example, the preceding "lib" and ".so" extension on the JNI library name on a Linux system.
Is there a standard way of dealing with this? If possible, I would like to avoid writing a bunch of platform-dependent Java code just to construct the correct JNI library name.
I believe you're looking for System.mapLibraryName, which is the method typically used by ClassLoader.findLibrary implementations. For example:
File lib = new File("../some_project/" + System.mapLibraryName("HelloWorld"));
System.load(lib.getAbsolutePath());
This will use libHelloWorld.so on Linux and HelloWorld.dll on Windows. Be aware that some operating systems support multiple extensions, and mapLibraryName can only support one, by design. The ones I'm aware of are MacOS (.dylib primarily and .jnilib for legacy) and AIX (.a and .so).
Well, as you have clearly explained, you can't use loadLibrary. And so that leaves load. But that requires an absolute path. So, expand your relative path into an absolute path using your preferred file path utility functions, and pass that on to load. This may seem inconvenient but it's much more robust than relying on the vagaries of library search paths.

Setting CLASSPATH during runtime

How do I set a CLASSPATH variable during runtime while using IKVM?
I've been trying to do it by using:
java.lang.System.setProperty("java.class.path", "whatever");
The class I'm calling requires a configuration file in the classpath to work - and I keep getting errors that seem to indicate that it hasn't gotten its settings.
Is the way I'm trying to add variable incorrect?
If you really can't set the classpath beforehand yourself using the java's -cp or -classpath argument (why not by the way? that's the normal approach), then you can try to use URLClassLoader instead. Here's a kickoff example:
URL url = new URL(whateverPath);
ClassLoader contextCL = Thread.currentThread().getContextClassLoader();
ClassLoader urlCL = URLClassLoader.newInstance(new URL[] { url }, contextCL);
Thread.currentThread().setContextClassLoader(urlCL);
// ...
You only need to be lucky if the class you're calling is actually loading its resources through Thread.currentThread().getContextClassLoader().getResource() and thus not through
SomeClass.class.getClassLoader().getResource().
I was trying to do the same thing. I had some jar files compiled to a .Net dll but some of those (3rd party) jar files were trying to load their configuration files from the java classpath.
I solved the problem by specifying the -classloader option for the ikvmc tool. Example:
ikvmc -out:mydotnetapp.dll -classloader:ikvm.runtime.ClassPathAssemblyClassLoader c:/myjavaapp/lib/*.jar
This worked for me!
Source for the solution: http://old.nabble.com/Not-able-to-load-files-from-ClassPath-td31141788.html

java.lang.UnsatisfiedLinkError no *****.dll in java.library.path

How can I load a custom dll file in my web application? I've tried the following:
Copied all required dlls in system32 folder and tried to load one of them in Servlet constructor System.loadLibrary
Copied required dlls into tomcat_home/shared/lib and tomcat_home/common/lib
All these dlls are in WEB-INF/lib of the web-application
In order for System.loadLibrary() to work, the library (on Windows, a DLL) must be in a directory somewhere on your PATH or on a path listed in the java.library.path system property (so you can launch Java like java -Djava.library.path=/path/to/dir).
Additionally, for loadLibrary(), you specify the base name of the library, without the .dll at the end. So, for /path/to/something.dll, you would just use System.loadLibrary("something").
You also need to look at the exact UnsatisfiedLinkError that you are getting. If it says something like:
Exception in thread "main" java.lang.UnsatisfiedLinkError: no foo in java.library.path
then it can't find the foo library (foo.dll) in your PATH or java.library.path. If it says something like:
Exception in thread "main" java.lang.UnsatisfiedLinkError: com.example.program.ClassName.foo()V
then something is wrong with the library itself in the sense that Java is not able to map a native Java function in your application to its actual native counterpart.
To start with, I would put some logging around your System.loadLibrary() call to see if that executes properly. If it throws an exception or is not in a code path that is actually executed, then you will always get the latter type of UnsatisfiedLinkError explained above.
As a sidenote, most people put their loadLibrary() calls into a static initializer block in the class with the native methods, to ensure that it is always executed exactly once:
class Foo {
static {
System.loadLibrary('foo');
}
public Foo() {
}
}
Changing 'java.library.path' variable at runtime is not enough because it is read only once by JVM. You have to reset it like:
System.setProperty("java.library.path", path);
//set sys_paths to null
final Field sysPathsField = ClassLoader.class.getDeclaredField("sys_paths");
sysPathsField.setAccessible(true);
sysPathsField.set(null, null);
Please, take a loot at: Changing Java Library Path at Runtime.
The original answer by Adam Batkin will lead you to a solution, but if you redeploy your webapp (without restarting your web container), you should run into the following error:
java.lang.UnsatisfiedLinkError: Native Library "foo" already loaded in another classloader
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1715)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1646)
at java.lang.Runtime.load0(Runtime.java:787)
at java.lang.System.load(System.java:1022)
This happens because the ClassLoader that originally loaded your DLL still references this DLL. However, your webapp is now running with a new ClassLoader, and because the same JVM is running and a JVM won't allow 2 references to the same DLL, you can't reload it. Thus, your webapp can't access the existing DLL and can't load a new one. So.... you're stuck.
Tomcat's ClassLoader documentation outlines why your reloaded webapp runs in a new isolated ClassLoader and how you can work around this limitation (at a very high level).
The solution is to extend Adam Batkin's solution a little:
package awesome;
public class Foo {
static {
System.loadLibrary('foo');
}
// required to work with JDK 6 and JDK 7
public static void main(String[] args) {
}
}
Then placing a jar containing JUST this compiled class into the TOMCAT_HOME/lib folder.
Now, within your webapp, you just have to force Tomcat to reference this class, which can be done as simply as this:
Class.forName("awesome.Foo");
Now your DLL should be loaded in the common classloader, and can be referenced from your webapp even after being redeployed.
Make sense?
A working reference copy can be found on google code, static-dll-bootstrapper .
You can use System.load() to provide an absolute path which is what you want, rather than a file in the standard library folder for the respective OS.
If you want native applications that already exist, use System.loadLibrary(String filename). If you want to provide your own you're probably better with load().
You should also be able to use loadLibrary with the java.library.path set correctly. See ClassLoader.java for implementation source showing both paths being checked (OpenJDK)
In the case where the problem is that System.loadLibrary cannot find the DLL in question, one common misconception (reinforced by Java's error message) is that the system property java.library.path is the answer. If you set the system property java.library.path to the directory where your DLL is located, then System.loadLibrary will indeed find your DLL. However, if your DLL in turn depends on other DLLs, as is often the case, then java.library.path cannot help, because the loading of the dependent DLLs is managed entirely by the operating system, which knows nothing of java.library.path. Thus, it is almost always better to bypass java.library.path and simply add your DLL's directory to LD_LIBRARY_PATH (Linux), DYLD_LIBRARY_PATH (MacOS), or Path (Windows) prior to starting the JVM.
(Note: I am using the term "DLL" in the generic sense of DLL or shared library.)
If you need to load a file that's relative to some directory where you already are (like in the current directory), here's an easy solution:
File f;
if (System.getProperty("sun.arch.data.model").equals("32")) {
// 32-bit JVM
f = new File("mylibfile32.so");
} else {
// 64-bit JVM
f = new File("mylibfile64.so");
}
System.load(f.getAbsolutePath());
For those who are looking for java.lang.UnsatisfiedLinkError: no pdf_java in java.library.path
I was facing same exception; I tried everything and important things to make it work are:
Correct version of pdf lib.jar ( In my case it was wrong version jar kept in server runtime )
Make a folder and keep the pdflib jar in it and add the folder in your PATH variable
It worked with tomcat 6.
If you believe that you added a path of native lib to %PATH%, try testing with:
System.out.println(System.getProperty("java.library.path"))
It should show you actually if your dll is on %PATH%
Restart the IDE Idea, which appeared to work for me after I setup the env variable by adding it to the %PATH%
The issue for me was naming:
The library name should begin with "lib..." such as libnative.dll.
So you might think you need to load "libnative": System.loadLibrary("libnative")
But you actually need to load "native": System.loadLibrary("native")
Poor me ! spent a whole day behind this.Writing it down here if any body replicates this issue.
I was trying to load as Adam suggested but then got caught with AMD64 vs IA 32 exception.If in any case after working as per Adam's(no doubt the best pick) walkthrough,try to have a 64 bit version of latest jre.Make sure your JRE AND JDK are 64 bit and you have correctly added it to your classpath.
My working example goes here:unstatisfied link error
I'm using Mac OS X Yosemite and Netbeans 8.02, I got the same error and the simple solution I have found is like above, this is useful when you need to include native library in the project. So do the next for Netbeans:
1.- Right click on the Project
2.- Properties
3.- Click on RUN
4.- VM Options: java -Djava.library.path="your_path"
5.- for example in my case: java -Djava.library.path=</Users/Lexynux/NetBeansProjects/NAO/libs>
6.- Ok
I hope it could be useful for someone.
The link where I found the solution is here:
java.library.path – What is it and how to use
It is simple just write java -XshowSettings:properties on your command line in windows and then paste all the files in the path shown by the java.library.path.
I had the same problem and the error was due to a rename of the dll.
It could happen that the library name is also written somewhere inside the dll.
When I put back its original name I was able to load using System.loadLibrary
First, you'll want to ensure the directory to your native library is on the java.library.path. See how to do that here. Then, you can call System.loadLibrary(nativeLibraryNameWithoutExtension) - making sure to not include the file extension in the name of your library.

loading from JAR files during deployment vs development

when i am loading some data into my java program, i usually use FileInputStream. however i deploy the program as a jar file and webstart, so i have to use getRessource() or getRessourceAsStream() to load the data directly from the jar file.
now it is quite annoying to always switch this code between development and deployment?
is there a way autmate this? i.e. is there a way to know if the code is run from a jar or not?
when i try to load it withoug jar like this:
InputStream is = this.getClass().getResourceAsStream("file.txt");
the returned inputstream is simply null, although the file is definitely in the root directory of the application.
thanks!
Why do you use FileInputStream during development? Why not just use getResourceAsStream from the very start? So long as you place your files in an appropriate place in your classpath, you shouldn't have any problems. It can still be a file in the local filesystem rather than in a jar file.
It's helpful to develop with the final deployment environment in mind.
EDIT: If you want something in the root directory of your classpath, you should either use:
InputStream x = getClass().getResourceAsStream("/file.txt");
or
InputStream x = getClass().getClassLoader().getResourceAsStream("file.txt");
Basically Class.getResourceAsStream will resolve relative resources to the package containing the class; ClassLoader.getResourceAsStream resolves everything relative to the "root" package.
You could read your data always as a ressource. You only have to add the path where the data lies to your classpath.
If your data stays in WEB-INF/somewhere/mydata.txt inside your jar file, you will access it with:
getClass().getResourceAsStream( "/WEB-INF/somewhere/mydata.txt" )
Now, if you create a development directory /devel/WEB-INF/somewhere/mydata.txt and put /devel to your classpath, your code will work in development and production.
EDIT after explanation in question:
In your case this.getClass().getResourceAsStream( "mydata.txt" ) the resource is taken from the same position where the classfile of this is taken from. If you want to keep this, then you have to create a directory /devel/<path of package>/mydata.txt and again add /devel to your classpath.
How about setting a system property in your dev environment, via the -D switch? e.g. java -D:mypropertyname=mypropertyvalue
You could set the property in ant scripts in your dev environment, other environments don't get the property:
e.g.
public static boolean isDevEnvironment(){ return System.getProperty("mypropertyname")!=null;}
You might find a better way to hack it from one of the existing System Properties
If a file is considered part of your deployed application (as opposed to be part of the installation specific files) and can be located through the classpath then consider simply always using getResourceAsStream since it works regardless of the actual deployment scheme as long as it is in the classpath.
You might also find the information available from the JVM relevant (if allowed by the security manager):
// Get the location of this class
Class cls = this.getClass();
ProtectionDomain pDomain = cls.getProtectionDomain();
CodeSource cSource = pDomain.getCodeSource();
URL loc = cSource.getLocation(); // file:/c:/almanac14/examples/
http://www.exampledepot.com/egs/java.lang/ClassOrigin.html?l=rel
There shouldn't be any difference between development vs deployment, IHMO.
Classloader.getResource or getResourceAsStream works well, you can read resources and even write them.You can write your own Protocol handles and access everything as an URL/URI, which allows you to read and write resources and also allows proper identification of who actually provide the resource.
The only problem is if an URLStreamHandlerFactory is already registered(in a J2EE application the container could install a factory and you don't have any way to go over and install your own) and you cannot use your handlers "anywhere".
Knowing that, it is preferred to implement your own "resources". At that time when I need it I couldn't find something like that so I had to implement my own ResourceManager. For me it looks more intuitive to access a resource like
Resource layout = ResourceManager.resolve("view://layout/main.jsp")
instead of
URL layout = Classloader.getResource("some_package/view/layout/main.jsp")

Categories

Resources