I have a JNI library (.so) shared between two web applications deployed in Tomcat7. I am loading the library using the System.loadLibrary only once in the first web application that is being deployed and then in the second I'm checking if it already was loaded to not load anymore (I tried loading it in both and I got UnsatisfiedLinkError - library was loaded by another classloader). I can make any call to the native library in the first application, but in the second one I get UnsatisfiedLinkError with the method name that I am trying to call.
I am running out of ideas of what I can do. Any ideas? I tried most of the solutions on SO.
Thank you.
EDIT
Yes, I tried adding the library in the tomcat lib folder and loading it from there. Initially it was in the bin folder and the same issue occurs.
Yes, this will happen when you try to load the library that has already loaded my another web application. Tomcat, uses separate class loaders for each of the web application, and it wont allow you load a same native library more than once to JVM via another class loader
Move any share jar files if any that consumes JNI from you sharedlib.so. Add the system path to sharedlib ,
export LD_LIBRARY_PATH=/path/to/whereyourlinklibrary
Write a simple class like this which enables you to load your shared library when tomcat starts. Just compile this class and drop it in tomcat lib folder
package msm;
public class DLLBootstrapper {
static {
System.loadLibrary("sharedlib");
}
public static void main(String args[]) {
System.out.println("Loaded");
}
}
you can now load this class from any of your web application ( probably in startup listener)
Class.forName("msm.DLLBootstrapper");
Good to go!
Have you tried putting shared JNI library just inside the lib directory of server.It should be shared by all the web applications deployed.
You need to load any native code from within the server classloader and not the webapp classloader. I recommend to write a Listener which loads the binary image of the shared object into the VM. This will happens only once. Please see the AprLifecycleListener on how to properly do that. It included a JNI compnent which likely represents you case exactly.
The shared object has to reside in ${catalina.home}/lib and LD_LIBRARY_PATH hat to point to it.
Tomcat has a built-in solution for this issue as of versions 9.0.13, 8.5.35, and 7.0.92:
1) Use the JniLifecycleListener to load the native library.
2) Use the loadLibrary() or load() from org.apache.tomcat.jni.Library instead of System.
See more details and examples in my answer at java.lang.UnsatisfiedLinkError: Native Library XXX.so already loaded in another classloader
Related
I can't seem to redeploy my spring boot webapp without restarting my entire Tomcat server.
Whenever I redeploy, the stacktrace tells me that opencv is already loaded in another classloader and it fails to deploy.
I am using OpenPnP's OpenCV package. https://github.com/openpnp/opencv.
I had this static method in my webapp
static{
nu.pattern.OpenCV.loadShared();
System.out.println("=====================LOADED CV================" + Core.VERSION);
}
Since the webapp was crashing everytime I redeployed it, I decided to jar up a separate program and uploaded it to my share/apache-tomcat-7.0.52/lib folder and run it as a main method to load it once
public class SeparateJarFromWebApp{
public static void main (String args[]){
System.out.println("==============RUNNING MAIN CLASS===========");
nu.pattern.OpenCV.loadShared();
System.out.println("=====================LOADED CV================" + Core.VERSION);
}
}
After running the command the run the main method of my jar I get the message:
You have loaded library /tmp/opencv_openpnp3438207847480914494/nu/pattern/opencv/linux/x86_64/libopencv_java320.so which might have disabled stack guard. The VM will try to fix the stack guard now.
Then i ran my webapp without running any commands to load openCv since it was already loaded by my separate jar. But I get this in my stacktrace:
java.lang.UnsatisfiedLinkError: org.opencv.core.Mat.n_Mat(III)J
I'm out of ideas
From a quick look at the code, it looks like it's initializing some native library. Unless the library allows unloading as well as loading, you're out of luck with regards to redeployment.
You might be able to deploy opencv to tomcat's lib directory, where it's at least in a place where it will be initialized only once (provided that only one webapp does so), and you'll have to be prepared that any second initialization (e.g. when you redeploy your webapp) will fail.
Why you expect some random main method in a jar on the classpath to be executed when deployed to a webserver is beyond me though.
I have deployed one web-application, which contains following code.
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
Now, I deployed another web-application which also have same code. When it tries to load library, it throwing following error.
Exception in thread "Thread-143" java.lang.UnsatisfiedLinkError:
Native Library /usr/lib/jni/libopencv_java248.so already loaded in
another classloader
I want to run these both application simultaneously.
Till now what I have tried:
Loaded library in one application and caught above exception into another application
Removed jars from both application and put opencv.jar into Tomcat's classpath(ie in /usr/share/tomcat7/lib).
But none of above worked, any suggestions by which I can do this ?
Edit: for option two,
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
This line works but gets exception when I am actually going to use that library. That is when I do following
Mat mat = Highgui.imread("/tmp/abc.png");
And I get this exception
java.lang.UnsatisfiedLinkError: org.opencv.highgui.Highgui.imread_1(Ljava/lang/String;)J
at org.opencv.highgui.Highgui.imread_1(Native Method)
at org.opencv.highgui.Highgui.imread(Highgui.java:362)
The problem is with how OpenCV handles the initialization of the native library.
Usually a class that uses a native library will have a static initializer that loads the library. This way the class and the native library will always be loaded in the same class loader. With OpenCV the application code loads the native library.
Now there's the restriction that a native library can only be loaded in one class loader. Web applications use their own class loader so if one web application has loaded a native library, another web application cannot do the same. Therefore code loading native libraries cannot be put in a webapp directory but must be put in the container's (Tomcat) shared directory. When you have a class written with the usual pattern above (loadLibrary in static initializer of using class) it's enough to put the jar containing the class in the shared directory. With OpenCV and the loadLibrary call in the web application code however, the native library will still be loaded in the "wrong" class loader and you will get the UnsatisfiedLinkError.
To make the "right" class loader load the native library you could create a tiny class with a single static method doing only the loadLibrary. Put this class in an extra jar and put this jar in the shared Tomcat directory. Then in the web applications replace the call to System.loadLibrary with a call to your new static method. This way the class loaders for the OpenCV classes and their native library will match and the native methods can be initialized.
Edit: example as requested by a commenter
instead of
public class WebApplicationClass {
static {
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
}
}
use
public class ToolClassInSeparateJarInSharedDirectory {
public static void loadNativeLibrary() {
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
}
}
public class WebApplicationClass {
static {
ToolClassInSeparateJarInSharedDirectory.loadNativeLibrary();
}
}
As of Tomcat versions 9.0.13, 8.5.35, and 7.0.92 we have added the following options to address this issue BZ-62830:
1) Use the JniLifecycleListener to load the native library.
e.g. to load the opencv_java343 library, you can use:
<Listener className="org.apache.catalina.core.JniLifecycleListener"
libraryName="opencv_java343" />
2) Use the load() or loadLibrary() from org.apache.tomcat.jni.Library instead of System.
e.g.
org.apache.tomcat.jni.Library.loadLibrary("opencv_java343");
Using either of those options will use the Common ClassLoader to load the native library, and therefore it will be available to all of the Web Apps.
I got stuck on this exact problem.
Adding listener in Tomcat (v8.5.58) server.xml file seems to successfully load the dll file (at least the log says so) when Tomcat starts, but when you call the the native method, it fails with java.lang.UnsatisfiedLinkError.
With or without calling "org.apache.tomcat.jni.Library.loadLibrary("TeighaJavaCore");" in my java code makes no difference, same error remain. I include tomcat-jni dependency in my project to enable the "org.apache.tomcat.jni.Library.loadLibrary("TeighaJavaCore")" call. While, I guess there is no need to call "org.apache.tomcat.jni.Library.loadLibrary("TeighaJavaCore")" in java code (at web application level) as the TeighaJavaCore.dll will be automatically loaded when Tomcat starts (because the listener above is defined for this purpose at Tomcat container level)
I also check the source code of "org.apache.tomcat.jni.Library.loadLibrary" here, it simply calls "System.loadLibrary(libname)".
https://github.com/apache/tomcat-native/blob/master/java/org/apache/tomcat/jni/Library.java
As of javacpp>=1.3 you may also change the cache folder (defined by system property) in your war deployment listener:
System.setProperty("org.bytedeco.javacpp.cachedir",
Files.createTempDirectory( "javacppnew" ).toString());
Note though that native libraries are always unpacked and will be loaded several times (because considered as different libs).
I have a a library that is bundled as an executable jar file and added to weblogic / tomcat classpath, how can I execute a main method from the jar file when the server is starting and loading the classes from the jar file.
what I want to is to have some initialization code to be executed first thing when the jar file is loaded and server is starting without any user intervention.
Note: I know I can bundle my jar in a war file, but I have some aspectj code in my library that I want to weave all running applications in the jvm, when I bundle my jar in war file, the aspectj code will only weave into the classes in the war file so I added my library jar file in the classpath.
Thanks in advance.
Add a class inside your JAR with the following code:
public class TomcatStartupListener implements org.apache.catalina.LifecycleListener {
public void lifecycleEvent(org.apache.catalina.LifecycleEvent event) {
if (event.getType().equals("after_start")) {
// call your main method here
}
}
}
Note: In order to compile this, you need to add <tomcat-dir>/lib/catalina.jar to your classpath. Otherwise when compiling it won't be able to find the necessary interfaces (org.apache.catalina.LifecycleListener and org.apache.catalina.LifecycleEvent). Once you're done with the compiling, put the JAR as usual under <tomcat-dir>/lib.
Now open <tomcat-dir>/conf/server.xml and add the following under the <Server> section:
<Listener className="com.yourpackage.TomcatStartupListener" />
Now whenever your Tomcat server starts, this TomcatStartupListener class inside your JAR will be called, and you can invoke your main method. There are a whole lot of other event types too! You can use any of these event types:
before_init
after_init
before_start
configure_start
start
after_start
before_stop
stop
configure_stop
after_stop
before_destroy
after_destroy
This approach is necessary because of the way the classloaders work in Tomcat (or even most JVMs). Here are the important points from that link:
There are three aspects of a class loader behavior
Lazy Loading
Class Caching
Separate Namespaces
The JVM will get very heavy if all classes inside all JARs get loaded indiscriminately. So the classes inside shared JARs are loaded only on-demand. The only way for you to invoke the main method is to add the above lifecycle listener.
Perhaps the simplest thing to do is to deploy a trivial servlet in a .war file that references your .jar file. The servlet can be configured to start up upon deployment/container start, and then it can invoke the class containing your main() method.
As application servers / servlet containers typically have a lot of different classloaders, you'll most likely need a different strategy for weaving aspects into your code than in standalone applications.
I would recommend to add the aspects to every war file deployed at build time. This might be following a common technique - as opposed to a server specific one.
Further, I'm not sure it can actually be done (properly & supported) on a server. Typically a server is built to separate all webapps from each other. You might get it to work, but it might break on the next update of the server.
It might be easier to suggest an alternative technique if you'd state the problem that you want to solve with your proposed approach.
Edit after your comment: Consider the standard web application lifecycle: You can execute some code, e.g. in a servlet, upon it being deployed. If you insist on your code being contained in main, you can call this method from your webapp's initialization code.
You need to register a Java Agent. See this link: java.lang.instrument.
java.lang.instrument provides services that allow Java programming language agents to instrument programs running on the JVM.
This is the right way to do this.
I have a dll file and I am trying to call functions of it through a Java program through JNA
But the problem is It is not able to locate my dll file and throwing the following exception:
java.lang.UnsatisfiedLinkError: Unable to load library 'UsbDll': The specified module could not be found.
at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:163)
at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:236)
at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:199)
at com.sun.jna.Native.register(Native.java:1018)
at com.MainClass.<clinit>(MainClass.java:15)
Exception in thread "main"
Below is my program:
package com;
import com.sun.jna.Native
public class MainClass {
static {
Native.register("UsbDll");
}
public native int method();
public static void main(String[] args) {
}
}
The name of my dll file is UsbDll.dll and my operating system is Windows.
============================ EDITED ================================
The location of my dll file is "c:\UsbDll.dll"
When I placed another dll file at the same location, JNA has located it so I think that the problem is with my "UsbDll.dll" file only.
When I tried to load both the dll files (UsbDll.dll and the another dll) with the following command
System.load("c:\\UsbDll.dll");
System.load("c:\\another.dll");
It loaded the "another.dll" successfully but for "UsbDll.dll", it throws the following exception:
java.lang.UnsatisfiedLinkError: C:\UsbDll.dll: Can't find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1803)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1699)
at java.lang.Runtime.load0(Runtime.java:770)
at java.lang.System.load(System.java:1003)
at com.MainClass.<clinit>(MainClass.java:16)
Exception in thread "main"
Q1. It looks like it is not finding some dependent libraries. But as I am totally new to these dlls, Is there anything I am missing or I need to ask the vendor to provide me the dependent libraries.
OR
Is it depends on some standard libraries which I can download from internet? If it is, then how to find the libraries name on which it depends?
============================ EDITED #2 ================================
I ran the Dependency Walker on my UsbDll.dll file to find the missing dependencies and found the following missing dependencies.
WER.DLL (referred by library WINNM.DLL found in my c:\windows\system32)
IESHIMS.DLL (referred by library WINNM.DLL found in my c:\windows\system32)
WDAPI920.DLL (referred directly by my UsbDll.dll)
Q1. As WINNM.DLL was found in my system32 folder, it seems as the standard dll. And if this standard dll is referring to 2 missing dlls (WER.DLL & IESHIMS.DLL), then I suspect how other applications are working who are using this WINNM.DLL file?
Q2. I googled for WDAPI920.dll (that was referred my UsbDll.dll directly) and many search results appeared with the same dll name. So it also looks like some standard library. So how to fix these dependencies? From where to download them? After downloading, Can I place them in the same directory in which my main dll (UsbDll.dll) is or I need to do something extra to load my dll (UsbDll.dll) sucessfully?
From your edited code it is quite evident that the UsbDll.dll depends on some standard modules which are not there on your system (for example if it uses ATL and if you have don't have proper runtime then it is guaranteed to fail). To resolve this you will need proper runtime environment.
Also it is possible that the dll in concern depends on some vendor specific module. For you the best option is (and fastest option would be) to contact the vendor. Otherwise, try to install proper runtime from the microsoft site (but its more of hit-and-trial)
Update
Use the below links for finding more about DLL dependency:
How do I determine the dependencies of a .NET application?
http://msdn.microsoft.com/en-us/library/ms235265.aspx
Command line tool to find Dll dependencies
Update 2
See the below mentioned link for the missing dll details (but it is specific to the windows version)
Dependency Walker reports IESHIMS.DLL and WER.DLL missing?
http://social.msdn.microsoft.com/Forums/en/vsx/thread/6bb7dcaf-6385-4d24-b2c3-ce7e3547e68b
From few simple google queries, WDAPIXXX.dll appears to be some win driver related thing (although i am not too sure). Check this link, they have something to say about WDAPI http://www.jungo.com/st/support/tech_docs/td131.html.
The DLL must by in the Path specified by LD_LIBRARY_PATH (see your env), or, in the case of JNA, in the current directory.
How can I load a custom dll file in my web application? I've tried the following:
Copied all required dlls in system32 folder and tried to load one of them in Servlet constructor System.loadLibrary
Copied required dlls into tomcat_home/shared/lib and tomcat_home/common/lib
All these dlls are in WEB-INF/lib of the web-application
In order for System.loadLibrary() to work, the library (on Windows, a DLL) must be in a directory somewhere on your PATH or on a path listed in the java.library.path system property (so you can launch Java like java -Djava.library.path=/path/to/dir).
Additionally, for loadLibrary(), you specify the base name of the library, without the .dll at the end. So, for /path/to/something.dll, you would just use System.loadLibrary("something").
You also need to look at the exact UnsatisfiedLinkError that you are getting. If it says something like:
Exception in thread "main" java.lang.UnsatisfiedLinkError: no foo in java.library.path
then it can't find the foo library (foo.dll) in your PATH or java.library.path. If it says something like:
Exception in thread "main" java.lang.UnsatisfiedLinkError: com.example.program.ClassName.foo()V
then something is wrong with the library itself in the sense that Java is not able to map a native Java function in your application to its actual native counterpart.
To start with, I would put some logging around your System.loadLibrary() call to see if that executes properly. If it throws an exception or is not in a code path that is actually executed, then you will always get the latter type of UnsatisfiedLinkError explained above.
As a sidenote, most people put their loadLibrary() calls into a static initializer block in the class with the native methods, to ensure that it is always executed exactly once:
class Foo {
static {
System.loadLibrary('foo');
}
public Foo() {
}
}
Changing 'java.library.path' variable at runtime is not enough because it is read only once by JVM. You have to reset it like:
System.setProperty("java.library.path", path);
//set sys_paths to null
final Field sysPathsField = ClassLoader.class.getDeclaredField("sys_paths");
sysPathsField.setAccessible(true);
sysPathsField.set(null, null);
Please, take a loot at: Changing Java Library Path at Runtime.
The original answer by Adam Batkin will lead you to a solution, but if you redeploy your webapp (without restarting your web container), you should run into the following error:
java.lang.UnsatisfiedLinkError: Native Library "foo" already loaded in another classloader
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1715)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1646)
at java.lang.Runtime.load0(Runtime.java:787)
at java.lang.System.load(System.java:1022)
This happens because the ClassLoader that originally loaded your DLL still references this DLL. However, your webapp is now running with a new ClassLoader, and because the same JVM is running and a JVM won't allow 2 references to the same DLL, you can't reload it. Thus, your webapp can't access the existing DLL and can't load a new one. So.... you're stuck.
Tomcat's ClassLoader documentation outlines why your reloaded webapp runs in a new isolated ClassLoader and how you can work around this limitation (at a very high level).
The solution is to extend Adam Batkin's solution a little:
package awesome;
public class Foo {
static {
System.loadLibrary('foo');
}
// required to work with JDK 6 and JDK 7
public static void main(String[] args) {
}
}
Then placing a jar containing JUST this compiled class into the TOMCAT_HOME/lib folder.
Now, within your webapp, you just have to force Tomcat to reference this class, which can be done as simply as this:
Class.forName("awesome.Foo");
Now your DLL should be loaded in the common classloader, and can be referenced from your webapp even after being redeployed.
Make sense?
A working reference copy can be found on google code, static-dll-bootstrapper .
You can use System.load() to provide an absolute path which is what you want, rather than a file in the standard library folder for the respective OS.
If you want native applications that already exist, use System.loadLibrary(String filename). If you want to provide your own you're probably better with load().
You should also be able to use loadLibrary with the java.library.path set correctly. See ClassLoader.java for implementation source showing both paths being checked (OpenJDK)
In the case where the problem is that System.loadLibrary cannot find the DLL in question, one common misconception (reinforced by Java's error message) is that the system property java.library.path is the answer. If you set the system property java.library.path to the directory where your DLL is located, then System.loadLibrary will indeed find your DLL. However, if your DLL in turn depends on other DLLs, as is often the case, then java.library.path cannot help, because the loading of the dependent DLLs is managed entirely by the operating system, which knows nothing of java.library.path. Thus, it is almost always better to bypass java.library.path and simply add your DLL's directory to LD_LIBRARY_PATH (Linux), DYLD_LIBRARY_PATH (MacOS), or Path (Windows) prior to starting the JVM.
(Note: I am using the term "DLL" in the generic sense of DLL or shared library.)
If you need to load a file that's relative to some directory where you already are (like in the current directory), here's an easy solution:
File f;
if (System.getProperty("sun.arch.data.model").equals("32")) {
// 32-bit JVM
f = new File("mylibfile32.so");
} else {
// 64-bit JVM
f = new File("mylibfile64.so");
}
System.load(f.getAbsolutePath());
For those who are looking for java.lang.UnsatisfiedLinkError: no pdf_java in java.library.path
I was facing same exception; I tried everything and important things to make it work are:
Correct version of pdf lib.jar ( In my case it was wrong version jar kept in server runtime )
Make a folder and keep the pdflib jar in it and add the folder in your PATH variable
It worked with tomcat 6.
If you believe that you added a path of native lib to %PATH%, try testing with:
System.out.println(System.getProperty("java.library.path"))
It should show you actually if your dll is on %PATH%
Restart the IDE Idea, which appeared to work for me after I setup the env variable by adding it to the %PATH%
The issue for me was naming:
The library name should begin with "lib..." such as libnative.dll.
So you might think you need to load "libnative": System.loadLibrary("libnative")
But you actually need to load "native": System.loadLibrary("native")
Poor me ! spent a whole day behind this.Writing it down here if any body replicates this issue.
I was trying to load as Adam suggested but then got caught with AMD64 vs IA 32 exception.If in any case after working as per Adam's(no doubt the best pick) walkthrough,try to have a 64 bit version of latest jre.Make sure your JRE AND JDK are 64 bit and you have correctly added it to your classpath.
My working example goes here:unstatisfied link error
I'm using Mac OS X Yosemite and Netbeans 8.02, I got the same error and the simple solution I have found is like above, this is useful when you need to include native library in the project. So do the next for Netbeans:
1.- Right click on the Project
2.- Properties
3.- Click on RUN
4.- VM Options: java -Djava.library.path="your_path"
5.- for example in my case: java -Djava.library.path=</Users/Lexynux/NetBeansProjects/NAO/libs>
6.- Ok
I hope it could be useful for someone.
The link where I found the solution is here:
java.library.path – What is it and how to use
It is simple just write java -XshowSettings:properties on your command line in windows and then paste all the files in the path shown by the java.library.path.
I had the same problem and the error was due to a rename of the dll.
It could happen that the library name is also written somewhere inside the dll.
When I put back its original name I was able to load using System.loadLibrary
First, you'll want to ensure the directory to your native library is on the java.library.path. See how to do that here. Then, you can call System.loadLibrary(nativeLibraryNameWithoutExtension) - making sure to not include the file extension in the name of your library.