I can't seem to redeploy my spring boot webapp without restarting my entire Tomcat server.
Whenever I redeploy, the stacktrace tells me that opencv is already loaded in another classloader and it fails to deploy.
I am using OpenPnP's OpenCV package. https://github.com/openpnp/opencv.
I had this static method in my webapp
static{
nu.pattern.OpenCV.loadShared();
System.out.println("=====================LOADED CV================" + Core.VERSION);
}
Since the webapp was crashing everytime I redeployed it, I decided to jar up a separate program and uploaded it to my share/apache-tomcat-7.0.52/lib folder and run it as a main method to load it once
public class SeparateJarFromWebApp{
public static void main (String args[]){
System.out.println("==============RUNNING MAIN CLASS===========");
nu.pattern.OpenCV.loadShared();
System.out.println("=====================LOADED CV================" + Core.VERSION);
}
}
After running the command the run the main method of my jar I get the message:
You have loaded library /tmp/opencv_openpnp3438207847480914494/nu/pattern/opencv/linux/x86_64/libopencv_java320.so which might have disabled stack guard. The VM will try to fix the stack guard now.
Then i ran my webapp without running any commands to load openCv since it was already loaded by my separate jar. But I get this in my stacktrace:
java.lang.UnsatisfiedLinkError: org.opencv.core.Mat.n_Mat(III)J
I'm out of ideas
From a quick look at the code, it looks like it's initializing some native library. Unless the library allows unloading as well as loading, you're out of luck with regards to redeployment.
You might be able to deploy opencv to tomcat's lib directory, where it's at least in a place where it will be initialized only once (provided that only one webapp does so), and you'll have to be prepared that any second initialization (e.g. when you redeploy your webapp) will fail.
Why you expect some random main method in a jar on the classpath to be executed when deployed to a webserver is beyond me though.
Related
I am trying to write a simple servlet which just calls a native function.
First I tried to load the shared object in the servlet but then i got an "UnsatisfiedLinkError" because the library was already loaded in the JVM. After searching for this problem I found this tomcat documentation.
It basically says that I must write a loader which gets executed when the server starts up. So I wrote the following class:
class MyLibLoader
{
static
{
System.loadLibrary("foo");
}
native void doFoo();
public static void main(String[] args)
{
System.out.println("Lib loaded");
}
}
compiled it into a .class file and put it in the "$CATALINA_HOME/shared/lib/" folder, edited the property "shared.loader" in the "catalina.properties" file to the following:
shared.loader="${catalina.home}/shared/lib"
after restarting the tomcat without errors i get the following Exception in the servlet:
java.lang.UnsatisfiedLinkError
Which means, by my understanding, that the JNI library is not loaded. I tried to change the library name to a name that definitely does not exist and there you go: I can restart the tomcat without errors that the library is not found. I also tried to put the loader in a .jar file an change the property to:
shared.loader="${catalina.home}/shared/lib/*.jar"
It didn't work too.
Based on this I assume that the tomcat just ignores the "shared.loader" property.
I have a JNI library (.so) shared between two web applications deployed in Tomcat7. I am loading the library using the System.loadLibrary only once in the first web application that is being deployed and then in the second I'm checking if it already was loaded to not load anymore (I tried loading it in both and I got UnsatisfiedLinkError - library was loaded by another classloader). I can make any call to the native library in the first application, but in the second one I get UnsatisfiedLinkError with the method name that I am trying to call.
I am running out of ideas of what I can do. Any ideas? I tried most of the solutions on SO.
Thank you.
EDIT
Yes, I tried adding the library in the tomcat lib folder and loading it from there. Initially it was in the bin folder and the same issue occurs.
Yes, this will happen when you try to load the library that has already loaded my another web application. Tomcat, uses separate class loaders for each of the web application, and it wont allow you load a same native library more than once to JVM via another class loader
Move any share jar files if any that consumes JNI from you sharedlib.so. Add the system path to sharedlib ,
export LD_LIBRARY_PATH=/path/to/whereyourlinklibrary
Write a simple class like this which enables you to load your shared library when tomcat starts. Just compile this class and drop it in tomcat lib folder
package msm;
public class DLLBootstrapper {
static {
System.loadLibrary("sharedlib");
}
public static void main(String args[]) {
System.out.println("Loaded");
}
}
you can now load this class from any of your web application ( probably in startup listener)
Class.forName("msm.DLLBootstrapper");
Good to go!
Have you tried putting shared JNI library just inside the lib directory of server.It should be shared by all the web applications deployed.
You need to load any native code from within the server classloader and not the webapp classloader. I recommend to write a Listener which loads the binary image of the shared object into the VM. This will happens only once. Please see the AprLifecycleListener on how to properly do that. It included a JNI compnent which likely represents you case exactly.
The shared object has to reside in ${catalina.home}/lib and LD_LIBRARY_PATH hat to point to it.
Tomcat has a built-in solution for this issue as of versions 9.0.13, 8.5.35, and 7.0.92:
1) Use the JniLifecycleListener to load the native library.
2) Use the loadLibrary() or load() from org.apache.tomcat.jni.Library instead of System.
See more details and examples in my answer at java.lang.UnsatisfiedLinkError: Native Library XXX.so already loaded in another classloader
I have deployed one web-application, which contains following code.
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
Now, I deployed another web-application which also have same code. When it tries to load library, it throwing following error.
Exception in thread "Thread-143" java.lang.UnsatisfiedLinkError:
Native Library /usr/lib/jni/libopencv_java248.so already loaded in
another classloader
I want to run these both application simultaneously.
Till now what I have tried:
Loaded library in one application and caught above exception into another application
Removed jars from both application and put opencv.jar into Tomcat's classpath(ie in /usr/share/tomcat7/lib).
But none of above worked, any suggestions by which I can do this ?
Edit: for option two,
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
This line works but gets exception when I am actually going to use that library. That is when I do following
Mat mat = Highgui.imread("/tmp/abc.png");
And I get this exception
java.lang.UnsatisfiedLinkError: org.opencv.highgui.Highgui.imread_1(Ljava/lang/String;)J
at org.opencv.highgui.Highgui.imread_1(Native Method)
at org.opencv.highgui.Highgui.imread(Highgui.java:362)
The problem is with how OpenCV handles the initialization of the native library.
Usually a class that uses a native library will have a static initializer that loads the library. This way the class and the native library will always be loaded in the same class loader. With OpenCV the application code loads the native library.
Now there's the restriction that a native library can only be loaded in one class loader. Web applications use their own class loader so if one web application has loaded a native library, another web application cannot do the same. Therefore code loading native libraries cannot be put in a webapp directory but must be put in the container's (Tomcat) shared directory. When you have a class written with the usual pattern above (loadLibrary in static initializer of using class) it's enough to put the jar containing the class in the shared directory. With OpenCV and the loadLibrary call in the web application code however, the native library will still be loaded in the "wrong" class loader and you will get the UnsatisfiedLinkError.
To make the "right" class loader load the native library you could create a tiny class with a single static method doing only the loadLibrary. Put this class in an extra jar and put this jar in the shared Tomcat directory. Then in the web applications replace the call to System.loadLibrary with a call to your new static method. This way the class loaders for the OpenCV classes and their native library will match and the native methods can be initialized.
Edit: example as requested by a commenter
instead of
public class WebApplicationClass {
static {
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
}
}
use
public class ToolClassInSeparateJarInSharedDirectory {
public static void loadNativeLibrary() {
System.loadLibrary(org.opencv.core.Core.NATIVE_LIBRARY_NAME);
}
}
public class WebApplicationClass {
static {
ToolClassInSeparateJarInSharedDirectory.loadNativeLibrary();
}
}
As of Tomcat versions 9.0.13, 8.5.35, and 7.0.92 we have added the following options to address this issue BZ-62830:
1) Use the JniLifecycleListener to load the native library.
e.g. to load the opencv_java343 library, you can use:
<Listener className="org.apache.catalina.core.JniLifecycleListener"
libraryName="opencv_java343" />
2) Use the load() or loadLibrary() from org.apache.tomcat.jni.Library instead of System.
e.g.
org.apache.tomcat.jni.Library.loadLibrary("opencv_java343");
Using either of those options will use the Common ClassLoader to load the native library, and therefore it will be available to all of the Web Apps.
I got stuck on this exact problem.
Adding listener in Tomcat (v8.5.58) server.xml file seems to successfully load the dll file (at least the log says so) when Tomcat starts, but when you call the the native method, it fails with java.lang.UnsatisfiedLinkError.
With or without calling "org.apache.tomcat.jni.Library.loadLibrary("TeighaJavaCore");" in my java code makes no difference, same error remain. I include tomcat-jni dependency in my project to enable the "org.apache.tomcat.jni.Library.loadLibrary("TeighaJavaCore")" call. While, I guess there is no need to call "org.apache.tomcat.jni.Library.loadLibrary("TeighaJavaCore")" in java code (at web application level) as the TeighaJavaCore.dll will be automatically loaded when Tomcat starts (because the listener above is defined for this purpose at Tomcat container level)
I also check the source code of "org.apache.tomcat.jni.Library.loadLibrary" here, it simply calls "System.loadLibrary(libname)".
https://github.com/apache/tomcat-native/blob/master/java/org/apache/tomcat/jni/Library.java
As of javacpp>=1.3 you may also change the cache folder (defined by system property) in your war deployment listener:
System.setProperty("org.bytedeco.javacpp.cachedir",
Files.createTempDirectory( "javacppnew" ).toString());
Note though that native libraries are always unpacked and will be loaded several times (because considered as different libs).
I made an app using javafx and packaged it as exe file. It works fine being started from eclipse but being installed it says
Failed due to exception in main class
no matter what pc it was started at. I've found a few same questions but everyone found solution by themselves but don't tell how. One question has an answer:
the issue were VM parameters I added in the build.xml in the
fx:platform / fx:jvmarg section. These params were put into the
package.cfg file which is called from the .exe file to initialize the
VM.
In my build.xml there is only one string related to the named fx:platform But in my case there is only <fx:platform basedir="${java.home}"/> and nothing else. What am I doing wrong?
I have a a library that is bundled as an executable jar file and added to weblogic / tomcat classpath, how can I execute a main method from the jar file when the server is starting and loading the classes from the jar file.
what I want to is to have some initialization code to be executed first thing when the jar file is loaded and server is starting without any user intervention.
Note: I know I can bundle my jar in a war file, but I have some aspectj code in my library that I want to weave all running applications in the jvm, when I bundle my jar in war file, the aspectj code will only weave into the classes in the war file so I added my library jar file in the classpath.
Thanks in advance.
Add a class inside your JAR with the following code:
public class TomcatStartupListener implements org.apache.catalina.LifecycleListener {
public void lifecycleEvent(org.apache.catalina.LifecycleEvent event) {
if (event.getType().equals("after_start")) {
// call your main method here
}
}
}
Note: In order to compile this, you need to add <tomcat-dir>/lib/catalina.jar to your classpath. Otherwise when compiling it won't be able to find the necessary interfaces (org.apache.catalina.LifecycleListener and org.apache.catalina.LifecycleEvent). Once you're done with the compiling, put the JAR as usual under <tomcat-dir>/lib.
Now open <tomcat-dir>/conf/server.xml and add the following under the <Server> section:
<Listener className="com.yourpackage.TomcatStartupListener" />
Now whenever your Tomcat server starts, this TomcatStartupListener class inside your JAR will be called, and you can invoke your main method. There are a whole lot of other event types too! You can use any of these event types:
before_init
after_init
before_start
configure_start
start
after_start
before_stop
stop
configure_stop
after_stop
before_destroy
after_destroy
This approach is necessary because of the way the classloaders work in Tomcat (or even most JVMs). Here are the important points from that link:
There are three aspects of a class loader behavior
Lazy Loading
Class Caching
Separate Namespaces
The JVM will get very heavy if all classes inside all JARs get loaded indiscriminately. So the classes inside shared JARs are loaded only on-demand. The only way for you to invoke the main method is to add the above lifecycle listener.
Perhaps the simplest thing to do is to deploy a trivial servlet in a .war file that references your .jar file. The servlet can be configured to start up upon deployment/container start, and then it can invoke the class containing your main() method.
As application servers / servlet containers typically have a lot of different classloaders, you'll most likely need a different strategy for weaving aspects into your code than in standalone applications.
I would recommend to add the aspects to every war file deployed at build time. This might be following a common technique - as opposed to a server specific one.
Further, I'm not sure it can actually be done (properly & supported) on a server. Typically a server is built to separate all webapps from each other. You might get it to work, but it might break on the next update of the server.
It might be easier to suggest an alternative technique if you'd state the problem that you want to solve with your proposed approach.
Edit after your comment: Consider the standard web application lifecycle: You can execute some code, e.g. in a servlet, upon it being deployed. If you insist on your code being contained in main, you can call this method from your webapp's initialization code.
You need to register a Java Agent. See this link: java.lang.instrument.
java.lang.instrument provides services that allow Java programming language agents to instrument programs running on the JVM.
This is the right way to do this.