I'm looking into using OpenCV for an Android application and am running into some problems compiling the samples which use Native OpenCV.
So, my question is:
When using OpenCV, when will I need to use Native OpenCV code? Can I get by without using it (as in Tutorial 1 within the samples)?
Apologies for the vague question, I can be more specific if that would help.
Thanks for any help in advance.
It depends on what you want to do, really. Native calls are very expensive in terms of speed, so if this is relevant for your application, you should reduce them to the minimum possible. Most of the Android OpenCV libraries rely on native calls, so going native or using OpenCV Java calls should not give a relevant increase in terms of performance time. But there is nothing like testing it.
You would also like to use native code if you already have C/C++ OpenCV implemented code. That would save you some time re-writing it in Java. You should also use native code, if the functions you want weren't already ported to Android (it is up to 42%, currently).
Related
Actually, I always believe the answer is no: an Android application must run over a separate instance of ART (like Java over JVM) to translate and take care of cross-platform needs.
However, I then feel confused about the figure of the android stack that Google puts. The way it draws seems to imply that, System Apps + Java API framework could go to the bottom through Native c/c++ libraries without Android Runtime. But I would like to argue that, if Java needs to call to native space, Android Runtime (JVM/legacy Dalvik) is still needed to interpret the source and make calls to native via JNI. Simply putting Java API Framework over Native Libraries is misleading to me.
Please comment and correct me if anything is logically incomplete or incorrect. Thanks
I've been doubted that it is not possible. However, If you decompile an apk to smali, you will see that smali is a more easily read java byte code. Convert smali from an apk to java source code and you have ART's java translation.
You can emulate/replace/recreate the Native/ART implementation and everything below that is self explanatory. It is linux based with hardware attributes that provide input to ART, retrieved by C++ only to be used by java producing the systems applications.
I am building an Android App that includes image processing techniques. From the Runtime point of view, which is better JavaCV or OpenCV ?
Their runtime overhead seems to be about the same, but the android-opencv wrappers do not give access to raw data via direct NIO buffers, rendering custom processing in Java a lot less efficient. JavaCV is more efficient for those tasks. Being the author of JavaCV, I also like its API better :) It's closer to the original C/C++ API than android-opencv.
EDIT: Another difference is that android-opencv forces users to use the NDK and JNI to write functions. Since JavaCV is based on JavaCPP, we can write native functions more easily without having to deal with neither the NDK nor JNI.
Writing native code using Android NDK will usually be much faster than using the SDK.
You can find nice tutorial here, using opencv with NDK. Notice that with NDK framework you have access to openGL functions.
JavaCV is just a Java wrapper around the native OpenCV library. OpenCV does provide a Java/SWIG binding. I have used both bindings and didn't see a significant difference between the two.
If you are writing a native only app go with native OpenCV.
I'm a complete newbie in the computer vision world and recently I implemented some examples using OpenCV with Java. I'm impressed with the potential this area has and wish to continue learning more.
I learned that OpenCV is written in C++ and while Java has a wrapper (JavaCV) I understood that the applications in Java are slower than in C++ and most enterprise application are written in C++.
My question is this: I have few years of experience in Java and I feel very comfortable to write any application with it; would it be smart to move to C++ to learn CV or should I stick with Java and use its wrapper.
Computer vision is a demanding area - and while it is true that you'd best stay with what you know, and move to opencv only if performance is needed, another truth is that you'll need to go deep into mathematics, pointers and algorithms to learn and build a good computer vision app. And to do that in Java can be more cumbersome than learning c++.
So, if all you want to do is to apply a filter over an image for some school project - go for Java. But if you want to stay more with OpenCV, to learn vision algorithms or to write your own, my strong suggestion is to learn C++ - isn't that scary!
A reason to write native code is flexibility - you'll want to do all kind of tricks that are difficult or performance-killers in Java.
Shortly speaking, learning C++ is much simpler than computer vision. And OpenCV is not just a library you want to call to do some processing out there. It's bleeding edge technology - you'll want to understand it, to hack into it, to build over it, to go through the code, much more than call someJNIfunc();
And if you do so, please be aware that OpenCV offers two interfaces - one for C and one for C++. And while they offer the same functionality, the C++ one is much like Java - with automatic memory management and more sweeties. You can refer to this post to see the differences
I suggest sticking with what you're comfortable with for now. Only switch to C++ when you find that it doesn't expose certain APIs you want or performance actually becomes a problem. Right now, you're in the learning phase.
JavaCV uses a wrapper called JavaCPP to call OpenCV from Java programs. JavaCPP automatically generates temporary native libraries that form a bridge used by JNI to let Java programs call the OpenCV native libraries.
The solution is elegant and it works well, but is quite finicky about installing just the right versions and having classpaths set correctly. You can get a glimpse of the difficulties people face at the JavaCV discussion forum, and at http://code.google.com/p/javacv/wiki/Windows7AndOpenCV.
I got this working with OpenCV-2.3.1 on XP, Windows 7, and Ubuntu 10.11, and still it took me several days to get it all updated to OpenCV-2.4.2. ffmpeg is especially tricky to get right across all platforms.
There is little or no speed overhead if you are using Java for high level program control because image objects and list-based data structures are maintained on the native side through pointers. One pitfall is knowing who is responsible for releasing allocated memory, so be prepared for VM crashes with complicated programs.
There is a bit overhead in transferring data objects to the Java side. I find that it takes about 1 microsecond to copy a keypoint location into a Java-side Point object. This doesn't sound like much but in a real-time application with thousands of keypoints it can make a difference. We also copy JavaCV IplImage objects to Java through ByteBuffers. This takes a millisecond or less so is quite feasible for real-time use.
In our case, we have a substantial body of Java code to leverage against OpenCV. And Java's garbage collection makes many things much much easier. I are satisfied that the overhead of learning JavaCV has been well worth it.
I found it necessary to build the project in Eclipse and compile JavaCV from source instead of using javacv.jar. (You'll need the other .jar files though.) This lets you examine exceptions to track down library version and classpath errors. And the JavaCV source is needed to understand how JavaCV exposes the OpenCV API.
How much time is spent in the OpenCV library and how much time is spend in your program? If your program entirely in C++ it cannot reduce the time spent in your program (outside the library) to less than nothing. e.g. if you spend 99% of your time in the library, using C++ cannot make it more than 1% faster.
For simple programs, Java and C++ doesn't make any significant difference in speed. But for a large code with lot of computational complexity, C++ turns out to be faster.
Wrappers do have a problem of overhead. But this will be negligible for a small program. If you write a complex code in Java, it wont be easy to rewrite it into C++. This is because of the large no of functions available in Java which will be so much different from C++.
Whether you should use Java or C++ for OpenCV depends on your motive.
If you seriously want to lean OpenCV and work on some big projects , I suggest you to move on to C++. But if you are looking for having just some fun with OpenCV, it will be better to stick to what you know.
Among C++ or Java, better use C++. I have almost no experience in Java, but the reason I would recommend C++ is its common usage among ML&CV libraries.
One of the best solution of powerful and flexible computer vision application may be next sequence:
training a model using flexible Python and popular machine learning libraries;
storing pre-trained weights;
rewriting the best model architechture to C++ using OpenCV library with useful, but not extensive Mat class or compiling to C++ ML model;
compiling with definite specification of your device processor.
Advances of this solution are application work speed, code safety, development speed and compatibility with many devices.
Moreover, Python Tensorflow and other models could be loaded to C++, e.g. PyTorch.
Besides, it is much more easier to get help from CV community with the C++ OpenCV code.
Disadvantages are tons of development problems and development speed (which may not be omitted with Java), sometimes strictful flexibility limits of OpenCV lib, and many others you may not think of at the beginning.
I've been playing around with the NDK recently, finding that many of the tutorials available online really don't help. I've been using this tutorial and I've got it running great.
However. Is this the correct use of the NDK? I mean if I have a game say with many classes all in C++ that I wish to port over to the android. Do I have to really manually change all my methods to the likes of:
JNIEXPORT jstring JNICALL Java_com_domain_ndk_NativeLib_hello
(JNIEnv * env, jobject obj) {
return (*env)->NewStringUTF(env, "Hello World!");
}
I can't see this being a very efficient way of porting my code over and I get the feeling I'm using the NDK wrong. I also have no idea how the NDK samples are supposed to work. Could anyone point me in the right direction?
Thanks
You do not need to change ANY of your existing methods to use the "auto-bound" JNI format. There aren't good examples in the Google NDK samples, but the Google SDK uses the other JNI binding method internally: Search for RegisterNatives() in the JNI docs. But even then, you really only need to create a wrapper for the few places that the Android SDK needs to talk to your library: user input/sensors in general, a draw callback, and possibly an "update" callback. Aside from that the rest of your app should remain untouched, except to get it to build.
To simplify things, I'd recommend you not use the native widgets; if you keep your game entirely in OpenGL, you can avoid going through Java to, e.g., draw textures to the screen, at least if you target 1.6 devices and newer, where the NDK has native bindings for OpenGL.
Contrary to some opinions (including those of Google), the NDK can and frequently is used for porting games to Android. I'm using it that way right now, in fact. You need to pass user events from Java to C++, and you need to have Java set up the basic GL context (there are no EGL bindings in the NDK), but you can do pretty much everything else in C++ with OpenGL and, once your basic Java messages are being passed to C++, you can (mostly) ignore Java. Thank goodness. :)
You'll still need to talk to Java for sound, and to get access to your files: Both still need to go through Java, and so you'll probably want to use "reverse JNI," where you call a Java function from C++. Also not hard, and there are examples all over the place.
I talk more about how to access your assets here: File Operations in Android NDK
Good luck.
I'd say your way isn't wrong, but be aware that passing data between Java and C/C++ code is a time consuming thing. Because of that I would suggest you write the most of the code actually in C/C++ and just call C/C++ functions from Java if it can't be avoided. For example you will need to pass data back for the GUI. Especially when you are programming a real time game I would avoid passing data between Java and C/C++ code too much. So just changing all the methods and then calling them in your Java code wouldn't be such a great idea.
Btw if you have to pass something back from C/C++ to Java, I would not pass it as a return value, but rather give the destination as a parameter in the function call.
The NDK is just a native C and C++ development environment for Android. Bionic (the Android version of libc) is more slightly more limited not by much. The only real complication is integrating you new component into the Android build tool chain. Android has a highly customized Makefile system. Downloading the source and looking at simple native components should give you an idea how to use it though.
Now if you want to integrate Java with C or C++ you will need to understand the Java Native Interface (JNI). You should be able to find good documentation of the JNI online.
The new release of the ndk (r5) now includes the capability to write apps entirely in native code.
Are there inexpensive or free gateways from .NET to Java? I'm looking at some data acquisition hardware which has drivers for C/C++ and .NET -- I really don't want to do any programming in .NET.
Update: I haven't done what I originally wanted to do, but I've done something similar, using JNA to encapsulate some functions from a DLL, in order to control a USB hardware device from Java. (the DLL comes from the device manufacturer) It works really nicely. Thanks!
You could also try to use JNA for accessing the native library. JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required. If their API is fairly straight foward, this might be the path of least resistance.
See their getting started guide where they call some native code (printf and GetSystemTime).
Well, there's JNBridge and EZ JCom, just from a Google search.
You could also use IKVM which is a slightly different approach.
(Any reason for not wanting to learn .NET, out of interest? It's a nice platform, and C# is a lovely language...)
If they have C++ versions of the drivers then you could write a wrapper around it using JNI and then load that in Java. JNI can be a bit of a pain, but it would let you use the C++ version of their drivers and not have to deal with .Net at all if you don't want.
I am partial to the recommendation to jump in the deep end with C# since it is so similar to Java. I did this and used IKVM to compile my favorite Java libs. to .NET assemblies and you get [nearly] all the core java runtime classes to boot, so if you tire of trying to find just the right C# collection type, you can always go back to java.util. (No generic collections though. Not sure why.)
Depending on what platform you're on, you have several choices for free IDEs too. For windows you can get Visual Studio Express for free but I also use SharpDevelop. You can also get the Mono IDE on Linux (and a few flavours of Unix, I think ?).
The C# learning curve is shallow if you already know Java. I only blew off 1.5 limbs on landmines that came out of nowhere for reasons I still don't understand, but workarounds were easy to come by. The worst thing about it was the darn developer docs which are AWFUL on account of being so slow. I really miss the snappiness of JavaDoc. Not only are the online docs incredibly slow, the problem is compounded by someones's iffy decision to put class summaries, constructors and methods/properties all on seperate pages so it just takes forever. Someone said to get the docs installer and install docs locally for a slightly improved experience. Not a bad idea I suppose.
I am author of jni4net, open source interprocess bridge between JVM and CLR. It's build on top of JNI and PInvoke. No C/C++ code needed. I hope it will help you.
If you have a Java application, the JNI mentioned by the others will be the way to go. You write some wrapper classes, and that's it.
If writing the wrappes is a too big task (depending on the number of methods you have to wrap), have a look at SWIG . I think it generates wrappers automatically, but I never actually used it.
If you want to code in the Java language, but you don't care if your program will run on the JRE/JVM, then you might as well use Microsoft J#. Basically, it's writing Java-Code wich is compiled to .NET-Bytecode and can use the .NET classes of the driver as well as your existing Java classes. With J# you will run into problems if your existing Java-code is newer than Java 1.4, look at this question on how to solve them.
From that point on, you could later add code in J#, C# or any other .NET language. However, you won't get back to the JRE/JVM easily.