I'm making software (for Android) to read out meter values from sensors.
The software is expected to receive updates very often in order to support more sensors. The functionality to read out the meter data voor these sensors is implemented in native libraries that are downloaded on runtime by the app.
I've got a list of shared objects and an entry function that should be called (the return value is saved):
libprintf.so,hookPrintf
libharx.so,hookHarx
Each library is loaded at runtime: System.loadLibrary("printf");
The signature of each function would look like this:
public native int hookPrintf();
public native int hookHarx();
The problem is that I don't know beforehand (at compile time) which shared objects will have to be loaded, and thus I can not add these signatures in the java source.
I tried using Reflection to call the methods dynamically as follows:
Method entryMethod = Expat.class.getMethod(hookMethod);
Object test = entryMethod.invoke(this);
System.out.writeln(test.toString());
However, if the signature is not present, this will still fail with a NoSuchMethodException. Is there another way to be able to call native methods without needing to have them defined beforehand?
Edit:
Using the RegisterNatives function, I can register my functions dynamically from the shared object, but that still requires the method signature to be defined in the class.
What does the registerNatives() method do?
You probably can have a Java class with only two native methods, and rely on RegisterNatives and UnregisterNatives and switch the loaded native implementations at runtime. With that, I would rather use a native wrapper library with native methods
private static native void hook(String libName, String hookName);
This wrapper will use dlopen() or LoadLibraryEx() and also unload the external libraries when necessary.
In some JVMs you can dynamically create new classes, but that is quite tricky.
You have a fundamental design issue. I would suggesting going back and create a JNI library that implements a single method like public native int createHook() or something equivalent. Then make the choice in your native code based on some conditional. If you still want/need to load the libraries at runtime, create non-jni libs for libprintf.so and libharx.so and use either dlopen to dynamically load the library or weakly link the libraries to the jni library.
Related
So I'm using a third party library in a Play application (namely the Echo Nest Java API). There are some oversights in how some methods are exposed by the library, and I need to modify one method in particular, which involves using a constructor with default access.
I had initially thought that creating a class in the same package that extends the class that I need to modify would do the trick, but it seems Java's runtime package handling is thwarting that attempt (that is, since different classloaders are being used for the two different classes, their being in the same package is not enough). I have some notion of potentially modifying the classloader for either my subclass or the Echo Nest library, but have just about no idea how to go about doing that or whether there's a better solution available.
Any pointers in the right direction would be appreciated!
So it turns out the solution was to load the library class using Play's default classloader, set the constructor to be accessible, and then use newInstance() to instantiate the library class. Some code, in case any one else runs into a similar problem:
// Use the default application classloader to load the library class
Class artistClazz = Play.application().classloader().loadClass("com.echonest.api.v4.Artist");
// Get the package private constructor that we need
Constructor<Artist> constructor = artistClazz.getDeclaredConstructor(EchoNestAPI.class, Map.class);
// Make sure it's accessible to this class
constructor.setAccessible(true);
return constructor.newInstance(this, (Map) mq.getObject("artist"));
I'm not by any means convinced that this was the best or cleanest solution to this problem, but it minimally affected the code outside of this modified subclass so I'll probably stick to it for now.
After some research I cannot come across the best approach for this. There will be certain color classes that I would like to share amongst multiple projects. Let's call one of them EncryptedColor. Since it is used across multiple projects I don't want multiple copies of it in existence of course. Otherwise I would need to make sure that an update in one location would need to be updated everywhere. However, these classes are needed in some released SDKs that we provide to customers.
How could I design it such that I can use these classes but not provide them with the classes that they don't need access to from their SDK. I don't want useless classes to become visible and flood the smaller subset of classes that they really need to be seeing.
A couple approaches I have thought of so far but aren't quite ideal:
Try and use a doclet structure that hides the calls within the javadoc such as doclava. Javadoc has not fully implemented its own hiding mechanism yet. As I understand this doesn't keep the functions from being visible, but it was mentioned in one spot that you would need reflection to use the calls. I don't see how just the javadoc does that so I must have been missing something.
Android has designed themselves it seems to force reflection from some #hide attributes included in methods that they have in source code. But from the sounds of it, the system hides those and then uses a different jar when it is loading to make those visible at launch time. Probably not useful here.
If I were to keep shared classes in the same package name I could access default and protected members, but...then I am keeping all my classes that use these in the same package name. Not quite ideal either, but it could be done in that manner if I needed to. Might get out of hand with large quantities of shared resources.
What approaches are taken typically in situations such as these? I haven't liked my findings and thought process thus far.
Short answer : you can't hide/remove these classes as they are needed at runtime by your application.
In my opinon, you have 3 alternatives :
Change the classes access to "package private". Yes, doing that doesn't make it impossible to access them, but these classes won't be accessible directly.
Remove the classes and create an API. You want to hide the logic ? Remove it and provide it through a REST API for instance. Depending or your architecture, it could be difficult or impossible.
Create all the instance of these classes in a dynamic way, with Class.forName, using Spring or as in #Steve K answer, with Java's ServiceLoader. As a result, you will be able to remove these classes from the main jar and make them more private, in a way. Again, classes will be here but a little less accessible.
My suggestion that could work would be to implement your color classes as a service using the Java ServiceLoader
You make an interface for your color classes, and implementations can be called using the ServiceLoader class. Then you simply separate your color classes into two packages - a public package you can jar up and distribute with your SDK, and a private package for those classes you want to be internal. The ServiceLoader will find all the color classes available so long as the jar files are in your project's classpath.
For example, if your color classes (as an example) had a common interface like this:
public interface MyAppColor {
public int getRed();
public int getGreen();
public int getBlue();
public int getAlpha();
public void setRed(int red);
public void setGreen(int green);
public void setBlue(int blue);
public void setAlpha(int alpha);
public boolean isValid();
public void doSomething(Object arg);
}
Then you could have a bunch of implementing classes in a jar file, with a service descriptor file included in the jar at the path:
META-INF/services/com.my.app.MyAppColor
The text of that file is simply the list of classes in the jar that implement the interface - one per line:
com.my.app.MyPublicAppColor
com.my.app.MyEncryptedPublicAppColor
com.my.app.MyOtherPublicAppColor
etc. Then all you have to do is make a factory for instantiating the correct type, which could be as simple as this:
public class MyAppColorFactory {
private static ServiceLoader<MyAppColor> serviceLoader = ServiceLoader.load(MyAppColor.class, null);
public static MyAppColor get(String className){
if (className != null){
for (MyAppColor c : serviceLoader){
if (className.equals(c.getClass().getName())){
return c;
}
}
}
return null;
}
}
Deploying only needed code:
- Use Only The Needed Source In Development (1) (2)
Since you have an entire library and many deployments which each use different components, the easiest way to do what you suggest is to use only the sources that you need; not a single library. You can ignore the unused sources. This will only ship the needed code.
- Make The Library "Package Private"
This will allow the access only for the public components of the library and everything else will not be callable. But, it will still ship all the code.
- Create an API as a REST SDK
This will require web access, not desirable for performance code {any code really}. You will ship no sdk code with this method.
- Obfuscate the code
Easy with the correct tools. Obfuscation will change the class and method names in production code to gibberish. This will make the library basically unusable to anyone but you. This will ship all the code but it will be obfuscated
- Native API
You can compile java to machine code and use it in production or as the api. You can also create the api in a native language {not desirable}.
I have a large data set. I am creating a system which allows users to submit java source files, which will then be applied to the data set. To be more specific, each submitted java source file must contain a static method with a specific name, let's say toBeInvoked(). toBeInvoked will take a row of the data set as an array parameter. I want to call the toBeInvoked method of each submitted source file on each row in the data set. I also need to implement security measures (so toBeInvoked() can't do I/O, can't call exit, etc.).
Currently, my implementation is this: I have a list of the names of the java source files. For each file, I create an instance of the custom secure ClassLoader which I coded, which compiles the source file and returns the compiled class. I use reflection to extract the static method toBeInvoked() (e.g. method = c.getMethod("toBeInvoked", double[].class)). Then, I iterate over the rows of the data set, and invoke the method on each row.
There are at least two problems with my approach:
it appears to be painfully slow (I've heard reflection tends to be slow)
the code is more complicated than I would like
Is there a better way to accomplish what I am trying to do?
There is no significantly better approach given the constraints that you have set yourself.
For what it is worth, what makes this "painfully slow" is compiling the source files to class files and loading them. That is many orders of magnitude slower than the use of reflection to call the methods.
(Use of a common interface rather than static methods is not going to make a measurable difference to speed, and the reduction in complexity is relatively small.)
If you really want to simplify this and speed it up, change your architecture so that the code is provided as a JAR file containing all of the compiled classes.
Assuming your #toBeInvoked() could be defined in an interface rather than being static (it should be!), you could just load the class and cast it to the interface:
Class<? extends YourInterface> c = Class.forName("name", true, classLoader).asSubclass(YourInterface.class);
YourInterface i = c.newInstance();
Afterwards invoke #toBeInvoked() directly.
Also have a look into java.util.ServiceLoader, which could be helpful for finding the right class to load in case you have more than one source file.
Personally, I would use an interface. This will allow you to have multiple instance with their own state (useful for multi-threading) but more importantly you can use an interface, first to define which methods must be implemented but also to call the methods.
Reflection is slow but this is only relative to other options such as a direct method call. If you are scanning a large data set, the fact you have to pulling data from main memory is likely to be much more expensive.
I would suggest following steps for your problem.
To check if the method contains any unwanted code, you need to have a check script which can do these checks at upload time.
Create an Interface having a method toBeInvoked() (not a static method).
All the classes which are uploaded must implement this interface and add the logic inside this method.
you can have your custom class loader scan a particular folder for new classes being added and load them accordingly.
When a file is uploaded and successfully validated, you can compile and copy the class file to the folder which class loader scans.
You processor class can lookup for new files and then call toBeInvoked() method on loaded class when required.
Hope this help. (Note that i have used a similar mechanism to load dynamically workflow step classes in Workflow Engine tool which was developed).
Can I enumerate all native methods in java, those that have to be
implemented in c/c++ using JNI?
Can I enumerate native methods by
name (there could be multiple overloads with the same name)?
How can I retrieve method signature to be able to generate the method
signature used by JNI?
Is there a way to check if all native jni methods have been bound properly, instead of trying to call them and get java.lang.UnsatisfiedLinkError exceptions. Sometimes method signature changes on either side without properly updating java or c++ side and I'd like to add some debugging code to detect these issues and handle them (perhaps by generating proper method signature and printing it to the log so I can easily fix the code).
I prefer JNI solution, but if something can be done with help on java side then it's ok also.
If I use registerNatives and register methods that weren't declared in java then it fails and prints it to logcat:
E/dalvikvm( 1445): ERROR: couldn't find native method
E/dalvikvm( 1445): Requested: Lcom/bla/bla/bla/Test;.nativeTestXX:()Z
but I'd like to catch this error and handle it myself. Is it possible to do it?
EDIT:
In my JNI code I have a static nativeInit (as suggested in Android JNI tips) that registers all native methods. In that same function I'd like to verify that all native methods are properly bound. That is, I don't need to wait till some uninitialized method is called and the app exists. The problem that I have: there is a lot of jni code written at different times by different ppl and some methods simply became incorrect, but they are used only in some obscure conditions. The best way for me, I think, is to check that all native methods are bound to some c++ function. The other problem, is that part of JNI code uses binding by exporting all these Long_java_names where method signature changes on either side cannot be detected.
There is no call to check for "unbound" native methods. Using RegisterNatives to perform explicit registration ensures that all methods you register have a matching declaration in the Java sources, but there is no way to check for native-declared methods for which there is no implementation (other than calling it and catching the exception).
At the point where a method with a native implementation is called, if nothing has yet been registered then Dalvik will search through the various shared libraries to find a match. What it sounds like you want is a way to force this search and check the result without actually calling the method. There is no such thing.
There are various ways to generate lists of native-declared methods, either statically or at runtime, but you also need a way to determine if an implementation is available. You're better off in the long run having unit tests that exercise the code.
I am looking for a way to dynamically define classes and instantiate them in Android, at runtime. From my understanding, this is already done in Android, I just need some help figuring it out.
I can a similiar result in Javascript and PHP. I know it can be done in Java using something like ASM, BCEL or CGlib. However, I do not know enough about any of these to understand if they will work on Android. Or, of they will work, what are the implications?
If, hypothetically, all three will work in Android, can someone point me in the correct direction as to where to start understanding which to use, and how to use it?
I haven't done much Java programming, and I have only just recently been working with it in Android, so, I appreciate all of the help/correction I can get. With that said, I would appreciate if your answer is NOT simply: Don't do this. I am looking for how to do this specifically, not how to do it right. At least, not until my app comes crashing down. :)
I believe that this already happens in Android in the following situations: (Not 100% SURE)
Creating an object from JSON.
AIDL
Don't do this :)
I actually doubt there are JSON libraries that behave this way; the two accepted ways I know (I am not an expert on this, though) are either to create some sort of data structure holding name-value pairs - i.e. add stuff to a data structure but not create a new class - or prepare a template of a class which will be populated from a JSON object.
Java, being statically-typed, is not really suitable for creating whole new classes at run-time, and there is no reflection support for that - though there is support for accessing objects of unknown types (e.g. querying for all their fields / methods).
What you can do is to manually write a java class to a file - either in Java code and then compile it somehow, or directly in bytecode - and then load that file at runtime. It's ugly, but it will work. Then it's just the same as any runtime loading of classes - either you rely on the base class / interface of the loaded class, or you have to use reflection to do anything meaningful with it.
For those who really do want to do this (for instance using Dalvik's JIT to create a fast interpreter for another language), there is this project:
http://code.google.com/p/dexmaker/
Which allows you to programatically create classes, variables and methods.
Generating Dalvik Bytecode at Runtime on-device Using ASM or BCEL
This example use ASM and BCEL to generete two classes on-device.
The classes are created into SD Card memory and then they are loaded into Android operating system dynamically.
The following class is the template of the example:
public class HelloWorld {
public static void hello(){
int a=0xabcd;
int b=0xaaaa;
int c=a-b;
String s=Integer.toHexString(c);
System.out.println(s);
}
}
Firstly I have used BCEL or ASM to create a new ad-hoc class in SD Card.
Secondly I have converted the Java Class to a Dex Class with the Dxclient utiliy in SD Card.
Finally I have created a jar file and then I have loaded this package into the device from SD Card
DXClient reference
https://github.com/headius/dexclient/blob/master/src/DexClient.java