Develop Reference

java.lang.VerifyError Precise Reference expected while trying to runBlocking Kotlin suspended fun in Android

I'm trying to write a unit test waiting for completion of a kotlin suspended function before checking results like this :
#Test
fun shouldSetupThingsProperly() {
val context = InstrumentationRegistry.getInstrumentation().context
runBlocking { MyObject.enable(context, false) }
Assert.assertTrue( /* whatever usefull */ true)
}
The suspending methods are as follow :
object MyObject {
#JvmStatic
suspend fun enable(context: Context, enable: Boolean) {
withContext(Dispatchers.IO) {
// ... do some work
wakeup(context)
}
}
private suspend fun wakeup(context: Context) {
withContext(Dispatchers.IO) {
try {
// setup things ...
} catch (ignore: Exception) {}
}
}
}
Test run ends with :
java.lang.VerifyError: Verifier rejected class MyObject: java.lang.Object MyObject.enable(android.content.Context, boolean, kotlin.coroutines.Continuation) failed to verify: java.lang.Object MyObject.enable(android.content.Context, boolean, kotlin.coroutines.Continuation): [0x16] register v7 has type Reference: android.content.Context but expected Precise Reference: MyObject (declaration of 'MyObject' appears in /data/app/test-_rphd0tDrOp0KM-Bz09NWA==/base.apk!classes2.dex)
at MyObject.enable(Unknown Source:0)
I'm not familiar with coroutine and I was wondering how to achieve waiting for completion of the enable suspended function inside the test properly or if error was due to some other mistake...

If it happend to coroutines- withContext on Android or Flutter, revert coroutines lib to 1.3.6 solved crash issue for me.
It seems that there is VerifyError bug in android coroutines lib version 1.3.7-1.3.8, and will be fixed after 1.4.0.
Details see links:
https://github.com/Kotlin/kotlinx.coroutines/issues/2049 https://github.com/Kotlin/kotlinx.coroutines/issues/2041

Testing coroutines is a trick, even after some experience.
If you can import, this will be very helpful: https://github.com/Kotlin/kotlinx.coroutines/tree/master/kotlinx-coroutines-test
If you have this dependency testing coroutines becomes much more manageable.
First off, if you can have the dispatcher you are running this a variable or parameter that can be set or overridden it will help you increase your testability.
As far as writing the test you can do something like:
#Before
fun before() {
Dispatchers.setMain(mainThreadSurrogate)
}
#Test
fun shouldSetupThingsProperly() = runBlockingTest {
val context = InstrumentationRegistry.getInstrumentation().context
MyObject.enable(context, false, Dispatchers.Main)
Assert.assertTrue( /* whatever useful */ true)
}
Your object itself will have I would say more of the changes
object MyObject {
#JvmStatic
suspend fun enable(context: Context, enable: Boolean, dispatcher: CoroutineDispatcher = Dispatchers.IO) {
// If you need a return feel free to use withContext such as:
// val result = withContext(dispatcher) { /* Return Value */ Any() }
CoroutineScope(dispatcher).run {
// ... do some work
wakeup(context)
}
}
private suspend fun wakeup(context: Context) {
// Another coroutine scope is unnecessary here, it will inherit the parent scope automatically, so you can call
// async functions here
delay(200)
try {
// setup things ...
} catch (exc: Exception) {
// We had an issue
}
}
}

Coroutines delegate exceptions

Currently, I have some scenario like this where I have java interface callback which looks something like this.
Java Callback
interface Callback<T> {
void onComplete(T result)
void onException(HttpResponse response, Exception ex)
}
Suspending function for the above look like this
suspend inline fun <T> awaitCallback(crossinline block: (Callback<T>) -> Unit) : T =
suspendCancellableCoroutine { cont ->
block(object : Callback<T> {
override fun onComplete(result: T) = cont.resume(result)
override fun onException(e: Exception?) {
e?.let { cont.resumeWithException(it) }
}
})
}
My calling function looks like this
fun getMovies(callback: Callback<Movie>) {
launch(UI) {
awaitCallback<Movie> {
// I want to delegate exceptions here.
fetchMovies(it)
}
}
What I'm currently doing to catch exception is this
fun getMovies(callback: CallbackWrapper<Movie>) {
launch(UI) {
try{
val data = awaitCallback<Movie> {
// I want to delegate exceptions here.
fetchMovies(it)
}
callback.onComplete(data)
}catch(ex: Exception) {
callback.onFailure(ex)
}
}
}
// I have to make a wrapper kotlin callback interface for achieving the above
interface CallbackWrapper<T> {
fun onComplete(result: T)
fun onFailure(ex: Exception)
}
Questions
The above works but is there any better way to do this?? One of the main thing is I'm currently migrating this code from callback so I have ~20 api calls and I don't want to add try/catch everywhere to delegate the result along with the exception.
Also, I'm only able to get exception from my suspending function is there any way to get both HttpResponse as well as the exception. Or is it possible to use existing JAVA interface.
Is there any better way to delegate the result from getMovies without using callback??

Is there any better way to delegate the result from getMovies without using callback?
Let me start with some assumptions:
you're using some async HTTP client library. It has some methods to send requests, for example httpGet and httpPost. They take callbacks.
you have ~20 methods like fetchMovies that send HTTP requests.
I propose to create an extension suspend fun for each HTTP client method that sends a request. For example, this turns an async client.httpGet() into a suspending client.awaitGet():
suspend fun <T> HttpClient.awaitGet(url: String) =
suspendCancellableCoroutine<T> { cont ->
httpGet(url, object : HttpCallback<T> {
override fun onComplete(result: T) = cont.resume(result)
override fun onException(response: HttpResponse?, e: Exception?) {
e?.also {
cont.resumeWithException(it)
} ?: run {
cont.resumeWithException(HttpException(
"${response!!.statusCode()}: ${response.message()}"
))
}
}
})
}
Based on this you can write suspend fun fetchMovies() or any other:
suspend fun fetchMovies(): List<Movie> =
client.awaitGet("http://example.org/movies")
My reduced example is missing the parsing logic that turns the HTTP response into Movie objects, but I don't think this affects the approach.
I'm currently migrating this code from callback so I have ~20 api calls and I don't want to add try/catch everywhere to delegate the result along with the exception.
You don't need a try-catch around each individual call. Organize your code so you just let the exception propagate upwards to the caller and have a central place where you handle exceptions. If you can't do that, it means you've got a specific way to handle each exception; then the try-catch is the best and idiomatic option. It's what you would write if you had a plain blocking API. Especially note how trivial it is to wrap many HTTP calls in a single try-catch, something you can't replicate with callbacks.
I'm only able to get exception from my suspending function is there any way to get both HttpResponse as well as the exception.
This is probably not what you need. What exactly do you plan to do with the response, knowing that it's an error response? In the example above I wrote some standard logic that creates an exception from the response. If you have to, you can catch that exception and provide custom logic at the call site.

I am not so sure whether you really need that awaitCallback or not.
If you really have lots of Callback already in place and that's why you used it then your functions will probably already have everything in place that works correctly with the Callback, e.g. I expect some methods as follows:
fun fetchMovies(callback : Callback<List<Movie>>) {
try {
// get some values from db or from a service...
callback.onComplete(listOf(Movie(1), Movie(2)))
} catch (e : Exception) {
callback.onFailure(e)
}
}
If you do not have something like this in place, you may not even need awaitCallback at all. So if your fetchMovies function rather has a signature as follows:
fun fetchMovies() : List<Movie>
and in getMovies you pass your Callback, then all you need is probably a simple async, e.g.:
fun getMovies(callback: Callback<List<Movie>>) {
GlobalScope.launch { // NOTE: this is now a suspend-block, check the parameters for launch
val job = async { fetchMovies() }
try {
callback.onComplete(job.await())
} catch (e: Exception) {
callback.onException(e)
}
}
}
This sample can of course be changed to many similar variants, e.g. the following will also work:
fun getMovies(callback: Callback<List<Movie>>) {
GlobalScope.launch { // NOTE: this is now a suspend-block, check the parameters for launch
val job = async { fetchMovies() } // you could now also cancel/await, or whatever the job
job.join() // we just join now as a sample
job.getCompletionExceptionOrNull()?.also(callback::onFailure)
?: job.getCompleted().also(callback::onComplete)
}
}
You could also add something like job.invokeOnCompletion. If you just wanted to pass any exception to your callback in your current code, you could just have used callback.onException(RuntimeException()) at the place where you put your comment I want to delegate exceptions here..
(note that I am using Kotlin 1.3 which is a RC now...)

Generic rxjava2 database access layer

I just started with java/rxjava2/android dev and managed to get the following working example:
Observable<Object> source3 = Observable.create(emitter-> {
cursor = app.dbh.getAlllTransactions2();
emitter.onNext(cursor);
emitter.onComplete();
}).subscribeOn(Schedulers.io());
source3.subscribe(c -> {
transactionAdapter = new TransactionCursorAdapter(this.getActivity(), (Cursor)c);
LSTVW_transactions.setAdapter(transactionAdapter);
});
Now I have 2 questions:
how is it that I am forced to use Object as a type. If I use anything else
android studio says it expects Object. Is it because of the lambda expression. I have done tests before and they allowed me to use any type.
I would like to make the below in a more generic fashion. The goal is to have Observable as the result with an arbitrary db function as a parameter which in then generically called. An older example I have found of this can be found here but I don't see how i could convert it to lambda/rxjava2 style (original link: https://dzone.com/articles/easy-sqlite-android-rxjava)
An example of such setup which I would like to convert:
private static <T> Observable<T> makeObservable(final Callable<T> func) {
return Observable.create(
new Observable.OnSubscribe<T>() {
#Override
public void call(Subscriber<? super T> subscriber) {
try {
subscriber.onNext(func.call());
} catch(Exception ex) {
Log.e(TAG, "Error reading from the database", ex);
}
}
});
}

Try this:
Observable.create((ObservableOnSubscribe<YourType>) e -> { ... }
I don't get exactly what do you want to achieve with the second snippet, but I think you can simplify just having this body, for the makeObservable method (I just removed the try-catch part):
return Observable.create(e -> e.onNext(func.call()));
About Rx abuse: I think that it is not a good idea to pass the Cursor as item of a stream. You would probably have a stream of data read from the database, so that your Observer can react properly.

Java Functions, Returns, and Optionals

I am trying to create a client library that reads JSON from an external file online. I already know about the function interfaces and optionals, but I was wondering if there is a way to allow users to supply callback functions such that the parent function exits completely. For JavaScript, such a function is as follows:
file.read('hello', function(err, data) {
// something here
});
Basically, I wish to do the same in Java. How can I do this such that the error callback supersedes the read function? What I mean is that in the event that the error callback is called, then read should not return a value at all. If the callback is not called then the read should return the value.

You could have the user pass in a function and then just not do anything with it if there is no error.
This example assumes that you have a custom class called Error that the caller is aware of and would like to interact with in case of an error.
public void read (String str, Function<Error,Void> errorFunc)
{
//interact w/ libraries, boolean error = true or false
//if there is an error, variable err of type Error contains information
if (error)
{
errorFunc.apply(err);
}
}

In Java upto 1.7 the only way to achieve javascript like callbacks is thru interface. The api user who calls your method read has the liberty of implementing what he feels needs to be done to handle the error by writing an implementation class for the interface at the invocation point.
public String read(String options,IErrorCallBack errorHandler) throws Exception {
try {
// When everything works fine return what you think should be returned.
return "Success";
}
catch(Exception e) {
// On Error call the function on the error handler.
errorHandler.doSomething();
throw e;
}
}
public interface IErrorCallBack {
public void doSomething();
}
// The invocation point.
read("myString", new IErrorCallBack() {
public void doSomething() {
// Your implementation.
}
});

How do you create a secure JEXL (scripting) sandbox?

I'm creating a sandbox for JEXL scripts to execute in so that a malicious user can't access data outside the variables we give them access to and also can't perform a DOS attack on the server. I'd like to document this for anybody else also doing this and also get other people's input into the approach.
The following is a list of the things I'm aware of that needs to be addressed:
Only allow instantiating classes using 'new' that are on a whitelist.
Do not allow accessing the getClass method on any class because then forName can be called and any class can be accessed.
Restrict access to resources such as files.
Allow an expression only a certain amount of time to execute so that we can limit the amount of resources it consumes.
This does not apply to JEXL but may apply to the scripting language you are using:
Do not allow an object to have a custom finalize method because the finalize method is called from the finalizer thread and will execute with the original AccessControlContext instead of the one being used to create the object and execute the code in it.

UPDATE: This was all done using JEXL 2.0.1. You may have to adapt this to make it work with newer versions.
Here is my approach for dealing with each of these cases. I've created unit tests to test each of these cases and I have verified that they work.
JEXL makes this pretty easy. Just create a custom ClassLoader. Override the two loadClass() methods. On JexlEngine call setClassLoader().
Again, JEXL makes this pretty easy. You must block both '.class' and '.getClass()'. Create your own Uberspect class which extends UberspectImpl. Override getPropertyGet, if identifier equals "class" return null. Override getMethod, if method equals "getClass" return null. When constructing JexlEngine pass a reference to your Uberspect implementation.
class MyUberspectImpl extends UberspectImpl {
public MyUberspectImpl(Log jexlLog) {
super(jexlLog);
}
#Override
public JexlPropertyGet getPropertyGet(Object obj, Object identifier, JexlInfo info) {
// for security we do not allow access to .class property
if ("class".equals(identifier)) throw new RuntimeException("Access to getClass() method is not allowed");
JexlPropertyGet propertyGet = super.getPropertyGet(obj, identifier, info);
return propertyGet;
}
#Override
public JexlMethod getMethod(Object obj, String method, Object[] args, JexlInfo info) {
// for security we do not allow access to .getClass() method
if ("getClass".equals(method)) throw new RuntimeException("Access to getClass() method is not allowed");
return super.getMethod(obj, method, args, info);
}
}
You do this using Java's AccessController mechanism. I'll give a quick run-down of doing this. Start java with -Djava.security.policy=policyfile. Make a file named policyfile containing this line:
grant { permission java.security.AllPermission; };
Set the default SecurityManager with this call: System.setSecurityManager(new SecurityManager()); Now you can control permissions and your app by default has all permissions. It would be better if you limit the permissions of your app to only what it requires of course. Next, create an AccessControlContext that limits the permissions to the bare minimum and call AccessController.doPrivileged() and pass the AccessControlContext, then execute the JEXL script inside doPrivileged(). Here is a small program that demonstrates this. The JEXL script calls System.exit(1) and if it isn't wrapped in doPrivileged() it would successfully terminate the JVM.
System.out.println("java.security.policy=" + System.getProperty("java.security.policy"));
System.setSecurityManager(new SecurityManager());
try {
Permissions perms = new Permissions();
perms.add(new RuntimePermission("accessDeclaredMembers"));
ProtectionDomain domain = new ProtectionDomain(new CodeSource( null, (Certificate[]) null ), perms );
AccessControlContext restrictedAccessControlContext = new AccessControlContext(new ProtectionDomain[] { domain } );
JexlEngine jexlEngine = new JexlEngine();
final Script finalExpression = jexlEngine.createScript(
"i = 0; intClazz = i.class; "
+ "clazz = intClazz.forName(\"java.lang.System\"); "
+ "m = clazz.methods; m[0].invoke(null, 1); c");
AccessController.doPrivileged(new PrivilegedExceptionAction<Object>() {
#Override
public Object run() throws Exception {
return finalExpression.execute(new MapContext());
}
}, restrictedAccessControlContext);
}
catch (Throwable ex) {
ex.printStackTrace();
}
The trick with this is interrupting the script before it finishes. One way I found to do this is to create a custom JexlArithmetic class. Then override each method in that class and before calling the real method in the super class check if the script should stop executing. I'm using an ExecutorService to create threads. When Future.get() is called pass the amount of time to wait. If a TimeoutException is thrown call Future.cancel() which interrupts the Thread running the script. Inside each overridden method in the new JexlArithmetic class check Thread.interrupted() and if true throw java.util.concurrent.CancellationException.
Is there a better location to put code which will get executed regularly as a script is being executed so that it can be interrupted?
Here is an excerpt of the MyJexlArithmetic class. You have to add all the other methods:
public class MyJexlArithmetic extends JexlArithmetic {
public MyJexlArithmetic(boolean lenient) {
super(lenient);
}
private void checkInterrupted() {
if (Thread.interrupted()) throw new CancellationException();
}
#Override
public boolean equals(Object left, Object right) {
checkInterrupted();
return super.equals(left, right); //To change body of generated methods, choose Tools | Templates.
}
#Override
public Object add(Object left, Object right) {
checkInterrupted();
return super.add(left, right);
}
}
Here is how I am instantiating JexlEngine:
Log jexlLog = LogFactory.getLog("JEXL");
Map <String, Object> functions = new HashMap();
jexlEngine = new JexlEngine(new MyUberspectImpl(jexlLog), new MyJexlArithmetic(false), functions, jexlLog);