Develop Reference

How can I get programatically any single Log generated by my App?

Any Android App produces Logs in the LogCat, even those not generated by developer's source code via Log.d, Log.i, Log.w and Log.e etc. etc. etc.. Perhaps Google Developers has some "automagic" thing for this, I don't know about that...
The point is I remember, years ago, I could somehow extend the class Application, override one or several of it's methods, and then:
Add my own code to process any single Log object generated by my
App in the LogCat
Do whatever I wanted with them (getting the label and the description strings, and then send them via mail, Slack etc., basically)
And then, calling super on that method and let the system do with that Log whatever Application by default does with it...
or something like that... if I recall correctly, I could do this with any log in my app's namespace. Or maybe it was just the crash handler? I can't remember...
It's been so long since I accomplished that (several years already!), so I don't remember how could I do that anymore... I search the internet like crazy trying to recall, but I am struggling to find it again... :-S
// ...public?? oO
[¿¿??] class MyApp extends Application [...] {
// [...]
#Override
public void whateverMethodItWasIDontRemember(params) {
// My coding stuff for the error reports
/* magic :D */
sendTheLogsMyWay();
// I bet this is important
super.whateverMethodItWasIDontRemember(params);
}
// [...]
}
I am about to launch the first Beta version of a new app, so I want beta testers to have a reliable way to send me LogCat's feed if anything has to be reported due to crashes, unexpected behaviour etc.
I mean, it would be ridiculous having to fill with CustomLogs every inch of source code for the beta version, when, in most cases, default logs are more than enough to see why it crashed (errors), or what optimization problems (usually warnings) might the Beta Tester have... not to mention that, if I forget to monitor something this way, the ridiculously big effort to log every single line of my code would be useless... oO
// -__- Mmm... perhaps extending Log itself
// would be more elegant...
import android.util.Log
public final class CustomLog {
public static void d(String label, String msg) {
// AKA My code to handle it
packItForNextErrorReport(label, msg);
Log.d(label, msg);
}
/*
* ... and so on with Log.i, w and e.
* ...I think you get the idea
*/
}

Restricting permissions with AccessController.doPrivileged() does not always work as expected

I managed to set up sort of a "Java sandbox" with the following code:
// #1
new File("xxx").exists();
// #2
PrivilegedExceptionAction<Boolean> untrusted = () -> new File("xxx").exists();
untrusted.run();
// #3
Policy.setPolicy(new Policy() {
#Override public boolean implies(ProtectionDomain domain, Permission permission) { return true; }
});
System.setSecurityManager(new SecurityManager());
AccessControlContext noPermissionsAccessControlContext;
{
Permissions noPermissions = new Permissions();
noPermissions.setReadOnly();
noPermissionsAccessControlContext = new AccessControlContext(
new ProtectionDomain[] { new ProtectionDomain(null, noPermissions) }
);
}
AccessControlContext allPermissionsAccessControlContext;
{
Permissions allPermissions = new Permissions();
allPermissions.add(new AllPermission());
allPermissions.setReadOnly();
allPermissionsAccessControlContext = new AccessControlContext(
new ProtectionDomain[] { new ProtectionDomain(null, allPermissions) }
);
}
// #4
try {
AccessController.doPrivileged(untrusted, noPermissionsAccessControlContext);
throw new AssertionError("AccessControlException expected");
} catch (AccessControlException ace) {
;
}
// #5
PrivilegedExceptionAction<Boolean> evil = () -> {
return AccessController.doPrivileged(untrusted, allPermissionsAccessControlContext);
};
try {
AccessController.doPrivileged(evil, noPermissionsAccessControlContext);
throw new AssertionError("AccessControlException expected"); // Line #69
} catch (AccessControlException ace) {
;
}
#1 and #2 should be self-explanatory.
#3 is the code that sets up the sandbox: It sets a totally unrestictive Policy (otherwise we'd lock ourselves out immediately), and then a system SecurityManager.
#4 then executes the "untrusted" code in a totally restrictive AccessControlContext, which causes an AccessControlException, which is what I want. Fine.
Now comes #5, where some evil code attempts to "escape" from the sandbox: It creates another, totally unrestricted AccessControlContext, and runs the untrusted code within that. I would expect that this would throw an AccessControlException as well, because the evil code successfully leaves the sandbox, but it doesn't:
Exception in thread "main" java.lang.AssertionError: AccessControlException expected
at sbtest.Demo.main(Demo.java:69)
From what I read in the JRE JAVADOC
doPrivileged
public static <T> T doPrivileged(PrivilegedExceptionAction<T> action, AccessControlContext context)
throws PrivilegedActionException
Performs the specified PrivilegedExceptionAction with privileges enabled and restricted by the
specified AccessControlContext. The action is performed with the intersection of the the permissions
possessed by the caller's protection domain, and those possessed by the domains represented by the
specified AccessControlContext.
If the action's run method throws an unchecked exception, it will propagate through this method.
Parameters:
action - the action to be performed
context - an access control context representing the restriction to be applied to the caller's
domain's privileges before performing the specified action. If the context is null,
then no additional restriction is applied.
Returns:
the value returned by the action's run method
Throws:
PrivilegedActionException - if the specified action's run method threw a checked exception
NullPointerException - if the action is null
See Also:
doPrivileged(PrivilegedAction), doPrivileged(PrivilegedExceptionAction,AccessControlContext)
, I would expect that the untrusted code would execute with no permissions and thus throw an AccessControlException, but that does not happen.
Why?? What can I do to mend that security hole?

Well, the untrusted code does execute with no permissions initially... until it requests that its permissions get restored (nested doPrivileged call with AllPermission AccessControlContext). And that request gets honored, because, according to your Policy, all your code, including the "evil" action, is fully privileged. Put otherwise, doPrivileged is not an "on-demand sandboxing tool". It is only useful as a means for its immediate caller to limit or increase its privileges, within the confines of what has already been granted to it by the policy decision point (ClassLoader + SecurityManager/Policy). Callers further down the line are absolutely free to "revert" that change if entitled to -- once again according to the policy decision point, not the opinion of any previous caller. So this is as-intended behavior and not a security hole.
What workarounds are there?
For one, there certainly is a "canonical" / sane way of using the infrastructure. According to that best practice, trusted code is to be isolated from untrusted code by means of packaging and class loader, resulting in the two being associated with distinct domains that can be authorized individually. If the untrusted code were then only granted permission to, say, read from a particular file system directory, no amount of doPrivileged calls would enable it to, say, open a URL connection.
That aside, one may of course come up with a hundred and two alternatives of creatively (but not necessarily safely) utilizing the different moving pieces of the infrastructure to their advantage.
Here for example I had suggested a custom protection domain with a thread-local for accomplishing roughly what you desire, i.e., on-demand sandboxing of a normally privileged domain throughout the execution of an untrusted action.
Another way of selectively sandboxing code within a single protection domain is by establishing a default blacklist and using DomainCombiners to whitelist trustworthy execution paths. Note that it is only applicable when the SecurityManager is set programmatically.
First one needs to ensure that no permissions are granted by default, neither via ClassLoader1 nor by Policy2.
Then a "special" AccessControlContext, coupled to a domain combiner that unconditionally yields AllPermission, is obtained as follows:
// this PD stack is equivalent to AllPermission
ProtectionDomain[] empty = new ProtectionDomain[0];
// this combiner, if bound to an ACC, will unconditionally
// cause it to evaluate to AllPermission
DomainCombiner combiner = (current, assigned) -> empty;
// bind combiner to an ACC (doesn't matter which one, since
// combiner stateless); note that this call will fail under
// a security manager (will check SecurityPermission
// "createAccessControlContext")
AccessControlContext wrapper = new AccessControlContext(
AccessController.getContext(), combiner);
// bind wrapper and thus combiner to current ACC; this will
// anew trigger a security check under a security manager.
// if this call succeeds, the returned ACC will have been
// marked "authorized" by the runtime, and can thus be
// reused to elevate permissions "on-demand" in the future
// without further a priori security checks.
AccessControlContext whitelisted = AccessController.doPrivileged(
(PrivilegedAction<AccessControlContext>) AccessController::getContext,
wrapper);
Now a standard security manager can be established to enforce the default blacklist. From this point onward all code will be blacklisted -- save for code holding a reference to the "whitelisted" / "backdoor" ACC, which it can leverage to escape the sandbox:
PrivilegedAction<Void> action = () -> {
System.getSecurityManager().checkPermission(new AllPermission());
return null;
};
// this will succeed, strange as it might appear
AccessController.doPrivileged(action, whitelisted);
// this won't
AccessController.doPrivileged(action);
// neither will this
action.run();
This leaves quite a bit of room for flexibility. One could directly call other trusted code within a "whitelisted" doPrivileged, and whitelisting would conveniently propagate up the call stack. One could alternatively expose the whitelisted ACC itself to trusted components so as to enable them to whitelist trusted execution paths of their own as they please. Or ACCs with limited permissions could be constructed, in the manner depicted above, and shared instead when it comes to code trusted less; or perhaps even specialized "pre-compiled" { action, limited pre-authorized ACC } capability objects.
Needless to say, whitelist propagation also widely opens up the door for bugs3. Unlike standard AccessController.doPrivileged(action) which is whitelist-opt-in, AccessController.doPrivileged(action, whitelisted) is effectively whitelist-opt-out; that is, for authorization to succeed under the former model, it is required that both the latest doPrivileged caller and every caller beyond it have the checked permission; whereas under the latter it suffices if merely the latest doPrivileged caller has it (provided no caller later on invokes standard doPrivileged, thereby reverting to the default blacklist).
Another prominent quirk4 to this approach lies with the fact that trusted third-party (library) code, reasonably expecting a call to standard doPrivileged to elevate its permissions, will be surprised to discover that it in fact causes the opposite.
1 The default application (aka "system") class loader only allows classes to read from their URL of origin (usually a JAR or directory on the local file system). Additionally permission to call System.exit is granted. If either is deemed "too much", a different class loader implementation will be necessitated.
2 Either by a custom subclass that unconditionally implies nothing, or by having the default sun.security.provider.PolicyFile implementation read an empty / "grant-nothing" configuration ("grant {};").
3 The convenient yet dangerous property of whitelist propagation could in theory be countered by use of a stateful combiner that emulates the behavior of standard doPrivileged by means of call stack inspection. Such a combiner might be provided a stable offset from the call stack's bottom upon instantiation, representing the invocation of some doWhitelistedSingleFrame(PrivilegedAction, Permission...) utility method exposed to trustworthy clients. Upon invocation of its combine method due to permission checks subsequently, the combiner would ensure that the stack has not gotten any deeper than offset + 1 (skipping frames associated with SecurityManager, AccessController, AccessControlContext, and itself), and, if it indeed hasn't, yield a modified PD stack evaluating to the desired permissions, while otherwise reverting to the default blacklist. Of course the combiner would have to be prepared to encounter synthetic frames (e.g. lambdas and bridges) at the anchored offset. Additionally it would have to safeguard itself from being used "out of context", via leakage to a different ACC or thread; and it would have to self-invalidate upon the utility method's return.
4 The only straightforward solution here would be to assign permissions to such code in the usual fashion (class loader and/or policy) -- which kind of defeats the goal of the exercise (circumventing the need for separate domains / class loaders / packaging).

jenkins plugin - Starting and stopping a Stage from inside a plugin

First, some background why I want this crazy thing. I'm building a Plugin in Jenkins that provides an API for scripts that are started from a pipeline-script to independently communicate with jenkins.
For example a shell-script can then tell jenkins to start a new stage from the running script.
I've got the communication between the script and Jenkins working, but the problem is that I now want to try and start a stage from a callback in my code but I can't seem to figure out how to do it.
Stuff I've tried and failed at:
Start a new StageStep.java
I can't seem to find a way to correctly instantiate and inject the step into the lifecycle. I've looked into DSL.java, but cant seem to get to an instance to call invokeStep(), nor was I able to find out how to instantiate DSL.java with the right environment.
Look at StageStepExecution.java and do what it does.
It seems to either invoke the body with an Environment Variable and nothing else, or set some actions and save the state in a config file when it has no body. I could not find out how the Pipeline: Stage View Plugin hooks into this, but it doesn't seem to read the config file. I've tried setting the Actions (even the inner class through reflection) but that did not seem to do anything.
Inject a custom string as Groovy body and call it with csc.newBodyInvoker()
A hacky solution I came up with was just generating the groovy script and running it like the ParallelStep does. But the sandbox does not allow me to call new GroovyShell().evaluate(""), and If I approve that call, the 'stage' step throws a MissingMethodException. So I also do not instatiate the script with the right environment. Providing the EnvironmentExpander does not make any difference.
Referencing and modifying workflow/{n}.xml
Changing the name of a stage in the relevant workflow/{n}.xml and rebooting the server updates the name of the stage, but modifying my custom stage to look like a regular one does not seem to add the step as a stage.
Stuff I've researched:
If some other plugin does something like this, but I couldn't find any example of plugins starting other steps.
How Jenkins handles the scripts and starts the steps, but It seems as though every step is directly called through the method name after the script is parsed, and I found no way to hook into this.
Other plugins using the StageView through other methods, but I could not find any.
add an AtomNode as a head onto the running thread, but I couldn't find how to replace/add the head and am hesitant to mess with jenkins' threading.
I've spent multiple days on this seemingly trivial call, but I can't seem to figure it out.

So the latest thing I tried actually worked, and is displayed correctly, but it ain't pretty.
I basically reimplemented the implementation of DSL.invokeStep(), which required me to use reflection A LOT. This is not safe, and will break with any changes of course so I'll open an issue in the Jenkins' ticket system in the hopes they will add a public interface for doing this. I'm just hoping this won't give me any weird side-effects.
// First, get some environment stuff
CpsThread cpsThread = CpsThread.current();
CpsFlowExecution currentFlowExecution = (CpsFlowExecution) getContext().get(FlowExecution.class);
// instantiate the stage's descriptor
StageStep.DescriptorImpl stageStepDescriptor = new StageStep.DescriptorImpl();
// now we need to put a new FlowNode as the head of the step-stack. This is of course not possible directly,
// but everything is also outside of the sandbox, so putting the class in the same package doesn't work
// get the 'head' field
Field cpsHeadField = CpsThread.class.getDeclaredField("head");
cpsHeadField.setAccessible(true);
Object headValue = cpsHeadField.get(cpsThread);
// get it's value
Method head_get = headValue.getClass().getDeclaredMethod("get");
head_get.setAccessible(true);
FlowNode currentHead = (FlowNode) head_get.invoke(headValue);
// crate a new StepAtomNode starting at the current value of 'head'.
FlowNode an = new StepAtomNode(currentFlowExecution, stageStepDescriptor, currentHead);
// now set this as the new head.
Method head_setNewHead = headValue.getClass().getDeclaredMethod("setNewHead", FlowNode.class);
head_setNewHead.setAccessible(true);
head_setNewHead.invoke(headValue, an);
// Create a new CpsStepContext, and as the constructor is protected, use reflection again
Constructor<?> declaredConstructor = CpsStepContext.class.getDeclaredConstructors()[0];
declaredConstructor.setAccessible(true);
CpsStepContext context = (CpsStepContext) declaredConstructor.newInstance(stageStepDescriptor,cpsThread,currentFlowExecution.getOwner(),an,null);
stageStepDescriptor.checkContextAvailability(context); // Good to check stuff I guess
// Create a new instance of the step, passing in arguments as a Map
Map<String, Object> stageArguments = new HashMap<>();
stageArguments.put("name", "mynutest");
Step stageStep = stageStepDescriptor.newInstance(stageArguments);
// so start the damd thing
StepExecution execution = stageStep.start(context);
// now that we have a callable instance, we set the step on the Cps Thread. Reflection to the rescue
Method mSetStep = cpsThread.getClass().getDeclaredMethod("setStep", StepExecution.class);
mSetStep.setAccessible(true);
mSetStep.invoke(cpsThread, execution);
// Finally. Start running the step
execution.start();

InvalidStackFrameException After Calling Method via ObjectReference#invokeMethod

I am currently working on an Eclipse plugin which enhances the debugging possibilities. During the debugging session, I invoke a Scala method via com.sun.jdi.ObjectReference#invokeMethod like that:
public int callMethod_Reactive_Level(final ObjectReference o) throws InvalidTypeException, ClassNotLoadedException, IncompatibleThreadStateException, InvocationException {
final Method m = apiValues.getMethod_Reactive_level(o); // com.sun.jdi.Method
final IntegerValue intVal = (IntegerValue) o.invokeMethod(thread, m, new LinkedList<Value>(), ObjectReference.INVOKE_SINGLE_THREADED);
return intVal.intValue();
}
After doing that, a call to org.eclipse.debug.core.model.IVariable#getValue leads to an InvalidStackFrameException. The whole error message is:
Status ERROR: org.scala-ide.sdt.debug code=5010 Exception while retrieving variable's value com.sun.jdi.InvalidStackFrameException
The message Exception while retrieving variable's value is shown when I inspect a variable in the variables view after calling a method as shown above.
Any idea how this problem can be solved? I do not understand why this is so problematic, since the JDI explicitely provides the possibility to do that.
Update: Since it may be a bug in the Scala IDE, there is a discussion and a tutorial how to reproduce the issue in the Scala IDE dev group.

The error message seems to come from the Scala IDE implementation of IVariable.getValue. It delegates to the JDI StackFrame.getValue, which throws. According to the docs, this can happen "if this stack frame has become invalid. Once the frame's thread is resumed, the stack frame is no longer valid."
My guess is that executing an invokeMethod on the same thread invalidates the stack frame.

Play Framework await() makes the application act wierd

I am having some strange trouble with the method await(Future future) of the Controller.
Whenever I add an await line anywhere in my code, some GenericModels which have nothing to do with where I placed await, start loading incorrectly and I can not access to any of their attributes.
The wierdest thing is that if I change something in another completely different java file anywhere in the project, play will try to recompile I guess and in that moment it starts working perfectly, until I clean tmp again.

When you use await in a controller it does bytecode enhancement to break a single method into two threads. This is pretty cool, but definitely one of the 'black magic' tricks of Play1. But, this is one place where Play often acts weird and requires a restart (or as you found, some code changing) - the other place it can act strange is when you change a Model class.
http://www.playframework.com/documentation/1.2.5/asynchronous#SuspendingHTTPrequests
To make it easier to deal with asynchronous code we have introduced
continuations. Continuations allow your code to be suspended and
resumed transparently. So you write your code in a very imperative
way, as:
public static void computeSomething() {
Promise delayedResult = veryLongComputation(…);
String result = await(delayedResult);
render(result); }
In fact here, your code will be executed in 2 steps, in 2 different hreads. But as you see it, it’s very
transparent for your application code.
Using await(…) and continuations, you could write a loop:
public static void loopWithoutBlocking() {
for(int i=0; i<=10; i++) {
Logger.info(i);
await("1s");
}
renderText("Loop finished"); }
And using only 1 thread (which is the default in development mode) to process requests, Play is able to
run concurrently these loops for several requests at the same time.
To respond to your comment:
public static void generatePDF(Long reportId) {
Promise<InputStream> pdf = new ReportAsPDFJob(report).now();
InputStream pdfStream = await(pdf);
renderBinary(pdfStream);
and ReportAsPDFJob is simply a play Job class with doJobWithResult overridden - so it returns the object. See http://www.playframework.com/documentation/1.2.5/jobs for more on jobs.
Calling job.now() returns a future/promise, which you can use like this: await(job.now())