Develop Reference

Android - dynamically load class into memory

DexClassLoader is great but works only by loading the compiled class as dex/jar file from the internal/external storage.
How can I load class directly into memory, without writing anything to the card first?
I am aware of the Java-Runtime-Compiler (compiles String to Class on-the-fly) from Peter Lawrey, which would be perfect, but it does not work in android.

The general principles for writing Java class loaders apply here as well, so basically what you need to do is write a class loader that can produce a Class instance, for example by invoking defineClass(). Of course this involves creating a valid array of dex bytecode. I have not done so yet, and besides very special occasions I would refraim from attempting to do so anyway. If you follow this road, remember to only use classloader features that have been present in Java 5 and 6.

As Thomas stated, you can reflectively invoke the protected defineClass() method of the ClassLoader into which you would like to load your class.
Here's an example of how this could be achieved:
public static Class<?> loadClass(byte[] code, ClassLoader loadInto) throws InvocationTargetException
{
try {
Method m = ClassLoader.class.getDeclaredMethod("defineClass", byte[].class, int.class, int.class);
m.setAccessible(true); // Make sure we can invoke the method
return (Class<?>) m.invoke(loadInto, code, 0, code.length);
}
// An exception should only be thrown if the bytecode is invalid
// or a class with the same name is already loaded
catch (NoSuchMethodException e) { throw new RuntimeException(e); }
catch (IllegalAccessException e){ throw new RuntimeException(e); }
}
Although, what I'm getting the feeling that you're referring to is runtime compilation of a String containing valid Java into bytecode, based on the link you included. Though I do not know of any way of doing this, I would recommend you take a look at this: https://github.com/linkedin/dexmaker

You can do it by InMemoryDexClassLoader

Is it a good idea to replace java.net.URL URLStreamHandlerFactory using reflection?

In Java we can register a custom protocol handler in at least two ways:
by setting system property 'java.protocol.handler.pkgs'
using URL.setURLStreamHandlerFactory
For more details please check http://accu.org/index.php/journals/1434
I can not go with the first option, as i would have to add a lot of jar files to the server (tomcat) classpath, to make the handler implementaion visible for bootstrap classloader. Moreover some initialization is needed, which has to be done in an application context.
The problem with the second option is that the Factory can be registered only once (check java.net.URL#setURLStreamHandlerFactory), and unfortunately it is done by Tomcat.
What i can do is to create a decorator factory which will extend the existing one by my protocol handler. Than using relfection set the static field URL#factory to null and register (again?) my "decoratorFactory" in a standard way using URL#setURLStreamHandlerFactory. I'm just wondering if it is a good idea to use the reflection here...? How about the security?
I would like to do something like that:
try {
Field factoryField = URL.class.getDeclaredField("factory");
factoryField.setAccessible(true);
// get current factory
Object currentFactory = factoryField.get(null);
// define a decorator factory
MyFactoryDecorator mfd = new MyFactoryDecorator(currentFactory);
// set the factory to null and register MyFactoryDecorator using URL#setURLStreamHandlerFactory.
factoryField.set(null, null);
URL.setURLStreamHandlerFactory(mfd);
} catch (NoSuchFieldException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
}

Idk if this would work, but what if you create a new URLStreamHandlerFactory that doesn't actually contain the implementation? It just sits there and then instantiates the real implementation at runtime when it is called via an app context lookup. Then in theory you can instantiate this new proxy handler via the system property at the beginning, but also use the one you want when it is actually called.
UPDATE:
Actually, I think your link above mentions this strategy:
Another approach is to use the factory registration, but to provide a factory class that itself supports registration of multiple different stream handlers using different names.
This approach supports code using the java.net.URL class but it does require a registration call for each protocol and so hence changes are needed to an application before it can make use of the new URLs. However the approach gets around the problems discussed above with multiple class loaders since the factory is loaded by the user code class loader rather than by the class loader for the URL class.
I'm not super familiar with what all you're doing and how the registration works, so this could be more or less complicated depending on what you're doing. Idk if you even need the extra registration or not, but it sounds like it might solve the .jar/app context problem.

Tomcat JSP/JSTL without HTTP

I have a pretty standard web app running under Tomcat 7.
What I'm now trying to do is leverage JSP/JSTL as a templating language independent of the HTTP/web serving aspects of Tomcat to produce HTML that can be emailed and converted to PDF.
Has anyone else tried to do this before and could help me with some pointers?
Thanks in advance.

In contrast to what Stephen C said, yes, JSP are Servlets, etc. etc. (And Velocity is quite good and easy to use)
But, what is a Servlet?
It's an interface. An interface with one major method:
service(ServletRequest req, ServletResponse res)
Locate the JSP class, cast it to a Servlet, create implementations of ServletRequest and ServletResponse, and then...
String jspClassName = findJspClassForJSP("your.jsp");
Class jspClass = Class.forName(jspClassName);
Servlet jspServlet = (Servlet)jspClass.newInstance();
MyServletRequest req = new MyServletRequest();
MyServletResponse resp = new MyServletResponse();
jspServlet.init();
jspServlet.service(req, resp);
jspServlet.destroy();
String results = reps.getContent();
Will this work? Well, after some work it will. Obviously you need to implement the minimum facades of the ServletRequest/Response as well as what ever your JSPs will need. But, likely you will probably need little more than the attributes and the streams. If you make your Response return a StringWriter, you're halfway there.
The next part is creating the servlet from the JSP. Handily, the Jasper compiler does that for you -- the game is invoking it. I have never done it directly, but it clearly can be done since both the servlet container does it, as well as the JSPC script/bat file, the ant task, as well as most of the Servlet containers out there use Jasper. So, that can be done. Once you know how to invoke that, you'll know the final generated class name for the JSP. (See the first line of the sample.)
Have I ever done this? No. But I bet within less than a day of messing around you'll know whether this is doable or not. I'm betting it is, especially if you don't run in to any class loader shenanigans. You'll possibly have an issue if you let your users change and regenerate a JSP (so MyEmail.jsp gets compiled in to MyEmail.class, MyEmail_2.class, etc.). But if you invoke Jasper yourself, you'll likely have more control over this.
The other hard part is determining the class name of the JSP. Most of the containers follow a basic pattern here, so if you poke around in the generated code from a WAR you'll likely find it.
Keep the JSPs reasonably simple (and an Email template shouldn't need to super complicated with embedded Java or anything making random calls), and it even more a good chance it will work.
Your solution may not be portable out of the box out of Tomcat, but you likely won't care. The folks that I've talked to that use JSP for templates, simply opened a socket to their own server and made a request. They didn't go this far either.
But on the surface, save some whacky class loader black hole hell, I bet you can get this to work pretty quick. Implement as little of the request and response as you need to, fight a few NPEs as the JSP and JSTL call stuff you weren't planning, and, as Santa says,
Hack away, Hack away, Hack away all!
Addenda:
So, for all the naysayers...
public void runJsp() {
JspC jspc = new JspC();
jspc.setUriroot("/tmp/app");
jspc.setOutputDir("/tmp/dest");
jspc.setJspFiles("newjsp.jsp");
jspc.setCompile(true);
try {
jspc.execute();
Class cls = Class.forName("org.apache.jsp.newjsp_jsp");
Servlet s = (Servlet) cls.newInstance();
MyRequest req = new MyRequest();
MyResponse resp = new MyResponse();
s.init(getServletConfig());
s.service(req, resp);
s.destroy();
System.out.println(resp.getSw().toString());
} catch (JasperException ex) {
throw new RuntimeException(ex);
} catch (ClassNotFoundException ex) {
throw new RuntimeException(ex);
} catch (InstantiationException ex) {
throw new RuntimeException(ex);
} catch (IllegalAccessException ex) {
throw new RuntimeException(ex);
} catch (ServletException ex) {
throw new RuntimeException(ex);
} catch (IOException ex) {
throw new RuntimeException(ex);
}
}
Amazing what source code and 1/2 hour in a debugger will do for you.
I created a simple JSP in /tmp/app/newjsp.jsp.
jspc.setUriroot tells the compiler where the base of your "web app" is located. jspc.setOutputDir tells jspc where to put the generated Java and Class files. jspc.setJspFiles tells jspc what files to compile, based off of the URI Root. jspc.setCompile told it to actually compile the code. Finally, jspc.execute() does the deed.
By default Jasper uses the package org.apache.jsp, and creates a new class based on the JSP file name. For my simple experiment, I simply put "/tmp/dest" on to the class path of my Glassfish container, so that the container would find the generated classes.
I load the class, and get an instance.
Finally, I created MyRequest, MyRequest, and, ultimately, MySession. My IDE conveniently created stubs for the respective interfaces. In this case I implemented: MyRequest.getSession(), MyResponse.setContentType(), MyResponse.setBufferSize(), and MyResponse.getWriter().
public PrintWriter getWriter() throws IOException {
if (sw == null) {
sw = new StringWriter();
pw = new PrintWriter(sw);
}
return pw;
}
Obviously sw and pw are instance variables of MyResponse.
MyRequest returned an instance of MySession. My implementation of MySession does -- nothing. But the runtime wanted a Session, it just doesn't use it on its own for my very simple JSP, and I wasn't motivated on stuffing in the the one from the Servlet.
I tested this on Glassfish v2.1. I simply added appserv_rt.jar (from glassfish/lib) to my build class path (so it could find the JspC jars), but I don't bundle it in the WAR (since it's already in the container).
And, shazam, it worked. In "real life", assuming the process that wanted to leverage the JSP was actually sourced from a web request, I would simply create an HttpServletResponseWrapper and override the earlier three methods, the rest would probably Just Work. If a web request isn't in the picture at all, then you'd need to create your own Session implementation (no big deal really, it's just a map).
I'd also use a private URLClassLoader to load the faux JSP classes. If I KNEW I'd never reload a JSP, then would just make the destination my WEB-INF/classes directory and give it it's own package and let the system load them.
But, yup, it worked. No big deal. It's just java.

This doesn't really make sense. A JSP is some nice syntax that results in the generation of a Java EE servlet class. Indeed, the "servlet" / "http" nature of a JSP is thoroughly intertwined through the APIs and the semantic model of JSPs and JSTL.
If you want to generate HTML independently of web requests, you would be better of using a different templating technology; e.g. Velocity or FreeMarker. If you then want the HTML to be delivered as web responses as well, arrange that your servlets call the templating engine to generate responses. (If you are using Spring there is existing infrastructure for this. Other frameworks may have similar support, but if not, it shouldn't be hard to implement some glue code yourself to do this.)

Integrating a third-party API into a Java application with clashing dependencies

I am working on an existing application which has quite a lot of external JAR dependencies. I need to enhance it to integrate with a third-party application which has an API. Sadly, the API is not well contained and also comes with a large number of its own dependencies some of which clash with mine.
I believe I should solve this using Classloaders, but I'm struggling to see how to structure them correctly.
To keep it simple, assume we have myapp.jar with a hibernate3.jar dependency, and vendor-api.jar with a hibernate2.jar dependency (and assume these are incompatible).
My new piece of code will reside in the myapp.jar library (although it could be in a separate jar if this would help). Due to the way the vendor API works, my new code needs to extend a class from the vendor-api.jar library.
How can I structure the Classloaders in such a way that anything within the vendor-api.jar accesses only its own dependencies, and anything on my side accesses only the myapp.jar and dependencies?
Thanks, Jon

I've not tried this myself, but from memory each clashing class needs to be in a sibling classloader and any communication between the two needs to go through a common ancestor. However, the ancestor cannot (AFAIK) "directly" reference classes from its children and must access them through the reflection API.
Something along these lines ought to work (dry-coded) YMMV. Comments and error-spotting welcome.
class Orchestrator {
URL[] otherAppClasspath = new URL[] { new URL("file:///vendor-api.jar"),
new URL("file:///hibernate2.jar"),
new URL("file:///code-extending-vendor-api.jar" };
URLClassLoader otherAppLoader = new URLClassLoader(otherAppClasspath);
URL[] yourAppClasspath = new URL[] { new URL("file:///myapp.jar"),
new URL("file:///hibernate3.jar") };
URLClassLoader yourAppLoader = new URLClassLoader(yourAppClasspath);
public void start() {
Method yourAppEntryPoint = yourAppLoader.findClass("com/company/Main").getMethod("start", new Class[] { Orchestrator.class } );
yourAppEntryPoint.invoke(null, new Object[] { this });
}
public static void main(String[] args) {
new Orchestrator().start();
}
// define some abstracted API here that can be called from your app
// and calls down into classes in the other app
public String getSomeResultFromOtherApp(int someArgument) {
Method otherAppAPI = otherAppLoader.findClass("com/company/ExtendingAPIClass").getMethod("getSomeResult", new Class[] { Integer.class });
return (String)otherAppAPI.invoke(null, new Object[] { someArgument });
}
}

#fd's answer gives a technical mechanism that ought to work - give or take some typos, exception handling, etc.
However, I think you would be better off not trying to do this ... unless your dependencies on the 3rd party APIs are restricted to a very small number of methods on a very small number of classes. Each class that you depend on has to be dynamically loaded and each method has to be looked up and invoked reflectively. Too much of that and your codebase will suffer.
If I were you, I'd try to resolve the dependency issue some other way:
Try to get the 3rd party vendor to use hibernate3.jar
Change your application to use hibernate2.jar
Refactor so that your application code and the 3rd party library are in separate JVMs or separate webapps.
From what you say, this might be hard.

Using OSGi may help you in the long term. Here is an implementation I am trying now- http://felix.apache.org

How to link to a package but only optionally execute an action based on existence/availability of package at runtime?

From the perspective of a cross application/applet java accessibility service, how would you link to a package but only optionally execute an action based on existence/availability of a package (being already loaded) at runtime?
I think what I'm interested in here is a way to resolve the class identity crisis but rather than the issue being between 2 apps sharing objects, being a service loaded at a higher level of the class loaders.
It seems like reflection is the way to go, but I am not sure how or if I can implement a derived class this way. I need to add a specific listener derived from the specific optional classes, I can load the listener using the applet class loader but the internals still fail. Say you wanted to add an JInternalFrameListener, but Swing wasn't guaranteed to be available, using reflection you can find the method to add the listener, but how can you create and have the frame listener work if it cannot find any of the related classes because they can't be found in the base classloader! Do I need to create a thread and use setContextClassLoader to the classloader that knows about swing so that I can get the class to be loaded reliably? simply trying to set the class loader on my existing thread didn't seem to work.
Earlier description of issues
Sorry, I'm not quite sure what to ask or how to make this clear, so it rambles on a bit.
Say a class uses some feature of another, but the other class may not always be available - say finding the website from JNLP if this is a JNLP app.
At one stage I thought that simply compiling against JNLP would mean that my class would not load unless JNLP was available, and so to identify this optional section I simply wrapped a try{} catch( NoClassDefFoundError ) around it.
Later something changed (perhaps changing jdk or ?? I don't recall) and it seemed that I should also use a try{} catch( ClassNotFoundException ).
Now I wanted to extend this idea to other optional features, but it doesn't seem to work consistently.
Say I wanted to add some feature to do something more advanced in a JRE1.6 runtime using the same jar and classes as I run in a JRE1.3, or say I want to handle some controls in a specific gui toolkit which may not always be used like SWT or oracle.forms.
Is there some way of doing this more reliably? It just seems wrong to cause an exception and catch it to ignore it all the time.
The current issue comes down to being able to compile against oracle.forms but then the accessibility component installed in ext is unable to access the oracle.forms classes even though objects from the oracle.forms package have been created. If I throw the frmall.jar into the ext directory to test then the accessibility component works up to the point that the whole lot gets flakey because of the different versions of the same package.
I seem to be caught up on an issue with the class loader not being the right one or something (??). How do I find the right one?
Edit:
The answers so far are kindof interesting but not quite getting me where I want to be.
In the case of the gui components I currently compile in the form of a factory something like...
import oracle.forms.ui.*;
import java.awt.*;
static public IComponentNode newNode( INode parent, Component component ) {
System.out.println( component.getClass().toString() );
try{
if( component instanceof FormDesktopContainer )
... does stuff here like return new FormDesktopNode( parent, (FormDesktopContainer) component )
} catch ( NoClassDefFoundError a ) {
System.out.println( a.getMessage() );
}
where it prints out class oracle.forms.ui.FormDesktopContainer and then throws and exception on the instanceof call with NoClassDefFound thus printing out oracle/forms/ui/FormDesktopContainer
So how can it have an instance of a class yet not be able to find it?

How about this? messy, but it ought to work:
public boolean exists(String className){
try {
Class.forName(className);
return true;
}
catch (ClassNotFoundException){
return false;
}
}

You can check the availability of a class by calling
ClassLoader.getSystemClassLoader().loadClass("my.package.MyClass")
if it throws a ClassNotFoundException, it's not available. If you get the Class object, it is. You can then choose behaviour based on whether or not the class is available.

I suggest compiling the majority of your code against your minimum target. Have code that uses particular optional libraries clearly separated, but dependent upon the bulk of your code. Dynamically load the code that uses optional libraries once. The main class should do something that checks for the presence of the required library/version in its static initialiser.
In the case of JNLP, your JNLP main class load the JNLP dependent code statically.
(Note that attempting to catch class loading related exceptions from normally linked code is unreliable.)

getSystemClass loader was not useful for this purpose as there where multiple possible class loaders to interact with based on which applet the given window was in. The accessibility components being loaded at a more base class loader cannot see the applet specific classes.
To interact with the objects reflection does the job, though it does add so much more to maintain.
// statically linking would be
return component.getText();
// dynamically is
try {
return (String)component.getClass().getMethod("getText", new Class [] {}).invoke(component, new Object [] {});
} catch (Throwable e) {
e.printStackTrace();
}
The trickier bit is in writing a class derived from an interface that is not directly accessible, using the Proxy service allows this to be accomplished, providing the proxy service the applet specific class loader and the dynamically loaded class for the interface.
public void addListener(Container parent) {
if (parent == null) { return; }
if ("oracle.forms".equals(parent.getClass().getName())) {
// Using the class loader of the provided object in the applet
// get the "class" of the interface you want to implement
Class desktopListenerClass = Class.forName( "oracle.DesktopListener"
, true, parent.getClass().getClassLoader());
// Ask the proxy to create an instance of the class,
// providing your implementation through the InvocationHandler::invoke
Object desktopListener = Proxy.newProxyInstance(
parent.getClass().getClassLoader()
, new Class[] { desktopListenerClass }, new InvocationHandler() {
public Object invoke(Object proxy, Method method, Object[] args)
throws Throwable {
if ("functionName".equals(method.getName())) {
// do stuff
}
return null;
}
});
// do something with your new object
Method addDesktopListener = parent.getClass().getMethod("");
addDesktopListener.invoke(parent, desktopListener);
}
}
examples cut down to show general method