I am building a dynamic Web Project (in Eclipse with Tomcat as server) using servlets and JSPs. The general purpose of the project is to let the user solve small code problems. In order to do that I take a method written by the user and use it to build a java file that I can run via Reflection. The problem I can't seem to figure out is that Tomcat (or Eclipse?) does not update the file at runtime. So when I create the file using the code of the current user and try to compile it, my program always executes the file as it was when I started the server using the code of the previous user. How can I tell it to update the file before running it?
Edit:
That's how I create the file:
public boolean writeFile() {
try {
PrintWriter writer = new PrintWriter(relativePath + "src\\testfiles\\TestFile.java");
writer.print(content);
writer.close();
return true; }...
Here I call the writer and try running the file:
FileWriter writer = new FileWriter(content);
if(writer.writeFile()){
Class<?> TestFile;
Method m;
try {
TestFile = cl.loadClass("testfiles.TestFile");
m = TestFile.getDeclaredMethod("testSolution");
m.invoke(null);
Thanks in advance!
Ok, it's now clear what the issue is. Your issue is not with Tomcat not reloading the file, but with the classloader not reloading the class.
Normal classloaders will only load a class once, and keep it cached forever. The only way for a class to get unloaded is by its classloader being garbage collected. To reload a class you either have to use a different classloader each time (with the previous one getting garbage collected or you'll run out of memory), or to have a custom loader thar doesn't cache.
See this article for an implementation of a custom classloader that does what you want.
You could theoretically just have a new class each time (by changing its name on each save) but you'd eventually run out of memory.
Maybe the easiest way is to instantiate a new classloader in the method itself, load a class, run the method on it, and don't keep any other references to the loader or the class. That way, you'll get a freshly loaded class each time, and the old instances of both classes and loaders will get garbage collected.
UPDATE: I was operating under the assumption that you already know how to compile a class at runtime but, based on the comments, that's not the case.
A classloader can, of course, only load a compiled class, so a source directly is not very useful.
Java internally provides a a compiler interface under javax.tools.JavaCompiler, but it's not easy to use and requires a different handling of Java versions before and after Java 9. So it is much easier to use a library like jOOR that hides the gory parts:
Class clazz = Reflect.compile("com.example.Test",
"package com.example;" +
"public class Test {\n" +
" public String hello() {\n" +
" return \"hello\";\n" +
" }\n" +
" }")
.type();
Instead of type() to simply get the class, you can actually keep using jOOR's fluent reflection API to trigger the methods on the generated class or whatever it is you'd normally do via regular reflection.
For direct JavaCompiler usage, see e.g. this or, even better, jOOR's source code.
Related
I want to generate the IR for some java bytecode. The project I am using is using Wala as the static analysis tool for other reasons. One of the steps that I need to do is to create class hierarchy for a given scope of interest. Now it seems that the creation of the class hierarchy is an expensive process, thus minimizing it is desirable specially if I need to call it repeatedly.
I tried to narrow the scope of the class hierarchy by using AnalysisScopeReader.readJavaScope(...) where I specify a scope file that contains only a single entry, mainly the class that I want to analyse, which is loaded using Application loader. However that does not seem to work because I get an error that com.ibm.wala.ipa.cha.ClassHierarchyException: failed to load root <Primordial,Ljava/lang/Object> of class hierarchy. It seems that Wala is expecting to have the jre.jar in the scope file too, so it can find Ljava/lang/Object. However that is problematic, because if I added with line to the scope file, the creation of class hierarchy will include so many classes that I am not going to use, and which i'd rather not spend time or memory on.
It seems that people have been struggling with this issue here too but was never resolved.
To be concrete, here is what I do
scope2 = AnalysisScopeReader.readJavaScope(somePath, null, urlcl);
cha = ClassHierarchyFactory.makeWithRoot(scope2);
I really do not want/need to create class hierarchy in my analysis, I am not using any call graph analysis. I need only to create the IR for a single class that I am analyzing without having to pay for creating hierarchy of classes which I am not going to use. Or at least, I want the class hierarchy is contains a single file that I am currently analyzing.
It seems that ClassHierarchyFactor.makeWithRoot() almost works except that it still requires Object class to use for missing superclasses, which means that I need to add the java lang class hierarchy into the analysis.
The solution here (How to get rid of the unrelated classes when using WALA to analyze Java bytecode?) seems to be a hack rather than a fix.
I am not sure if there is a way to do that, can anyone help? Is there anyway in Wala that I can get ssa IR for a single class without having to construct a class hierarchy that is so big java lang class hierarchy?
Many thanks!
The following should get you minimal class hierarchy for a single targetClass
private ClassHierarchy getClassHierarchy(Class<?> targetClass) {
AnalysisScope scope = getAnalysisScope(targetClass);
return ClassHierarchyFactory.makeWithRoot(scope);
}
where
private AnalysisScope getAnalysisScope(Class<?> targetClass) {
String targetFile = targetClass.getResource(targetClass.getSimpleName() + ".class").getFile();
FileSystem fs = FileSystems.getFileSystem(URI.create("jrt:/"));
byte[] bytes = Files.readAllBytes(fs.getPath("modules", "java.base", "java/lang/Object.class"));
Path path = Path.of(System.getProperty("user.dir"), "target", "test-classes", "Object.class");
Files.write(path, bytes);
AnalysisScope scope = AnalysisScope.createJavaAnalysisScope();
scope.addClassFileToScope(ClassLoaderReference.Primordial, path.toFile());
scope.addClassFileToScope(ClassLoaderReference.Application, new File(targetFile));
return scope;
}
I am developing a component based game engine in java, right now when i have changes to the components I need to rebuild and restart the editor for the changes to take effect (or i can use some limited hot code injection if the application is running in debug mode).
I am looking for a way to allow the user to modify the source of the components and reload them without having to restart the application (maybe just exit and enter game mode instead). Also a important feature that I need is that the final exported code should be native Java code(so no interpreter should be used in the final result)).
Can you give me any pointers on how to integrate the beanshell interpreter in the project? I can manually monitor the source folder for changes and feed the updated java classes to it, but how is the hotswap going to occur really?
First of all, the title is a bit confusing. You don't need to integrate a BeanShell. What you actually need are:
to define a proper architecture
to use Java Compiler API in order to work with java classes
Architecture
Let's say you have an object graph. There are lots of objects, references, etc. So it will be really a tricky task to replace some instance with the new one. Instead of solving this problem you can hide dynamic part behind a "static" proxy. Proxy will handle all reloading stuff (including source folder monitoring).
Before reload:
After reload:
Having that done you can easily track changes and update dynamic part when needed.
Java Compiler API
Instead of using interpreted languages you can use Java, compiling it on the fly and loading using 'Class.forName()'. There are a lot of different examples due to the fact this approach was there for a while.
Here are some details:
Add dynamic Java code to your application
Compiling fully in memory with javax.tools.JavaCompiler
Basically you want to implement extensibility or plugin design pattern. There are multiple ways to implement this scenario.
Which ever the component you want to allow someone else to reload the module, define an interface and implement your own implementation as a default one. For example, Here I am trying to provide a HelloInterface which each country can implement and load anytime,
public interface HelloInterface {
public String sayHello(String input);
..
}
public class HelloImplDefault implements HelloInterface {
public String sayHello(String input) {
return "Hello World";
}
}
Now allow user to add a plugin (custom implementation) files to a pre-configured path. You can either user FileSystemWatcher or a background thread to scan this path and try to compile and load the file.
To compile java file,
private void compile(List<File> files) throws IOException{
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
Iterable<? extends JavaFileObject> compilationUnits = fileManager
.getJavaFileObjectsFromFiles(files);
JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, null,
null, compilationUnits);
boolean success = task.call();
fileManager.close();
}
To load class file,
private void load(List<File> files) throws MalformedURLException, InstantiationException, IllegalAccessException, ClassNotFoundException{
ClassLoader cl = Thread.currentThread().getContextClassLoader();
for(File f: files){
if(f.getName().endsWith(".class") && !loadedClass.contains(f.getName())){
URL url = f.toURL();
URL[] urls = new URL[]{url};
Object obj = cl.loadClass(f.getName().replace(".class", "")).newInstance();
if(obj instanceof HelloInterface){
HelloProviders.addProvider((HelloInterface)obj);
System.out.println("Loaded "+ ((HelloInterface)obj).getProviderName());
}else{
//Add more classes if you want
}
loadedClass.add(f.getName());
}
}
}
At this point you can read custom implementation and loaded in system class loader. Now you are ready to go. There are security implications to this approach which you need learn from internet.
I implemented one sample code and posted in github, please take a look. Happy coding!
Take a look at the tapestry-ioc inversion of control container which supports live-reloading.
When in development mode (tapestry.production-mode=false) you can live reload your services. Note that if a service interface changes, you will need a restart. But any changes to the service implementation that do not alter the service interface can be live-reloaded.
I want to create an auto-generated resource file similar to R from android. I know how to do the parsing and creation of the file and essentially creating a new class file. What I don't know how to do is start this auto-generation process.
So, using eclipse (although if there is a way to make this happen in an agnostic fashion, I would prefer it), how can I trigger an auto-generation session to read a properties file and create a .java file holding static variables and the "keys" from this parsed file, that I can then reference from my code?
A few examples of how to generate java source files have already been provided. It should be relatively easy to read the properties file and invoke one of these APIs.
To trigger the code generation process, you need to add a custom build step. With Ant, just add a custom task. And then hook it up to an Eclipse Builder: project Properties -> Builders -> New.
Subsequently, Eclipse should find and refresh this file on its own. If it doesn't, then check your configs: Preferences -> General -> Workspace -> find "Refresh using native hooks or polling" and similar ones and check them. (Note that I'm not 100% sure that this last part will work.)
The path of least resistance is to run this build step separately. If your properties file is not changing that often, then it shouldn't be that big a deal. This is similar to what you'd do if you use Protocol Buffers, JAXB, wsdl2java, etc. If you want everything to work magically like R.java does, then you probably have to do something a little more complicated:
- Try to use Eclipse Builder options to control when the Ant task is executed
- If you can't figure that out, then I'd check out how Eclipse hooks up to the above projects (that is, to Protocol Buffers, JAXB, wsdl2java, etc.)
- Look at the ADT custom PreCompilerBuilder class
- Check out the build-helper-plugin
It is common to use a ResourceBundle to create an object that allows you to lookup properties by key. You can learn about the ResourceBundle on the Java Trail.
The basic idea is that you have a text file with a .properties extension. You point the ResourceBundle object to that file. You can then use the getString() or getObject() method passing in the key to the property you want. That is all there is to it. You just need to load the ResourceBundle when you start your program (or sometime before you need it).
If you create you own class that has a ResourceBundle as a member value, you can use a simple bit of code like this to have a simple get() method to get the property value:
public String get(String aString)
{
String sVal = null;
try
{
sVal = (String)myProperties.getObject(aString);
}
catch (MissingResourceException e)
{
log.debug("Missing Property Value: "+aString);
}
return sVal;
}
I hope that is useful.
I have a case where I need to create a lot of class loaders in my application to temporarily make some code visible while user supplied scripts are running. I'm using an URLClassLoader for this and it works pretty well.
When the script terminates, I want to "unload" or "close" the class loader to free the resources.
Is it enough to set the reference to the class loader to null? I'm especially wondering if I'll eventually run out of file handles because the extra classes are in JAR files.
PS: Must work with Java 5 and up. Yeah, I know...
A little late, but hopefully this'll be helpful for those who come to this Question later (like me).
With Java 7, a close() method has been added to URLClassLoader, which is exactly what OP was asking for.
EDIT (thanks to #Hot Licks): OK, so it isn't exactly what the OP has asked for. It doesn't free up all the resources, or make the resources and the loader collectible. It simply prevents the loading of more resources using the class loader. It does, however, close the jar file that was loaded with the URLClassLoader.
If you can't use Java7 and it's close() method, use reflection to close all open JAR archives of a classloader, like so:
public void close() {
try {
Class clazz = java.net.URLClassLoader.class;
java.lang.reflect.Field ucp = clazz.getDeclaredField("ucp");
ucp.setAccessible(true);
Object sun_misc_URLClassPath = ucp.get(this);
java.lang.reflect.Field loaders =
sun_misc_URLClassPath.getClass().getDeclaredField("loaders");
loaders.setAccessible(true);
Object java_util_Collection = loaders.get(sun_misc_URLClassPath);
for (Object sun_misc_URLClassPath_JarLoader :
((java.util.Collection) java_util_Collection).toArray()) {
try {
java.lang.reflect.Field loader =
sun_misc_URLClassPath_JarLoader.getClass().getDeclaredField("jar");
loader.setAccessible(true);
Object java_util_jar_JarFile =
loader.get(sun_misc_URLClassPath_JarLoader);
((java.util.jar.JarFile) java_util_jar_JarFile).close();
} catch (Throwable t) {
// if we got this far, this is probably not a JAR loader so skip it
}
}
} catch (Throwable t) {
// probably not a SUN VM
}
return;
}
When all classes that the class loader loaded no longer have any references, and all references to the class loader itself have been erased, the class loader and the classes it loaded will be garbage collected as a group.
Note that this is dependent on having the JVM attribute set that causes unreferenced classes to be unloaded. It's set by default in most environments, but may not be in some embedded cases.
[Note that it's a non-trivial matter to remove references to a class. Any other class that references it by name will of course prevent removal. So the class must be loaded using ClassLoader.findClass or something similar.]
If you do not longer have classes (and object) loaded from that classloader, and if you do not keep any reference to that classloader, it will automatically handled by the garbage collector.
There are no close() methods in URL class loader or any of its parent classes, so you're out of luck.
Shouldn't GC handle this?
I extended URLClassLoader and made a close method based on Java 7s. I wanted to develop my IRC bot on my iPad 2, so I did what was needed. Now my plugin system is stable on Java 6 and 7, hurray.
I am writing a static analysis tool for an assignment, it analyses Java bytecode using the ASM library. One of the parts of ASM that we use requires (or at least, appears to require) that the class be loaded from the ClassLoader.
We were hoping the tool would be able to analyse .class files without requiring them on the classpath. We already load the .classes from a specified directory at run time and read them in using an InputStream. This is acceptable for ASM in most cases. There are some classes, such as SimpleVerifier, which attempt to load the classes though.
Is it possible, under this scenario, to register the .class files to be loaded so that calls to Class.forName() will load them? Or is there an easy way to extend the ClassLoader to allow this?
Edit: the information on URLClassLoader was useful. Unfortunately, using Thread.currentThread().setContextClassLoader() to an instance of that didn't work in this scenario. The library code I'm calling into uses a loader it retrieves on instance initialisation using getClass().getClassLoader().
By the time I set the URLClassLoader the class hasn't been initialised so I guess the contextClassLoader does not load that class.
Have I understand the responses correctly? Would using the URLClassLoader to load the 3rd party class be a possibility?
Almost.
If you have classes compiled somewhere, you can load them with a URLClassLoader. You can then set this ClassLoader to be the ClassLoader for the current Thread: Thread.setContextClassLoader(ClassLoader)
Users can that get the current threads context class loader and use that to access the class definition.
First of all, ASM can be used in a such way that it won't use ClassLoader to obtain information about classes.
There are several places in ASM framework where it loads classes by default but all those places can be overridden in your own subclasses. Out of the top of my head:
ClassWriter.getCommonSuperClass() method is called only when ClassWriter.COMPUTE_FRAMES flag is used and can be overwriten to not use ClassLoader to get inforamtion about classes. You can find an example of that in ClassWriterComputeFramesTest that introduces a ClassInfo abstraction
Similarly SimpleVerifier.getClass() method is used by SimpleVerifier.isAssignableFrom() and you can overwrite the latter and use the ClassInfo abstraction to find the common super type. If I am not mistaken, AspectWerkz project had implemented similar thing in its type pattern matching code. Also note that there is SimpleVerifier.setClassLoader() method, which you can use if you still want to load your own classes.
On a side note, on a Sun's JVMs, loaded classes gets to PermGen area and can't be unloaded, so it is not a good idea to load classes only for static code analysis purposes if you can avoid that, especially if tool would be integrated into a long-live process, such as IDE.
You can't, as far as I know, extend the System class loader at runtime, but you can dynamically load classes from an arbitrary location (jar or directory) using URLClassLoader.
You could try to setup a "launcher" in the startup of your application that creates an URLClassLoader passing it the locations on the classpath and your own .class locations and start the application from that classloader.
When the SimpleVerifier is loaded by the URLClassLoader it will also be able to load the classes from the extra locations.
Yes, you can use URLClassLoader
I have a test where I do load the class at runtime. This class is not in the classpath (nor even exist when the test is run for that matter ), later is it loaded and works great.
Here's the code.
void testHello() throws MalformedURLException, ClassNotFoundException {
URL[] url = {
new URL("file:/home/oreyes/testwork/")
};
try {
new URLClassLoader(url).loadClass("Hello");
throw new AssertionError("Should've thrown ClassNotFoundException");
} catch ( ClassNotFoundException cnfe ){}
c.process();// create the .class file
new URLClassLoader(url).loadClass("Hello");
// it works!!
}
Taken from this question.
I created my own ClassLoader its quite simple.
/**
* Used to hold the bytecode for the class to be loaded.
*/
private final static ThreadLocal<byte[]> BYTE_CODE = new ThreadLocal<byte[]>();
#Override
protected Class<?> findClass(final String name) throws ClassNotFoundException {
final byte[] bytes = BYTE_CODE.get();
if (null == bytes) {
throw new ClassNotFoundException(name);
}
return this.defineClass(null, bytes, 0, bytes.length);
}