I'm using Java Compiler API to compile in-memory classes. That is, classes are compiled to bytecode (no .classes files stored in disk) and then loaded by reconstructing the bytecode.
Sometimes, I need to compile a class that depends on another, also in-memory compiled, class. For instance: Compile Class A, then compile Class B which depends on Class A.
To solve this, I pass both Class A and Class B as the compilation units needed by the getTask method of the compiler API.
However, I really don't like this solution, as it makes me recompile Class A which was already compiled.
Is there a way to get around this?
EDIT: I found a solution through this link: http://www.ibm.com/developerworks/java/library/j-jcomp/index.html
Yes, this is totally possible as long as you properly implement the ForwardingJavaFileManager. The two most important methods are inferBinaryName() and list(). If you set these two up properly, the compiler will be able to resolve classes that you've previously compiled.
inferBinaryName() must return the class' simple name (e.g. the inferred binary name for com.test.Test would be just Test). Here is my implementation (my subclass of JavaFileObject is called InAppJavaFileObject):
#Override
public String inferBinaryName(Location location, JavaFileObject javaFileObject) {
if(location == StandardLocation.CLASS_PATH && javaFileObject instanceof InAppJavaFileObject) {
return StringUtils.substringBeforeLast(javaFileObject.getName(), ".java");
}
return super.inferBinaryName(location, javaFileObject);
}
Note that I'm stripping off ".java" from the end. When constructing a JavaFileObject, the file name must end in ".java", but if you don't strip the suffix later, the compiler won't find your class.
list() is a little bit more complicated because you have to be careful to play along nicely with your delegate file manager. In my implementation, I keep a map of fully-qualified class name to my subclass of JavaFileObject that I can iterate over:
#Override
public Iterable<JavaFileObject> list(Location action, String pkg, Set<JavaFileObject.Kind> kind, boolean recurse) throws IOException {
Iterable<JavaFileObject> superFiles = super.list(action, pkg, kind, recurse);
// see if there's anything in our cache that matches the criteria.
if(action == StandardLocation.CLASS_PATH && (kind.contains(JavaFileObject.Kind.CLASS) || kind.contains(JavaFileObject.Kind.SOURCE))) {
List<JavaFileObject> ourFiles = new ArrayList<JavaFileObject>();
for(Map.Entry<String,InAppJavaFileObject> entry : files.entrySet()) {
String className = entry.getKey();
if(className.startsWith(pkg) && ("".equals(pkg) || pkg.equals(className.substring(0, className.lastIndexOf('.'))))) {
ourFiles.add(entry.getValue());
}
}
if(ourFiles.size() > 0) {
for(JavaFileObject javaFileObject : superFiles) {
ourFiles.add(javaFileObject);
}
return ourFiles;
}
}
// nothing found in our hash map that matches the criteria... return
// whatever super came up with.
return superFiles;
}
Once you have those methods properly implemented, the rest just works. Enjoy!
That leads to the obvious question of why you want to compile class A separately first. Why not just compile everything in one go?
How if you maintain the modified time of the files and the (in-memory) compiled byte code?
I don't think you can avoid compiling both classes. In fact, if you don't compile both of them, there is a chance that you will end up with binary compatibility problems, or problems with incorrect inlined constants.
This is essentially the same problem as you'd get if you compiled one class and not the other from the command line.
But to be honest, I wouldn't worry about trying to optimize the compilation like that. (And if your application needs to be able to dynamically compile one class and not the other, it has probably has significant design issues.)
Related
On my current project, I've felt the need to create a sort of simulated callback system in Java using reflection. However, I'm having issues getting my reflection to actually function. The code at fault follows:
public Callback(Object parentObj, String methodName, Class<?>...parameters)
{
if(parentObj == null)
throw new IllegalArgumentException("parentObj cannot be null", new NullPointerException());
Class<?> clazz = parentObj.getClass();
// Trace debugging, see output
for(Method m : clazz.getDeclaredMethods())
if(m.getName().equals("myMethod")) System.out.println (m);
try { this.method = clazz.getMethod(methodName, parameters); }
catch(NoSuchMethodException nsme) { nsme.printStackTrace(); } // Exception caught
catch(SecurityException se) { se.printStackTrace(); }
this.parentObj = parentObj;
this.parameters = parameters;
}
When I construct the Callback object, I'm using syntax like this:
new Callback(this, "myMethod", boolean.class)
When I try to create my pseudo-callback, it hits the NoSuchMethodException catch block. I've included some trace debugging above to show the output of one of my methods failing. The output:
private void my.package.MyClass.myMethod(boolean)
java.lang.NoSuchMethodException: my.package.MyClass.myMethod(boolean)
at java.lang.Class.getMethod(Class.java:1605)
at my.package.other.Callback.<init>(Callback.java:63)
I couldn't figure the problem out, so I started hunting, to little avail. The best I could find was mention of versioning conflict between the compiled JAR and the runtime. However, MyJar.jar/META-INF/MANIFEST.MF contains Created-By: 1.6.0_02 (Sun Microsystems Inc.). My IDE is running C:\Program Files\Java\jdk1.6.0_02\bin\javac.exe to compile my project. I'm using C:\Program Files\Java\jdk1.6.0_02\bin\java.exe to run my JAR.
I'm at a loss why Class.getMethod is claiming the method doesn't exist, but Class.getMethods seems to have no problem finding it. Help? :(
Your method is private but getMethod() only returns public method.
You need to use getDeclaredMethod().
You need the parameter list to be absolutely correct for the method you want for the call to succeed.
I've found that tiny steps are important when doing reflection because the compiler doesn't help. Write a small snippet which actually invokes exactly the method you want to in this particular case, and then when that works, generalize it into the framework here. I would focus on the parameters passed.
The Javadoc for getMethod isn't explicit, but it looks like it might throw a NoSuchMethodException for methods that aren't public, and your method is private.
The versioning issue that can cause NoSuchMethodException isn't a difference between the compiler versions. It's a difference in the version of (in your case) MyClass at compile time versus runtime.
Since you're using reflection you issue might have nothing to do with versioning, though. Certainly that would not explain different behavior between getMethod and getDeclaredMethods, because you're running them against the same Class instance, hence a version difference isn't really possible.
Are you sure that the parameters match your actual method?
Trying to upgrade to JDK8 on a big project, compilation goes really slow on JDK8 compared to JDK7.
Running the compiler in verbose mode, JDK8 compiler stops at a big generated converter class(Mapping) for entities from server to client.
The converter methods in several cases call other converter methods from the same Mapping class.
As a workaround tried to split the Mapping file into multiple files. This visibly improved performance when only compiling the Mapping class or it's containing project(projectA). But compile time was very slow for other projects which invoke converter methods from projectA.
Another workaround was to make all convert methods return null, not calling anything else. Again, the performance was good for projectA but not for depending projects.
ProjectA uses generics but since it is compatible with JDK6, which didn't have generalized type inference introduced, maybe it's another JDK8 bug that causes this slowdown.
So possibly out of context but for generalized type inference, some threads like below suggest an upgrade to JDK9. But since it's not yet released, it's not a viable option as upgrade.
It'd be ideal if a backport of the fix would be done to JDK8. This was requested in the following StackOverflow thread but no reply from Oracle team yet.
Slow compilation with jOOQ 3.6+, plain SQL, and the javac compiler
I've attached 2 screenshots of how the heap looks in JDK7 vs JDK8. Could this be a cause for the JDK8 slowdown?
Thank you!
Update 20160314
The converter methods from Mapping class look like:
public static ResponseItemVO convert (ResponseItem pArg0){
if(pArg0==null){
return null;
}
ResponseItemVO ret = new ResponseItemVO();
ret.setErrorDetails(pArg0.getErrorDetails());
ret.setResult(Mapping.convert(pArg0.getResult()));
ret.setIdentifier(Mapping.convert(pArg0.getIdentifier()));
return ret;
}
And the VO looks like:
public class ResponseItemVO extends ResultVO<IdentifierVO, DetailsVO > {
public ResponseItemVO() {}
}
JDK7 Heap:
JDK8 Heap:
You've noticed already, there's a severe performance regression in Java 8 when it comes to overload resolution based on generic target typing. One of the reasons in your case might be the fact that the compiler needs to find the appropriate method from an assignment type
ResultVO<Something, Something> result = Mapping.convert(...);
// heavy lookup here ---------------------------^^^^^^^
If you're in control of the code generator, and not constrained by backwards compatibility, it might be worth thinking about avoiding the overloading of the convert() method. Without overloading, the compiler doesn't have to do the overload resolution work, neither inside of your mapping code, nor at the call site. This will certainly be much much faster.
Attempt 1: By using the parameter type in the method name:
class Mapping {
public static ResponseItemVO convertResponseItem(ResponseItem pArg0){
if (pArg0==null){
return null;
}
ResponseItemVO ret = new ResponseItemVO();
ret.setErrorDetails(pArg0.getErrorDetails());
ret.setResult(Mapping.convertResult(pArg0.getResult()));
ret.setIdentifier(Mapping.convertIdentifier(pArg0.getIdentifier()));
return ret;
}
}
Attempt 2: By moving the convert method elsewhere, e.g. into the VO type
class ResponseItemVO {
public static ResponseItemVO from(ResponseItem pArg0){
if (pArg0==null){
return null;
}
ResponseItemVO ret = new ResponseItemVO();
ret.setErrorDetails(pArg0.getErrorDetails());
ret.setResult(ResultVO.from(pArg0.getResult()));
ret.setIdentifier(IdentifierVO.from(pArg0.getIdentifier()));
return ret;
}
}
Or better...
class ResponseItem {
public ResponseItemVO toVO(){
ResponseItemVO ret = new ResponseItemVO();
ret.setErrorDetails(getErrorDetails());
ret.setResult(getResult().toVO());
ret.setIdentifier(getIdentifier().toVO());
return ret;
}
}
I am trying to replace a certain class file with my own in an obfuscated jar. The original class file has methods named "new" and "null" so a quick decompile + compile doesn't work.
I tried compiling and using jbe to add new methods named "new" that relayed everything to "new_symbol" functions (with new_symbol beeing the decompiled version of the original "new" function).
This did not work. ("code segment has wrong length in class file")
Does anyone know of a way to refractor method names in class files? And if that isn't possible, a way to reliably create those "proxy functions"?
From google I learned that there are about 1000+ different backend library's but only jbe as fronted for bytecode editing?
EDIT:
Let me try to illustrate it.
Let's say that there is a jar file with a class that provides a function that logs everything you give it to a database.
I'd like to replace that class file with my own, and it should not only log everything to a database, but also print whatever data it gets to the command line.
The problem is, that class file was obfuscated and the obfuscator gave it public method names like "new" or "null". If you try:
public class replacement{
public void new (string data){
...
}
}
And compile that, you get compilation errors.
My idea was to create this :
public class replacement{
public void newsymbol (string data){
...
}
}
And use a bytecode editor to create a function named "new" that calls "newsymbol" with the same arguments. (but I get "code segment wrong length" and other errors going down this route.
My question therefore could be better frased as "give me a way to intercept calls to a class file who's public methods are named "new" "null" "weird_unicode_symbols""....
Scala allows you to use identifiers in names if you surround them by `.
class f{
def `new`():Int = {
return 3
}
}
jd-gui output
import scala.reflect.ScalaSignature;
#ScalaSignature(bytes=/* snip */)
public class f
{
public int jdMethod_new()
{
return 3;
}
}
I assume that jdMethod_ is prefixed in order to make the identifier valid. There is no jdMethod_ when looking at the class file using a hex editor.
However, this does have a flaw when you need to use public fields; scalac never generates public fields, it always makes them private and creates accessors.
So, what turned out to be the best solution for me was to use a hex editor (as suggested by user60561).
Apparantly, the name of every function and field is only saved once in the class file so if you use names with the same amount of bytes you can hexedit your way to victory.
For me it came down to replacing "new" by "abc" and every strange unicode character with a two-char sequence.
Thanks for all the suggestions.
I recently discovered the library kryonet, which is super awesome and fits my needs excellently.
However, the one problem that I am having is developing a good strategy for registering all of the classes that can be transferred.
I know that I can write a static method in each object that will return a list of all of the classes that it uses, but I would really rather not have to do that (for my own time purposes, as well as those who will be extending these objects).
I was playing around with trying to see if there was a way to get all of the classes that an object references (in it's fields as well as it's inheritance) from the getClass() method, but I was unable to have any success.
Finally, I know that kryo has kryo.setRegistrationOptional(true) but I am having a very difficult time trying to figure out how to use it. When this option is turned on, kryo still seems to throw exceptions if I haven't registered classes. Also, this method supposed is much slower than being able to register all of the classes. I'm fine if the first time you need to send an object using this method is slow, but I don't know if I'm okay with serious performance degradation every time that I want to send an object.
Ideally, I'll have a package of objects that I will want to send using kryonet. If there was just some was to scan that package and determine all of the classes that I need to register, that would be excellent. Now not all of my clients would need to register every object, but that's something of a separate issue, and I don't know if there is a solution to that.
If anyone could point me in the right direction that would be excellent.
Classes may come from different places such as disk, network, memory (dynamically generated). Therefore, obtaining information about classes to be registered with Kryo has to be handled separately for each specific case.
If you can read classes from a jar file then the following snippet should get you started.
private static List<Class<?>> getFromJarFile(final String jar, final String packageName) throws ClassNotFoundException, IOException {
final List<Class<?>> classes = new ArrayList<Class<?>>();
final JarInputStream jarFile = new JarInputStream(new FileInputStream(jar));
JarEntry jarEntry = null;
do {
jarEntry = jarFile.getNextJarEntry();
if (jarEntry != null) {
String className = jarEntry.getName();
if (className.endsWith(".class")) {
className = className.substring(0, className.lastIndexOf('.')); // strip filename extension
if (className.startsWith(packageName + "/")) { // match classes in the specified package and its subpackages
classes.add(Class.forName(className.replace('/', '.')));
}
}
}
} while (jarEntry != null);
return classes;
}
I'm trying to use Groovy to create an interactive scripting / macro mode for my application. The application is OSGi and much of the information the scripts may need is not know up front. I figured I could use GroovyShell and call eval() multiple times continually appending to the namespace as OSGi bundles are loaded. GroovyShell maintains variable state over multiple eval calls, but not class definitions or methods.
goal: Create a base class during startup. As OSGi bundles load, create derived classes as needed.
I am not sure about what you mean about declared classes not existing between evals, the following two scripts work as expected when evaled one after another:
class C {{println 'hi'}}
new C()
...
new C()
However methods become bound to the class that declared them, and GroovyShell creates a new class for each instance. If you do not need the return value of any of the scripts and they are truly scripts (not classes with main methods) you can attach the following to the end of every evaluated scrips.
Class klass = this.getClass()
this.getMetaClass().getMethods().each {
if (it.declaringClass.cachedClass == klass) {
binding[it.name] = this.&"$it.name"
}
}
If you depend on the return value you can hand-manage the evaluation and run the script as part of your parsing (warning, untested code follows, for illustrative uses only)...
String scriptText = ...
Script script = shell.parse(scriptText)
def returnValue = script.run()
Class klass = script.getClass()
script.getMetaClass().getMethods().each {
if (it.declaringClass.cachedClass == klass) {
shell.context[it.name] = this.&"$it.name"
}
}
// do whatever with returnValue...
There is one last caveat I am sure you are aware of. Statically typed variables are not kept between evals as they are not stored in the binding. So in the previous script the variable 'klass' will not be kept between script invocations and will disappear. To rectify that simply remove the type declarations on the first use of all variables, that means they will be read and written to the binding.
Ended up injecting code before each script compilation. End goal is that the user written script has a domain-specific-language available for use.
This might be what you are looking for?
From Groovy in Action
def binding = new Binding(x: 6, y: 4)
def shell = new GroovyShell(binding)
def expression = '''f = x * y'''
shell.evaluate(expression)
assert binding.getVariable("f") == 24
An appropriate use of Binding will allow you to maintain state?