a colleague pointed me the other day to BCEL which , as best I can tell from his explanation and a quick read, a way to modify at run time the byte code. My first thought was that it sounded dangerous, and my second thought was that it sounded cool. Then I gave it some more thought and I recalled the codinghorror post on monkey-patching and realized that this was basically the same thing. Has anyone ever used BCEL for anything practical? Am I right that this is basically run time monkey patching, or am I missing something?
From BCEL's FAQ:
Q: Can I create or modify classes
dynamically with BCEL?
A: BCEL contains useful classes in the
util package, namely ClassLoader and
JavaWrapper.Take a look at the
ProxyCreator example.
But monkeypatching is... uhm... controversial, and you probably shouldn't use it if your language doesn't support it.
If you have a good use case for it, may I suggest embbededing Jython?
It's a bit more low-level than classic monkey patching, and from what I read, the classes already loaded into the VM are not updated. It only supports saving it to class files again, not modifying run time classes.
You might look at it as monkey patching. I prefer not to use it (maybe I never faced a good use case for it?), but be familiar with it (to have an idea how Spring and Hibenrate use it and why).
See this realworld example: Jawk - Compiler Module. BCEL is useful for "compilation" ur custom language.
BCEL does not support monkey patching, it just manipulates with bytecode and possibly loads it in a custom classloader. However you can implement monkeypatching on JVM using library like BCEL and Java agent. The Java agent (loaded by -javaagent argument) can access the Instrumentation API and modify loaded classes. It is not hard to implement it via some bridges.
But remember:
I am not sure if having to use -javaagent is something you want.
In any language, monkey patching can lead to badly predictable behavior.
You can modify a method. In theory, you can also add some method, but you need to compile the project against modified (patched) classes. I think this would cause a lot of pain and it is not worth of it. There are alternative languages that support it (e.g. Groovy) or suppport something similar (e.g. implicit conversions in Scala).
It is better to design your API well than to use monkey patching. It may be rather useful for third party libraries.
Related
It is possible to access bits of MATLAB's internal java code to programmatically change MATLAB itself. For example, you can programmatically open a document in the editor using
editorServices = com.mathworks.mlservices.MLEditorServices;
editorServices.newDocument() %older versions of MATLAB seem to use new()
You can see the method signatures (but not what they do) using methodsview.
methodsview(com.mathworks.mlservices.MLEditorServices)
I have a few related questions about using these Java methods.
Firstly, is there any documentation on these things (either from the Mathworks or otherwise)?
Secondly, how do you find out what methods are available? The ones I've come across appear to be contained in JAR files in matlabroot\java\jar, but I'm not sure what the best way to inspect a JAR file is.
Thirdly, are there functions for inspecting the classes, other than methodsview?
Finally, are there any really useful methods that anyone has found?
There is no official documentation nor support for these classes. Moreover, these classes and internal methods represent internal implementation that may change without notice in any future Matlab release. This said, you can use my uiinspect and checkClass utilities to investigate the internal methods, properties and static fields. These utilities use Java reflection to do their job, something which is also done by the built-in methodsview function (I believe my utilities are far more powerful, though). In this respect, I believe we are not crossing the line of reverse-engineering which may violate Matlab's license.
If you are looking for documentation, then my UndocumentedMatlab.com website has plenty of relevant resources, and more is added on a regular basis so keep tuned.
I am also working on a book that will present a very detailed overview of all these internal classes, among other undocumented stuff - I hope to have publication news later this year.
I am an eclipse fan. If you use that as your IDE, the jar can be imported into one of your projects and you can inspect the methods in there.
To find out more about java objects, I use uiinspect.
The only place I know that is documenting the Matlab hidden Java stuff is Undocumented Matlab by Yair Altman. His site lists plenty of very useful tricks. Being able to use Java to format text in list boxes has come in very handy for me, for example.
EDIT
The man has spoken. Listen to him, since I don't think there's anyone outside MathWorks who knows more about Matlab's internal java code.
Undocumented Matlab is a great place to start looking.
First, I have no experience doing this. But like the beginning of any good program, I have problem that I need to fix, so I'm willing to learn.
So many of you are probably already familiar with pdftk, the handy utility for handling various pdf-related tasks. So far as I can tell, most of these features are available in much newer, lighter libraries/extensions, except the one I need (and probably the only reason it still exists): merging form data files (fdf and xfdf) with a form PDF and getting a new file as the output.
The problem is that my server doesn't have gcj, which is fundamental to build/compile pdftk. I don't know if it's because I'm on Solaris or if it's for some other sysadmin-level reason, but I'm not getting gcj anytime soon. And there are no pre-compiled binaries for Solaris as far as I can find.
So I'm thinking that the MAKE file and C code can be rewritten to import the Java library (very ancient version of itext) directly, via javac.
But I'm not sure where to really start. All I know is:
I want a binary when I'm done, so that there won't be a need for a Java VM on every use.
The current app uses GCJ.
So my first thought was "Oh this is easy, I can probably just call the classes with some other C-based method", but instead of finding a simple method for doing this, I'm finding tons of lengthy posts on the various angles that this can be approached, etc.
Then I found a page on Sun's site on how to call other languages (like C) in a Java class. But the problems with that approach are:
I'd have to write a wrapper for the wrapper
I'd probably be better off skipping that part and writing the whole thing in Java
I ain't ready for that just yet if I can just import the classes with what is already there
I'm not clear on if I can compile and get a binary at the end or if I'm trapped in Java being needed every time.
Again, I apologize for my ignorance. I just need some advice and examples of how one would replace GCJ dependent C code with something that works directly with Java.
And of course if I'm asking one of those "if we could do that, we'd be rich already" type questions, let me know.
I'm not sure what you are looking for exactly, so I provided several answers.
If you have java code that needs to run, you must:
Run it in a jvm. You can start that vm within your own custom c-code, but it is still using a jvm
Rewrite it in another language.
Compile with an ahead-of-time compiler (eg gcj)
Incidentally, you could compile a copy of gcj in your home folder and use that. I believe the magic switch is --enable-languages=java,c (see: here for more)
If you have c-code you want to call from java, you have four options:
Java Native Interface (JNI). It seems you found this
Java Native Access (JNA). This is slower than JNI, but requires less coding and no wrapper c-code. It does require a jar and a library
Create a CLI utility and use Runtime.Exec(...) to call it.
Use some sort of Inter Process Communication to have the Java code ask the c-code to perform the operation and return the result.
Additional platform dependent options
Use JACOB (win32 only: com access)
I am not sure if I understand what you are looking for.
If you are looking to incorporate the C code into Java to make a native binary without the gcj, I think you are out of luck. You can include the C in Java, but it would be a primarily Java program meaning you would need the JVM on each run. Is there anything stopping you from compiling the gcj yourself?
I am just curious but I want to know if it is feasible to remove totally the Java standard class libraries coming with the JVM and start a new one from the scratch [à la ClassPath].
If that is possible, what classes MUST be implemented as minimum? (Object and String come to my mind, but... I do not know).
Such thing breaks some license? Is there any way to say to the "java" command to "not use the rt.jar"?
Thanks in advance,
Ernesto
You can use the -Xbootclasspath option to specify your own set of core classes.
If you do go down this path, you will probably end up with a lot of problems if you intend to also use third party libraries as they will depend on the core API and any inconsistencies in your version will likely cause bugs.
As an absolute minimum you'd probably have to reimplement everything in the java.lang package. As well as Object and String, the primitive wrapper classes need to be present in order for auto-boxing to work. I don't think you can replace java.lang without a fair bit of native code to make things like threads work.
In theory, "yes" it is possible, though you might also need to implement your own JVM! The relationships between the JVM and some of the low level classes (Object, Class, Thread, etc) are such that these classes have to be implemented as part of the JVM.
In practice, it is such a big task that you'd be working on it for the rest of your life, and the chances are that nobody would use your code even if you succeeded. That doesn't sound like "fun" to me.
Such thing breaks some license?
Not per-say. But if you ever tried to release it calling it "Java", Sun's lawyers would be after you for trademark infringement. You can only legally call your implementation Java if it has been validated against the Sun TCK.
But I don't want to be totally discouraging. If you want to hack on the internals of a JVM or stuff like that, the JNode project is always looking for keen new people.
No, it is not feasible at all. I mean, sure, you could do it, but you aren't going to do it better than a large corporation or open source project with years of experience and large numbers of Java gurus. It might be fun to geek it up though.
Normally boot strap loaders are written using assembly languages.
Is it possible to dot it using java with native interface.
There are some possible solutions:
It exists hardware, that is able to
execute Java-bytecode.
Also you can compile Java-code to
native code with different
compilers.
Finally you can have a mini-loader, that
starts the rest of the process in
Java.
Neither of this solutions is easy or does make too much sense. But possible - yes.
For completeness: JNode is a project targeting to create a PC-Operating-system in Java.
Also answers to this question may be helpful. It ask for device-drivers in Java, and that's not completely the same question, but some of the solutions presented in the answers may be helpful with your question too.
I don't want to say an outright no, because I'm sure if I did someone would come up with a way of doing it, but this would certainly be VERY difficult (and possibly fruitless).
For Java to run on a JVM, you'd need to natively bootstrap a sufficient amount of the OS natively that then switching to java would be a bit of a waste of time (it really wouldn't accomplish much other than adding complexity).
There are devices that can "natively" run bytecode where it's conceivably possibly, but I don't think that's viable most of the time.
Of course it is. A better question would be whether it is practical or not, and if it would offer any benefits worth the implementation time compared to existing methods.
#Mnementh mentioned JNode. For the record, JNode uses GRUB as its bootstrap loader.
I agree with #Mnementh that implementing a boot loader in Java would be technically possible though probably difficult. But my question to the OP is "why would you want to do it?". I don't see the point.
I am a fan of static metaprogramming in C++. I know Java now has generics. Does this mean that static metaprogramming (i.e., compile-time program execution) is possible in Java? If so, can anyone recommend any good resources where one can learn more about it?
No, this is not possible. Generics are not as powerful as templates. For instance, a template argument can be a user-defined type, a primitive type, or a value; but a generic template argument can only be Object or a subtype thereof.
Edit: This is an old answer; since 2011 we have Java 7, which has Annotations that can be used for such trickery.
The short answer
This question is nearly more than 10 years old, but I am still missing one answer to this. And this is: yes, but not because of generics and note quite the same as C++.
As of Java 6, we have the pluggable annotation processing api. Static metaprogramming is (as you already stated in your question)
compile-time program execution
If you know about metaprogramming, then you also know that this is not really true, but for the sake of simplicity, we will use this. Please look here if you want to learn more about metaprogramming in general.
The pluggable annotation processing api is called by the compiler, right after the .java files are read but before the compiler writes the byte-code to the .class files. (I had one source for this, but i cannot find it anymore.. maybe someone can help me out here?).
It allows you, to do logic at compile time with pure java-code. However, the world you are coding in is quite different. Not specifically bad or anything, just different. The classes you are analyzing do not yet exist and you are working on meta data of the classes. But the compiler is run in a JVM, which means you can also create classes and program normally. But furthermore, you can analyze generics, because our annotation processor is called before type erasure.
The main gist about static metaprogramming in java is, that you provide meta-data (in form of annotations) and the processor will be able to find all annotated classes to process them. On (more easy) example can be found on Baeldung, where an easy example is formed. In my opinion, this is quite a good source for getting started. If you understand this, try to google yourself. There are multiple good sources out there, to much to list here. Also take a look at Google AutoService, which utilizes an annotation processor, to take away your hassle of creating and maintaining the service files. If you want to create classes, i recommend looking at JavaPoet.
Sadly though, this API does not allow us, to manipulate source code. But if you really want to, you should take a look at Project Lombok. They do it, but it is not supported.
Why is this important (Further reading for the interested ones among you)
TL;DR: It is quite baffling to me, why we don't use static metaprogramming as much as dynamic, because it has many many advantages.
Most developers see "Dynamic and Static" and immediately jump to the conclusion that dynamic is better. Nothing wrong with that, static has a lot of negative connotations for developers. But in this case (and specifically for java) this is the exact other way around.
Dynamic metaprogramming requires reflections, which has some major drawbacks. There are quite a lot of them. In short: Performance, Security, and Design.
Static metaprogramming (i.e. Annotation Processing) allows us to intersect the compiler, which already does most of the things we try to accomplish with reflections. We can also create classes in this process, which are again passed to the annotation processors. You then can (for example) generate classes, which do what normally had to be done using reflections. Further more, we can implement a "fail fast" system, because we can inform the compiler about errors, warnings and such.
To conclude and compare as much as possible: let us imagine Spring. Spring tries to find all Component annotated classes at runtime (which we could simplify by using service files at compile time), then generates certain proxy classes (which we already could have done at compile time) and resolves bean dependencies (which, again, we already could have done at compile time). Jake Whartons talk about Dagger2, in which he explains why they switched to static metaprogramming. I still don't understand why the big players like Spring don't use it.
This post is to short to fully explain those differences and why static would be more powerful. If you want, i am currently working on a presentation for this. If you are interested and speak German (sorry about that), you can have a look at my website. There you find a presentation, which tries to explain the differences in 45 minutes. Only the slides though.
Take a look at Clojure. It's a LISP with Macros (meta-programming) that runs on the JVM and is very interoperable with Java.
What do you exactly mean by "static metaprogramming"? Yes, C++ template metaprogramming is impossible in Java, but it offers other methods, much more powerful than those from C++:
reflection
aspect-oriented programming (#AspectJ)
bytecode manipulation (Javassist, ObjectWeb ASM, Java agents)
code generation (Annotation Processing Tool, template engines like Velocity)
Abstract Syntax Tree manipulations (APIs provided by popular IDEs)
possibility to run Java compiler and use compiled code even at runtime
There's no best method: each of those methods has its strengths and weaknesses.
Due to flexibility of JVM, all of those methods in Java can be used both at compilation time and runtime.
No. Even more, generic types are erased to their upper bound by the compiler, so you cannot create a new instance of a generic type T at runtime.
The best way to do metaprogamming in Java is to circumvent the type erasure and hand in the Class<T> object of your type T. Still, this is only a hack.
If you need powerful compile-time logic for Java, one way to do that is with some kind of code generation. Since, as other posters have pointed out, the Java language doesn't provide any features suitable for doing compile-time logic, this may be your best option (iff you really do have a need for compile-time logic). Once you have exhausted the other possibilities and you are sure you want to do code-generation, you might be interested in my open source project Rjava, available at:
http://www.github.com/blak3mill3r
It is a Java code generation library written in Ruby, which I wrote in order to generate Google Web Toolkit interfaces for Ruby on Rails applications automatically. It has proved quite handy for that.
As a warning, it can be very difficult to debug Rjava code, Rjava doesn't do much checking, it just assumes you know what you're doing. That's pretty much the state of static metaprogramming anyway. I'd say it's significantly easier to debug than anything non-trivial done with C++ TMP, and it is possible to use it for the same kinds of things.
Anyway, if you were considering writing a program which outputs Java source code, stop right now and check out Rjava. It might not do what you want yet, but it's MIT licensed, so feel free to improve it, deep fry it, or sell it to your grandma. I'd be glad to have other devs who are experienced with generic programming to comment on the design.
Lombok offers a weak form of compile time metaprogramming. However, the technique they use is completely general.
See Java code transform at compile time for a related discussion
You can use a metaprogramming library for Java such as Spoon: https://github.com/INRIA/spoon/
No, generics in Java is purely a way to avoid casting of Object.
In a very reduced sense, maybe?
http://michid.wordpress.com/2008/08/13/type-safe-builder-pattern-in-java/