It is possible to access bits of MATLAB's internal java code to programmatically change MATLAB itself. For example, you can programmatically open a document in the editor using
editorServices = com.mathworks.mlservices.MLEditorServices;
editorServices.newDocument() %older versions of MATLAB seem to use new()
You can see the method signatures (but not what they do) using methodsview.
methodsview(com.mathworks.mlservices.MLEditorServices)
I have a few related questions about using these Java methods.
Firstly, is there any documentation on these things (either from the Mathworks or otherwise)?
Secondly, how do you find out what methods are available? The ones I've come across appear to be contained in JAR files in matlabroot\java\jar, but I'm not sure what the best way to inspect a JAR file is.
Thirdly, are there functions for inspecting the classes, other than methodsview?
Finally, are there any really useful methods that anyone has found?
There is no official documentation nor support for these classes. Moreover, these classes and internal methods represent internal implementation that may change without notice in any future Matlab release. This said, you can use my uiinspect and checkClass utilities to investigate the internal methods, properties and static fields. These utilities use Java reflection to do their job, something which is also done by the built-in methodsview function (I believe my utilities are far more powerful, though). In this respect, I believe we are not crossing the line of reverse-engineering which may violate Matlab's license.
If you are looking for documentation, then my UndocumentedMatlab.com website has plenty of relevant resources, and more is added on a regular basis so keep tuned.
I am also working on a book that will present a very detailed overview of all these internal classes, among other undocumented stuff - I hope to have publication news later this year.
I am an eclipse fan. If you use that as your IDE, the jar can be imported into one of your projects and you can inspect the methods in there.
To find out more about java objects, I use uiinspect.
The only place I know that is documenting the Matlab hidden Java stuff is Undocumented Matlab by Yair Altman. His site lists plenty of very useful tricks. Being able to use Java to format text in list boxes has come in very handy for me, for example.
EDIT
The man has spoken. Listen to him, since I don't think there's anyone outside MathWorks who knows more about Matlab's internal java code.
Undocumented Matlab is a great place to start looking.
Related
Is there any tool that allows you to "query" (rather than simply search) the JDK documentation? For example:
Show me all deprecated methods.
Show me all subclasses of a particular class (rather than only the direct subclasses, which the documentation provides).
Show me all methods that return (say) a Thread.
Show me all instances of a given method name, regardless of signature.
What prompted this question is that I was muddling up two completely unrelated methods that I only use occasionally: Pattern.matches() and Matcher.matches(). And then I found that there is also PathMatcher.matches(), which also has a completely unrelated purpose. And that got me wondering how many other "matches()" methods there are in the JDK. And then I thought that there may be other useful queries that could also be run against the JDK documentation.
The only motivation for having such a tool is to help me improve my own knowledge of Java with information that is interesting or useful (to me at least), but is not otherwise easy to obtain. This question is similar, but I am looking for something more sophisticated than a simple search.
ETA: Marcel's suggestion below of using the Doclet API provides a great solution, without too much effort.
ETA2: Re determining deprecated methods, I've just found out that Oracle already address this in the JavaDoc API here
Could it be that you're approaching this from the wrong angle? Rather than parsing the docs, which is an already transformed representation of the source, why not parse source code or byte code of the JDK directly?
parsing byte code
parsing source code
hook into the Javadoc tool (i.e. let Javadoc parse the code for you) by using the Doclet API
Depending on your needs you might also want to take the really easy road and have your classpath scanned by the reflections library.
Reflections reflections = new Reflections("some.package");
Set<Method> voidMethods = reflections.getMethodsReturn(Thread.class);
That having said don't forget that any good IDE can dig up a lot of the info you seem to be looking for (e.g. searching for methods called matches).
I'm writing an applet, which uses ~10 external libraries. Together they occupy more than 2 megabytes. In some libs we use only 1-2 classes, so a lot of others can be safely deleted. So the question is how to remove unused classes from jar libraries?
A lot of other questions link to Proguard. But it doesn't process libraries (or I am doing something wrong) and also ruins parts of code which use reflection.
You could use the maven-shade-plugin and tell it to build a minimized jar file that combines your code and libs.
You could use something like ClassDep, which statically identifies which classes you will use.
However it's possible to easily fool this. Imagine some of your code contains:
Class.forName(className);
so you can dynamically build a classname and load that class. Tools like ClassDep can't identify these cases, so you'd need to perform comprehensive testing on your shrunken jars.
ProGuard can process your code together with the libraries (with the option -injars). You can still keep external libraries that you don't want to process (with the option -libraryjars).
Any automatic shrinking tool will have problems with reflection. ProGuard recognizes some basic reflection and it allows you to specify the parts of the internal API that should be preserved for the sake of reflection. ProGuard supports some powerful configuration, but depending on the amount of reflection in the libraries, it may still require trial and error.
You can simply "unzip" the JAR's, take only the classes you want from each, and place them in a custom archive. Brian A. gave a good suggestion on how to identify those classes and some caveats. I would add they you may be violating licenses as well...
There are few open source projects/APIs/libraries that we use in our project (Spring, Struts, iBatis etc.) and I want to understand their design and how they work internally.
What is the best way to understand these projects? Note that I am already using these libraries in my project. And I know the input-output interaction/configurations for these libraries. What I don't understand is how these APIs/libraries work internally.
The problems I face is:
Finding the entry class of the library. Is there any way I can know the entry class for the library - something which is kicking the whole API?
Tools/Plugins to use in Eclipse to get an overview of the design of the library. Going through each and every class of the library, can be a very daunting task. Is there any tool you would like to recommend which can generate the class diagrams of the API in Eclipse.
Thanks in advance!!
UPDATE: I need some inputs on eclipse plugins which can help me in getting an overview/class diagram of the library
I always use the same strategy for this: I never try to "understand" the code base as a whole, and I usually try to follow the request flow. I read enough of the documentation to determine what is necessary to use the application, and I read that code (Keep all source code loaded in your IDE).
For example, in struts you'll be installing a servlet filter in web.xml. Start reading the filter and follow the path a single request takes through your stack.
Likewise for spring, there are two main entry points, the filter and "getBean", both of which are mentioned real early in the documentation. Read those two.
For both of these cases you'll find one or two classes that represent the "core" of the framework real quickly. Read those really well and let actual use cases & needs drive your further exploration.
Approaching "understanding" of an open source library (or any other code base for that matter) by trying to find all the pieces is usually not a very good way of approaching these things, it will usually just lead nowhere because a lot of these things contain too much code. When following the request flow I find making diagrams can also be quite distracting, it tends to draw attention/focus away from understanding (and since my understanding increases rapidly most of them are out-of-date even before they reach the printer).
Nice question!!!, what I've done, specially in the case of Spring, apart from consulting the Documentation and their API's is to attach the sources of the project to my project on Eclipse, that way I'm able to navigate through the source code, not just the API. Its been quite helpful specially in the case of the Spring-Security project, there were some concepts that I just couldn't understand until I inspected the source code.
That's one of the advantages of using Open Source libraries.
Regards.
Tools like Structure101 (http://www.headwaysoftware.com/products/structure101/index.php), and Lattix (http://www.lattix.com/) let you analyze code and produce architecture diagrams / dependency matrices.
This is not exactly class diagram - the main focus is on layering. So the entry point is usually the topmost layer.
But then again, as I specified above, you will notice that some libs are just a mess, and these tools will not be helpful enough.
See the S101 online demo: http://www.structure101.com/java/
This for example is the Sonar project architecture: http://www.structure101.com/java/tracker/sonar/1.11.1/arch.html
Your best bet for those three would be to consult the official documentation (make sure you are looking at the version you are using) or to get a book on the technology.
Most APIs don't have a class with a main method; they're running in the webserver called by the server itself. Unless they're running as their own server, they won't have a main method.
I'm trying to code an application which runs un different java platforms like J2SE, J2ME, Android, etc. I already know that I'll have to rewrite most of the UI for each platform, but want to reuse the core logic.
Keeping this core portable involves three drawbacks that I know of:
Keeping to the old Java 1.4 syntax, not using any of the nice language features of Java 5.0
only using external libraries that are known to work on those platforms (that is: don't use JNI and don't have dependencies to other libs which violate this rules)
only using the classes which are present on all those platforms
I know of ways to overcome (1): code in 5.0 style and automatically convert it to 1.4 (retroweaver - haven't tried it yet, but seems ok).
I think (2) is a problem that I just have to accept.
Now I'd like to know what's the best workarround for (3), especially collection classes, which I miss the most. I can think of those:
Most programmers I know just don't use Set, Map, List, etc. and fallback to Vector and plain Arrays. I think this makes code ugly in the first place. But I also know that the right choice between TreeSet/Hashset or LinkedList/ArrayList is crucial for performance, and always using Vector and Arrays can't be right.
I could code my own implementations of that classes. This seems to be reinventing the wheel, and I think I could not do it as good as others have done.
Since Java is open source, I could grab the sourcecode of the J2SE Collections framework and include into my application when building for J2ME. I don't know if this is a good idea, though. Pherhaps there are good reasons not to do this.
Maybe there already are libraries out there, which rebuild the most important features of the collections framework, but are optimized for low end systems, pherhaps by not implementing functionality that is used infrequently. Do you know any?
Thanks for your answers and opinions!
Edit: I finally found a (complex, but nice) solution, and I thought by providing my own answer and accepting it, the solution would become visible at the top. But to the contrary, my answer is still at the very bottom.
J2ME is brutal, and you're just going to have to resign yourself to doing without some of the niceties of other platforms. Get used to Hashtable and Vector, and writing your own wrappers on top of those. Also, don't make the mistake of assuming that J2ME is standard either, as each manufacturer's JVM can do things in profoundly different ways. I wouldn't worry much about performance initially, as just getting correctness on J2ME is enough of a challenge. It is possible to write an app that runs across J2ME, J2SE and Android, as I've done it, but it takes a lot of work. One suggestion that I'd have is that you write the core of your application logic and keep it strictly to java.lang, java.util and java.io. Anywhere where you're going to be doing something that might interact with the platform, such as the file system or network, you can create an interface that your core application code interacts with, that you have different implementations for the different environments. For example, you can have an interface that wraps up HTTP stuff, and uses javax.microedition.io.HttpConnection with J2ME and java.net.HttpURLConnection on Android. It's a pain, but if you want to maintain an app running on all three of those environments, it can get you there. Good luck.
It's been a while since I asked this question, and I while since I found a nice, working solution for the problem, but I had since forgotton to tell you.
My main focus was the Java Collections Framework, which is part of the java.util package.
I've finally taken the source code of Suns Java 6.0 and copied all the classes that belong to the Collections framework into a project of my own. This was a Java 6.0 project, but I used the jars from J2ME as classpath. Most of those classes that I copied depend on other J2SE classes, so there are broken dependencies. Anyway, it was quite easy to cut those depensencies by leaving out everything that deals with serialization (which is not a priority for me) and some minor adjustments.
I compiled the whole thing with a Java 6 compiler, and retrotranslator was used to port the resulting bytecode back to Java 1.2.
Next problem is the package name, because you can't deliver classes from java.util with a J2ME application and load them - the bootstrap class loader won't look into the applications jar file, the other bootloaders aren't allowed to load something with that package name, and on J2ME you can't define custom classloaders. Retrotranslator not only converts bytecode, it also helps to change name references in existing bytecode. I had to move and rename all classes in my project, e.g. java.util.TreeMap became my.company.backport.java.util.TreeMap_.
I was than able to write actual J2ME application in a second Java 6.0 project which referenced the usual java.util.TreeMap, using the generic syntax to create type-safe collections, compile that app to Java 6.0 byte code, and run it through retrotranslator to create Java 1.2 code that now references my.company.backport.java.util.TreeMap_. Note that TreeMap is just an example, it actually works for the whole collections framework and even for 3rd party J2SE Jars that reference that framework.
The resulting app can be packaged as a jar and jad file, and runs fine on both J2ME emulators and actual devices (tested on a Sony Ericsson W880i).
The whole process seems rather complex, but since I used Ant for build automation, and I needed retranslator anyway, there only was a one-time overhead to setup the collection framework backport.
As stated above, I've done this nearly a year ago, and writing this mostly from the top of my head, so I hope there are no errors in it. If you are interested in more details, leave me a comment. I've got a few pages of German documentation about that process, which I could provide if there is any demand.
We faced exactly this situation in developing zxing. If J2ME is in your list of targets, this is your limiting factor by far. We targeted MIDP 2.0 / CLDC 1.1. If you have a similar requirement, you need to stick to Java 1.2. Java 1.4 language features are definitely not present (like assert) and in general you won't find anything after 1.2 in J2ME.
We did not use external libraries, but, you could package them into your deployed .jar file with little trouble. It would make the resulting .jar bigger, and that could be an issue. (Then you can try optimizers/shrinkers like ProGuard to mitigate that.)
I did end up reimplementing something like Collections.sort() and Comparator since we needed them and they are not in J2ME. So yeah you might consider doing this in cases, though only where necessary.
We used Vector and Hashtable and arrays since there is no other choice, really, in J2ME. I would just use them unless you have a reason not to, and that would be performance I guess. In theory JVM makers are already optimizing their implementation but that doesn't mean you couldn't make a better one... I guess I would be surprised if it is worth it in the vast majority of cases. Just make sure you really need to do this before putting in the effort.
To answer part of your question another collections library would be Javolution which can be built for j2me.
a colleague pointed me the other day to BCEL which , as best I can tell from his explanation and a quick read, a way to modify at run time the byte code. My first thought was that it sounded dangerous, and my second thought was that it sounded cool. Then I gave it some more thought and I recalled the codinghorror post on monkey-patching and realized that this was basically the same thing. Has anyone ever used BCEL for anything practical? Am I right that this is basically run time monkey patching, or am I missing something?
From BCEL's FAQ:
Q: Can I create or modify classes
dynamically with BCEL?
A: BCEL contains useful classes in the
util package, namely ClassLoader and
JavaWrapper.Take a look at the
ProxyCreator example.
But monkeypatching is... uhm... controversial, and you probably shouldn't use it if your language doesn't support it.
If you have a good use case for it, may I suggest embbededing Jython?
It's a bit more low-level than classic monkey patching, and from what I read, the classes already loaded into the VM are not updated. It only supports saving it to class files again, not modifying run time classes.
You might look at it as monkey patching. I prefer not to use it (maybe I never faced a good use case for it?), but be familiar with it (to have an idea how Spring and Hibenrate use it and why).
See this realworld example: Jawk - Compiler Module. BCEL is useful for "compilation" ur custom language.
BCEL does not support monkey patching, it just manipulates with bytecode and possibly loads it in a custom classloader. However you can implement monkeypatching on JVM using library like BCEL and Java agent. The Java agent (loaded by -javaagent argument) can access the Instrumentation API and modify loaded classes. It is not hard to implement it via some bridges.
But remember:
I am not sure if having to use -javaagent is something you want.
In any language, monkey patching can lead to badly predictable behavior.
You can modify a method. In theory, you can also add some method, but you need to compile the project against modified (patched) classes. I think this would cause a lot of pain and it is not worth of it. There are alternative languages that support it (e.g. Groovy) or suppport something similar (e.g. implicit conversions in Scala).
It is better to design your API well than to use monkey patching. It may be rather useful for third party libraries.