So here's my situation:
I am running a Java Client/Server architecture that has high CPU usage and I'm trying to reduce the lag time on the main "server" thread. I have been profiling the server with YourKit to identify the CPU-hogging code.
The problem is:
I am using someone else's code, and because of the way it is written, it is impossible to decompile, then recompile without using a special obfuscator which I do not have access to (no I am not violating any copyrights or anything).
What I am currently doing:
To modify the class files without worrying about obfuscation, I have been using Storyyeller's amazing Krakatau decompiler (https://github.com/Storyyeller/Krakatau) to disassemble class files into assembly files.
I manually edit the .j assembly files while looking at a Jasmin reference page (which takes FOREVER and I often mess up), then reassemble them into class files and run them again.
What I want to do:
Instead of painstakingly editing the assembly, I was wondering, does anyone know of a way to convert .java Source Code to .j Assembly code?
Also, if I simply decompile the .class files, is it possible for me to simply recompile them even though the packages for the imports do not exist?
import com.bazinga.*;
public class MainThread{}//compile this even though package com.bazinga doesn't exist?
If anyone knows ANY WAY I could do this, I would really appreciate it!
Instead of painstakingly editing the assembly, I was wondering, does anyone know of a way to convert .java Source Code to .j Assembly code?
Yes and no. The obvious answer is that you can just compile your code and then disassemble the resulting classes. However, this is not always helpful, because compilation can be context dependent (such as inlining static final constants, or handling of nested classes). Additionally, if you plan to add your code into an existing method, you have to be careful to not use existing local variable slots or clobber the operand stack.
My best advice is to try to isolate your modifications as much as possible. For example, if you want to add code to the jar, instead of inserting it into an existing class, just write the code you want to add in Java, compile it and add the classfiles in. Then modify the target class to call into your own class.
As for imports, you can compile against stubs. Just create a dummy class with the name you want, and optionally dummy methods for anything you need to call. The implementations can just be {throw null;} or similar, since you won't actually be executing them ever, they just need to exist to satisfy the compiler during compilation.
Related
Do certain Java compilers prefer a different layout of a Java file before it is compiled unto a class file for the JVM?
What I mean is, by first writing your main Class and then writing all your other classes in the following lines bring or not a faster compile time?
Does the compiler take longer because it has not yet encountered what it is it needs for the relevant information in the main Class?
If I recall correctly, Java doesn't use explicit pointers either so I don't see that being an issue.
In other words, if you write your Classes outside of main first does this speed up compile time?
If any such difference exists, it would be so negligible you won't notice it.
In other words - you should focus on organizing the classes in a way that would make sense and would be easy for you to maintain, not on helping the compiler.
It is pretty simple: you specify the order of classes.
In other words: you give a list of file or directory names to the compiler. And then the compiler processes those in the order given, to then walk through each file. Sometimes it will make forward reference to understand that types are used that aren't defined yet.
I guess: when you ask the compiler to go for a complete directory, it will simply read the files in the order that the file system uses (like alphabetical).
Finally: this is definitely an implementation detail of the compiler (or even the build tool that generates the commands running the compiler). So a different tool, or tool version might lead to different results. So again: don't waste your time to "optimize" for this.
I'm working on a sandbox feature for my java antivirus, and I've come into a question: Does the specified package on a class matter for compilation?
Example:
I'm running a program that wants to use Runtime.getRuntime().exec(), when the classloader attempts to load that to run a method, does it check the package qualified in the file, if they exist? I would prefer not to try and change files in the JVM, but to simply load ones from a different package. I can accomplish the loading and such, but my only dilemma, will it crash and burn? Inside the java, it would be registered as say, java.lang.Runtime, but the compiled code will say for example pkg.pkg.Runtime and will it need to extend the old runtime? My guess is that extending the old runtime would just break it. Does anyone know anything about this? I'm working on making a testable example, but I'm still a bit away and wanted to get some answers, as well as this might benefit some people.
Does the specified package on a class matter for compilation?
Yes it does matter. A class called pkg.pkg.Runtime() cannot be loaded as if it was java.lang.Runtime.
Furthermore, if my memory is correct, the JVM has some additional security measures in it to prevent normal applications from injecting classes into core packages such as java.lang.
If you need to change the behaviour of the java.lang.Runtime class (for experimental purposes!) then I think you will need to put your modified version on the boot classpath, ahead of the "rt.jar" file.
However:
This level of tinkering can easily result in JVM instability; i.e. hard JVM crashes that are difficult to diagnose.
If your aim is to produce a "production quality" tool, then you will find that things that involve tinkering with the JVM are not considered acceptable. People are going to be very suspicious of installation instructions that say things like "add this to your installed JVM's bootclasspath".
Distributing a "tinkered with" JVM may fall foul of Oracle's Java licensing agreement.
My advice would be to look for a less intrusive way of doing what you are trying to do. For instance, if you are trying to do virus checking, either do it outside of the JVM, or in a custom application classloader.
You commented:
I have a custom classloader, my question is: If I compile a class that is labelled as say, pkg.pkg.Runtime, can I register in my classloader as java.lang.Runtime?
As I said above, no you can't. A bytecode file has the classname embedded in it. If you attempt to "pull a swifty" by loading a class with a different name, the JVM will throw an Error.
And:
If not, then how can I replace the class? If the compiled package name has to equal the request referenced naming, then can I modify the .class file to to match, or perhaps compile it as if it were in the java.lang package?
That's what you would have to do. You need to name the class java.lang.Runtime in the source code and compile it as such.
But what I meant by my advice above is that you should use do the virus checking in the class loader. Forget about trying to replace / modify the behaviour of Runtime. It is a bad idea for the reasons I listed above.
I'm looking for some ideas on how to compile Java code with some other pieces of code missing (method calls). I am fully aware that javac will not allow you to compile Java files if cannot find all dependencies. But maybe there is some way how to bypass it, something like force compile.
My bytecode knowledge is not so good but I think some method invoke is just full package definition of class and method name with parameters. So if compiler just puts this data to class file and assume in running process dependency will be available (if not simple NoSuchMethodExp).
Only workaround so far I found is to create empty missing class files with empty methods to "cheat" compiler. Works perfectly but there should be easier way :)
Any ideas?
Use Interfaces.
Create the interfaces that have the methods you need. At runtime, inject (Spring, Guice, etc.) or generate (cglib ...) classes that implement the interface.
If you're modifying a jar, you can extract the class files you are not modifying to another directory and include that in the classpath. That way they will be available to the compiler.
Bad luck! Probably all you can do is to create mock objects for missing parts of code just to compile your code (empty methods, so the compiler can find it).
Another question - if you miss some classes, how will you execute that code?
UPDATED according to information provided:
Well, there is another option to modify classes in jar, you can use AOP, and to make it done read about AspectJ - actually for me this is the easiest option (typically you need to spend time mocking objects, writing empty methods, so I would contribute that time to study new technology, which will help you many times ;)
And btw the easiest way to implement it, if you use Eclipse, is:
install AJDT
create aspect project
create aspect which modifies code (depending on what you need to change)
add jar file you want to modify
immediately get modified code in
another already packed jar file
Sounds magically :)
In this case you don't need any dependencies in classpath, except for libraries which are needed for new code you add!
Methods aren't dependencies. They are part of the class definition. The only places the java runtime looks for method definitions is in the class def that was compiled at compile time and in its parent classes. If you're problem is that a super class is incomplete, I don't think I can help you.
If not, you could define some of these methods as abstract and than have a child class implement them.
What kind of code is missing? Normally this happens if you refer to libraries your compiler can't find. Maybe you simply need to extend the classpath the compiler is searching for classes.
If you really refer to code that is not available yet you need to implement at least those methods you refer to. But that sounds strange... maybe you can clear things up.
Is there a tool to deobfuscate java obfuscated codes?
The codes is extracted from a compiled class but they are obfuscated and non-readable.
First step would be to learn with which tool it was obfuscated. Maybe there's already a "deobfuscator" around for the particular obfuscator.
On the other hand, you can also just run an IDE and use its refactoring powers. Rename the class, method and variable names to something sensitive. Use your human logical thinking powers to figure what the code actually represents and name them sensitively. And the picture would slowly but surely grow.
Good luck.
Did you try to make the code less obscure with Java Deobfuscator (aka JDO), a kind of smart decompiler?
Currently JDO does the following:
renames obfuscated methods, variables, constants and class names
to be unique and more indicative of
their type
propogates changes throughout the entire source tree (beta)
has an easy to use GUI
allow you to specify the name for a field, method and class (new feature!)
Currently JDO does not do the
following (but it might one day)
modify method bytecode in any way
Not to gravedig but I wrote a tool that works on most commercial obfuscators
https://github.com/Contra/JMD
I used Java Deobfuscator (aka JDO) but it has a few bugs. It can't work with case sensitive file names.
So I've changed the source and uploaded a patch for that in sourceforge.
The patch, Download
Most likely only human mindpower to make sense of it. Get the best decompiler available and ponder on its output.
Maybe it will work on Unix/Linux/MacOS?
If so, you could move one step of your process to a VM, in where you unpack the code, before you rename the too long names. How long is the file name limit on Windows?
It may not be best practice but are there ways of removing unsused classes from a third party's jar files. Something that looks at the way in which my classes are using the library and does some kind of coverage analysis, then spits out another jar with all of the untouched classes removed.
Obviously there are issues with this. Specifically, the usage scenario I put it though may not use all classes all the time.
But neglecting these problems, can it be done in principle?
There is a way.
The JarJar project does this AFAIR. The first goal of the JarJar project is to allow one to embed third party libraries in your own jar, changing the package structure if necessary. Doing so it can strip out the classes that are not needed.
Check it out at http://code.google.com/p/jarjar/.
Here is a link about shrinking jars: http://sixlegs.com/blog/java/jarjar-keep.html
There is a tool in Ant called a classfileset. You specify the list of root classes that you know you need, and then the classfileset recursively analyzes their code to find all dependencies.
Alternatively, you could develop a good test suite that exercises all of the functions that you need, then run your tests under a test coverage tool. The tool will tell you which classes (and statement in them) were actually utilized. This could give you an even smaller set of code than what you'd find with static analysis.
I use ProGuard for this. As well as being an excellent obfuscator, it has a code shrinking phase which can combine multiple JARs and then strip out any unused classes or class members. It does an excellent job at shrinking.
At a previous job, I used a Java obfuscator that as well as obfuscating the code, also removed classes and methods that weren't being used. If you were doing "Class.byName" or any other type of reflection stuff, you needed to tell the obfuscator because it couldn't tell by inspecting the code what classes or methods called by reflection.
The problem, of course, is that you don't know if other parts of the third party library are doing any reflection, and so removing an "unused" class might cause things to break in an obscure case that you haven't tested.
jar is just a zip file, so I guess you can. If you could get to the source, it's cleaner. Maybe try disassembling the class?
Adding to this question, can that improve performance? Since the classes not used would not be JIT compiled improving startup time or does the java automatically detect that while compiling to bytecode and do not even deal with the code that is not used?
This would be an interesting project (has anyone done it already?)
I presume you'd give the tool your jar(s) as a starting point, and the library jar to clean up. It could use reflection to determine which classes your jar(s) reference directly, and which are used indirectly down the call tree (this is not trivial at all, but doable). If it encounters any reflection code in any of the two places, it should give a very loud warning.