How to deobfuscate jar file using mapping.txt generated by Proguard? - java

This may sound suspicious, but I need to deobfuscate my own app. Some time ago I published my app that I obfuscated with Proguard. Unfortunately I did not save that version of the app and changed the code a lot.
Now I need the code of that older version. I still have mapping.txt file generated by Proguard.
I have converted the classes.dex file to jar (using dex2jar tool), now I need to deobfuscate it according to mapping.txt. Are there any tools that can help me? Maybe Proguard itself?
UPD. I need a tool that can automatically rename all the classes, functions etc, so I dont have to do it manually. Thank you

If you have the mappings then you can get every class, field, and method name back.You can use java-asm (A bytecode modification library) to remap the class and member names. Most of the work will be updating member descriptors.
I made one for myself and it took about a day to get it working reliably.
Edit: It's not perfect, but it's pretty darn close. Link.
Give it a proguarded jar file, the mappings and it'll rename all it can find. There's a few issues but for just looking at the source it should do the job. Just dump the jar it makes (FileName + "-Re.jar") into a decompiler such as Luyten/JD-GUI. Also the file inputs are only changed via the buttons. The textboxes are just for visual-feedback/confirmation.
Edit 2: Fixed NPE occurring when class entry in mappings doesn't exist in the given jar.
Edit 3: Fixed a lot of issues resulting in methods not being renamed.
Edit 4: Uploaded compiled jar to github (releases, in the zip file it's in the mapping util), added support for multiple mapping types.

I suggest to use proguard tool as suggested in above answer along with Enigma. The solution for your problem can't be very straightforward and you need to use a combination of available tools to resolve it.

Related

How to import JAR file as a project into an IDE?

I have a Maven project. After I make changes to the project, I package the project up into an Uber Jar file and upload that to my backups. I am wondering, if my computer were to break or I lose all the code to the project, can I get the project back using only the Uber Jar file or would the project be gone forever?
How can I open this Jar file as a project and view all the Java code inside of it?
It would be entirely gone; that jar file contains only class files, not your source files, and you can't 'recover' source files from class files (you can decompile them which is mostly useless for this purpose; all comments, most names, most structure - all gone. You can't feasibly continue your project with this).
As it isn't in there, it's not possible to 'open a jar file and see all java code inside it'.
You'll need to set up backups.
The proper way forward is to first set up version control; this ensures that you don't just have a backup of the latest state of your source files, but every state of it - you can travel back to any time. Protects against code you deleted by accident, and means you can freely remove code you think you no longer need without that nagging feeling of: Yeah but what if.... maybe later? - so stop commenting out stuff you don't need, just get rid of it. It also means if there's a bug, you can time travel to the exact point in time when you added the line, and you can review the changes made by others in a project (presumably you're writing this on your own for now, but at some point you'll work with more people than just yourself!)
Read up on git which is the version control system that 80%+ or so of the community uses, probably best not to spring for exotic options if you aren't familiar. There are a billion tutorials out there to find if you search the web.
Host your git on a site like github or bitbucket which therefore also takes care of backups. Alternatively, host it on your own server (it's not complicated; git is open source). If you can't do that either, just let git write to a local directory and then use e.g. backblaze or something similar to ensure that file is backed up.
Yes, you can view your code by using a decompiler. I have experience mostly with IntelliJ, and this IDE includes a decompiler of its own. In case you lose everything and have only the jar file. You can use this included decompiler to get your source back from your .class files. But instructions on doing that are a story for another question...
If you want to secure your code use GIT. A version control tool that is a must when it comes to programming. Google about it and after a few days of playing around with it, you will never worry about such things.
Right-click on your project.
Select Build Path.
Click on Configure Build Path.
Click on Libraries, select Modulepath and select Add External JARs.
Select the jar file from the required folder.
Click and Apply and Ok.

IntelliJ IDEA - Decompiling/Editing/Recompiling JAR

I realize this may seem like a completely stupid question and this question is a "wall of text", but I'm feeling like I've gotten way out of my depth here and need some guidance. I've read several SO questions on this topic, but still can't seem to find one that addresses this particular situation. (please reference update(s) at the end of this post)
BACKGROUND
Our company has an application that was built in Java and released as an executable JAR package by a developer who passed away a couple of years ago. Since then, there has been minimal need for even looking at that code, let alone making any changes. That's been really good because I do my programming in VB.NET (Visual Studio) and, while I can read and make sense of Java code, I'm not proficient in actually writing/debugging that code.
However, a recent issue has come up that forced me to try to take a look at this Java code due to some internal changes in organization and data structure. I've looked in his "src\" directory and found older versions of his original code but wasn't able to find the most recent version anywhere. I found some code that made it possible for me to extract the JAR that's currently being used to a local directory on CodeProject (JarsVB), so I've been able to look over some of the .java files when trying to figure out what query is being used for some random operation. I don't know what tool(s) the original developer used to create the project/JAR, so I've installed the IntelliJ IDEA Community Edition 2018 as an IDE, and it's worked for me so far to simply view the code so I can understand a bit about what it's doing.
PROBLEM/GOALS
Unfortunately, now there is a need for me to make a change to the Java code and update the JAR, and this is where I can't seem to make heads or tails of what I need to do. I have my local copy of the code from the "unzipped" JAR containing all the .java and .class files (along with all the other resources), but I'm not sure how to go from that to modifying the code and recompiling the executable JAR.
So, my goals here are as follows:
(properly) Decompile the existing executable JAR. (If the JarsVB solution I mentioned above did what it was supposed to do, I should already have this part handled, but if there's a better, more "integrated" way of doing it, I'd be open to that as well.)
Modify one or more .java files. (I believe I can figure out the syntax well enough to get this part done.)
Debug/test my changes.
Recompile the changes into an updated executable JAR for release. (THIS is where I'm experiencing the most confusion.)
WHAT I'VE DONE
I've been looking at the documentation for IntelliJ to try to figure out how to make this all happen, but I feel like I'm missing stuff. I set my "Project Structure" settings to point to a specific folder, but I can't seem to get it to actually build anything in my specified path.
I went into one of the .java files and made/saved a small change to test, then went to the Build menu and tried all the building options available to me: "Build Project", "Build Module", and "Rebuild Project". Each time, the event log shows that "All files are up-to-date" (even though I changed one), so I go to my output directory to see what it built, but the folder is empty.
I looked at the IntelliJ documentation for Packaging a Module into a JAR File, which says to
On the main menu, choose Build | Build Artifact
But that option is disabled in my Build menu, and when I look in the Project Structure dialog, the Artifacts section is empty, so I try to add a new setting (Add -> JAR -> From modules with dependencies...), but I'm not sure how to properly set that up either.
I tried to select a Main Class from my classes/ directory, but it wouldn't actually accept any of the .class files I selected, so I just left it blank. Then, I did a search for a MANIFEST file, but couldn't find one anywhere so I left that blank as well. The Build menu option is enabled now, but when I tried to Build Artifact, again, I get nothing in my output directory. In fact, the only files I can find that have changed are in my local working directory. I'm just dumbfounded at this point.
FINAL THOUGHTS/QUESTIONS
I've tried to provide as much detail here as I could think of about all the things I've tried and gone through to get this JAR updated, but if there's a question about anything, please let me know. I'm not looking for a "handout" and I don't expect anyone to do this for me, but I'm also not wanting to become a Java developer just for the sake of making some minor changes to an application that will eventually be replaced by a .NET application. I simply am not familiar enough with the tools or Java development in general to know how to get to where I want to be from where I am.
My decompiled source files are in a totally separate directory from the original, production JAR file because, when I recompile this time, I want to completely recreate the JAR. Am I understanding the Java development process correctly in editing one of the .java files I got from decompiling with the JarsVB and then recompiling the JAR?
Thanks in advance for any assistance in finding the resources I need to accomplish these goals.
EDIT/UPDATE
So, looking at the link in the accepted answer in another SO question, How to build jars from IntelliJ properly?, I figured out at least one part of my issue: Leaving the Main Class setting of my Artifacts configuration blank was a problem. Since I hadn't been able to select a .class file and I wasn't sure how to correctly populate that field, I hadn't given the IDE enough information to operate correctly. I entered the namespace and class (namespace.class) I found in the class file that apparently defines the main method for the application, then set the path for the MANIFISET.MF file to my output directory and tried again to Build Artifact.
This time, at least, it DID produce a JAR in my defined output directory, but it's only 1KB in size. As stated above, the source files are in a completely separate directory from the original JAR file from which they were decompiled. The output directory, too, is completely separate from the location of the original JAR file. Do I need a copy of the original JAR file in the output path for recompiling to work correctly?
I'm making progress, but I'm sure I'm overlooking something "stupid" at this point that's primarily due to my unfamiliarity with the IDE and developing Java applications in general.
UPDATE 2
Looking at another SO question - how to create jar of java package in intelliJ idea - I learned that I have to individually add the necessary files for repackaging into the JAR. However, this brings up the question, what files do I add? The linked question identifies the .class files, but when I go look at the .class files in my working directory, none of those have been updated recently, so it looks like I'm still missing a step somewhere.
Another question this brings up is, are there certain conventions for Java development of which I need to be aware when preparing my environment? I have my output path set to a completely separate folder than any working or production code, so I'm wondering if something in that setup might potentially cause issues.
As I said before, I made a small change to one of the .java files, then tried both the Build Module and Rebuild Project options, but those are still telling me that "All files are up-to-date". Even so, I tried adding just the .class files from under my classes\ directory to my Artifact configuration and tried again to Build Artifact. I got a bit larger file (approx. 5MB), but when I try to execute the JAR, it just doesn't appear to do anything, let alone actually launch the application.
I tried again by adding the root folder of my local copy of the source adding everything the root folder contains. (yes, the directory probably needs some "spring cleaning", but that's for another day)
This time, I got a much larger file this time (approx. 21MB), so I thought I might have fixed the problem. Unfortunately, no such luck. The JAR still doesn't appear to execute.
For reference, the original JAR file from which the code was decompiled is approx. 59MB in size so, either IntelliJ is doing an incredible job with compression, or there's yet another step I haven't yet found. I'm sure this is all a matter of getting my IDE configured correctly, but I just can't seem to find the right combination of settings.

My jar resource files in intellij are read only and I need to edit them

I have tried unsuccessfully for a few hours now to edit the java files in a jar I am using as a library. I have marked the resource as a content root and as a source root but I am still unable to edit the code in the jars. The project compiles and runs correctly but I need to make an adjustment to a resource file and cannot; I have tried every project structure I could think of. Is it just impossible? All help is appreciated.
It is not recommended to edit JAR files. From the perspective of reproducibility1, it is better to:
Get hold of the source tree for the library
Check it into your version control (or fork it on Github)
Modify and build it
Use the resulting JAR instead of the original JAR
Another approach is to "overlay" the changes you want to make by creating a another JAR with the alternative version of the resources and placing it earlier in the application classpath.
But if neither of those works for you, you can use the jar command from the command line to modify a JAR file:
Use jar -x ... to extract the files to a temporary directory tree
Apply what ever changes need to be made to the tree
Use jar -c .... to create a new JAR from the tree.
Read the manual entry for the jar command for more details. Signing the new JAR with the original keys would be an issue if you are not the original signer, but I doubt that that is relevant to you.
1 - The point is that the next guy maintaining your code needs to know what you did to the library JAR that you "edited", in case he needs to do the same procedure with another version of the JAR. If you do it by hand, he has no choice but to do a forensic comparison of the differences between the original and your edited version. And that assumes that the original JAR can still be obtained. Note that "the next guy" could be you ... in a couple of months or years time, when you have forgotten exactly what you did.

Why JAR Files Do Not Contain Documentation?

I'm in the process of writing a small Java library that contains a related code that I usually include in most of my android app. I decided to export the library as a jar file then drop the file in the libs folder of my future projects.
Using Android Studio:
I created a Java Library module and put my code in it. And I added some comments to some of the method, following this.
Then, I ran the jar task in gradle, which gave me the .jar file in build/libs directory of my module.
Now, when I used this jar in one of my android apps, Everything works as expected, except the Doc part. When I hover over the classes and methods of my library, I don't see the Doc comments that I wrote.
Q1: Am I missing another step?
Q2: Are jar files supposed to have no comments?
The javadocs are the documents that are generated from the javadoc comments in your source code. They are not part of a normal JAR file because that would unnecessarily bloat the JAR files ... with stuff that someone running to code doesn't need.
The javadocs can be generated by a Gradle task, by the javadoc command (if you have a Java SDK installed) and by various other tools. You can then read them using a web browser.
On the other hand, IDEs can often render the javadoc comments in source code and display them as pop-ups, etcetera. (Some people would call this "javadocs", but I think that is an overstatement, since you typically can't navigate the documentation ... like you can with read javadoc documents.)
In order to render the javadoc comments, the IDE needs the source code. JAR files don't (normally) contain any source code or javadocs. Instead, the normal way to deal with this is to tell the IDE where the source code is, either by pointing it at a source code directory, a ZIP file containing source code, or URL for downloading the source code.
(I don't use Android Studio, so I can tell you exactly how to do this. However, I imagine that the IDE's online help explains how to do it ...)
It seems that your end goal here is to distribute your libraries in a way that allows programmers to see the javadoc comments.
The simple way to do that is to distribute source code. This Q&A describes how to get Gradle to generate a separate archive containing the source code, or add the source code to the JAR containing your compiled code1.
If that isn't acceptable, you may need to generate the javadocs as HTML2 and provide the HTML tree as a separate ZIP file that a programmer can unzip and read with a web browser. Alternatively, put the javadocs up on a website.
1 - I would not recommend this. People who just want to use the JAR as a binary are liable to complain about "bloat".
2 - If neither providing source code or javadoc HTML documentation is acceptable, I don't think there is a pragmatic solution.
There is a separate Gradle task to generate javadoc. Try adding the following:
task javadocJar(type: Jar, dependsOn:javadoc) {
classifier = 'javadoc'
from javadoc.destinationDir }
And then run:
gradle javadocJar
See if that helps.
In addition to the above, you can try and add the following to make to generate a single jar with both compiled classes and javadoc:
jar {
from javadoc.destinationDir
}
jar.dependsOn javadoc
I don't know if that's the right decision to bundle everything in the same jar. I prefer keeping the jars separate and maybe find another way to make the IDE use the javadoc jar file. Maybe try adding the javadoc jar as another dependency of the module.
Yes This is possible
Hi, This is possible but with a small change like in the jar file.
First of all, from a code point of view jar file contains only compiled ".class" files and not source files ".java"
So if you need a doc to be applied with a jar by this I mean not the index.html which gets created but the comment that appears whenever a person uses the jar API and calls a method with a suggestion.
Example :
For that, we need to also add a source file while generating .jar file.
Steps for the same:
Type comments/java docs in code
Generate Docs
This will create a doc folder in project folder
Now create jar file
Make sure you choose this option as shown below
Almost done just test it by importing jar to another project and it should the suggestions as per docs
Very Important this can be harmful as you are including source files.java in your jar so before making make sure if you need this or not.!!!!
Hope this gave your answer
Any questions you can contact me over: VaibhavMojidra.com

How to step onto/debug imported java code in netbeans

I have been searching for a long time on how to step through imported code classes in netbeans with no solution.
When debugging or stepping into the code, netbeans does not highlight and stop at the imported code classes being used. This is extremely inconvenient since the code I am working on is complex and stepping through is the only way to fully understand it.
I created a new project with existing sources. I have made sure that under window>debugging>sources I have added my working folders.
I also made sure the sources were added in the "runtime" classpath.
The compile classpath appears to be empty, I'm not sure whether that has something to do with it.
MORE INFO: In the debugging window while the program is paused, the main method profile from my imported class says it is reading from "Hidden source calls", but it is really just the normal imported code.
Why is this happening? Can anyone please help?
Folks,
One of the reasons you can get Hidden Source calls is adding in existing packages and java source code at the wrong directory level. You MUST add in the parent directory of the package not the child directory that represent the package name,
example if you have /home/torsi/java/src/torsipackage/com/main.java, you need to add in src, not torsipackage, you will now you made this mistake if you see listed in the top-level package name for your package.
Tom
Well I've not heard the term "Hidden source calls" before but I wonder if you're trying to step into third party code? If you want to step into compiled code it must have source associated I believe. I think it's possible to step into native code because they include the source somehow in the distribution for that exact reason. If you want to step into a library that's been developed by some other private entity you will likely not be afforded that luxury unless it's open source, and even then you may have to retrieve the source first.
Ultimately my point is that there is a difference between a .java file and a .class file. I believe it's possible to reverse engineer .class files but haven't done research on the topic.
If you want to step into code you didn't write odds are you'll need to change your code to work with the imported library or alternatively file a bug report with the developer.

Categories

Resources