I need a way to essentially secure my jar library to allow registered apps to use it in their projects and deny usage to apps that weren't approved by me.
It is fine if I hard code things in the lib for each distribution. I currently have this jar obfuscated.
What are good approaches to restrict the usage of a jar?
One idea was to lock the lib to a specific package so if the developer tries to use it in another project they can't. But I'm not sure if they can easily provide a custom fake Context to make it work...
To me the best approach if you would like your library to stay standalone (without involving the network for checking or downloading pieces of the library, I mean) would be to make mandatory the use of an initializer class that would receive a token from the client application.
This would be crackable as the token validity test would be performed by your lib: one may modify the lib in a way is would just skip that test, but this would be made harder by the obfuscation. But this is probably sufficient, unless using your lib without having registered it is a really critical issue.
So you would have something like:
boolean Initializer.initLib(String passcode)
That would prevent the lib to work unless passcode is correct.
You can make the obfuscation more efficient by avoiding checking that way:
public void initLib(String passcode) {
if (passcode == A_GIVEN_PUBLIC_STATIC_THAT_STORESTHE_CODE) {
// do the proper initializations
}
else {
throw new RuntimeException("Bad passcode, sorry!");
}
}
But doing that way instead:
public void initLib(String passcode) {
final char[] PASS_ENCRYPTED = "f5uhjgf56ik8kv214d5".toCharArray();
final char[] PASS_MINUSMASK = "bc".toCharArray();
final int PASS_SHIFT = 11;
final int PASS_MASK_MINUS = 2;
for (int ctr = 0; ctr < PASS_MINUSMASK.length; ++ctr) {
final char next = PASS_ENCRYPTED[PASS_SHIFT + ctr - PASS_MASK_MINUS];
if (passcode.charAt(ctr) != next - (PASS_MINUSMASK[ctr] - 'a')) {
// make the lib unusable by some inits. But it should look as a proper initialization
return;
}
}
// make the lib usable by some inits.
}
This looks stupid, but if you have a look at the obfuscated code, you will see a big difference. This code is just an example (it accepts "hi" as a valid passcode), any algorithm would be fine as long as its obfuscated version is not too straightforward to reverse.
Now the question is: what passcode?
As the library's protection concerns the developpers of the client apps that will use it, and not the final users of these apps, you cannot rely on any piece of data specific to the devices on which the applications will run. So no IMEI or anything like that.
If these developpers are trustworthy that's fine. A fixed passcode is sufficient.
But if they are subject to give this passcode to other people to allow them using your library, this is more difficult. In this case I don't think you can solve it without a real "industrial" process such as registering the client apps and their code checksums, for example. Such a process needs a specific design and cannot be solved "just by the code", but as it also has a cost (time, resources, involvment of the client...) you can only consider this if the use of library is very critical.
Can't you make your jar call your server with a specific code and the application name, to check if they are registered ?
When you build an Android app with a jar, that jar is compiled into the app and becomes a part of it. You can't just copy the jar out of the package and use it elsewhere. Unless I'm not understanding the question, this shouldn't be an issue you need to worry about.
Related
TLDR: File.exists() is buggy and i would like to understand why!
I am facing a weird issue (as so often happens) in my Android App. I will try to be as brief as i can.
First, i will show you the code and then provide some additional info. This is not the full code. Just the core of the issue.
Example code:
String myPath = "/storage/emulated/0/Documents";
File directory= new File(myPath);
if (!directory.exists() && !directory.mkdirs()) {
throw new IllegalArgumentException("Could not create the specified directory: " + directory.getAbsolutePath() + ".");
}
Most of the time this works fine. A few times however the exception is thrown which means that the directory did not exist and could not be created. Out of every 100 runs, it works fine on 95-96 times and fails 4-5 times.
I have declared the permissions for storage/read external storage/write external storage in my manifest and asked for the permissions on runtime. The problem does not lie there. (If anything i have too many permissions at this point :D ). After all, if it was a permission issue it would fail every time but in my case it fails at a rate of 4% or 5%.
With the above code i am attempting to create a file that points to the 'Documents' folder. In my app i am actually using String myPath = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOCUMENTS).getPath();
In the specific device where the error occurs this path happens to be "/storage/emulated/0/Documents" and this is why i hardcoded it in the example code i gave you.
If i use a file explorer app on the device (i.e. 'Astro file manager' i can see that the folder does exist and has some contents and also confirm that the path really is "/storage/emulated/0/Documents".
This has never happened to me locally. Only the users of the app experience the issue and i know the issue exists thanks to Firebase/Crashlytics. The users have the exact same tablet as the one i am using for development, namely a Lenovo TB-8504X. (I work for a company and we provide both the software and the hardware).
So, do you have any thoughts on why this issue occurs?
Has anyone ever experienced something similar?
Could the path to the 'Documents' folder sometimes be "/storage/emulated/0/Documents" and sometimes become something else on the same physical device?
I am an experienced Android developer but i am quite novice in Android architecture and the Android filesystem. Could it be that on start-up (when device is powered on or after a reboot) the filesystem has not yet 'mounted' the 'disk' at the point when my code checks if the directory exists? Here i am using the terms 'mount' and 'disk' as loosely as possible. Also my app is actually a launcher/parental control app so it is the first thing that gets fired when device starts. I am almost conviced that this does not make sense at all but at this point i am trying to see the greater picture and explore solutions that transcend typical Android development.
I would really appreciate your help as this issue is starting to get on my nerves.
Looking forward to any helpful responses.
Thanks in advance.
EDIT (27/08/2019) :
I came across this Java Bug Report although it is pretty outdated. According to this, when operating on NFS-mounted volumes, java.io.File.exists ends up performing a stat(2). If the stat fails (which it may do for several reasons), then File.exists (mistakenly) assumes that the file being stat'ed does not exist. Could this be the source of my troubles?
EDIT (28/08/2019) :
Today i am able to add a bounty to this question in an attempt to draw some more attention. I would encourage you to read the question carefully, look through the comments disregarding the one that claims that this has to do with costumer support from Realm. Realm code is indeed the one using the unreliable method but what i want to know is why the method is unreliable. Whether or not Realm can work around this and use some other code instead, is beyond the scope of the question. I simply want to know if one can safely use File.exists() and if not, why?
Once again, thank you all in advance. It would be really important to me to get an answer even if it is overly technical and involves a deeper understanding of NFS file systems, Java, Android, Linux, or whatever!
EDIT (30/08/2019) :
Because some users suggest replacing File.exists() with some other method, i'd like to state that what i am interested in at this point is understating why the method fails and not what one could use instead as a workaround.
Even if i wanted to replace File.exists() with something else, i am not able to do that because this piece of code resides in RealmConfiguration.java file (Read-only) which is part of the Realm Library that i use in my app.
To make things even more clear i will provide two pieces of code. The code i use in my activity and the method that get's called in RealmConfiguration.java as a consequence:
Code i use in my activity :
File myfile = new File("/storage/emulated/0/Documents");
if(myFile.exists()){ //<---- Notice that myFile exists at this point.
Realm.init(this);
config = new RealmConfiguration.Builder()
.name(".TheDatabaseName")
.directory(myFile) //<---- Notice this line of code.
.schemaVersion(7)
.migration(new MyMigration())
.build();
Realm.setDefaultConfiguration(config);
realm = Realm.getDefaultInstance();
}
At this point myFile exists and the code that resides in RealmConfiguration.java get's called.
The RealmConfiguration.java method that crashes :
/**
* Specifies the directory where the Realm file will be saved. The default value is {#code context.getFilesDir()}.
* If the directory does not exist, it will be created.
*
* #param directory the directory to save the Realm file in. Directory must be writable.
* #throws IllegalArgumentException if {#code directory} is null, not writable or a file.
*/
public Builder directory(File directory) {
//noinspection ConstantConditions
if (directory == null) {
throw new IllegalArgumentException("Non-null 'dir' required.");
}
if (directory.isFile()) {
throw new IllegalArgumentException("'dir' is a file, not a directory: " + directory.getAbsolutePath() + ".");
}
------> if (!directory.exists() && !directory.mkdirs()) { //<---- Here is the problem
throw new IllegalArgumentException("Could not create the specified directory: " + directory.getAbsolutePath() + ".");
}
if (!directory.canWrite()) {
throw new IllegalArgumentException("Realm directory is not writable: " + directory.getAbsolutePath() + ".");
}
this.directory = directory;
return this;
}
So, myFile exists in my activity, the Realm code get's called and suddenly myFile no longer exists.. Again i wish to point out that this is not consistent. I am noticing crashes at a rate of 4-5% meaning that most of the time myFile exists both in the activity and when the realm code makes it's check.
I hope this will be helpful.
Again thanks in advance!
First of all, if you are using Android, bug reports in the Java Bugs database are not relevant. Android does not use the Sun / Oracle codebase. Android started out as a clean-room re-implementation of the Java class libraries.
So if there are bugs in File.exists() on Android the bugs would be in the Android codebase, and any reports would be in the Android issue tracker.
But when you say this:
According to this, when operating on NFS-mounted volumes, java.io.File.exists ends up performing a stat(2). If the stat fails (which it may do for several reasons), then File.exists (mistakenly) assumes that the file being stat'ed does not exist.
Unless you are using NFS, that bug report is not directly relevant.
It is not a mistake / bug. It is a limitation.
At the file system level, it is a fact of life that Linux supports many different kinds of file system, and that many of them behave in unexpected ways ... compared to an "ordinary" file system. It is not possible for the JVM to hide all of the weird filesystem-specific edge cases at the Java API level.
On the API level, File.exists cannot report any errors. The signature doesn't allow it to throw an IOException, and throwing an unchecked exception would be a breaking change. All it can say is true or false.
If you want to distinguish the various reasons for a false, you should use the newer Files.exists(Path, LinkOptions...) method instead.
Could this be the source of my troubles?
Yes it could, and not just in the NFS case! See below. (With Files.exist, an NFS stat failure would most likely be an EIO, and that would raise an IOException rather than returning false.)
The File.java code in the Android codebase (version android-4.2.2_r1) is:
public boolean exists() {
return doAccess(F_OK);
}
private boolean doAccess(int mode) {
try {
return Libcore.os.access(path, mode);
} catch (ErrnoException errnoException) {
return false;
}
}
Note how it turns any ErrnoException into a false.
A bit more digging reveals that the os.access call is performing a native call which makes an access syscall, and throws ErrnoException if the syscall fails.
So now we need look at the documented behavior of the access syscall. Here's what man 2 access says:
F_OK tests for the existence of the
file.
On error (at least one bit in mode
asked for a permission that is denied, or mode is F_OK and the file
does not exist, or some other error occurred), -1 is returned, and
errno is set appropriately.
access() shall fail if:
EACCES The requested access would be denied to the file, or search per‐
mission is denied for one of the directories in the path prefix
of pathname. (See also path_resolution(7).)
ELOOP Too many symbolic links were encountered in resolving pathname.
ENAMETOOLONG
pathname is too long.
ENOENT A component of pathname does not exist or is a dangling symbolic
link.
ENOTDIR
A component used as a directory in pathname is not, in fact, a
directory.
EROFS Write permission was requested for a file on a read-only
filesystem.
access() may fail if:
EFAULT pathname points outside your accessible address space.
EINVAL mode was incorrectly specified.
EIO An I/O error occurred.
ENOMEM Insufficient kernel memory was available.
ETXTBSY
Write access was requested to an executable which is being executed.
I have struck out the errors that I think are technically impossible or implausible, but the still leaves quite few to consider.
Another possibility is something (e.g. some other part of your application) is deleting or renaming the file or a (hypothetical) symlink, or changing file permissions ... behind your back.
But I don't think that File.exist() is broken1, or that the host OS is broken. It is theoretically possible, but you would need some clear evidence to support the theory.
1 - It is not broken in the sense that it is not behaving differently to the known behavior of the method. You could argue until the cows come home about whether the behavior is "correct", but it has been like that since Java 1.0 and it can't be changed in OpenJDK or in Android without breaking thousands of existing applications written over the last 20+ years. It won't happen.
What to do next?
Well my recommendation would be to use strace to track the syscalls that your app is making and see if you can get some clues as to why some access syscalls are giving you unexpected results; e.g. what the paths are and what the errno is. See https://source.android.com/devices/tech/debug/strace .
I have had a similar issue, but with a higher trouble rate, where the Anti Virus was locking FileSystem, and thus failing any requests (almost instantly)
the workaround was using java.nio.Files.exists() instead.
SPSSReader reader = new SPSSReader(args[0], null);
Iterator it = reader.getVariables().iterator();
while (it.hasNext())
{
System.out.println(it.next());
}
I am using this SPSSReader to read the spss file. Here,every string is printed with some junk characters appended with it.
Obtained Result :
StringVariable: nameogr(nulltpc{)(10)
NumericVariable: weightppuo(nullf{nd)
DateVariable: datexsgzj(nulllanck)
DateVariable: timeppzb(null|wt{l)
DateVariable: datetimegulj{(null|ns)
NumericVariable: commissionyrqh(nullohzx)
NumericVariable: priceeub{av(nullvlpl)
Expected Result :
StringVariable: name (10)
NumericVariable: weight
DateVariable: date
DateVariable: time
DateVariable: datetime
NumericVariable: commission
NumericVariable: price
Thanks in advance :)
I tried recreating the issue and found the same thing.
Considering that there is a licensing for that library (see here), I would assume that this might be a way of the developers to ensure that a license is bought as the regular download only contains a demo version as evaluation (see licensing before the download).
As that library is rather old (copyright of the website is 2003-2008, requirement for the library is Java 1.2, no generics, Vectors are used, etc), I would recommend a different library as long as you are not limited to the one used in your question.
After a quick search, it turned out that there is an open source spss reader here which is also available through Maven here.
Using the example on the github page, I put this together:
import com.bedatadriven.spss.SpssDataFileReader;
import com.bedatadriven.spss.SpssVariable;
public class SPSSDemo {
public static void main(String[] args) {
try {
SpssDataFileReader reader = new SpssDataFileReader(args[0]);
for (SpssVariable var : reader.getVariables()) {
System.out.println(var.getVariableName());
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
I wasn't able to find stuff that would print NumericVariable or similar things but as those were the classnames of the library you were using in the question, I will assume that those are not SPSS standardized. If they are, you will either find something like that in the library or you can open an issue on the github page.
Using the employees.sav file from here I got this output from the code above using the open source library:
resp_id
gender
first_name
last_name
date_of_birth
education_type
education_years
job_type
experience_years
monthly_income
job_satisfaction
No additional characters no more!
Edit regarding the comment:
That is correct. I read through some SPSS stuff though and from my understanding there are only string and numeric variables which are then formatted in different ways. The version published in maven only gives you access to the typecode of a variable (to be honest, no idea what that is) but the github version (that does not appear to be published on maven as 1.3-SNAPSHOT unfortunately) does after write- and printformat have been introduced.
You can clone or download the library and run mvn clean package (assuming you have maven installed) and use the generated library (found under target\spss-reader-1.3-SNAPSHOT.jar) in your project to have the methods SpssVariable#getPrintFormat and SpssVariable#getWriteFormat available.
Those return an SpssVariableFormat which you can get more information from. As I have no clue what all that is about, the best I can do is to link you to the source here where references to the stuff that was implemented there should help you further (I assume that this link referenced to in the documentation of SpssVariableFormat#getType is probably the most helpful to determine what kind of format you have there.
If absolutely NOTHING works with that, I guess you could use the demo version of the library in the question to determine the stuff through it.next().getClass().getSimpleName() as well but I would resort to that only if there is no other way to determining the format.
I am not sure, but looking at your code, it.next() is returning a Variable object.
There has to be some method to be chained to the Variable object, something like it.next().getLabel() or it.next().getVariableName(). toString() on an Object is not always meaningful. Check toString() method of Variable class in SPSSReader library.
Today I came across an obfuscated class (well a lot of obfuscated classes in a jar) and I do not have a clue on how this kind of obfuscation is done.
An example:
protected void a(ChannelHandlerContext ☃, ByteBuf ☃, ByteBuf ☃)
throws Exception
{
int ☃ = ☃.readableBytes();
if (☃ < this.c)
{
☃.b(0);
☃.writeBytes(☃);
}
else
{
byte[] ☃ = new byte[☃];
☃.readBytes(☃);
☃.b(☃.length);
this.b.setInput(☃, 0, ☃);
this.b.finish();
while (!this.b.finished())
{
int ☃ = this.b.deflate(this.a);
☃.writeBytes(this.a, 0, ☃);
}
this.b.reset();
}
}
}
As you see above, all the parameter variables are a snow-man. How can this be undone? Also how is it done in the first place; how is the JVM able to "process" those and execute the code without any problem?
To clarify, I am not going to use this code, it is just for educational purposes. I am taking the Computer Science course at school so since we are learning Java and talking of limitations such as decompilations. I am interested in learning more, so I decided to have a look into bigger projects especially servers. This piece of code is pulled out of the Spigot server for Minecraft (A game) that is a fork of Bukkit server for Minecraft that was supposed to be open source.
First of all, you should note that it is the parameters which have this unicode and not the methods. Why is this important?
Parameters do not need to have names specified, as they are mostly indexed by a number reference. However it can be specified and I assume that most java runtimes do in fact not check this name as it is not needed for execution.
In the opposite, class names, method names, and field names are however needed.
About you mentioning Spigot, Spigot is indeed open source. However you most likely decompiled a class which is originally from the original Mojang Minecraft server, which is not open source and is indeed obfuscated.
Edit: In the case you want to investigate these classes, I recently found a tool called Bytecode Viewer, which is available at https://github.com/Konloch/bytecode-viewer
This tool has multiple decompilers as well as some options to view a more bytecode like version of the class file.
An example of a function I found contains the following bytecode data:
<localVar:index=1 , name=☃ , desc=D, sig=null, start=L1, end=L2>
<localVar:index=3 , name=☃ , desc=D, sig=null, start=L1, end=L2>
<localVar:index=5 , name=☃ , desc=D, sig=null, start=L1, end=L2>
Indeed as is visible, the unicode name has been set the same, but it does not matter as in the end the indexes (1,3,5) are used to reference these variables.
protected void a(ChannelHandlerContext ☃, ByteBuf ☃, ByteBuf ☃)
This isn't valid. You cannot have multiple parameters with the same name. It could be that you are not reading the unicode text with the right text format.
Your Text editor is showing the value of the unicode character.
I just tested on eclipse and names with unicode characters are acceptable.
public String publicationXmlUrl(int \u9090currentPage) {
But writing with values are not:
public String publicationXmlUrl(int ♥currentPage) {
I apologize if this is a duplicate, I couldn't find anything describing exactly what I wanted. I'm building a webapp that has a number of different properties that need to change depending on the environment in addition to a number of .properties configuration files that need to change as well. Right now I have a global enum (DEVELOPMENT, STAGING, and PRODUCTION) that is used to determine which string constants are used in the application and then I utilize a bunch of comments in the configuration files to switch between database servers, etc. There has got to be a better way to do this...I'd ideally like to be able to make one change in one file (A large block comment would be fine...) to adjust these configurations. I saw this post where the answer is to utilize JNDI which I really like, but it would seem as though I would need to call that from a servlet that starts up or a bean that gets initialized on start in order to use it for my log4j or JDBC configuration files.
Does anybody have any strategies for dealing with this?
Thanks!
I'm not sure if this strategy will apply to your situation, but in the past I've successfully used our build tool (ant in that case) to build different war files depending on the profile. So you would have multiple log4j configuration files in your source tree, and then delete the ones you don't want from the final build depending on the profile that was used to build it.
Traceability becomes slightly hard (i.e. difficult sometimes to figure out which one was used to build it), but it's a very clean solution, from your code perspective, since it's all done in your build script.
We store all our default configuration values in a single XML file. During deployment we apply an XML patch (RFC-5261) with values specific to the environment.
https://www.rfc-editor.org/rfc/rfc5261
http://xmlpatch.sourceforge.net/
I am going to assume that your properties files are made up of 95% name=value pairs that are identical across all your deployment environments and 5% of name=value pairs that change from one deployment environment to another.
If this assumption is correct, then you could try something like the following pseudocode.
void generateRuntimeConfigFiles(int deploymentMode)
{
String[] searchAndReplacePairs;
if (deploymentMode == Constants.PRODUCTION) {
searchAndReplacePairs = ...
} else if (deploymentMode == Constants.STAGING) {
searchAndReplacePairs = ...
} else { // Constants.DEVELOPMENT
searchAndReplacePairs = ...
}
String[] filePairs = new String[] {
"log4j-template.properties", "log4j.properties",
"jdbc-template.properties", "jdbc.properties",
"foo-template.xml", "foo.xml",
...
};
for (int i = 0; i < filePairs.length; i += 2) {
String inFile = filePairs[i + 0];
String ouFile = filePairs[i + 1];
searchAndReplaceInFile(inFile, outFile,
searchAndReplacePairs);
}
}
Your application calls generateRuntimeConfigFiles() before initialising anything else that might rely on properties/XML files.
Now the only problem you have to deal with is how to store and retrieve different settings for searchAndReplacePairs. Perhaps you could obtain them from files with names such as production.properties, staging.properties and development.properties.
If the above approach is appealing to you, then email me for the source code of searchAndReplaceInFile() to save you having to re-invent the wheel. You can find my email address from the "info" box in my Stackoverflow profile.
I suggest using Apache Commons Configuration. It provides all the plumbing for dealing with different configurations depending on your environment.
http://commons.apache.org/configuration
Is there any way to completely disable Java security manager?
I'm experimenting with source code of db4o. It uses reflection to persist objects and it seems that security manager doesn't allow reflection to read and write private or protected fields.
My code:
public static void main(String[] args) throws IOException {
System.out.println("start");
new File( DB_FILE_NAME ).delete();
ObjectContainer container = Db4o.openFile( DB_FILE_NAME );
String ob = new String( "test" );
container.store( ob );
ObjectSet result = container.queryByExample( String.class );
System.out.println( "retrieved (" + result.size() + "):" );
while( result.hasNext() ) {
System.out.println( result.next() );
}
container.close();
System.out.println("finish");
}
Output:
start
[db4o 7.4.68.12069 2009-04-18 00:21:30]
AccessibleObject#setAccessible() is not available. Private fields can not be stored.
retrieved (0):
finish
This thread suggests modifying java.policy file to allow reflection but it doesn't seem to work for me.
I'm starting JVM with arguments
-Djava.security.manager -Djava.security.policy==/home/pablo/.java.policy
so specified policy file will be the only policy file used
The file looks like this:
grant {
permission java.security.AllPermission;
permission java.lang.reflect.ReflectPermission "suppressAccessChecks";
};
I spent last 3 hrs on this and don't have any ideas how to make this work.
Any help appreciated.
You could try adding this to the main() of your program:
System.setSecurityManager(null);
Worked for me for a "trusted" WebStart application when I was having security manager issues. Not sure if it will work for your db4o case, but it might be worth a try.
EDIT: I'm not suggesting that this is a general solution to security manager problems. I was just proposing it as a way to help debug the original poster's problem. Clearly, if you want to benefit from a security manager then you should not disable it.
Do you really have two '=' signs in your java.security.policy command line option? That won't work. Make sure you are setting the property as
-Djava.security.policy=/home/pablo/.java.policy
To actually disable the SecurityManager, simply leaving off the java.security.manager system property altogether should be enough.
Update: As I was reading the documentation for policy files to learn more about the "==" syntax, I noticed that unless the policy file is in the current working directory, it needs to be specified as a URL (including scheme). Have you tried prefixing the policy path with the "file:" scheme?
I was also puzzled because (assuming you are running as user "pablo"), it looks like that policy should be loaded by default from your home directory, so you shouldn't need to specify it at all. On the other hand, if you are not running as the user "pablo", maybe the file is not readable.
I found this example of how to make private fields and methods accessible to your code. Basically, it distills down to the use of Field.setAccessible(true) and Method.setAccessible(true)
Field example:
Field privateStringField = PrivateObject.class.
getDeclaredField("privateString");
privateStringField.setAccessible(true);
Method example:
Method privateStringMethod = PrivateObject.class.
getDeclaredMethod("getPrivateString", null);
privateStringMethod.setAccessible(true);
You could also look at using Groovy with your Java code as it (currently) circumvents much of the access level restrictions of Java code. Although, this message board posting seems to suggest this 'feature' may change in future versions of Groovy.