Develop Reference

Java - make a library and import optional

I have a library that I'm using in an Java application - it's important for certain functionality, but it's optional. Meaning that if the JAR file is not there, the program continues on without issue. I'd like to open source my program, but I can not include this library, which is necessary to compile the source code as I have numerous import statements to use the API. I don't want to maintain two code sets. What is the best way to remove the physical jar file from open source release, but still maintain the code to support it where other people could still compile it?

the typical approach taken is to define the wrapper API (i.e. interfaces) and include those interfaces in the open sourced code, and then provide configuration options where one can specify class names of classes that implement certain interfaces.
You will import API interfaces instead of importing classes directly into your open sourced code. This way, you are open sourcing the API but not the implementation of the parts that you do not want to open source or you cannot open source.
There are many examples, but take a look at JDBC API (interfaces) and JDBC drivers (implementation classes) for starters.

I was pretty much typing the same thing as smallworld with one addition. If this API were necessary you can use a project build tool like Maven to handle the dependencies on you project. If someone checks it out from source control with the pom they can download the dependencies for themselves and you don't have to include them in a source repo.

There's probably a number of ways to fix this, here's a couple I can think of:
If you have only a couple of methods you need to invoke in the 3rd party library, you could use reflection to invoke those methods. It creates really verbose code, that is hard to read though.
If you don't have too much of the API in the 3rd party library you use, you could also create a separate JAR file, containing just a non-functional shell of the classes in the library (just types with the same names and methods with the same signatures). You can then use this JAR to distribute and compile against. At run-time you'd replace it with the real JAR if available.
The most common way is probably to just create a wrapper API in a separate module/project for the code that is dependent on the 3rd party library, and possibly distribute a pre-built JAR. This might go against your wish to not maintain two code sets, but may prove to be the best and less painful solution in the long run.

How to encrypt a .jar file

I'm working in a project where we need to encrypt the .jar file so no one can access to the .class files which inside the jar file.... is there any java coding which can help me to encrypt the .jar file ?

Even if you encrypt the jar file, it must be decrypted before the JVM is able to run it, so you'll need another jar file containing classes that decrypt and loads in the JVM.
Since this second jar file cannot be itself encrypted, a malicious user wanting to see you class files, can simply look at classes in this second jar file, and then decrypt your super-secret jar file and have access to it.
Maybe you can increase security of your code using an obfuscator, but it will eventually protect (make it harder but not impossible) your class files from decompilation, not from being used.
If obfuscation is not enough, you could consider compiling your jar file to a DLL for windows or a SO for unix/linux, that will make it much harder to decompile, but it's not always possible to do that correctly and it's generally a PITA. GCJ is able to do this somehow, and there are other commercial products that will actually compile .class/.jar directly to machine code.
However please consider that it does not matter how much security you put in it, since the client computer MUST be able to execute it, it must be able to read it, so no matter what your code will be exposed, you can only make it harder.
If you really have an algorithm so secret you don't want to disclose no matter what, consider converting it to a web service, hosting it on your server, so that you don't have to send the actual code to the client machines and can also better prevent unauthorized copies of your application by checking access to that vital part of it.

I assume you are aware of the fact that any skilled java coder can reverse-engineer the Java tool you use (or write) and still decode the app's jars? Also writing custom classloaders which read your "encrypted" code can be decompiled and a tool could be written to bypass it.
Even with obfuscation and bytecode modification and custom classloaders, java is hackable/decompileable and the source can almost always be brought to a somewhat readable state.

You want to obfuscate, not encrypt, the jar file.
A popular choice for doing this in Java is ProGuard.

No. Since your program needs to be able to run the code it would be pointless anyway.
You can obfuscate your code though so decompiling the .class files results in less readable code (meaningless variable/class names etc).

As far as I know this is not supported by standard JVM. But you can do the following. Separate your application into 2 parts. First will not be encrypted. It will be a simple loader that will instantiate the rest using custom class loader. This class loader will get Classes as arrays of bytes, decrypt and load them.

if you don't want to provide an access to the class files inside the jar, why should you supply your jar with the application?
It feels like your question is kind of wrong conceptually...
If you need some custom way of loading the classes, consider to use custom classloader.

if you are packaging in jar -> just rename it to jarname.ABCD or any misleading extension or even take off the extension, and accordingly specify the jar name in your application.

i prefer jCrypt!
It is a simple tool where you can crypt the classes(and ressources)

Does Proguard remove unused code (on Android)

I was just wondering, I am designing a library to use with my Android projects. Now I am starting to include things like the apache FTP jar to support some debug file uploads.
I know that not all projects will use all parts of the library (e.g. some project will not have an FTP upload at all, but wants to use the UI tools from the library).
Now I got three questions:
Does Proguard remove unsused sources (own code), like e.g. my UI tool classes, if they are never references from the main project - meaning not used in the application.
Does Proguard remove external libraries (e.g. apache ftp jar) if never used. If not, I may include the source, if option 1 applies.
How about resource files (not really Proguards job, more IntelliJ or Eclipse) - Example: I write a google map extension using default markers, stored in the library project. If I do not need the maps anyway, do the files get included in each android project - and is there an easy way to prevent that.
Some more backround: I try to keep all my library stuff in one project as long as possible. I don't know a good point to split the library yet, so I don't want to overkill to create seperate libs for everything (did that in the past, and most of the time it was way to much modularization)...
Thanks for any insights,
Chris

Does Proguard remove unsused sources
It tries to, when configured properly.
Does Proguard remove external libraries (e.g. apache ftp jar) if never used.
It tries to, when configured properly.
How about resource files (not really Proguards job, more IntelliJ or Eclipse) - Example: I write a google map extension using default markers, stored in the library project. If I do not need the maps anyway, do the files get included in each android project - and is there an easy way to prevent that.
No.

Using serviceloader on android

I am very new to java and android development and to learn I am trying to start with an application to gather statistics and information like munin does. I am trying to be able to load "plugins" in my application. These plugins are already in the application but I don't want to have to invoke them all separately, but be able to iterate over them. I was trying to use serviceloader but could never get the META-INF/services into my apk. So I am wondering if it is possible to use serviceloader on android
Thanks
EDIT: I am asking about java.util.ServiceLoader, I think it should, but I can't figure out how to get my services folder into META-INF on the apk

There is an open bug report against this issue. See https://code.google.com/p/android/issues/detail?id=59658

The META-INF folder is deliberately excluded from the APK by ApkBuilder; the only comment in ApkBuilder.java is "we need to exclude some other folder (like /META-INF)" but there is no other explanation.
Even after adding META-INF with ant, you will still get in trouble if you want to use Proguard, which refuses to replace the content of META-INF/services/* files or rename them (that's another story, the author wants to keep Proguard agnostic).
However, people using maven may want to check https://github.com/pa314159/maven-android-plugin (the branch named "modified"), that tries to solve both issues. It is a fork from the original "android-maven-plugin" I modified one month ago for my own Android projects.
It also provides a patch for Proguard-4.7
Hope this helps, any feedback is welcome.

I've figured out a solution that may work for some situations. Instead of ServiceLoader, I'm using the org.openide.util.Lookup class / library that comes with NetBeans - it is a superset of ServiceLoader. It does not require NetBeans itself and seems to work ok with Eclipse. It is necessary to replace whatever ServiceLoader functionality you are using in your application with Lookup equivalents, and add the org-openide-util-lookup library. Then, you can just do something like this:
Lookup lookup = new ProxyLookup(Lookup.getDefault(),
Lookups.metaInfServices(myClass.getClassLoader(), "services/"));
And move your ServiceLoader files from META-INF/services/ to services/.
Note that, because of the ProxyLookup, this will continue to work on standard Java environments unchanged (i.e., in those cases it will continue to look in META-INF/services).
Here is a link to the documentation for the library: http://bits.netbeans.org/dev/javadoc/org-openide-util-lookup/org/openide/util/lookup/Lookups.html
UPDATE
After working with this for a couple of days, it seems to function well - I move between environments (standard Java and Android) and it works properly in each location. The primary downside is having to manually copy the files to the /services directory.

It is possible. You may want to check http://developer.android.com/reference/java/util/ServiceLoader.html

ServiceLoader is stuff from the Java language that is not really relevant on Android. I recommend not using it. If you just want to find a list of classes within your .apk to load, there are all kinds of ways to do this -- put in XMl file in res/xml that lists them, use reflection, annotations, etc.

How to create my own java library(API)?

I created a program in Java and I designed it so that methods that I want them to appear (getter methods) in the main, I can call them easily after initiate the class that holds these methods.
The question is that, I need to make this application (that holds the getter methods) to be like an API so that I can give my application for developers to use my functions (the getter methods) if they need them, and only what they need is to add this file (I think the API after is done shown as .jar file).
How can I make it so that I can make my code reusable with other application? It's similar to the .dll, I think.
Thanks a lot ;)

Create a JAR. Then include the JAR. Any classes in that JAR will be available. Just make sure you protect your code if you are giving out an API. Don't expose any methods / properties to the end user that shouldn't be used.
Edit: In response to your comment, make sure you don't include the source when you package the JAR. Only include the class files. That's the best you can really do.

To be useable as an API, your classes should:
Use a unique package (ideally following the convention, i.e. the reverse of a domain you own as prefix). This prevents naming conflicts
Have only those classes and methods public or protected that are intended to be used by others. This makes it easier to use.
Have extensive Javadoc comments.
Be available as a JAR file - ideally separate JARs for binary distribution, source code and javadoc files.

You need to package your application as a jar file. You can use ant jar task to create jar files or you can use the jar command.
For ant tasks look at this link.
For creating it manually look at this link.

Make sure you write and publish javadocs for all your public and protected classes and methods.

To create the jar:
jar cf <jar_name> <sources>

There are several ways you can expose your code. Creating a jar and distributing that may be the easiest as other developers will just have to include your jar. However, if you are talking about "anyone" accessing your code, a web service may make more sense as you can provide access to the data without providing all of the necessary code. You mention providing access to your getters - if you just create a class that has getters, the other developers can use them, but how are they going to be populated? If your application is self contained in that it gets the necessary data and provides the getters, that should work, but if you are talking about providing access to data from your running application, a web service makes more sense as your application can retrieve the data and provide access via publicly accessible methods.
You most likely want to create interfaces as well so developers can code against the interface and you can change the internal workings without impacting them. Any API that will be used by others should be extensively documented as well.

Well, depends on your IDE. I use Netbeans, so I just hit build project, and viola! A jar file is created in my directory specified. Now, that's just for compiling. All anyone has to do is download your .jar file, and if in Netbeans, right click libraries, add jar/folder, and select the downloaded file.

You can also consider:
Include some samples that demonstrate how to use your library
Build your jar using Apache Maven
Put your jar in a public maven repository
Publish a new version of your library as you find/fix bugs
If you want to hide your implementation, you can pack your jar with obfuscation, so that if someone decompiles your classes, the code will be difficult to read