Develop Reference

Make a Java class visible from any ClassLoader

I'm using a Java Agent (Agent.class) to transform a method in a program (Program.class) in a way that includes a call to the Agent class.
public Program.getMultiplier()F:
ALOAD 1
ALOAD 2
FDIV
+ INVOKESTATIC Agent.getCustomMultiplier(F)F
FRETURN
I've inspected the class loaders and their parents of both Agent and Program classes, and their hierarchy looks like this:
Agent.class: AppClassLoader <- PlatformClassLoader <- null
Program.class: URLClassLoader <- PlatformClassLoader <- null
When the Program executes the added INVOKESTATIC instruction, it throws a ClassNotFoundException -- it cannot find the Agent class as it was loaded by a different class loader.
As a temporary solution, I've tried forcing AppClassLoader to become a parent of URLClassLoader with reflection, which works in older Java versions but has been removed since Java 12.
Is there a more reliable way to make sure my Agent class is visible from any class loader?

You can add classes to the bootstrap class loader using appendToBootstrapClassLoaderSearch. This makes the classes of the specified jar file available to all classes whose defining class loader follows the standard delegation pattern.
But this requires the classes to be packaged in a jar file. When you specify the Agent’s own jar file, you have to be aware that classes loaded through the bootstrap loader are distinct from the classes loaded through the app loader, even when they originate from the same jar file. Further, the classes loaded by the bootstrap loader must not have dependencies to classes loaded by by other class loaders.
If your getCustomMultiplier method is supposed to interact with the running Agent, you have to separate the agent and the class containing this method.

Have your Agent listen to the creation of new ClassLoaders and then attach instances of them to the new ClassLoaders.
This way you preserve your "Agent listens to ClassLoader" interface, even if it now extends beyond the one platform class loader you expected the Agent to listen to.

You may be able to do something specific that works for URLClassLoader, but not all classes are loaded by an instance of URLClassLoader. Any OSGi project won't, most web servers also use their own classloaders in order to support hot reload, etc.
As far as I know there's no way to just casually update some 'global parent of all classloaders' or inject one; there's no such parent, and even if there was, a classloader is free to ignore its parent entirely.
Therefore the general answer is: No, you can't do that.
But, let's get our hacking hats on!
You're an agent already. One of the things you get to do as agent is to 'witness' classes as they are being loaded. Just invoke .addTransformer on the instance of Instrumentation you get in your agentmain and register one.
When you notice the Program class being loaded, do the following:
Take the bytecode and toss it through ASM, BCEL, Bytecode Buddy, or any other java 'class file reader/transformer' framework.
Also open up a class from within your agent's code (I wouldn't use Agent itself, I'd make a class called ProgramAddonMethods or whatnot as a container - everything inside is for the program to use / for your agent to 'inject' into that program.
Add every static member in ProgramAddonMethods directly to Program. As you do so, modify the typename on all accesses (both INVOKESTATIC and the read/write field opcodes) where the etypename is ProgramAddonMethods and make it the fully qualified name of the targeted class instead.
inject the INVOKESTATIC as you already do, but, rewrite it so that it's going to its own class, as you just copied all the static methods and fields over there.
Then return the bytecode of that modified class from your transformer.
This 100% guarantees you cannot possibly run into any module or classpath boundary issues and it will work with any classloader abstraction, guaranteed, but there are some caveats:
Just don't attempt to futz with instance anything. Make it all static methods and fields. You can make fake instance fields using an IdentityHashMap if you must (e.g. a static IdentityHashMap<Foo, String> names; is effectively identical to adding private String name; to the Foo class.. except it's a bit slower of course; presumably as you're already in a mess o reflection that's acceptable here).
Your code has to be 'dependency free'. It cannot rely on anything else, no libraries other than java.*, not even a helper class. This idea quickly runs out of steam if the job you're injecting becomes complicated. If you must, make a classloader for your own agent jar using the appropriate 'thread-safely initialize it only once' guards, and have that load in a bundle that does have the benefit of allowing dependencies.
This is all highly complicated stuff but you appear to have already worked out how to inject INVOKESTATIC calls, so, I think you know how to do this.
This is precisely what lombok does to 'patch' some methods in eclipse to ensure that things like save actions, auto-formatting, and syntax highlighting don't break - lombok injects knowledge of generated notes where appropriate and does it in this exact manner because eclipse uses a classloader platform called Equinox which makes any other solution problematic. You can look at it for inspiration or guidelines, though it's not particularly well documented. You're looking in particular at:
The lombok.eclipse.agent package in the eclipseAgent source root.
The lombok.patcher project which is lombok's only actual dependency, in particular the lombok.patcher.PatchScript.transplantMethod method.
Note that the next method may also interest you: lombok.patcher's 'insert' doesn't move the method - it injects the body of the method directly in there (it 'inlines'). This requires some serious finagling of the stack and is only advised for extremely simple one-liner-esque methods, and probably is excessive and unneccessary firepower for this problem.
DISCLAIMER: I wrote most of that.

Using shared class libraries internally, but not providing access to them in released SDK

After some research I cannot come across the best approach for this. There will be certain color classes that I would like to share amongst multiple projects. Let's call one of them EncryptedColor. Since it is used across multiple projects I don't want multiple copies of it in existence of course. Otherwise I would need to make sure that an update in one location would need to be updated everywhere. However, these classes are needed in some released SDKs that we provide to customers.
How could I design it such that I can use these classes but not provide them with the classes that they don't need access to from their SDK. I don't want useless classes to become visible and flood the smaller subset of classes that they really need to be seeing.
A couple approaches I have thought of so far but aren't quite ideal:
Try and use a doclet structure that hides the calls within the javadoc such as doclava. Javadoc has not fully implemented its own hiding mechanism yet. As I understand this doesn't keep the functions from being visible, but it was mentioned in one spot that you would need reflection to use the calls. I don't see how just the javadoc does that so I must have been missing something.
Android has designed themselves it seems to force reflection from some #hide attributes included in methods that they have in source code. But from the sounds of it, the system hides those and then uses a different jar when it is loading to make those visible at launch time. Probably not useful here.
If I were to keep shared classes in the same package name I could access default and protected members, but...then I am keeping all my classes that use these in the same package name. Not quite ideal either, but it could be done in that manner if I needed to. Might get out of hand with large quantities of shared resources.
What approaches are taken typically in situations such as these? I haven't liked my findings and thought process thus far.

Short answer : you can't hide/remove these classes as they are needed at runtime by your application.
In my opinon, you have 3 alternatives :
Change the classes access to "package private". Yes, doing that doesn't make it impossible to access them, but these classes won't be accessible directly.
Remove the classes and create an API. You want to hide the logic ? Remove it and provide it through a REST API for instance. Depending or your architecture, it could be difficult or impossible.
Create all the instance of these classes in a dynamic way, with Class.forName, using Spring or as in #Steve K answer, with Java's ServiceLoader. As a result, you will be able to remove these classes from the main jar and make them more private, in a way. Again, classes will be here but a little less accessible.

My suggestion that could work would be to implement your color classes as a service using the Java ServiceLoader
You make an interface for your color classes, and implementations can be called using the ServiceLoader class. Then you simply separate your color classes into two packages - a public package you can jar up and distribute with your SDK, and a private package for those classes you want to be internal. The ServiceLoader will find all the color classes available so long as the jar files are in your project's classpath.
For example, if your color classes (as an example) had a common interface like this:
public interface MyAppColor {
public int getRed();
public int getGreen();
public int getBlue();
public int getAlpha();
public void setRed(int red);
public void setGreen(int green);
public void setBlue(int blue);
public void setAlpha(int alpha);
public boolean isValid();
public void doSomething(Object arg);
}
Then you could have a bunch of implementing classes in a jar file, with a service descriptor file included in the jar at the path:
META-INF/services/com.my.app.MyAppColor
The text of that file is simply the list of classes in the jar that implement the interface - one per line:
com.my.app.MyPublicAppColor
com.my.app.MyEncryptedPublicAppColor
com.my.app.MyOtherPublicAppColor
etc. Then all you have to do is make a factory for instantiating the correct type, which could be as simple as this:
public class MyAppColorFactory {
private static ServiceLoader<MyAppColor> serviceLoader = ServiceLoader.load(MyAppColor.class, null);
public static MyAppColor get(String className){
if (className != null){
for (MyAppColor c : serviceLoader){
if (className.equals(c.getClass().getName())){
return c;
}
}
}
return null;
}
}

Deploying only needed code:
- Use Only The Needed Source In Development (1) (2)
Since you have an entire library and many deployments which each use different components, the easiest way to do what you suggest is to use only the sources that you need; not a single library. You can ignore the unused sources. This will only ship the needed code.
- Make The Library "Package Private"
This will allow the access only for the public components of the library and everything else will not be callable. But, it will still ship all the code.
- Create an API as a REST SDK
This will require web access, not desirable for performance code {any code really}. You will ship no sdk code with this method.
- Obfuscate the code
Easy with the correct tools. Obfuscation will change the class and method names in production code to gibberish. This will make the library basically unusable to anyone but you. This will ship all the code but it will be obfuscated
- Native API
You can compile java to machine code and use it in production or as the api. You can also create the api in a native language {not desirable}.

SecurityManager frameworks that offer finer control over resources

I always thought that SecurityManagers included a check method which was called when Method/Field.setAccessible() was attempted that included a Permission that included the name of the method/field enclosing class and member name etc. Apparently it does not which is a shock.
I had an idea that it would be possible to solve this problem by using a ClassLoader that rewrote attempts such as
Method.setAccessible()
to
MethodHelper.setAccessible( Method );
The MethodHelper method could set a thread local which my security manager look at and clear to get the actual Method.
This of course has some potential flaws as it requires class file rewriting which of course can only happen for non system classes.
The same approach could be taken for retrieving methods, fields, etc which today do make the member available to the SecurityManager in any form.
Are there any FOSS libraries that package the above functionality ?

Java .policy file - how to prevent java.util.Date() from being accessible

I am playing around with the java .policy file and was wondering how I could go about doing something like preventing calls to java.util.Date(), as an example.
I just want to get a better sense of the .policy file works and how it can be used to sandbox code.

You'll be out of luck there I'm afraid.
As Paŭlo Ebermann says, package.access can block out package hierarchies. You can be more precise about this with a custom SecurityManager, which is usually a damn good indication you are doing something really dodgy.
In general you can make a ClassLoader that doesn't always delegate to its parent. Technically against the current Java SE spec, although the Java EE spec encourages it. You could block out java.util.Date. It's still accessible through reflection if any other class references it, or you can get an instance of it. You could block out the transitive closure of uses of Date, including those that in some way return a Date. However, to complete the scheme with your minimal date you'd have to load a java.util.Date in your class loader, which you can't along with all other java.* classes.
So, err, replace the java.util.Date class in rt.jar (possibly using a Java Agent), and substitute in any class you don't want to restrict new Date() with new Date(System.currentTimeMillis()).
(Btw, +1 to anything that reduces the dependency on System.currentTimeMillis() and other magic methods.)

To restrict access to certain packages, you have actually to change not the .policy file, but the security.properties. There is an entry package.access=... which lists the packages for which RuntimePermissions are needed. So, you can't restrict specifically the access to a single class, only to a whole package (including subpackages, if needed), i.e. java.util.
(You can alternatively access this by the Security.?etProperty methods.)
If you did this, you later can add the right RuntimePermission to the Policy to let the "good" code use it.
I think quite a good part of the JRE would cease working if you restrict access to java.util, so better try another class for testing.

The way the sandbox mostly works is that there are calls from classes that do security-sensitive stuff to the current SecurityManager to check whether or not such a call should succeed. Since the Date class isn't perceived to be security-sensitive, no such calls exist in it's code and that's why - as explained by Tom and Paulo - it is very difficult to restrict it.
For example, in contrast: File operations are perceived to be security sensitive and that's why the File class has calls to the SecurityManager. As an example the delete method:
public boolean delete() {
SecurityManager security = System.getSecurityManager();
if (security != null) {
security.checkDelete(path);
}
return fs.delete(this);
}
And thanks to the SecurityManager check in the File class you can restrict File delete operations in the .policy file with more ease.

How can I run my code upon class load?

Is there a feasible way to get my own code run whenever any class is loaded in Java, without forcing the user explicitly and manually loading all classes with a custom classloader?
Without going too much into the details, whenever a class implementing a certain interface read its annotation that links it with another class, and give the pair to a third class.
Edit: Heck, I'll go to details: I'm doing an event handling library. What I'm doing is having the client code do their own Listener / Event pairs, which need to be registered with my library as a pair. (hm, that wasn't that long after all).
Further Edit: Currently the client code needs to register the pair of classes/interfaces manually, which works pretty well. My intent is to automate this away, and I thought that linking the two classes with annotations would help. Next, I want to get rid of the client code needing to keeping the list of registrations up to date always.
PS: The static block won't do, since my interface is bundled into a library, and the client code will create further interfaces. Thus, abstract classes won't do either, since it must be an interface.

If you want to base the behavior on an interface, you could use a static initializer in that interface.
public interface Foo{
static{
// do initializing here
}
}
I'm not saying it's good practice, but it will definitely initialize the first time one of the implementing classes is loaded.
Update: static blocks in interfaces are illegal. Use abstract classes instead!
Reference:
Initializers (Sun Java Tutorial)
But if I understand you right, you want the initialization to happen once per implementing class. That will be tricky. You definitely can't do that with an interface based solution. You could do it with an abstract base class that has a dynamic initializer (or constructor), that checks whether the requested mapping already exists and adds it if it doesn't, but doing such things in constructors is quite a hack.
I'd say you cleanest options are either to generate Code at build time (through annotation processing with apt or through bytecode analysis with a tool like asm) or to use an agent at class load time to dynamically create the mapping.
Ah, more input. Very good. So clients use your library and provide mappings based on annotations. Then I'd say your library should provide an initializer method, where client code can register classes. Something like this:
YourLibrary.getInstance().registerMappedClasses(
CustomClass1.class,
CustomClass2.class,
CustomClass3.class,
CustomClass4.class
)
Or, even better, a package scanning mechanism (example code to implement this can be found at this question):
YourLibrary.getInstance().registerMappedClassesFromPackages(
"com.mycompany.myclientcode.abc",
"com.mycompany.myclientcode.def"
)
Anyway, there is basically no way to avoid having your clients do that kind of work, because you can't control their build process nor their classloader for them (but you could of course provide guides for classloader or build configuration).

If you want some piece of code to be run on any class loading, you should:
overwrite the ClassLoader, adding your own custom code at the loadClass methods (don't forget forwarding to the parent ClassLoader after or before your custom code).
Define this custom ClassLoader as the default for your system (here you got how to do it: How to set my custom class loader to be the default?).
Run and check it.
Depending on what kind of environment you are, there are chances that not all the classes be loaded trouugh your custom ClassLoader (some utility packages use their own CL, some Java EE containers handle some spacific areas with specific classLoaders, etc.), but it's a kind of aproximation to what you are asking.