Develop Reference

How can I list all methods of all imported classes in a file using Java?

My objective is to look at some lines of codes of an external file and count the number of functions of a class are called then.
For example, if I have the following code:
import java.io.BufferedReader;
import whatever.MyClass;
import java.util.ArrayList;
...
...
public void example(){
InputStreamReader isr = new InputStreamReader (whatever);
MyClass object = new MyClass();
someArrayList.add(whatever2)
someArrayList.add(whatever3)
}
In this case, BufferedReader and MyClass functions were called once, and ArrayList functions were called twice.
My solution for that is get a list of all methods inside the used classes and try to match with some string of my code.
For classes created in my project, I can do the following:
jar -tf jarPath
which returns me the list of classes inside a JAR . And doing:
javap -cp jarPath className
I can get a list of all methods inside a JAR whit a specific class name. However, what can I do to get a external methods names, like add(...) of an "external" class java.util.ArrayList?
I can't access the .jar file of java.util.ArrayList correct? Anyone have another suggestion to reach the objective?

The compiler doesn't put the imports into the object file. It throws them away. Import is just a shorthand to the compiler.(Imports are a compile-time feature ).
first step :
use Qdox https://github.com/paul-hammant/qdox to get all the imports in a class :
String fileFullPath = "Your\\java\\ file \\full\\path";
JavaDocBuilder builder = new JavaDocBuilder();
builder.addSource(new FileReader( fileFullPath ));
JavaSource src = builder.getSources()[0];
String[] imports = src.getImports();
for ( String imp : imports )
{
System.out.println(imp);
}
second step :
inspire from that code , loop through your imports (String array) and apply the same code and you will get the methods .
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class Tes {
public static void main(String[] args) {
Class c;
try {
c = Class.forName("java.util.ArrayList");
Arrays.stream(getAccessibleMethods(c)).
forEach(System.out::println);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
public static Method[] getAccessibleMethods(Class clazz) {
List<Method> result = new ArrayList<Method>();
while (clazz != null) {
for (Method method : clazz.getDeclaredMethods()) {
result.add(method);
}
clazz = clazz.getSuperclass();
}
return result.toArray(new Method[result.size()]);
}
}
Output :
public void java.util.ArrayList.add(int,java.lang.Object)
public boolean java.util.ArrayList.add(java.lang.Object)
public boolean java.util.ArrayList.remove(java.lang.Object)
public java.lang.Object java.util.ArrayList.remove(int)
public java.lang.Object java.util.ArrayList.get(int)
public java.lang.Object java.util.ArrayList.clone()
public int java.util.ArrayList.indexOf(java.lang.Object)
public void java.util.ArrayList.clear()
.
.
.
All the code - one class :
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import com.thoughtworks.qdox.JavaDocBuilder;
import com.thoughtworks.qdox.model.JavaSource;
public class Tester {
public static void main(String[] args) {
// put your .java file path
// CyclicB is a class within another project in my pc
String fileFullPath =
"C:\\Users\\OUSSEMA\\Desktop\\dev\\OCP_Preparation\\src\\w\\CyclicB.java";
JavaDocBuilder builder = new JavaDocBuilder();
try {
builder.addSource(new FileReader( fileFullPath ));
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
JavaSource src = builder.getSources()[0];
String[] imports = src.getImports();
for ( String imp : imports )
{
Class c;
try {
c = Class.forName(imp);
Arrays.stream(getAccessibleMethods(c)).
forEach(System.out::println);
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
}
public static Method[] getAccessibleMethods(Class clazz) {
List<Method> result = new ArrayList<Method>();
while (clazz != null) {
for (Method method : clazz.getDeclaredMethods()) {
result.add(method);
}
clazz = clazz.getSuperclass();
}
return result.toArray(new Method[result.size()]);
}
}
Output all the methods within the classes imported in the file CyclicB.java :
private void java.lang.Throwable.printStackTrace(java.lang.Throwable$PrintStreamOrWriter)
public void java.lang.Throwable.printStackTrace(java.io.PrintStream)
public void java.lang.Throwable.printStackTrace()
public void java.lang.Throwable.printStackTrace(java.io.PrintWriter)
public synchronized java.lang.Throwable java.lang.Throwable.fillInStackTrace()
.
.
.

You may look into OpenJDK project that has a Java compiler. Learn to build the modified versions. Investigate the syntax analysis layer of this compiler and find where the method calls are handled. Put the logging into these locations and now you only need to build your java file with the modified compiler to get the information about the calls.
The build is complex, but you will likely only need a careful editing in a few files. It is not exactly very low hanging fruit but I think it should be possible to discover these files and make changes in them, and still may be a simpler/cleaner approach than to implement the own Java syntax parser (also doable with JavaCC).
If you also need to track calls from the external libraries, build them with the modified compiler as well and you will have the needed records.
GNU Classpath is another open source project where you can do the similar thing, and it may be easier to build. However, unlike OpenJDK, GNU Classpath java system library is not complete.
This approach may not discover some methods called during reflection. But it would discover that reflection framework methods have been called. If it is a security - related project, the simplest would be to agree that reflection is not allowed. It is uncommon to use reflection in a normal Java application that is not a framework.

Java Security Manager: Restrictions on code from external jar loaded via ServiceLoader

What am I trying to achieve?
I am working on a java application that can be extended by additional jars that get integrated via ServiceLoader. These loaded extensions should run with some restrictions by the SecurityManager, of course simply to improve the security. As an example each Extension shall get one specific directory where it can store whatever, but access to any other file/folder should be restricted. The main application is trusted code and can therefore run without any restrictions. Furthermore the main application provides some api implementations for each extension that shall also run without restrictions. That means an extension mustn't access a file outside of its directory but when the extension is calling an api method that tries to access any other file, the access should be granted.
Question
How can I achieve the mentioned behaviour that only 'direct' calls from extension classes get restricted but not any code from the main application?
Running extensions in different threads/threadGroups might be a good solution anyway but since calls to the api might run under the same thread(group) it might not help to identify whether access should be restricted or not based only on the thread.
Example
I created a simplified test environment. On one hand there are these two interfaces:
public interface Extension {
void doSomethingRestricted();
void doSameViaApi(ExtensionApi api);
}
public interface ExtensionApi {
void doSomethingWithHigherPermissions();
}
For testing I created a jar containing this extension:
public class SomeExtension implements Extension {
public void doSomethingRestricted() {
System.out.println(System.getProperty("user.home"));
}
public void doSameViaApi(final ExtensionApi api) {
api.doSomethingWithHigherPermissions();
}
}
In the main application I would like do something like this:
final ExtensionApi api = () -> System.out.println(System.getProperty("user.home"));
try {
final URLClassLoader urlClassLoader = new URLClassLoader(new URL[] { jarFile.toURI().toURL() });
for(final Extension extension : ServiceLoader.load(Extension.class, urlClassLoader)) {
extension.doSomethingRestricted();
extension.doSameViaApi(api);
}
}
So when I call extension.doSomethingRestricted(); it should result in a SecurityException but calling extension.doSameViaApi(api); should work just fine.
So both methods try to do the same but one does try to do it via the api call. The only approach I could think of is iterating through the call history and checking the classloaders to analyze whether the access request is based on trusted code or extension code. But I feel like this might be a nasty error-prone solution so maybe I missed some better approaches?

First ensure your "main" JAR's classes get to enjoy full privileges. Programmatically this may be accomplished as follows:
package q46991566;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.Policy;
import java.util.Collections;
public class Main {
public static void main(String... args) throws Exception {
// policy configuration contents: this JAR gets all permissions, others get nothing
StringBuilder sb = new StringBuilder("grant {};\n\ngrant codebase \"")
.append(Main.class.getProtectionDomain().getCodeSource().getLocation())
.append("\" {\n\tpermission java.security.AllPermission;\n};\n");
// temp-save the policy configuration
Path policyPath = Files.createTempFile(null, null);
Files.write(policyPath, Collections.singleton(sb.toString()));
// convey to the default file-backed policy provider where to obtain its configuration from;
// leading equals ensures only the specified config file gets processed
System.setProperty("java.security.policy", "=".concat(policyPath.toUri().toURL().toString()));
// establish a policy; "javaPolicy" is the default provider's standard JCA name
Policy.setPolicy(Policy.getInstance("javaPolicy", null));
// policy loaded; backing config no longer needed
Files.delete(policyPath);
// establish a security manager for enforcing the policy (the default implementation is more than
// sufficient)
System.setSecurityManager(new SecurityManager());
// ...
}
}
Alternatively, you will either have to a) modify the JRE distribution's java.policy (or specify a different configuration via the policy.url.n properties in java.security), or b) replace the implementation of the System ClassLoader with one that statically grants AllPermission to the ProtectionDomain associated with classes loaded from the "main" JAR.
Secondly, when loading Extensions from some JAR, employ a URLClassLoader subclass that a) manages extension-specific directories and b) includes a java.io.FilePermission in the permission collection being statically accorded to the protection domain mapped to its defined classes. Crude sample implementation (note that there is no persistent relationship between an extension JAR and a directory; also note that two Extensions originating from the same JAR (but loaded by different class loaders, of course) will get different directories):
package q46991566;
import java.io.FilePermission;
import java.io.IOException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.CodeSource;
import java.security.Permission;
import java.security.PermissionCollection;
import java.security.Permissions;
import java.security.cert.Certificate;
import java.util.Enumeration;
import java.util.Objects;
public final class ExtensionLoader extends URLClassLoader {
private static void copyPermissions(PermissionCollection src, PermissionCollection dst) {
for (Enumeration<Permission> e = src.elements(); e.hasMoreElements();) {
dst.add(e.nextElement());
}
}
private final CodeSource origin;
private final PermissionCollection perms = new Permissions();
private final Path baseDir;
public ExtensionLoader(URL extensionOrigin) {
super(new URL[] { extensionOrigin });
origin = new CodeSource(Objects.requireNonNull(extensionOrigin), (Certificate[]) null);
try {
baseDir = Files.createTempDirectory(null);
perms.add(new FilePermission(baseDir.toString().concat("/-"), "read,write,delete"));
copyPermissions(super.getPermissions(origin), perms);
perms.setReadOnly();
}
catch (IOException ioe) {
throw new RuntimeException(ioe);
}
}
#Override
protected PermissionCollection getPermissions(CodeSource cs) {
return (origin.implies(cs)) ? perms : super.getPermissions(cs);
}
// ExtensionApiImpl (or ExtensionImpl directly -- but then ExtensionLoader would have to be relocated
// into a separate, also fully privileged JAR, accessible to the extension) can call this to relay to
// extensions where they can persist their data
public Path getExtensionBaseDir() {
return baseDir;
}
// optionally override close() to delete baseDir early
}
Lastly, for unprivileged Extensions to be able to execute privileged operations via ExtensionApi, the latter's implementation must wrap privileged method (methods issuing SecurityManager::checkXXX requests) invocations within Privileged(Exception)Actions and pass them to AccessController::doPrivileged; e.g.:
ExtensionApi api = () -> {
AccessController.doPrivileged((PrivilegedAction<Void>) () -> {
try {
Files.write(Paths.get("/root/Documents/highly-sensitive.doc"), Collections.singleton("trusted content"),
StandardOpenOption.CREATE, StandardOpenOption.WRITE, StandardOpenOption.APPEND);
return null;
}
catch (IOException ioe) {
throw new RuntimeException(ioe);
}
});
};
For details on the (proper) use of "privileged blocks", refer to the AccessController documentation and the "Secure Coding Guidelines for Java SE" document.

Detecting remote files in Java

I'm using Watcher in JDK7 which relies on inotify events. If the file is on a NFS, I want my program to fallback and use polling instead. Is there a way to detect if a file is on a remote drive (other than using Runtime.exec and parsing the mount table)? I'm only concerned with Linux compatibility for now.
I suppose one option is to use both inotify and polling when the program starts, but then disable the polling thread if an inotify event for my file is created.

You should be able to get relatively reliable info about the underlying file system type with FileStore.type().
It will definitely tell you if it's an NFS, or CIFS, not sure about other network mount types.
However I have no info about how reliable it is, #hoaz's suggestion to check if events are coming through might be a good idea.

I had the same problem. I have solved it by creating a new thread in de main class and touching the files periodically so a new change event gets fired.
The sample polls the dir for every 10 seconds does a touch.
Here a sample of the code:
package com.ardevco.files;
import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.attribute.FileTime;
import java.util.ArrayList;
import java.util.List;
public class Touch implements Runnable {
private Path touchPath;
public Touch(Path touchPath) {
this.touchPath = touchPath;
this.checkPath = checkPath;
}
public static void touch(Path file) throws IOException {
long timestamp = System.currentTimeMillis();
touch(file, timestamp);
}
public static void touch(Path file, long timestamp) throws IOException {
if (Files.exists(file)) {
FileTime ft = FileTime.fromMillis(timestamp);
Files.setLastModifiedTime(file, ft);
}
}
List<Path> listFiles(Path path) throws IOException {
final List<Path> files = new ArrayList<>();
try (DirectoryStream<Path> stream = Files.newDirectoryStream(path)) {
for (Path entry : stream) {
if (Files.isDirectory(entry)) {
files.addAll(listFiles(entry));
}
files.add(entry);
}
}
return files;
}
#Override
public void run() {
while (true) {
try {
for (Path path : listFiles(touchPath)) {
touch(path);
}
} catch (IOException e) {
System.out.println("Exception: " + e);
}
try {
Thread.sleep(10000L);
} catch (InterruptedException e) {
System.out.println("Exception: " + e);
}
}
}
}

How to use custom PolicySpi

I'm trying to implement a custom java.security.Permission type, which should be checked at runtime (so no policy file, but in code). This checking is done by a java.security.Policy. I understood I should implement my own java.security.PolicySpi for this.
I cannot find any explanation on how to initialise and use a PolicySpi, or is there a better way to do this?

Checking permissions
In your question you stated that you then want to check the permission with java.security.Policy, but without using a spi.policy file.
From the PolicySpi API, you can see that a PolicySpi object features 4 methods:
engineGetPermissions(CodeSource codesource)
engineGetPermissions(ProtectionDomain domain)
engineImplies(ProtectionDomain domain, Permission permission)
engineRefresh()
However, you might not need PolicySpi as there are easier solutions to check permissions.
See:
Security Manager vs Access Controller
AccessController usage
Since you haven't specified what kind of permission you will grant, I will assume it is a permission concerning a java.security.CodeSource object.
To check all current permissions for a file:
public static void main(String[] args) {
CodeSource source;
try {
source = new CodeSource(new URL("file:/c:/*"), (java.security.cert.Certificate[]) null);
Policy policy = Policy.getPolicy();
System.out.println(policy.getPermissions(source));
} catch (IOException e) {
e.printStackTrace();
}
}
A nice example for the SecurityManager checkPermission() is this tutorial.
For checking specific FilePermissions, you can use:
FilePermission perm = new FilePermission("path/file", "read");
AccessController.checkPermission(perm);
Granting permissions
Granting permissions at runtime can be done with java.lang.RuntimePermission.
For other examples of how to grant permissions to a file, I suggest you read the following:
Access Control Mechanisms and Algorithms
Configuring spi.policy files
Security Managers and Permissions
That should bring you a long way! Good luck!

The previous answer lists alternatives to using PolicySpi (and more generally custom Policy implementations ). This answer will instead give a simplistic example on how a PolicySpi implementation can actually be used as a replacement of the system-default Policy.
Author a JCA Provider.
package com.example;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.security.Provider;
public final class TestProvider extends Provider {
private static final long serialVersionUID = 5544432861418770903L;
public TestProvider() {
super("TestProvider", 1, "TestProvider 1.0");
AccessController.doPrivileged((PrivilegedAction<Void>) () -> {
putService(new TestPolicySpiService(this));
return null;
});
}
}
Author the sole Service descriptor encapsulated by the provider.
package com.example;
import java.security.Policy.Parameters;
import java.security.PolicySpi;
import java.security.Provider;
import java.security.Provider.Service;
import java.util.Collections;
final class TestPolicySpiService extends Service {
TestPolicySpiService(Provider p) {
super(p, "Policy", "TestPolicy", PolicySpi.class.getName(), Collections.emptyList(), Collections.emptyMap());
}
#Override
public PolicySpi newInstance(Object constructorParameter) {
Parameters policyParams = null;
if (constructorParameter instanceof Parameters) {
policyParams = (Parameters) constructorParameter;
}
return new TestPolicySpi(policyParams);
}
#Override
public boolean supportsParameter(Object parameter) {
return parameter instanceof Parameters;
}
}
Author the actual service (the PolicySpi implementation in this case) that the service descriptor produces.
package com.example;
import java.security.Permission;
import java.security.Policy.Parameters;
import java.security.PolicySpi;
import java.security.ProtectionDomain;
final class TestPolicySpi extends PolicySpi {
TestPolicySpi(Parameters policyParams) {}
#Override
protected boolean engineImplies(ProtectionDomain domain, Permission permission) {
// deny unconditionally
return false;
}
}
Register the provider either statically, by modifying the security.provider.n properties in JAVA_HOME/lib/security/java.security, or programmatically, via java.security.Security.addProvider(Provider) / java.security.Security.insertProviderAt(Provider, int).
Replace the default Policy.
package com.example;
import java.security.NoSuchAlgorithmException;
import java.security.Policy;
public class Main {
public static void main(String... args) throws NoSuchAlgorithmException {
// the following assumes that the provider has been statically registered
Policy.setPolicy(Policy.getInstance("TestPolicy", null));
System.setSecurityManager(new SecurityManager());
// test
System.out.println(System.getProperty("user.home")); // should raise AccessControlException
}
}
Is there a better way to do this?
There certainly is a less involved way, as long as the consequent tight coupling between application and policy does not irk you too badly: Just subclass Policy directly and pass an instance of your implementation to Policy.setPolicy(Policy).
Further reading:
Java Cryptography Architecture (JCA) Reference Guide
How to Implement a Provider in the Java Cryptography Architecture
Standard Algorithm Name Documentation for JDK 8
Troubleshooting Security

As of Java 6, the default implementation for PolicySpi is sun.security.provider.PolicySpiFile. You can get inspired from the source code of PolicySpiFile:
package sun.security.provider;
import java.security.CodeSource;
import java.security.Permission;
import java.security.PermissionCollection;
import java.security.Policy;
import java.security.PolicySpi;
import java.security.ProtectionDomain;
import java.security.URIParameter;
import java.net.MalformedURLException;
/**
* This class wraps the PolicyFile subclass implementation of Policy
* inside a PolicySpi implementation that is available from the SUN provider
* via the Policy.getInstance calls.
*
*/
public final class PolicySpiFile extends PolicySpi {
private PolicyFile pf;
public PolicySpiFile(Policy.Parameters params) {
if (params == null) {
pf = new PolicyFile();
} else {
if (!(params instanceof URIParameter)) {
throw new IllegalArgumentException
("Unrecognized policy parameter: " + params);
}
URIParameter uriParam = (URIParameter)params;
try {
pf = new PolicyFile(uriParam.getURI().toURL());
} catch (MalformedURLException mue) {
throw new IllegalArgumentException("Invalid URIParameter", mue);
}
}
}
protected PermissionCollection engineGetPermissions(CodeSource codesource) {
return pf.getPermissions(codesource);
}
protected PermissionCollection engineGetPermissions(ProtectionDomain d) {
return pf.getPermissions(d);
}
protected boolean engineImplies(ProtectionDomain d, Permission p) {
return pf.implies(d, p);
}
protected void engineRefresh() {
pf.refresh();
}
}

Using reflections with onejar plugin

I am using reflections with onejar maven plugin, and I have a problem when running the resulting jar:
20204 [main] ERROR org.reflections.Reflections - given scan urls are
empty. set urls in the configuration
It seems to me that reflections doesn't support the class loader that onejar is using. I've been using the following configuration of Reflections in my code:
Reflections reflections = new Reflections("path.to.my.package");
Any ideas? Thanks.

Yep a known issue. There is a work around...
Use Custom UrlType and custom Vfs.Dir; Uses OneJarURLConnection to correctly read content from embedded JARs; uses 'standard' Vfs.File.
Issue and solution discussed here : http://code.google.com/p/reflections/issues/detail?id=128
Source if you need it:
OneJarDir
package com.sdl.ws.integration.profserv.shared.onejar;
import com.simontuffs.onejar.OneJarURLConnection;
import org.reflections.vfs.Vfs;
import org.reflections.vfs.ZipDir;
import org.reflections.vfs.ZipFile;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Enumeration;
import java.util.List;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
public class OneJarDir implements Vfs.Dir {
private JarFile oneJarFile = null;
private List<Vfs.File> oneJarClassFiles = new ArrayList<Vfs.File>();
private OneJarURLConnection oneJarConnection;
public OneJarDir(OneJarURLConnection oneJarConnection) {
this.oneJarConnection = oneJarConnection;
try {
this.oneJarConnection.connect();
this.oneJarFile = this.oneJarConnection.getJarFile();
Enumeration<JarEntry> entries = oneJarFile.entries();
while (entries.hasMoreElements()) {
oneJarClassFiles.add(new ZipFile(new ZipDir(oneJarFile), entries.nextElement()));
}
} catch (IOException e) {
throw new RuntimeException("Can't create One-Jar VFS directory", e);
}
}
public String getPath() {
return oneJarConnection.getURL().getPath();
}
public Iterable<Vfs.File> getFiles() {
return oneJarClassFiles;
}
public void close() {
try {
if (oneJarConnection != null)
oneJarConnection.getInputStream().close();
} catch (IOException e) {
throw new RuntimeException("Can't close VFS JAR stream", e);
}
}
}
OneJarUrlType
package com.sdl.ws.integration.profserv.shared.onejar;
import com.simontuffs.onejar.OneJarURLConnection;
import org.reflections.vfs.Vfs;
import java.io.IOException;
import java.net.URL;
public class OneJarUrlType implements Vfs.UrlType {
private static final String _JAR_DIR = "jar!";
public boolean matches(URL url) {
// check if "double-jarred' by one-jar; this would appear to conflict with the standard JAR loader, so it either needs to be first (which it is)
// OR the standard needs to be removed. This match assumes a nested JAR, unlike the standard JAR type.
String externalForm = url.toExternalForm();
// ugly, but should be much faster than regex.
int idx1 = externalForm.indexOf(_JAR_DIR);
return (idx1 > 0 && externalForm.indexOf(_JAR_DIR, idx1 + _JAR_DIR.length()) > 0);
}
public Vfs.Dir createDir(URL url) {
try {
return new OneJarDir(new OneJarURLConnection(url));
} catch (IOException e) {
throw new RuntimeException("Can't open One-Jar embedded JAR", e);
}
}
}