Develop Reference

NoSuchMethodException when attempting to instantiate a dynamically loaded class with a constructor that includes a custom class

I am attempting to instantiate a dynamically loaded class (with a custom class loader) with a constructor that contains both a String object and a custom class I have created.
I have re-created the issue outside of my project, this is assuming that A.class is already in the target folder.
Main method:
public class simplifiedMainMethod {
static boolean propagate = false;
static Class<?> projectActions = null;
static Object oInstance;
public static void main (String[] args)
{
String projectName = "A";
String baseUrl = "https://www.google.com/";
simplifiedReport reporter = new simplifiedReport();
try
{
ClassLoader cl = Thread.currentThread().getContextClassLoader();
Reloader r = new Reloader(cl);
if (propagate)
Thread.currentThread().setContextClassLoader(r);
projectActions = r.loadClass("autobots_framework."+projectName);
Constructor<?> c = projectActions.getConstructor(String.class, simplifiedReport.class);
oInstance = c.newInstance(baseUrl, reporter);
}
catch (Exception e) {
e.printStackTrace();
}
}
}
simplifiedReport.java:
public class simplifiedReport
{
public simplifiedReport()
{
}
public void testMethod()
{
System.out.println("Hello World");
}
}
A.java:
public class A
{
public A(String testString, simplifiedReport testReport)
{
System.out.println(testString);
testReport.testMethod();
}
}
Here is the CustomClassLoader I am using:
public class ClassLoaderCompiler
{
public ClassLoaderCompiler()
{}
public static boolean compileCode(String fileDirectory, String projectName) throws IOException
{
JavaCompiler compiler = new EclipseCompiler();
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
StandardJavaFileManager fileManager = compiler.getStandardFileManager(diagnostics, null, null);
Iterable<? extends JavaFileObject> compilationUnits = fileManager.getJavaFileObjectsFromStrings(Arrays.asList(fileDirectory));
JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, null, null, compilationUnits);
try
{
task.call();
fileManager.close();
return true;
}
catch(Exception e)
{
fileManager.close();
return false;
}
}
public static void moveClassFile(String projectName)
{
try
{
Files.deleteIfExists(new File(System.getProperty("user.dir")+"/target/classes/autobots_framework/"+projectName+".class").toPath());
Files.deleteIfExists(new File(System.getProperty("user.dir")+"/target/classes/autobots_framework/L_"+projectName+".class").toPath());
if (Files.exists(Paths.get(System.getProperty("user.dir")+"/projects/"+projectName+"/"+projectName+".class")))
{
Path temp = Files.move (Paths.get(System.getProperty("user.dir")+"/projects/"+projectName+"/"+projectName+".class"),Paths.get(System.getProperty("user.dir")+"/target/classes/autobots_framework/"+projectName+".class"));
Path tempLocator = Files.move (Paths.get(System.getProperty("user.dir")+"/projects/"+projectName+"/L_"+projectName+".class"),Paths.get(System.getProperty("user.dir")+"/target/classes/autobots_framework/L_"+projectName+".class"));
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
/**
* adapted from http://stackoverflow.com/a/3971771/7849
*/
class Reloader extends ClassLoader {
static URL url;
ClassLoader orig;
Reloader(ClassLoader orig) {
this.orig = orig;
}
#Override
public Class<?> loadClass(String s) {
return findClass(s);
}
#Override
public Class<?> findClass(String s) {
try {
byte[] bytes = loadClassData(s);
return defineClass(s, bytes, 0, bytes.length);
} catch (IOException ioe) {
try {
return super.loadClass(s);
} catch (ClassNotFoundException ignore) {
ignore.printStackTrace(System.out);
}
ioe.printStackTrace(System.out);
return null;
}
}
private byte[] loadClassData(String className) throws IOException {
try {
/*
* get the actual path using the original classloader
*/
Class<?> clazz = orig.loadClass(className);
url = clazz.getResource(clazz.getSimpleName() + ".class");
/*
* force reload
*/
File f = new File(url.toURI());
int size = (int) f.length();
byte buff[] = new byte[size];
FileInputStream fis = new FileInputStream(f);
DataInputStream dis = new DataInputStream(fis);
dis.readFully(buff);
dis.close();
return buff;
} catch (Exception ex) {
throw new IOException(ex);
}
}
}
I am getting this error as a result:
java.lang.NoSuchMethodException: autobots_framework.A.<init>(java.lang.String, autobots_framework.simplifiedReport)
at java.lang.Class.getConstructor0(Unknown Source)
at java.lang.Class.getConstructor(Unknown Source)
I figured this was due to the parameters not matching, so to confirm I tried finding what the constructor is implicitly looking for:
for (Constructor c : projectActions.getDeclaredConstructors())
{
System.out.print(c.toGenericString());
}
Yielding this result:
public autobots_framework.A(java.lang.String,autobots_framework.simplifiedReport)
Which is frustratingly looking a lot like what I have put here:
Constructor<?> c = projectActions.getConstructor(String.class, simplifiedReport.class);
oInstance = c.newInstance(baseUrl, reporter);
I understand that this has something to do with my class, as when I leave out simplifiedReport in the constructor and only instantiating it with String.class, I am successful. What modifier am I missing in my class to make this work?
This is all a function of my application to dynamically compile code and then instantiate them to run custom code and methods.

Java Dynammic Class loading inside webapp

I made a java project, the project only contais this class:
package test.processor;
public abstract class Processor {
public abstract void loadData(String objectId);
public abstract void processData();
public abstract void saveData(String objectId);
}
The project is exported as a jar file (processor.jar)
Then I made another project that imports processor.jar and there is a class that extends Processor:
package test.process;
import test.processor.Processor;
public class Process extends Processor{
#Override
public void loadData(String objectId) {
System.out.println("LOAD DATAAAAAAAAAAAA");
}
#Override
public void processData() {
System.out.println("PROCESS DATAAAAAAAAAAAA");
}
#Override
public void saveData(String objectId) {
System.out.println("SAVE DATAAAAAAAAAAAA");
}
}
This project is also exported as jar (plugin.jar).
Finally, I coded something to load the plugins dynamically:
import test.processor.Processor;
public class Test {
public void testPlugins(){
Processor plugin = (Processor) loadJar(
"C:\\Users\\...\\Desktop\\plugin.jar",
"test.process.Process");
processor.loadData("dada");
}
private Object loadJar(String jar, String className){
File jarFile = new File(jar);
Object instance = null;
try {
URL jarpath = jarFile.toURI().toURL();
String jarUrl = "jar:" + jarpath + "!/";
URL urls[] = { new URL(jarUrl) };
URLClassLoader child = new URLClassLoader(urls);
Class classToLoad = Class.forName(nomeClasse, true, child);
instance = classToLoad.newInstance();
} catch (Exception ex) {
ex.printStackTrace();
}
return instance;
}
}
If I run that code inside a main method it works correctly, once I try to run it in the server there is a problem when loading the class, I get a ClassNotFoundException (Processor).
I tried putting the jar in the tomcat/lib, project/WEB-INF/lib and nothing changed.
Any idea of what Im doing wrong?

I didn't solve it the way I wanted, but I solved it:
First I tried loading the process.jar manually:
private Object loadJars(String processJar, String pluginJar, String className){
File processJarFile = new File(processJar);
File pluginJarFile = new File(pluginJar);
Object instance = null;
try {
URL processJarPath = processJarFile.toURI().toURL();
String processJarUrl = "jar:" + processJarPath + "!/";
URL pluginJarPath = pluginJarFile.toURI().toURL();
String pluginJarUrl = "jar:" + pluginJarPath + "!/";
URL urls[] = { new URL(processJarUrl), new URL(pluginJarUrl) };
URLClassLoader child = new URLClassLoader(urls);
Class classToLoad = Class.forName(nomeClasse, true, child);
instance = classToLoad.newInstance();
} catch (Exception ex) {
ex.printStackTrace();
}
return instance;
}
That loads the Process class correctly, the problem happens in the testPlugins mehod, once it tries to cast to Processor (ClassCastException, can't cast Process to Processor):
public void testPlugins(){
Processor plugin = (Processor) loadJars("C:\\Users\\...\\Desktop\\processor.jar",
"C:\\Users\\...\\Desktop\\plugin.jar",
"test.process.Process");
processor.loadData("dada");
}
Still need to read a lot about classloading but I guess the problem is that it doesn't recognize the Processor loaded from C:\Users\...\Desktop\processor.jar as the same as the Processor loaded from the webapp context or it "forgets" Process extends Processor.
I was in a hurry so I didn't have time to research, to solve the problem I invoked the methods using reflection:
public void modifiedTestPlugins(){
Object plugin = loadJar("C:\\Users\\...\\Desktop\\processor.jar",
"C:\\Users\\...\\Desktop\\plugin.jar",
"test.process.Process");
try {
Method processData = findMethod(obj.getClass(), "processData");
//here I invoke the processData method, it prints: PROCESS DATAAAAAAAAAAAA
loadData.invoke(processData, new Object[]{});
} catch (Exception e) {
e.printStackTrace();
}
}
private static Method findMethod(Class clazz, String methodName) throws Exception {
Method[] methods = clazz.getMethods();
for (int i = 0; i < methods.length; i++) {
if (methods[i].getName().equals(methodName))
return methods[i];
}
return null;
}

Unit test code with WatchService

Below is a short simple example of using a WatchService to keep data in sync with a file. My question is how to reliably test the code. The test fails occasionally, probably because of a race condition between the os/jvm getting the event into the watch service and the test thread polling the watch service. My desire is to keep the code simple, single threaded, and non blocking but also be testable. I strongly dislike putting sleep calls of arbitrary length into test code. I am hoping there is a better solution.
public class FileWatcher {
private final WatchService watchService;
private final Path path;
private String data;
public FileWatcher(Path path){
this.path = path;
try {
watchService = FileSystems.getDefault().newWatchService();
path.toAbsolutePath().getParent().register(watchService, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
} catch (Exception ex) {
throw new RuntimeException(ex);
}
load();
}
private void load() {
try (BufferedReader br = Files.newBufferedReader(path, Charset.defaultCharset())){
data = br.readLine();
} catch (IOException ex) {
data = "";
}
}
private void update(){
WatchKey key;
while ((key=watchService.poll()) != null) {
for (WatchEvent<?> e : key.pollEvents()) {
WatchEvent<Path> event = (WatchEvent<Path>) e;
if (path.equals(event.context())){
load();
break;
}
}
key.reset();
}
}
public String getData(){
update();
return data;
}
}
And the current test
public class FileWatcherTest {
public FileWatcherTest() {
}
Path path = Paths.get("myFile.txt");
private void write(String s) throws IOException{
try (BufferedWriter bw = Files.newBufferedWriter(path, Charset.defaultCharset())) {
bw.write(s);
}
}
#Test
public void test() throws IOException{
for (int i=0; i<100; i++){
write("hello");
FileWatcher fw = new FileWatcher(path);
Assert.assertEquals("hello", fw.getData());
write("goodbye");
Assert.assertEquals("goodbye", fw.getData());
}
}
}

This timing issue is bound to happen because of the polling happening in the watch service.
This test is not really a unit test because it is testing the actual implementation of the default file system watcher.
If I wanted to make a self-contained unit test for this class, I would first modify the FileWatcher so that it does not rely on the default file system. The way I would do this would be to inject a WatchService into the constructor instead of a FileSystem. For example...
public class FileWatcher {
private final WatchService watchService;
private final Path path;
private String data;
public FileWatcher(WatchService watchService, Path path) {
this.path = path;
try {
this.watchService = watchService;
path.toAbsolutePath().getParent().register(watchService, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
} catch (Exception ex) {
throw new RuntimeException(ex);
}
load();
}
...
Passing in this dependency instead of the class getting hold of a WatchService by itself makes this class a bit more reusable in the future. For example, what if you wanted to use a different FileSystem implementation (such as an in-memory one like https://github.com/google/jimfs)?
You can now test this class by mocking the dependencies, for example...
import static java.nio.file.StandardWatchEventKinds.ENTRY_CREATE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_DELETE;
import static java.nio.file.StandardWatchEventKinds.ENTRY_MODIFY;
import static org.fest.assertions.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.nio.file.FileSystem;
import java.nio.file.Path;
import java.nio.file.WatchEvent;
import java.nio.file.WatchKey;
import java.nio.file.WatchService;
import java.nio.file.spi.FileSystemProvider;
import java.util.Arrays;
import org.junit.Before;
import org.junit.Test;
public class FileWatcherTest {
private FileWatcher fileWatcher;
private WatchService watchService;
private Path path;
#Before
public void setup() throws Exception {
// Set up mock watch service and path
watchService = mock(WatchService.class);
path = mock(Path.class);
// Need to also set up mocks for absolute parent path...
Path absolutePath = mock(Path.class);
Path parentPath = mock(Path.class);
// Mock the path's methods...
when(path.toAbsolutePath()).thenReturn(absolutePath);
when(absolutePath.getParent()).thenReturn(parentPath);
// Mock enough of the path so that it can load the test file.
// On the first load, the loaded data will be "[INITIAL DATA]", any subsequent call it will be "[UPDATED DATA]"
// (this is probably the smellyest bit of this test...)
InputStream initialInputStream = createInputStream("[INITIAL DATA]");
InputStream updatedInputStream = createInputStream("[UPDATED DATA]");
FileSystem fileSystem = mock(FileSystem.class);
FileSystemProvider fileSystemProvider = mock(FileSystemProvider.class);
when(path.getFileSystem()).thenReturn(fileSystem);
when(fileSystem.provider()).thenReturn(fileSystemProvider);
when(fileSystemProvider.newInputStream(path)).thenReturn(initialInputStream, updatedInputStream);
// (end smelly bit)
// Create the watcher - this should load initial data immediately
fileWatcher = new FileWatcher(watchService, path);
// Verify that the watch service was registered with the parent path...
verify(parentPath).register(watchService, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
}
#Test
public void shouldReturnCurrentStateIfNoChanges() {
// Check to see if the initial data is returned if the watch service returns null on poll...
when(watchService.poll()).thenReturn(null);
assertThat(fileWatcher.getData()).isEqualTo("[INITIAL DATA]");
}
#Test
public void shouldLoadNewStateIfFileChanged() {
// Check that the updated data is loaded when the watch service says the path we are interested in has changed on poll...
WatchKey watchKey = mock(WatchKey.class);
#SuppressWarnings("unchecked")
WatchEvent<Path> pathChangedEvent = mock(WatchEvent.class);
when(pathChangedEvent.context()).thenReturn(path);
when(watchKey.pollEvents()).thenReturn(Arrays.asList(pathChangedEvent));
when(watchService.poll()).thenReturn(watchKey, (WatchKey) null);
assertThat(fileWatcher.getData()).isEqualTo("[UPDATED DATA]");
}
#Test
public void shouldKeepCurrentStateIfADifferentPathChanged() {
// Make sure nothing happens if a different path is updated...
WatchKey watchKey = mock(WatchKey.class);
#SuppressWarnings("unchecked")
WatchEvent<Path> pathChangedEvent = mock(WatchEvent.class);
when(pathChangedEvent.context()).thenReturn(mock(Path.class));
when(watchKey.pollEvents()).thenReturn(Arrays.asList(pathChangedEvent));
when(watchService.poll()).thenReturn(watchKey, (WatchKey) null);
assertThat(fileWatcher.getData()).isEqualTo("[INITIAL DATA]");
}
private InputStream createInputStream(String string) {
return new ByteArrayInputStream(string.getBytes());
}
}
I can see why you might want a "real" test for this that does not use mocks - in which case it would not be a unit test and you might not have much choice but to sleep between checks (the JimFS v1.0 code is hard coded to poll every 5 seconds, have not looked at the poll time on the core Java FileSystem's WatchService)
Hope this helps

I created a wrapper around WatchService to clean up many issues I have with the API. It is now much more testable. I am unsure about some of the concurrency issues in PathWatchService though and I have not done thorough testing of it.
New FileWatcher:
public class FileWatcher {
private final PathWatchService pathWatchService;
private final Path path;
private String data;
public FileWatcher(PathWatchService pathWatchService, Path path) {
this.path = path;
this.pathWatchService = pathWatchService;
try {
this.pathWatchService.register(path.toAbsolutePath().getParent());
} catch (IOException ex) {
throw new RuntimeException(ex);
}
load();
}
private void load() {
try (BufferedReader br = Files.newBufferedReader(path, Charset.defaultCharset())){
data = br.readLine();
} catch (IOException ex) {
data = "";
}
}
public void update(){
PathEvents pe;
while ((pe=pathWatchService.poll()) != null) {
for (WatchEvent we : pe.getEvents()){
if (path.equals(we.context())){
load();
return;
}
}
}
}
public String getData(){
update();
return data;
}
}
Wrapper:
public class PathWatchService implements AutoCloseable {
private final WatchService watchService;
private final BiMap<WatchKey, Path> watchKeyToPath = HashBiMap.create();
private final ReadWriteLock lock = new ReentrantReadWriteLock();
private final Queue<WatchKey> invalidKeys = new ConcurrentLinkedQueue<>();
/**
* Constructor.
*/
public PathWatchService() {
try {
watchService = FileSystems.getDefault().newWatchService();
} catch (IOException ex) {
throw new RuntimeException(ex);
}
}
/**
* Register the input path with the WatchService for all
* StandardWatchEventKinds. Registering a path which is already being
* watched has no effect.
*
* #param path
* #return
* #throws IOException
*/
public void register(Path path) throws IOException {
register(path, ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);
}
/**
* Register the input path with the WatchService for the input event kinds.
* Registering a path which is already being watched has no effect.
*
* #param path
* #param kinds
* #return
* #throws IOException
*/
public void register(Path path, WatchEvent.Kind... kinds) throws IOException {
try {
lock.writeLock().lock();
removeInvalidKeys();
WatchKey key = watchKeyToPath.inverse().get(path);
if (key == null) {
key = path.register(watchService, kinds);
watchKeyToPath.put(key, path);
}
} finally {
lock.writeLock().unlock();
}
}
/**
* Close the WatchService.
*
* #throws IOException
*/
#Override
public void close() throws IOException {
try {
lock.writeLock().lock();
watchService.close();
watchKeyToPath.clear();
invalidKeys.clear();
} finally {
lock.writeLock().unlock();
}
}
/**
* Retrieves and removes the next PathEvents object, or returns null if none
* are present.
*
* #return
*/
public PathEvents poll() {
return keyToPathEvents(watchService.poll());
}
/**
* Return a PathEvents object from the input key.
*
* #param key
* #return
*/
private PathEvents keyToPathEvents(WatchKey key) {
if (key == null) {
return null;
}
try {
lock.readLock().lock();
Path watched = watchKeyToPath.get(key);
List<WatchEvent<Path>> events = new ArrayList<>();
for (WatchEvent e : key.pollEvents()) {
events.add((WatchEvent<Path>) e);
}
boolean isValid = key.reset();
if (isValid == false) {
invalidKeys.add(key);
}
return new PathEvents(watched, events, isValid);
} finally {
lock.readLock().unlock();
}
}
/**
* Retrieves and removes the next PathEvents object, waiting if necessary up
* to the specified wait time, returns null if none are present after the
* specified wait time.
*
* #return
*/
public PathEvents poll(long timeout, TimeUnit unit) throws InterruptedException {
return keyToPathEvents(watchService.poll(timeout, unit));
}
/**
* Retrieves and removes the next PathEvents object, waiting if none are yet
* present.
*
* #return
*/
public PathEvents take() throws InterruptedException {
return keyToPathEvents(watchService.take());
}
/**
* Get all paths currently being watched. Any paths which were watched but
* have invalid keys are not returned.
*
* #return
*/
public Set<Path> getWatchedPaths() {
try {
lock.readLock().lock();
Set<Path> paths = new HashSet<>(watchKeyToPath.inverse().keySet());
WatchKey key;
while ((key = invalidKeys.poll()) != null) {
paths.remove(watchKeyToPath.get(key));
}
return paths;
} finally {
lock.readLock().unlock();
}
}
/**
* Cancel watching the specified path. Cancelling a path which is not being
* watched has no effect.
*
* #param path
*/
public void cancel(Path path) {
try {
lock.writeLock().lock();
removeInvalidKeys();
WatchKey key = watchKeyToPath.inverse().remove(path);
if (key != null) {
key.cancel();
}
} finally {
lock.writeLock().unlock();
}
}
/**
* Removes any invalid keys from internal data structures. Note this
* operation is also performed during register and cancel calls.
*/
public void cleanUp() {
try {
lock.writeLock().lock();
removeInvalidKeys();
} finally {
lock.writeLock().unlock();
}
}
/**
* Clean up method to remove invalid keys, must be called from inside an
* acquired write lock.
*/
private void removeInvalidKeys() {
WatchKey key;
while ((key = invalidKeys.poll()) != null) {
watchKeyToPath.remove(key);
}
}
}
Data class:
public class PathEvents {
private final Path watched;
private final ImmutableList<WatchEvent<Path>> events;
private final boolean isValid;
/**
* Constructor.
*
* #param watched
* #param events
* #param isValid
*/
public PathEvents(Path watched, List<WatchEvent<Path>> events, boolean isValid) {
this.watched = watched;
this.events = ImmutableList.copyOf(events);
this.isValid = isValid;
}
/**
* Return an immutable list of WatchEvent's.
* #return
*/
public List<WatchEvent<Path>> getEvents() {
return events;
}
/**
* True if the watched path is valid.
* #return
*/
public boolean isIsValid() {
return isValid;
}
/**
* Return the path being watched in which these events occurred.
*
* #return
*/
public Path getWatched() {
return watched;
}
#Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final PathEvents other = (PathEvents) obj;
if (!Objects.equals(this.watched, other.watched)) {
return false;
}
if (!Objects.equals(this.events, other.events)) {
return false;
}
if (this.isValid != other.isValid) {
return false;
}
return true;
}
#Override
public int hashCode() {
int hash = 7;
hash = 71 * hash + Objects.hashCode(this.watched);
hash = 71 * hash + Objects.hashCode(this.events);
hash = 71 * hash + (this.isValid ? 1 : 0);
return hash;
}
#Override
public String toString() {
return "PathEvents{" + "watched=" + watched + ", events=" + events + ", isValid=" + isValid + '}';
}
}
And finally the test, note this is not a complete unit test but demonstrates the way to write tests for this situation.
public class FileWatcherTest {
public FileWatcherTest() {
}
Path path = Paths.get("myFile.txt");
Path parent = path.toAbsolutePath().getParent();
private void write(String s) throws IOException {
try (BufferedWriter bw = Files.newBufferedWriter(path, Charset.defaultCharset())) {
bw.write(s);
}
}
#Test
public void test() throws IOException, InterruptedException{
write("hello");
PathWatchService real = new PathWatchService();
real.register(parent);
PathWatchService mock = mock(PathWatchService.class);
FileWatcher fileWatcher = new FileWatcher(mock, path);
verify(mock).register(parent);
Assert.assertEquals("hello", fileWatcher.getData());
write("goodbye");
PathEvents pe = real.poll(10, TimeUnit.SECONDS);
if (pe == null){
Assert.fail("Should have an event for writing good bye");
}
when(mock.poll()).thenReturn(pe).thenReturn(null);
Assert.assertEquals("goodbye", fileWatcher.getData());
}
}

Error with Java WebStart Signed Jars on 1.7.0_40

We are aware of the issue with jar softlinker
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6967414
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6805618
and have used following class (found on web and modified to take care of JAVA 7 as well.)
import java.io.IOException;
import java.lang.ref.SoftReference;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.JarURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.*;
import java.util.jar.JarFile;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* A utility class for working around the java webstart jar signing/security bug
* <p/>
* see http://bugs.sun.com/view_bug.do?bug_id=6967414 and http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6805618
*
* #author Scott Chan
*/
public class JarSignersHardLinker {
private static final String JRE_1_DOT = "1.";
private static final String DOT_ZERO_UNDERSCORE = ".0_";
/**
* the 1.6.0 update where this problem first occurred
*/
private static final int PROBLEM_JRE_UPDATE = 19;
private static final int PROBLEM_JRE_MAJOR_VERSION = 6;
public static final List sm_hardRefs = new ArrayList();
protected static void makeHardSignersRef(JarFile jar) throws java.io.IOException {
if (jar != null && jar.getClass().getName().equals("com.sun.deploy.cache.CachedJarFile")) {
Logger.info("Making hard refs for: " + jar.getName());
//lets attempt to get at the each of the soft links.
//first need to call the relevant no-arg method to ensure that the soft ref is populated
//then we access the private member, resolve the softlink and throw it in a static list.
callNoArgMethod("getSigners", jar);
makeHardLink("signersRef", jar);
callNoArgMethod("getSignerMap", jar);
makeHardLink("signerMapRef", jar);
// callNoArgMethod("getCodeSources", jar);
// makeHardLink("codeSourcesRef", jar);
callNoArgMethod("getCodeSourceCache", jar);
makeHardLink("codeSourceCacheRef", jar);
}
}
/**
* if the specified field for the given instance is a Softreference
* That soft reference is resolved and the returned ref is stored in a static list,
* making it a hard link that should never be garbage collected
*
* #param fieldName
* #param instance
*/
private static void makeHardLink(String fieldName, Object instance) {
//System.out.println("attempting hard ref to " + instance.getClass().getName() + "." + fieldName);
try {
Field signersRef = instance.getClass().getDeclaredField(fieldName);
signersRef.setAccessible(true);
Object o = signersRef.get(instance);
if (o instanceof SoftReference) {
SoftReference r = (SoftReference) o;
Object o2 = r.get();
sm_hardRefs.add(o2);
} else {
Logger.warn(fieldName + ": is not an instance of soft reference");
}
} catch (NoSuchFieldException e) {
e.printStackTrace();
return;
} catch (IllegalAccessException e) {
e.printStackTrace();
}
}
/**
* Call the given no-arg method on the given instance
*
* #param methodName
* #param instance
*/
private static void callNoArgMethod(String methodName, Object instance) {
// System.out.println("calling noarg method hard ref to " + instance.getClass().getName() + "." + methodName + "()");
try {
Method m = instance.getClass().getDeclaredMethod(methodName);
m.setAccessible(true);
m.invoke(instance);
} catch (SecurityException e1) {
e1.printStackTrace();
} catch (NoSuchMethodException e1) {
e1.printStackTrace();
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
}
/**
* is the preloader enabled. ie: will the preloader run in the current environment
*
* #return
*/
public static boolean isHardLinkerEnabled() {
boolean isHardLinkerDisabled = false; //change this to use whatever mechanism you use to enable or disable the preloader
return !isHardLinkerDisabled && isRunningOnJre1_6_0_19OrHigher() && isRunningOnWebstart();
}
/**
* is the application currently running on webstart
* <p/>
* detect the presence of a JNLPclassloader
*
* #return
*/
public static boolean isRunningOnWebstart() {
ClassLoader cl = Thread.currentThread().getContextClassLoader();
while (cl != null) {
if (cl.getClass().getName().equals("com.sun.jnlp.JNLPClassLoader")) {
return true;
}
cl = cl.getParent();
}
return false;
}
/**
* Is the JRE 1.6.0_19 or higher?
* TBFI-5349: Java has a bug, sometimes Jars get garbage collected. To resolve this we are making
* hard references to the Jars.
*
* This method checks for java version. The bug is in 1.6.0_19 and above release hence checking for version 19
* #return
*/
public static boolean isRunningOnJre1_6_0_19OrHigher() {
String javaVersion = System.getProperty("java.version");
String updateStr = null;
String javaMajorVersionStr = null;
boolean isHardReferenceRequired = false;
// Problem persist in JAVA 7 and probable in JAVA 8 as well. So changing the patter.
Pattern pattern = Pattern.compile(JRE_1_DOT + "([6-9]+)" + DOT_ZERO_UNDERSCORE + "([0-9]+)(.*)");
Matcher matcher = pattern.matcher(javaVersion);
while (matcher.find()) {
javaMajorVersionStr = matcher.group(1);
updateStr = matcher.group(2);
break;
}
Logger.info("Java version: " + javaMajorVersionStr + " update string: " + updateStr);
try {
if (javaMajorVersionStr != null) {
int java_version = Integer.parseInt(javaMajorVersionStr);
if (java_version > PROBLEM_JRE_MAJOR_VERSION) {
isHardReferenceRequired = true;
} else if (java_version == PROBLEM_JRE_MAJOR_VERSION && Integer.parseInt(updateStr) >= PROBLEM_JRE_UPDATE) {
isHardReferenceRequired = true;
}
}
return isHardReferenceRequired;
} catch (NumberFormatException e) {
//then unable to determine java Major version or update level
e.printStackTrace();
return isHardReferenceRequired;
}
}
/**
* get all the JarFile objects for all of the jars in the classpath
*
* #return
*/
public static Set<JarFile> getAllJarsFilesInClassPath() {
Set<JarFile> jars = new LinkedHashSet<JarFile>();
for (URL url : getAllJarUrls()) {
try {
jars.add(getJarFile(url));
} catch (IOException e) {
Logger.error("unable to retrieve jar at URL: " + url);
}
}
return jars;
}
/**
* Returns set of URLS for the jars in the classpath.
* URLS will have the protocol of jar eg: jar:http://HOST/PATH/JARNAME.jar!/META-INF/MANIFEST.MF
*/
static Set<URL> getAllJarUrls() {
try {
Set<URL> urls = new LinkedHashSet<URL>();
Enumeration<URL> mfUrls = Thread.currentThread().getContextClassLoader().getResources("META-INF/MANIFEST.MF");
while (mfUrls.hasMoreElements()) {
URL jarUrl = mfUrls.nextElement();
// System.out.println(jarUrl);
if (!jarUrl.getProtocol().equals("jar")) {
continue;
}
urls.add(jarUrl);
}
return urls;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* get the jarFile object for the given url
*
* #param jarUrl
* #return
* #throws IOException
*/
public static JarFile getJarFile(URL jarUrl) throws IOException {
URLConnection urlConnnection = jarUrl.openConnection();
if (urlConnnection instanceof JarURLConnection) {
// Using a JarURLConnection will load the JAR from the cache when using Webstart 1.6
// In Webstart 1.5, the URL will point to the cached JAR on the local filesystem
JarURLConnection jcon = (JarURLConnection) urlConnnection;
return jcon.getJarFile();
} else {
throw new AssertionError("Expected JarURLConnection");
}
}
/**
* Spawn a new thread to run through each jar in the classpath and create a hardlink
* to the jars softly referenced signers infomation.
*/
public static void go() {
if (!isHardLinkerEnabled()) {
return;
}
Logger.info("Starting Resource Preloader Hardlinker");
Thread t = new Thread(new Runnable() {
public void run() {
try {
Set<JarFile> jars = getAllJarsFilesInClassPath();
for (JarFile jar : jars) {
makeHardSignersRef(jar);
}
} catch (Exception e) {
Logger.warn("Problem preloading resources", e);
} catch (Error e) {
Logger.error("Error preloading resources", e);
}
}
});
t.start();
}
}
When we launch the application with JRE6 it works fine. But the problem is with JRE7. When the application is launched with JRE7 we get below exception in the log. From the exception we know that the jars are not hard referenced and that the user can have problems if the jars get garbage collected. We have a release next week and need to find a work around for this issue.
java.lang.NoSuchMethodException: com.sun.deploy.cache.CachedJarFile.getSigners()
at java.lang.Class.getDeclaredMethod(Unknown Source)
at com.XXXXXX.ui.main.JarSignersHardLinker.callNoArgMethod(JarSignersHardLinker.java:96)
at com.XXXXXX.ui.main.JarSignersHardLinker.makeHardSignersRef(JarSignersHardLinker.java:45)
at com.XXXXXX.ui.main.JarSignersHardLinker$1.run(JarSignersHardLinker.java:262)
at java.lang.Thread.run(Unknown Source)
java.lang.NoSuchFieldException: signersRef
at java.lang.Class.getDeclaredField(Unknown Source)
at com.XXXXXX.ui.main.JarSignersHardLinker.makeHardLink(JarSignersHardLinker.java:69)
at com.XXXXXX.ui.main.JarSignersHardLinker.makeHardSignersRef(JarSignersHardLinker.java:46)
at com.XXXXXX.ui.main.JarSignersHardLinker$1.run(JarSignersHardLinker.java:262)
at java.lang.Thread.run(Unknown Source)
This exception is repeated 52 times (for all the JAR's).
We have made sure that all the 52 jars/files (except the JNLP itself) are signed properly and that the java cache is cleard before the application is launched.
JAVA version used is JDK 7u40 on windows machine.
Options tried are:
Removing the jdk.certpath.disabledAlgorithms=MD2, RSA keySize < 1024
from java.securites file.
Checking the jar signer certificate. Signer certificate uses SHA1withRSA as signing algorithm.
Note:
THE SOURCE CODE IS COMPILED IN JAVA 5u11 AND RUN IN JAVA 7u40
We have observed one more difference. With JRE6, when we run the same
piece of code, it first loades JAVAWS.jar, Deploy.jar and plugin .jar from java/jre6/lib path but with JRE7 these jars are not loaded.
This has been tried in both 64 and 32 bits java version with no luck.
Any help here is really appreciated.

#jorge_B: We sign the jars using ant task. But due to a problem in JAVA 6u19 (where the softreferenced jars are sometimes garbage collected) we have decided to hardlink the jars as soon as we hit a java version 6u19 or above. This issue is not with jar signing. The issue is when we try to hard reference the jars. our jars are not getting hard referenced, insted we are getting nosuchmethod and nosuchfield exception in the log.

import java.io.IOException;
import java.lang.ref.SoftReference;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.JarURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.*;
import java.util.jar.JarFile;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* A utility class for working around the java webstart jar signing/security bug
* <p/>
* see http://bugs.sun.com/view_bug.do?bug_id=6967414 and http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6805618
*
* #author Scott Chan
*/
public class JarSignersHardLinker {
private static final String JRE_VERSION_START = "1.";
private static final int PROBLEM_JRE_VERSION = 6;
private static final String DOT_ZERO = ".0_";
/**
* the 1.6.0 update where this problem first occurred
*/
private static final int PROBLEM_JRE_UPDATE = 19;
private static String majorVersionStr = null;
public static final List sm_hardRefs = new ArrayList();
protected static void makeHardSignersRef(JarFile jar) throws java.io.IOException {
Logger.info("Making hard refs for: " + (jar != null ? jar.getName() : null) + " with Java Version: "+majorVersionStr);
if (jar != null && jar.getClass().getName().equals("com.sun.deploy.cache.CachedJarFile")) {
//lets attempt to get at the each of the soft links.
//first neet to call the relevant no-arg method to ensure that the soft ref is populated
//then we access the private member, resolve the softlink and throw it in a static list.
if (majorVersionStr != null && Integer.parseInt(majorVersionStr) > PROBLEM_JRE_VERSION) {
callNoArgMethod("getSigningData", jar);
makeHardLink("signingDataRef", jar);
} else {
callNoArgMethod("getSigners", jar);
makeHardLink("signersRef", jar);
callNoArgMethod("getSignerMap", jar);
makeHardLink("signerMapRef", jar);
// callNoArgMethod("getCodeSources", jar);
// makeHardLink("codeSourcesRef", jar);
callNoArgMethod("getCodeSourceCache", jar);
makeHardLink("codeSourceCacheRef", jar);
}
}
}
/**
* if the specified field for the given instance is a Softreference
* That soft reference is resolved and the returned ref is stored in a static list,
* making it a hard link that should never be garbage collected
*
* #param fieldName
* #param instance
*/
private static void makeHardLink(String fieldName, Object instance) {
Logger.info("attempting hard ref to " + instance.getClass().getName() + "." + fieldName);
try {
Field signersRef = instance.getClass().getDeclaredField(fieldName);
signersRef.setAccessible(true);
Object o = signersRef.get(instance);
if (o instanceof SoftReference) {
SoftReference r = (SoftReference) o;
Object o2 = r.get();
sm_hardRefs.add(o2);
} else {
Logger.warn(fieldName + ": is not an instance of soft reference");
}
} catch (NoSuchFieldException e) {
e.printStackTrace();
return;
} catch (IllegalAccessException e) {
e.printStackTrace();
}
}
/**
* Call the given no-arg method on the given instance
*
* #param methodName
* #param instance
*/
private static void callNoArgMethod(String methodName, Object instance) {
Logger.info("calling noarg method hard ref to " + instance.getClass().getName() + "." + methodName + "()");
try {
Method m = instance.getClass().getDeclaredMethod(methodName);
m.setAccessible(true);
m.invoke(instance);
} catch (SecurityException e1) {
e1.printStackTrace();
} catch (NoSuchMethodException e1) {
e1.printStackTrace();
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
}
/**
* is the preloader enabled. ie: will the preloader run in the current environment
*
* #return
*/
public static boolean isHardLinkerEnabled() {
boolean isHardLinkerDisabled = false; //change this to use whatever mechanism you use to enable or disable the preloader
return !isHardLinkerDisabled && isRunningOnJre1_6_0_19OrHigher() && isRunningOnWebstart();
}
/**
* is the application currently running on webstart
* <p/>
* detect the presence of a JNLPclassloader
*
* #return
*/
public static boolean isRunningOnWebstart() {
ClassLoader cl = Thread.currentThread().getContextClassLoader();
while (cl != null) {
if (cl.getClass().getName().equals("com.sun.jnlp.JNLPClassLoader")) {
return true;
}
cl = cl.getParent();
}
return false;
}
/**
* Is the JRE 1.6.0_19 or higher?
*
* #return
*/
public static boolean isRunningOnJre1_6_0_19OrHigher() {
String javaVersion = System.getProperty("java.version");
// Sometimes java releases version 1.X.0_YY-rev for specific issue and specific user,
// to resolve this we use patter instead of string split.
//ERP-6460: Checking whether JRE is 6 update 19 or higher
String updateStr = null;
majorVersionStr = null;
boolean isHardReferenceRequired = false;
Pattern pattern = Pattern.compile(JRE_VERSION_START + "([6-9]+)" + DOT_ZERO + "([0-9]+)(.*)");
Matcher matcher = pattern.matcher(javaVersion);
while (matcher.find()) {
majorVersionStr = matcher.group(1);
updateStr = matcher.group(2);
break;
}
try {
if (majorVersionStr != null) {
int java_version = Integer.parseInt(majorVersionStr);
if (java_version > PROBLEM_JRE_VERSION) {
isHardReferenceRequired = true;
} else if (java_version == PROBLEM_JRE_VERSION && Integer.parseInt(updateStr) >= PROBLEM_JRE_UPDATE) {
isHardReferenceRequired = true;
}
}
return isHardReferenceRequired;
} catch (NumberFormatException e) {
e.printStackTrace();
return isHardReferenceRequired;
}
}
/**
* get all the JarFile objects for all of the jars in the classpath
*
* #return
*/
public static Set<JarFile> getAllJarsFilesInClassPath() {
Set<JarFile> jars = new LinkedHashSet<JarFile>();
for (URL url : getAllJarUrls()) {
try {
jars.add(getJarFile(url));
} catch (IOException e) {
Logger.error("unable to retrieve jar at URL: " + url);
}
}
return jars;
}
/**
* Returns set of URLS for the jars in the classpath.
* URLS will have the protocol of jar eg: jar:http://HOST/PATH/JARNAME.jar!/META-INF/MANIFEST.MF
*/
static Set<URL> getAllJarUrls() {
try {
Set<URL> urls = new LinkedHashSet<URL>();
Enumeration<URL> mfUrls = Thread.currentThread().getContextClassLoader().getResources("META-INF/MANIFEST.MF");
while (mfUrls.hasMoreElements()) {
URL jarUrl = mfUrls.nextElement();
// System.out.println(jarUrl);
if (!jarUrl.getProtocol().equals("jar")) {
continue;
}
urls.add(jarUrl);
}
return urls;
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* get the jarFile object for the given url
*
* #param jarUrl
* #return
* #throws IOException
*/
public static JarFile getJarFile(URL jarUrl) throws IOException {
URLConnection urlConnnection = jarUrl.openConnection();
if (urlConnnection instanceof JarURLConnection) {
// Using a JarURLConnection will load the JAR from the cache when using Webstart 1.6
// In Webstart 1.5, the URL will point to the cached JAR on the local filesystem
JarURLConnection jcon = (JarURLConnection) urlConnnection;
return jcon.getJarFile();
} else {
throw new AssertionError("Expected JarURLConnection");
}
}
/**
* Spawn a new thread to run through each jar in the classpath and create a hardlink
* to the jars softly referenced signers infomation.
*/
public static void go() {
if (!isHardLinkerEnabled()) {
return;
}
Logger.info("Starting Resource Preloader Hardlinker");
Thread t = new Thread(new Runnable() {
public void run() {
try {
Set<JarFile> jars = getAllJarsFilesInClassPath();
for (JarFile jar : jars) {
makeHardSignersRef(jar);
}
} catch (Exception e) {
Logger.error("Problem preloading resources", e);
} catch (Error e) {
Logger.error("Error preloading resources", e);
}
}
});
t.start();
}
}

Custom ClassLoader with JBoss

I am trying to compile and load a class dynamically during runtime using the JavaCompiler API. I store the compiled bytecode in memory. So I use a custom class loader to load the class.
public class CompilerAPITest {
static String sourceCode = "package in.test;" +
"public class DynamicCompilationHelloWorld implements TestInterface{" +
"public void test (){" +
"System.out.println (\"Hello, dynamic compilation world!\");" +
"new in.test.another.SomeClass().fun();" +
"}" +
"}" ;
public void doCompilation (){
SimpleJavaFileObject fileObject = new DynamicJavaSourceCodeObject ("in.test.DynamicCompilationHelloWorld", sourceCode) ;
JavaFileObject javaFileObjects[] = new JavaFileObject[]{fileObject} ;
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
JavaFileManager stdFileManager = new
CompilerAPITest.ClassFileManager(compiler
.getStandardFileManager(null, null, null));
Iterable<? extends JavaFileObject> compilationUnits = Arrays.asList(javaFileObjects);
DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
List<String> options = Arrays.asList("-cp", System.getProperty("java.class.path")
+ ":" + getPath(CompilerAPITest.class));
CompilationTask compilerTask = compiler.getTask(null, stdFileManager, diagnostics, options, null, compilationUnits) ;
boolean status = compilerTask.call();
if (!status){//If compilation error occurs
/*Iterate through each compilation problem and print it*/
for (Diagnostic<? extends JavaFileObject> diagnostic : diagnostics.getDiagnostics()){
System.out.format("Error on line %d in %s", diagnostic.getLineNumber(), diagnostic);
}
}
try {
stdFileManager.close() ;//Close the file manager
} catch (IOException e) {
e.printStackTrace();
}
try {
stdFileManager.getClassLoader(null)
.loadClass("in.test.DynamicCompilationHelloWorld").asSubclass(TestInterface.class).newInstance().test();
} catch (ClassNotFoundException e) {
e.printStackTrace();
//This does nothing.
} catch (InstantiationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static void main(String args[]){
new CompilerAPITest ().doCompilation() ;
}
class DynamicJavaSourceCodeObject extends SimpleJavaFileObject{
private String qualifiedName ;
private String sourceCode ;
protected DynamicJavaSourceCodeObject(String name, String code) {
super(URI.create("string:///" +name.replaceAll("\\.", "/") + Kind.SOURCE.extension), Kind.SOURCE);
this.qualifiedName = name ;
this.sourceCode = code ;
}
#Override
public CharSequence getCharContent(boolean ignoreEncodingErrors)
throws IOException {
return sourceCode ;
}
public String getQualifiedName() {
return qualifiedName;
}
public void setQualifiedName(String qualifiedName) {
this.qualifiedName = qualifiedName;
}
public String getSourceCode() {
return sourceCode;
}
public void setSourceCode(String sourceCode) {
this.sourceCode = sourceCode;
}
}
private static class ClassFileManager extends
ForwardingJavaFileManager<JavaFileManager> {
private JavaClassObject jclassObject;
public ClassFileManager(StandardJavaFileManager
standardManager) {
super(standardManager);
}
#Override
public ClassLoader getClassLoader(Location location) {
return new java.security.SecureClassLoader() {
#Override
protected Class<?> findClass(String name)
throws ClassNotFoundException {
byte[] b = jclassObject.getBytes();
return super.defineClass(name, jclassObject
.getBytes(), 0, b.length);
}
};
}
#Override
public JavaFileObject getJavaFileForOutput(Location location,
String className, Kind kind, FileObject sibling)
throws IOException {
jclassObject = new JavaClassObject(className, kind);
return jclassObject;
}
}
private static class JavaClassObject extends SimpleJavaFileObject {
protected final ByteArrayOutputStream bos =
new ByteArrayOutputStream();
public JavaClassObject(String name, Kind kind) {
super(URI.create("string:///" + name.replace('.', '/')
+ kind.extension), kind);
}
public byte[] getBytes() {
return bos.toByteArray();
}
#Override
public OutputStream openOutputStream() throws IOException {
return bos;
}
}
}
This works fine when run in a standalone setup. However when i call doCompilation() in my production setup, which runs on JBoss, i get the following exception.
java.lang.NoClassDefFoundError:
in/test/TestInterface(wrong name:
in/test/DynamicCompilationHelloWorld)
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at java.lang.ClassLoader.defineClass(ClassLoader.java:466)
at in.test.CompilerAPITest$ClassFileManager$1.findClass(CompilerAPITest.java:126)
What could be the problem here?

Found the issue after some googling.
Parent class loader needs to be set for the custom class loader used.
Class loading hierarchy is slightly different in JBoss. Here UnifiedClassLoader is used, which is used for checking even the peer class loaders, in addition to the parents, before throwing ClassNotFoundException. So when a custom class loader is used, it needs to delegate the defineClass calls to the UnifiedClassLoader, when it can not load the class.
Here is an example code snippet for a custom class loader.
private static class ByteClassLoader extends ClassLoader{
private Map<String, JavaFileObject> store = new HashMap<String, JavaFileObject>();
public ByteClassLoader(Map<String, JavaFileObject> str)
{
super( ByteClassLoader.class.getClassLoader() ); // set parent
store = str;
}
protected Class<?> findClass(String name)
throws ClassNotFoundException{
JavaFileObject jfo = store.get(name);
if (jfo == null){
throw new ClassNotFoundException(name);
}
byte[] bytes = ((JavaClassObject)jfo).getBytes();
Class<?> cl = defineClass(name, bytes, 0, bytes.length);
if (cl == null){
throw new ClassNotFoundException(name);
}
return cl;
}
}
And for loading the class,
ByteClassLoader cl = new ByteClassLoader(store);
cl.loadClass(className);
needs to be used.
Here is an excellent link on dynamic compilation and class loading.
http://fivedots.coe.psu.ac.th/~ad/jg/javaArt1/onTheFlyArt1.pdf