Develop Reference

To generate Java Class on Fly and add to class path

I am generating a java class on fly and trying to invoke a method on it. For this, seems like I have to do the following
Compile the class (javac filename will not work as it depends on may other dependencies)
Add the class to the class path at runtime
How can I achieve this?

I made it work with JavaCompiler and Custom class loader like below.
private Path compileSource(Path javaFile, String contractFileNameWithoutExtension) {
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
compiler.run(null, null, null, javaFile.toFile().getAbsolutePath());
return javaFile.getParent().resolve(contractFileNameWithoutExtension+".class");
}
public Class findClass(String name) {
String filePath = sourceCodeLocation +"/"+ name.replace(".", "/")+".class";
byte[] b = loadClassFromFile(filePath);
return defineClass(name, b, 0, b.length);
}
private byte[] loadClassFromFile(String fileName) {
try {
InputStream inputStream = FileUtils.getFileInputStream.apply(fileName);
byte[] buffer;
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
int nextValue = 0;
try {
while ((nextValue = inputStream.read()) != -1) {
byteStream.write(nextValue);
}
} catch (IOException e) {
e.printStackTrace();
}
buffer = byteStream.toByteArray();
return buffer;
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e);
}
}

Parent Last Classloader to solve Java Class path hell?

I have a project which uses two versions of bouncyCastle jars bcprov-jdk15 and bcprov-jdk16. The jvm loads the older version but there is a feature I wrote which needs the newer version to run. I tried to solve this classpath hell by using a custom class loader. After some googling and with the help of some previous Stackoverflow answers[1] [2] and this blog, I wrote the following Parent Last Class loader to load the classes from the newer jar before delegating to the parent class loader.
public class ParentLastClassLoader extends ClassLoader {
private String jarFile; //Path to the jar file
private Hashtable classes = new Hashtable(); //used to cache already defined classes
public ParentLastClassLoader(ClassLoader parent, String path)
{
super(parent);
this.jarFile = path;
}
#Override
public Class<?> findClass(String name) throws ClassNotFoundException
{
System.out.println("Trying to find");
throw new ClassNotFoundException();
}
#Override
protected synchronized Class<?> loadClass(String className, boolean resolve) throws ClassNotFoundException
{
System.out.println("Trying to load");
try
{
System.out.println("Loading class in Child : " + className);
byte classByte[];
Class result = null;
//checks in cached classes
result = (Class) classes.get(className);
if (result != null) {
return result;
}
try {
JarFile jar = new JarFile(jarFile);
JarEntry entry = jar.getJarEntry(className + ".class");
InputStream is = jar.getInputStream(entry);
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
int nextValue = is.read();
while (-1 != nextValue) {
byteStream.write(nextValue);
nextValue = is.read();
}
classByte = byteStream.toByteArray();
result = defineClass(className, classByte, 0, classByte.length, null);
classes.put(className, result);
return result;
} catch (Exception e) {
throw new ClassNotFoundException(className + "Not found", e);
}
}
catch( ClassNotFoundException e ){
System.out.println("Delegating to parent : " + className);
// didn't find it, try the parent
return super.loadClass(className, resolve);
}
}
}
I loaded the main class in the feature with this class loader but the BouncyCaslte classes used in the feature are not loaded by my custom classloader.
ClassLoader loader = new ParentLastClassLoader(Thread.currentThread().getContextClassLoader(), pathToJar);
Class myClass = loader.loadClass("MainClassOfTheFeature");
Method mainMethod = myClass.getMethod("MainMethod");
mainMethod.invoke(myClass.getConstructor().newInstance());
Jvm still uses the classes it loaded from the older version. How can I make the JVM to load the classes from my class loader when running the feature and use the already loaded older classes in the older jar when the feature is not running?
Edit:
The problem remains even after setting the custom classloader as the Thread context classloader in the MainMethod of the feature Main class.
Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());

I managed to solve this problem. Modified the code of the ParentLastClassLoader to get an array of all the Jarfile paths which are needed by the feature. So when a class is loaded, all the jarfiles needed by the feature will be searched for the .class files. If a class file cannot be found, it will be delegated to the parent.
private class ParentLastClassLoader extends ClassLoader {
private String[] jarFiles; //Paths to the jar files
private Hashtable classes = new Hashtable(); //used to cache already defined classes
public ParentLastClassLoader(ClassLoader parent, String[] paths)
{
super(parent);
this.jarFiles = paths;
}
#Override
public Class<?> findClass(String name) throws ClassNotFoundException
{
System.out.println("Trying to find");
throw new ClassNotFoundException();
}
#Override
protected synchronized Class<?> loadClass(String className, boolean resolve) throws ClassNotFoundException
{
System.out.println("Trying to load");
try
{
System.out.println("Loading class in Child : " + className);
byte classByte[];
Class result = null;
//checks in cached classes
result = (Class) classes.get(className);
if (result != null) {
return result;
}
for(String jarFile: jarFiles){
try {
JarFile jar = new JarFile(jarFile);
JarEntry entry = jar.getJarEntry(className.replace(".","/") + ".class");
InputStream is = jar.getInputStream(entry);
ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
int nextValue = is.read();
while (-1 != nextValue) {
byteStream.write(nextValue);
nextValue = is.read();
}
classByte = byteStream.toByteArray();
result = defineClass(className, classByte, 0, classByte.length, null);
classes.put(className, result);
} catch (Exception e) {
continue;
}
}
result = (Class) classes.get(className);
if (result != null) {
return result;
}
else{
throw new ClassNotFoundException("Not found "+ className);
}
}
catch( ClassNotFoundException e ){
System.out.println("Delegating to parent : " + className);
// didn't find it, try the parent
return super.loadClass(className, resolve);
}
}
}
The ParentLastClassLoader is instantiated as follows.
ClassLoader loader = new ParentLastClassLoader(Thread.currentThread().getContextClassLoader(), paths);
Once the ParentLastClassLoader is instantiated, the MainClassOfTheFeature will be loaded and its MainMethod will be invoked.

Ok, you created your own classloader and then loaded a class using it. The question is - how does the thread classloader will know about that?
So, you must load the class using some classloader, and then set this classloader as thread context classloader.

How to copy file from one location to another location?

I want to copy a file from one location to another location in Java. What is the best way to do this?
Here is what I have so far:
import java.io.File;
import java.io.FilenameFilter;
import java.util.ArrayList;
import java.util.List;
public class TestArrayList {
public static void main(String[] args) {
File f = new File(
"D:\\CBSE_Demo\\Demo_original\\fscommand\\contentplayer\\config");
List<String>temp=new ArrayList<String>();
temp.add(0, "N33");
temp.add(1, "N1417");
temp.add(2, "N331");
File[] matchingFiles = null;
for(final String temp1: temp){
matchingFiles = f.listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) {
return name.startsWith(temp1);
}
});
System.out.println("size>>--"+matchingFiles.length);
}
}
}
This does not copy the file, what is the best way to do this?

You can use this (or any variant):
Files.copy(src, dst, StandardCopyOption.REPLACE_EXISTING);
Also, I'd recommend using File.separator or / instead of \\ to make it compliant across multiple OS, question/answer on this available here.
Since you're not sure how to temporarily store files, take a look at ArrayList:
List<File> files = new ArrayList();
files.add(foundFile);
To move a List of files into a single directory:
List<File> files = ...;
String path = "C:/destination/";
for(File file : files) {
Files.copy(file.toPath(),
(new File(path + file.getName())).toPath(),
StandardCopyOption.REPLACE_EXISTING);
}

Update:
see also
https://stackoverflow.com/a/67179064/1847899
Using Stream
private static void copyFileUsingStream(File source, File dest) throws IOException {
InputStream is = null;
OutputStream os = null;
try {
is = new FileInputStream(source);
os = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
while ((length = is.read(buffer)) > 0) {
os.write(buffer, 0, length);
}
} finally {
is.close();
os.close();
}
}
Using Channel
private static void copyFileUsingChannel(File source, File dest) throws IOException {
FileChannel sourceChannel = null;
FileChannel destChannel = null;
try {
sourceChannel = new FileInputStream(source).getChannel();
destChannel = new FileOutputStream(dest).getChannel();
destChannel.transferFrom(sourceChannel, 0, sourceChannel.size());
}finally{
sourceChannel.close();
destChannel.close();
}
}
Using Apache Commons IO lib:
private static void copyFileUsingApacheCommonsIO(File source, File dest) throws IOException {
FileUtils.copyFile(source, dest);
}
Using Java SE 7 Files class:
private static void copyFileUsingJava7Files(File source, File dest) throws IOException {
Files.copy(source.toPath(), dest.toPath());
}
Or try Googles Guava :
https://github.com/google/guava
docs:
https://guava.dev/releases/snapshot-jre/api/docs/com/google/common/io/Files.html

Use the New Java File classes in Java >=7.
Create the below method and import the necessary libs.
public static void copyFile( File from, File to ) throws IOException {
Files.copy( from.toPath(), to.toPath() );
}
Use the created method as below within main:
File dirFrom = new File(fileFrom);
File dirTo = new File(fileTo);
try {
copyFile(dirFrom, dirTo);
} catch (IOException ex) {
Logger.getLogger(TestJava8.class.getName()).log(Level.SEVERE, null, ex);
}
NB:- fileFrom is the file that you want to copy to a new file fileTo in a different folder.
Credits - #Scott: Standard concise way to copy a file in Java?

public static void copyFile(File oldLocation, File newLocation) throws IOException {
if ( oldLocation.exists( )) {
BufferedInputStream reader = new BufferedInputStream( new FileInputStream(oldLocation) );
BufferedOutputStream writer = new BufferedOutputStream( new FileOutputStream(newLocation, false));
try {
byte[] buff = new byte[8192];
int numChars;
while ( (numChars = reader.read( buff, 0, buff.length ) ) != -1) {
writer.write( buff, 0, numChars );
}
} catch( IOException ex ) {
throw new IOException("IOException when transferring " + oldLocation.getPath() + " to " + newLocation.getPath());
} finally {
try {
if ( reader != null ){
writer.close();
reader.close();
}
} catch( IOException ex ){
Log.e(TAG, "Error closing files when transferring " + oldLocation.getPath() + " to " + newLocation.getPath() );
}
}
} else {
throw new IOException("Old location does not exist when transferring " + oldLocation.getPath() + " to " + newLocation.getPath() );
}
}

Copy a file from one location to another location means,need to copy the whole content to another location.Files.copy(Path source, Path target, CopyOption... options) throws IOException this method expects source location which is original file location and target location which is a new folder location with destination same type file(as original).
Either Target location needs to exist in our system otherwise we need to create a folder location and then in that folder location we need to create a file with the same name as original filename.Then using copy function we can easily copy a file from one location to other.
public static void main(String[] args) throws IOException {
String destFolderPath = "D:/TestFile/abc";
String fileName = "pqr.xlsx";
String sourceFilePath= "D:/TestFile/xyz.xlsx";
File f = new File(destFolderPath);
if(f.mkdir()){
System.out.println("Directory created!!!!");
}
else {
System.out.println("Directory Exists!!!!");
}
f= new File(destFolderPath,fileName);
if(f.createNewFile()) {
System.out.println("File Created!!!!");
} else {
System.out.println("File exists!!!!");
}
Files.copy(Paths.get(sourceFilePath), Paths.get(destFolderPath, fileName),REPLACE_EXISTING);
System.out.println("Copy done!!!!!!!!!!!!!!");
}

You can do it with the Java 8 Streaming API, PrintWriter and the Files API
try (PrintWriter pw = new PrintWriter(new File("destination-path"), StandardCharsets.UTF_8)) {
Files.readAllLines(Path.of("src/test/resources/source-file.something"), StandardCharsets.UTF_8)
.forEach(pw::println);
}
If you want to modify the content on-the-fly while copying, check out this link for the extended example https://overflowed.dev/blog/copy-file-and-modify-with-java-streams/

I modified one of the answers to make it a bit more efficient.
public void copy(){
InputStream in = null;
try {
in = new FileInputStream(Files);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
try {
OutputStream out = new FileOutputStream();
try {
// Transfer bytes from in to out
byte[] buf = new byte[1024];
while (true) {
int len = 0;
try {
if (!((len = in.read(buf)) > 0)) break;
} catch (IOException e) {
e.printStackTrace();
}
try {
out.write(buf, 0, len);
} catch (IOException e) {
e.printStackTrace();
}
}
} finally {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
} finally {
try {
in.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
private void moveFile() {
copy();
File dir = getFilesDir();
File file = new File(dir, "my_filename");
boolean deleted = file.delete();
}

Files.exists()
Files.createDirectory()
Files.copy()
Overwriting Existing Files:
Files.move()
Files.delete()
Files.walkFileTree()
enter link description here

You can use
FileUtils.copy(sourceFile, destinationFile);
https://commons.apache.org/proper/commons-io/apidocs/org/apache/commons/io/FileUtils.html

Implementing a selective ClassLoader

I want to instrument the bytecode of some classes on the classpath at loading time. Since these are 3rd party libraries, I know exactly when they are loaded. The problem is that I need to do the instrumentation selectively, i.e. instrument only some classes. Now if I do not load a class with my classloader but with its parent, this parent gets set as the classes classloader and all succinct classes are loaded by that parent, effectively putting my classloader out of use. So I need to implement a parent-last classloader (see How to put custom ClassLoader to use?).
So I need to load classes myself. If those classes are system classes (starting with "java" or "sun") I delegate to the parent. Otherwise I read the bytecode and call defineClass(name, byteBuffer, 0, byteBuffer.length);. But now a java.lang.ClassNotFoundException: java.lang.Object is thrown.
Here is the code, any comment highly appreciated:
public class InstrumentingClassLoader extends ClassLoader {
private final BytecodeInstrumentation instrumentation = new BytecodeInstrumentation();
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
Class<?> result = defineClass(name);
if (result != null) {
return result;
}
result = findLoadedClass(name);
if(result != null){
return result;
}
result = super.findClass(name);
return result;
}
private Class<?> defineClass(String name) throws ClassFormatError {
byte[] byteBuffer = null;
if (instrumentation.willInstrument(name)) {
byteBuffer = instrumentByteCode(name);
}
else {
byteBuffer = getRegularByteCode(name);
}
if (byteBuffer == null) {
return null;
}
Class<?> result = defineClass(name, byteBuffer, 0, byteBuffer.length);
return result;
}
private byte[] getRegularByteCode(String name) {
if (name.startsWith("java") || name.startsWith("sun")) {
return null;
}
try {
InputStream is = ClassLoader.getSystemResourceAsStream(name.replace('.', '/') + ".class");
ByteArrayOutputStream buffer = new ByteArrayOutputStream();
int nRead;
byte[] data = new byte[16384];
while ((nRead = is.read(data, 0, data.length)) != -1) {
buffer.write(data, 0, nRead);
}
buffer.flush();
return buffer.toByteArray();
} catch (IOException exc) {
return null;
}
}
private byte[] instrumentByteCode(String fullyQualifiedTargetClass) {
try {
String className = fullyQualifiedTargetClass.replace('.', '/');
return instrumentation.transformBytes(className, new ClassReader(fullyQualifiedTargetClass));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
The code can be executed e.g. with:
InstrumentingClassLoader instrumentingClassLoader = new InstrumentingClassLoader();
Class<?> changedClass = instrumentingClassLoader.loadClass(ClassLoaderTestSubject.class.getName());
The ClassLoaderTestSubject should call some other classes, where the called classes are target of instrumentation, but the ClassLoaderTestSubject itself is not...

I'd recommend you to use regular class loader strategy, i.e. parent first. But put all classes that you want to instrument into separate jar file and do not add it to the classpath of the application. Instantiate these classes using your class loader that extends URL class loader and knows to search jars in other location. In this case all JDK classes will be known automatically and your code will be simpler. You do not have to "think" whether to instrument the class: if it is not loaded by parent class loader it is your class that has to be instrumented.

Stupid mistake. The parent classloader is not the parent as in the inheritance hierarchy. It is the parent as given to the constructor. So the correct code looks like this:
public InstrumentingClassLoader() {
super(InstrumentingClassLoader.class.getClassLoader());
this.classLoader = InstrumentingClassLoader.class.getClassLoader();
}
#Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
[... as above ...]
result = classLoader.loadClass(name);
return result;
}

java classloader and runtime compilation

Despite warnings to drop my present course of action, I currently see no better way to solve my problem. I must generate Java code at runtime, then compile it, load it and reference it.
Problem is that the generated code imports code that has already been loaded by the system class loader (I suppose) - that is, code present in one of the jars on my classpath.
(I run inside a Tomcat 6 web container over Java 6.) You may ask yourselves why that is a problem - well I sure don't know - but fact is that I get compilation errors:
/W:/.../parser/v0.5/AssignELParser.java:6:
package com.xxx.yyy.zzz.configuration
does not exist
Following some examples off the internet I have defined the following classes:
class MemoryClassLoader extends ChainedAction {
private static final Logger LOG = Logger.getLogger(MemoryClassLoader.class);
private LoaderImpl impl;
private class LoaderImpl extends ClassLoader {
// The compiler tool
private final JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
// Compiler options
private final Iterable<String> options = Arrays.asList("-verbose");
// DiagnosticCollector, for collecting compilation problems
private final DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
// Our FileManager
private final MemoryFileManager manager = new MemoryFileManager(this.compiler);
public LoaderImpl(File sourceDirectory) {
List<Source> list = new ArrayList<Source>();
File[] files = sourceDirectory.listFiles(new FilenameFilter() {
#Override
public boolean accept(File dir, String name) {
return name.endsWith(Kind.SOURCE.extension);
}
});
for (File file : files) {
list.add(new Source(file));
}
CompilationTask task = compiler.getTask(null, manager, diagnostics, options, null, list);
Boolean compilationSuccessful = task.call();
LOG.info("Compilation has " + ((compilationSuccessful) ? "concluded successfully" : "failed"));
// report on all errors to screen
for (Diagnostic<? extends JavaFileObject> diagnostic : diagnostics.getDiagnostics()) {
LOG.warn(diagnostic.getMessage(null));
}
}
#Override
protected Class<?> findClass(String name) throws ClassNotFoundException {
synchronized (this.manager) {
Output output = manager.map.remove(name);
if (output != null) {
byte[] array = output.toByteArray();
return defineClass(name, array, 0, array.length);
}
}
return super.findClass(name);
}
}
#Override
protected void run() {
impl = new LoaderImpl(new File(/* Some directory path */));
}
}
class MemoryFileManager extends ForwardingJavaFileManager<JavaFileManager> {
final Map<String, Output> map = new HashMap<String, Output>();
MemoryFileManager(JavaCompiler compiler) {
super(compiler.getStandardFileManager(null, null, null));
}
#Override
public Output getJavaFileForOutput(Location location, String name, Kind kind, FileObject source) {
Output output = new Output(name, kind);
map.put(name, output);
return output;
}
}
class Output extends SimpleJavaFileObject {
private final ByteArrayOutputStream baos = new ByteArrayOutputStream();
Output(String name, Kind kind) {
super(URI.create("memo:///" + name.replace('.', '/') + kind.extension), kind);
}
byte[] toByteArray() {
return this.baos.toByteArray();
}
#Override
public ByteArrayOutputStream openOutputStream() {
return this.baos;
}
}
class Source extends SimpleJavaFileObject {
public Source(File file) {
super(file.toURI(), Kind.SOURCE);
}
#Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
StringBuilder sb = new StringBuilder("");
try {
File file = new File(uri);
FileReader fr = new FileReader(file);
BufferedReader br = new BufferedReader(fr);
sb = new StringBuilder((int) file.length());
String line = "";
while ((line = br.readLine()) != null) {
sb.append(line);
sb.append("\n");
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return sb.toString();
}
}
It seems that the inner class LoaderImpl by extending the ClassLoader class and by not calling an explicit super constructor should reference as its parent class loader the system class loader.
If it does so then why do I get the "runtime" compilation error - above? Why does it not find the code for the imported class?

Not sure if it can help, but have you tried to specify classpath explicitly?
getClassPath()
{
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
URL[] urls = ((URLClassLoader) classLoader).getURLs();
StringBuilder buf = new StringBuilder(1000);
buf.append(".");
String separator = System.getProperty("path.separator");
for (URL url : urls) {
buf.append(separator).append(url.getFile());
}
}
classPath = buf.toString();
and then
options.add("-classpath");
options.add(getClassPath());
I also can't see where do you pass LoaderImpl instance to the compiler. Shouldn't it be done explicitly?