Develop Reference

Scala URLClassLoader is not reloading class file

I am running a scala project where I need to execute some rules. The rules will be dynamically added or removed from scala class file at runtime.
So, I want whenever the rules class modify, it should reload to get the changes without stopping the execution process.
I used runtime.getruntime.exec() to compile it
and am URL Class loader to get the modified code from classes
The exec() run fines. and in target folder classes gets modifies also, even when I am using URL Class Loader, not getting any error.
But it is giving me same result which i have on starting of the project. It's not giving me modification code.
Below is the code which I am using.
package RuleEngine
import akka.actor._
import akka.http.scaladsl.Http
import akka.http.scaladsl.server.Directives._
import akka.stream.ActorMaterializer
import akka.util.Timeout
import scala.io.StdIn
import Executor.Compute
import scala.concurrent.{Await, ExecutionContextExecutor}
import scala.concurrent.duration._
object StatsEngine {
def main(args: Array[String]) {
implicit val system: ActorSystem = ActorSystem("StatsEngine")
implicit val materializer: ActorMaterializer = ActorMaterializer()
implicit val executionContext: ExecutionContextExecutor = system.dispatcher
implicit val timeout = Timeout(10 seconds)
val computeDataActor = system.actorOf(Props[Compute],"ComputeData")
val route = {
post {
path("computedata"/) {
computeDataActor ! "Execute"
complete("done")
}
}
}
val bindingFuture = Http().bindAndHandle(route , "localhost", 9000)
println(s"Server online at http://localhost:9000/\nPress RETURN to stop...")
}
}
This is the main object file where I have created Akka HTTP to make API's
It will call computeDataActor whose code is below.
package Executor
import java.io.File
import java.net.URLClassLoader
import CompiledRules.RulesList
import akka.actor.Actor
class Compute extends Actor{
def exceuteRule(): Unit ={
val rlObj = new RulesList
rlObj.getClass.getDeclaredMethods.map(name=>name).foreach(println)
val prcs = Runtime.getRuntime().exec("scalac /home/hduser/MStatsEngine/Test/RuleListCollection/src/main/scala/CompiledRules/RuleList.scala -d /home/hduser/MStatsEngine/Test/RuleListCollection/target/scala-2.11/classes/")
prcs.waitFor()
val fk = new File("/home/hduser/MStatsEngine/Test/RuleListCollection/target/scala-2.11/classes/").toURI.toURL
val classLoaderUrls = Array(fk)
val urlClassLoader = new URLClassLoader(classLoaderUrls)
val beanClass = urlClassLoader.loadClass("CompiledRules.RulesList")
val constructor = beanClass.getConstructor()
val beanObj = constructor.newInstance()
beanClass.getDeclaredMethods.map(x=>x.getName).foreach(println)
}
override def receive: Receive ={
case key:String => {
exceuteRule()
}
}
}
Rules are imported which is mentioned below.
package CompiledRules
class RulesList{
def R1 : Any = {
return "executing R1"
}
def R2 : Any = {return "executing R2"}
// def R3 : Any = {return "executing R3"}
//def R4 : Any = {return "executing R4"}
def R5 : Any = {return "executing R5"}
}//Replace
So, whene i execute code, and on calling API, I will get ouput as
R1
R2
R5
Now, without stopping the project, I will uncomment R3 and R4. And I will call API again,
As I am executing code again, using
runtime.getruntime.exec()
it will compile the file and update classes in target
So, i used URLClassLoader to get new object of modification code.
But Unfortunately I am getting same result always which i have on starting of the project
R1
R2
R5
Below is link for complete project
Source Code

val beanClass = urlClassLoader.loadClass("CompiledRules.RulesList")
val constructor = beanClass.getConstructor()
val beanObj = constructor.newInstance()
Is just creating the newInstance of Already loaded class.
Java's builtin Class loaders always checks if a class is already loaded before loading it.
loadClass
protected Class<?> loadClass(String name,
boolean resolve)
throws ClassNotFoundException
Loads the class with the specified binary name. The default implementation of this method searches for classes in the following order:
Invoke findLoadedClass(String) to check if the class has already been loaded.
Invoke the loadClass method on the parent class loader. If the parent is null the class loader built-in to the virtual machine is used, instead.
Invoke the findClass(String) method to find the class.
To reload a class you will have to implement your own ClassLoader subclass as in this link

Jython - Calling Python Class in Java

I want to call my Python class in Java, but I get the error message:
Manifest
com.atlassian.tutorial:myConfluenceMacro:atlassian-plugin:1.0.0-SNAPSHOT
: Classes found in the wrong directory
I installed Jython on my pc via jar. And added it in my pom (because I am using a Maven Project). What am I doing wrong? How can I call a python method inside my java class?
I am using python3.
POM
<!-- https://mvnrepository.com/artifact/org.python/jython-standalone -->
<dependency>
<groupId>org.python</groupId>
<artifactId>jython-standalone</artifactId>
<version>2.7.1</version>
</dependency>
JAVA CLASS
package com.atlassian.tutorial.javapy;
import org.python.core.PyInstance;
import org.python.util.PythonInterpreter;
public class InterpreterExample
{
PythonInterpreter interpreter = null;
public InterpreterExample()
{
PythonInterpreter.initialize(System.getProperties(),
System.getProperties(), new String[0]);
this.interpreter = new PythonInterpreter();
}
void execfile( final String fileName )
{
this.interpreter.execfile(fileName);
}
PyInstance createClass( final String className, final String opts )
{
return (PyInstance) this.interpreter.eval(className + "(" + opts + ")");
}
public static void main( String gargs[] )
{
InterpreterExample ie = new InterpreterExample();
ie.execfile("hello.py");
PyInstance hello = ie.createClass("Hello", "None");
hello.invoke("run");
}
}
Python Class
class Hello:
__gui = None
def __init__(self, gui):
self.__gui = gui
def run(self):
print ('Hello world!')
Thank you!

You have wrong indentation in your Python class. Correct code is:
class Hello:
__gui = None
def __init__(self, gui):
self.__gui = gui
def run(self):
print ('Hello world!')
so that __init__() and run() are methods of your Hello class, not global functions. Otherwise your code works nicely.
And please remember that the latest version of Jython is 2.7.1 - it is not Python3 compatible.

Get file from class path so tests can run on all machines

I am using Vertx and trying to test some parameters that i am getting data from jsonfile, currently it works but i want get this file just through class path so it can be tested from a different computer.
private ConfigRetriever getConfigRetriever() {
ConfigStoreOptions fileStore = new ConfigStoreOptions().setType("file").setOptional(true)
.setConfig(new JsonObject()
.put("path", "/home/user/MyProjects/MicroserviceBoilerPlate/src/test/resources/local_file.json"));
ConfigStoreOptions sysPropsStore = new ConfigStoreOptions().setType("sys");
ConfigRetrieverOptions options = new ConfigRetrieverOptions().addStore(fileStore).addStore(sysPropsStore);
return ConfigRetriever.create(Vertx.vertx(), options);
}
My path as written above starts from /home / dir which makes it impossible to be tested on another machine. My test below uses this config
#Test
public void tourTypes() {
ConfigRetriever retriever = getConfigRetriever();
retriever.getConfig(ar -> {
if (ar.failed()) {
// Failed to retrieve the configuration
} else {
JsonObject config = ar.result();
List<String> extractedIds = YubiParserServiceCustomImplTest.getQueryParameters(config, "tourTypes");
assertEquals(asList("1", "2", "3", "6"), extractedIds);
}
});
}
I want to make the path a class path so i can test it on all environment.
I tried to access class path like this but not sure how it should be
private void fileFinder() {
Path p1 = Paths.get("/test/resources/local_file.json");
Path fileName = p1.getFileName();
}

If you have stored the file inside "src/test/resources" then you can use
InputStream confFile = getClass().getResourceAsStream("/local_file.json");
or
URL url = getClass().getResource("/local_file.json");
inside your test class (example)
IMPORTANT!
In both cases the file names can start with a / or not. If it does, it starts at the root of the classpath. If not, it starts at the package of the class on which the method is called.

Put .json file to /resources folder of your project (here an example).
Then access it via ClassLoader.getResourceAsStream:
InputStream configFile = ClassLoader.getResourceAsStream("path/to/file.json");
JsonObject config = new JsonParser().parse(configFile);
// Then provide this config to Vertx

As I understand, considering the location of your json file, you simply need to do this:
.setConfig(new JsonObject().put("path", "local_file.json"));
See this for reference.

Add custom folders to classpath in bazel java tests

I'm trying to migrate a large codebase from maven to bazel and I've found that some of the tests write to target/classes and target/test-classes and the production code reads it as resources on the classpath. This is because maven surefire/failsafe run by default from the module directory and add target/classes and target/test-classes to the classpath.
For me to migrate this large codebase the only reasonable solution is to create target, target/classes and target/test-classes folders and add the last two to the classpath of the tests.
Any ideas on how this can be achieved?
Thanks

Another line of approach. Instead of generating a test suite, create a custom javaagent and a custom class loader. Use jvm_flags to setup and configure it.
The javaagent has a premain method. This sounds like a natural place to do things that happen before the regular main method, even if they don't have anything to do with class instrumentation, debugging, coverage gathering, or any other usual uses of javaagents.
The custom javaagent reads system property extra.dirs and creates directories specified there. It then reads property extra.link.path and creates the symbolic links as specified there, so I can place resources where the tests expect them, without having to copy them.
Classloader is needed so that we can amend the classpath at runtime without hacks. Great advantage is that this solution works on Java 10.
The custom classloader reads system property extra.class.path and (in effect) prepends it before what is in java.class.path.
Doing things this way means that standard bazel rules can be used.
BUILD
runtime_classgen_dirs = ":".join([
"target/classes",
"target/test-classes",
])
java_test(
...,
jvm_flags = [
# agent
"-javaagent:$(location //tools:test-agent_deploy.jar)",
"-Dextra.dirs=" + runtime_classgen_dirs,
# classloader
"-Djava.system.class.loader=ResourceJavaAgent",
"-Dextra.class.path=" + runtime_classgen_dirs,
],
,,,,
deps = [
# not runtime_deps, cause https://github.com/bazelbuild/bazel/issues/1566
"//tools:test-agent_deploy.jartest-agent_deploy.jar"
],
...,
)
tools/BUILD
java_binary(
name = "test-agent",
testonly = True,
srcs = ["ResourceJavaAgent.java"],
deploy_manifest_lines = ["Premain-Class: ResourceJavaAgent"],
main_class = "ResourceJavaAgent",
visibility = ["//visibility:public"],
)
tools/ResourceJavaAgent.java
import java.io.File;
import java.io.IOException;
import java.lang.instrument.Instrumentation;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.LinkedList;
import java.util.List;
// https://stackoverflow.com/questions/60764/how-should-i-load-jars-dynamically-at-runtime
public class ResourceJavaAgent extends URLClassLoader {
private final ClassLoader parent;
public ResourceJavaAgent(ClassLoader parent) throws MalformedURLException {
super(buildClassPath(), null);
this.parent = parent; // I need the parent as backup for SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
System.out.println("initializing url classloader");
}
private static URL[] buildClassPath() throws MalformedURLException {
final String JAVA_CLASS_PATH = "java.class.path";
final String EXTRA_CLASS_PATH = "extra.class.path";
List<String> paths = new LinkedList<>();
paths.addAll(Arrays.asList(System.getProperty(EXTRA_CLASS_PATH, "").split(File.pathSeparator)));
paths.addAll(Arrays.asList(System.getProperty(JAVA_CLASS_PATH, "").split(File.pathSeparator)));
URL[] urls = new URL[paths.size()];
for (int i = 0; i < paths.size(); i++) {
urls[i] = Paths.get(paths.get(i)).toUri().toURL(); // important only for resource url, really: this url must be absolute, to pass getClass().getResource("/users.properties").toURI()) with uri that isOpaque == false.
// System.out.println(urls[i]);
}
// this is for spawnVM functionality in tests
System.setProperty(JAVA_CLASS_PATH, System.getProperty(EXTRA_CLASS_PATH, "") + File.pathSeparator + System.getProperty(JAVA_CLASS_PATH));
return urls;
}
#Override
public Class<?> loadClass(String s) throws ClassNotFoundException {
try {
return super.loadClass(s);
} catch (ClassNotFoundException e) {
return parent.loadClass(s); // we search parent second, not first, as the default URLClassLoader would
}
}
private static void createRequestedDirs() {
for (String path : System.getProperty("extra.dirs", "").split(File.pathSeparator)) {
new File(path).mkdirs();
}
}
private static void createRequestedLinks() {
String linkPaths = System.getProperty("extra.link.path", null);
if (linkPaths == null) {
return;
}
for (String linkPath : linkPaths.split(",")) {
String[] fromTo = linkPath.split(":");
Path from = Paths.get(fromTo[0]);
Path to = Paths.get(fromTo[1]);
try {
Files.createSymbolicLink(from.toAbsolutePath(), to.toAbsolutePath());
} catch (IOException e) {
throw new IllegalArgumentException("Unable to create link " + linkPath, e);
}
}
}
public static void premain(String args, Instrumentation instrumentation) throws Exception {
createRequestedDirs();
createRequestedLinks();
}
}

If you could tell the tests where to write these files (in case target/classes and target/test-classes are hardcoded), and then turn the test run into a genrule, then you can specify the genrule's outputs as data for the production binary's *_binary rule.

I solved the first part, creating the directories. I still don't know how to add the latter two to classpath.
Starting from https://gerrit.googlesource.com/bazlets/+/master/tools/junit.bzl, I modified it to read
_OUTPUT = """import org.junit.runners.Suite;
import org.junit.runner.RunWith;
import org.junit.BeforeClass;
import java.io.File;
#RunWith(Suite.class)
#Suite.SuiteClasses({%s})
public class %s {
#BeforeClass
public static void setUp() throws Exception {
new File("./target").mkdir();
}
}
"""
_PREFIXES = ("org", "com", "edu")
# ...
I added the #BeforeClass setUp method.
I stored this as junit.bzl into third_party directory in my project.
Then in a BUILD file,
load("//third_party:junit.bzl", "junit_tests")
junit_tests(
name = "my_bundled_test",
srcs = glob(["src/test/java/**/*.java"]),
data = glob(["src/test/resources/**"]),
resources = glob(["src/test/resources/**"]),
tags = [
# ...
],
runtime_deps = [
# ...
],
],
deps = [
# ...
],
)
Now the test itself is wrapped with a setUp method which will create a directory for me. I am not deleting them afterwards, which is probably a sound idea to do.
The reason I need test resources in a directory (as opposed to in a jar file, which bazel gives by default) is that my test passes the URI to new FileInputStream(new File(uri)). If the file resides in a JAR, the URI will be file:/path/to/my.jar!/my.file and the rest of the test cannot work with such URI.

Calling java function from .py file in Pydev

I have a file.py as follows :
import unittest
from com.bahmanm import Greeter
class A(unittest.TestCase, Greeter):
def test_A(self):
self.greet("Bahman")
if __name__ == "__main__":
unittest.main()
In above case, Greeter is a java file as:
package com.bahmanm;
public class Greeter
{
private String msg;
public Greeter()
{
msg = "Hello, ";
}
public void greet(String name)
{
System.out.println(msg + name);
}
}
The code executes successfully, but I am not able to navigate from python code to java code (in PyDev) at line self.greet("Bahman") in the file.py code.
Though, I am able to view the contents of Greeter file from line
from com.bahmanm import Greeter. But unable to check the code flow at the function call.
I am using jython interpreter (grammar 2.5, default interpreter,jython.jar 2.5.3). I have also added the Java src path to PYTHONPATH in Eclipse. Also I have added Java project in the
PYTHON PROJECT->RIGHT CLICK->PROPERTIES-> PROJECT REFERENCES
Any suggestion regarding the above navigation will be of great help.