I have a java code where I am calling a scala object's method (of a dependent library). For writing unit test I want to mock the scala object's method invocation
Operations.scala
object Operations {
def load(spark: SparkSession, path: String): Dataset[Row] = {
}
}
Main.java
public class Main {
public void callMethod() {
Dataset<Row> df = Operations.load(sparkSession, path);
}
}
I want to write a Unit test for callMethod() and hence mock the Operations.load scala call. I did not find a way to do it.
Any help will be appreciated.
If you are willing to change the signature of your callMethod method, one possible solution is to use interfaces and dependency injection.
Create an interface which gives you a Dataset[Row]:
trait ObtainDataSet {
def obtain: DataSet[Row]
}
And then have your callMethod have a parameter that takes that interface:
public void callMethod(ObtainDataSet obtainDataSet) {
Dataset<Row> df = obtainDataSet.obtain;
}
Then on production code you inject the real implementation using the Operations.load method:
class ObtainDataSetImpl(spark: SparkSession, path: String) extends ObtainDataSet {
override def obtain: Dataset[Row] = Operations.load(sparkSession, path)
}
But you can easily swap that production implementation for something ad-hoc on the tests. You don't even need to use mocking frameworks:
class MyMock extends ObtainDataSet {
override def obtain: DataSet[Row] = ??? // something useful for tests.
}
But if you want, ScalaMock is normally used
Here is how this is usually done ... I'll write it in scala, because java syntax is soooo much more verbose (and frankly, I don't remember it very well), but it is the same idea.
trait Operations {
def load(s: SparkSession, path: String): DataSet[Row]
}
object Operations extends Operations {
def load(s: SparkSession, path: String) = ??? // implementation here
}
class Main(operations: Operations = Operations) {
callMethod(): Unit = {
val df = operations.load(sparkSession, path);
...
// do stuff
}
}
and then in test, you write (this is with Mockito/scalatest but, again, same idea with whatever library you use for testing):
val ops = mock[Operations]
when(ops.load(any, any)).thenReturn(whatever)
new Main(ops).callMethod(mock[SparkSession], "/foo/bar")
This approach is called "dependency injection": Your Main class has a "dependency" on Operations, that is now "injected" at run time rather than than being hardcoded as in your snippet. This allows you to use different dependency instances for production (actual Operations object) and test (the mock) runs.
Also, this is a separate topic, but I thought it is still worth mentioning: Unit (void) return type for the method you are testing doesn't seem like the best idea, and will likely cause you more problems testing it down the road (how do you make sure the method actually produced whatever side effects you are expecting from it?).
It would be better to make this method return the actual result, and have the caller take care of the side effects. Then you could easily test that it produces the result you expect:
new Main(ops).callMethod(mock[SparkSession], "/foo/bar") shouldBe "ExpectedFooResult"
Related
I'm new to annotation processing and code generation. I want to find out how can I perform such operation like appending new method to existing class. Here is an example of what I want to do:
Assume that we have a class with with custom annotations like this one:
class SourceClass {
#CustomAnnotation
fun annotatedFun1(vararg argument: Any) {
//Do something
}
#CustomAnnotation
fun annotatedFun2(vararg argument: Any) {
//Do something
}
fun someOtherFun() {
//Do something
}
}
And the result I want to get - extended copy of that class:
class ResultClass {
fun hasFunWithName(name: String): Boolean {
return (name in arrayOf("annotatedFun1", "annotatedFun2"))
}
fun callFunByName(name: String, vararg arguments: Any) {
when (name) {
"annotatedFun1" -> annotatedFun1(*arguments)
"annotatedFun2" -> annotatedFun2(*arguments)
}
}
fun annotatedFun1(vararg argument: Any) {
//Do something
}
fun annotatedFun2(vararg argument: Any) {
//Do something
}
fun someOtherFun() {
//Do something
}
}
I've already found out how to create annotation processor. I'm looking for a method to save all existing fields, properties and methods in source class and to append a few more methods to it.
If it is possible to modify class without creating new one - it would be perfect, but in all tutorials only new classes are created and I didn't find any example where all contents of source class are being copied to another one.
Please, do not advise to use reflection. I need this for android and so reflection is not the option cause of resources cost. I'm looking for compile-time solution.
It is required for custom script language implemented in app and should be used to simplify wrapper classes structure. When this job is done directly in code - it looks awful when such method count exceeds 20 per class.
Here is a good example of Java Annotation Processing I recently worked with.
It's an implementation of #Immutable annotation.
Check out ByteBuddy or Kotlin Poet to understand how additional code generation works.
For Kotlin you do almost the same, check this manual for Kotlin-specific steps.
With Kotlin, you can use extension functions and that is the recommended way of adding new functionality to existing classes that you don't control. https://kotlinlang.org/docs/reference/extensions.html
You may be abel to follow the pattern used by Project Lombok. See How does lombok work? or the source code for details.
Another option would be to write a new class that extends your source class:
class ResultClass : SourceClass {
fun hasFunWithName(name: String): Boolean {
return (name in arrayOf("annotatedFun1", "annotatedFun2"))
}
fun callFunByName(name: String, vararg arguments: Any) {
when (name) {
"annotatedFun1" -> annotatedFun1(*arguments)
"annotatedFun2" -> annotatedFun2(*arguments)
}
}
}
Or perhaps use composition instead and implemnent cover methods for all the public methods in SourceClass.
If you are not tied to doing this using annotation processing, you could use a separate piece of custom code to process the source code files before compiling. Maybe use a regular expression like /#CustomAnnotation\s+.*fun (\w+)\s*\(([^)]*)\)/gm (Test on Regex101) to find the annotated methods.
If I understood the requirement correctly, the goal is to implement something like described below.
You have a source file C.java that defines the class C like this:
public final class C
{
#Getter
#Setter
private int m_IntValue;
#Getter
#Constructor
private final String m_Text;
}
And now you want to know how to write an annotation processor that jumps in during compilation and modifies the source from C.java that the compiler sees to something like this:
public final class C
{
private int m_IntValue;
public final int getIntValue() { return m_IntValue; }
public final void setIntValue( final int intValue ) { m_IntValue = intValue; }
private final String m_Text;
public final String getText() { return m_Text; }
public C( final String text ) { m_Text = text; }
}
The bad news is, that this is not possible … not with an annotation processor, not for Java 15.
For Java 8 there was a way, using some internal classes with reflection to convince the AP to manipulate the already loaded source code in some way and let the compiler compile it a second time. Unfortunately, it failed more often than it worked …
Currently, an annotation processor can only create a new (in the sense of additional) source file. So one solution could be to extend the class (of course, that would not work for the sample class C above, because the class itself is final and all the attributes are private …
So writing a pre-processor would be another solution; you do not have a file C.java on your hard drive, but one named C.myjava that will be used by that preprocessor to generate C.java, and that in turn is used by the compiler. But that is not done by an annotation processor, but it may be possible to abuse it in that way.
You can also play around with the byte code that was generated by the compiler and add the missing (or additional) functionality there. But that would be really far away from annotation processing …
As a summary: today (as of Java 15), an annotation processor does not allow the manipulation of existing source code (you cannot even exclude some source from being compiled); you can only generate additional source files with an annotation processor.
Can somebody tell me if this is a bug or intended behavior.
I know in Spock I can test private methods:
def "test with private"() {
given:
FileContentValidator fileContentValidator = new FileContentValidator(1)
when:
fileContentValidator.validateCustomerSiteId("") // this is a private method
then:
true // succeeds
}
But when I try the same thing using a Spock Spy, it fails:
def "test with private on spy"() {
given:
FileContentValidator fileContentValidator = Spy(FileContentValidator, constructorArgs: [1])
when:
fileContentValidator.validateCustomerSiteId("") // this is a private method
then:
true // does not get here
}
I get an exception:
groovy.lang.MissingMethodException: No signature of method: com.shoppertrak.device.management.web.validator.ophour.FileContentValidator$$EnhancerByCGLIB$$7ff6a42.validateCustomerSiteId() is applicable for argument types: (java.lang.String) values: []
I think this is due to how the cglib works. When testing an existing concrete class, Spock doesn't get involved in the byte code, so you're taking advantage of a flaw in Groovy that give you access to private methods. When you spy or mock the same class, the Spock/cglib manipulation steps in and changes the resulting byte code. The end product is a method that is truly private, thus you can't access it.
There are probably hacks you can use to get around it, but you're probably better off adding something like a CustomerSiteIdValidator class with a public validateCustomerSiteId() that gets injected into your FileContentValidator class. Then you can easily mock it and isolate responsibilities.
Someone suggested adding the ability to Spy private methods, but the ticket was closed as Won't Fix
https://github.com/spockframework/spock/issues/403
Trying to broaden the appeal of Spock at work and run into this issue. Actually trying to write Unit Tests for a Groovy class, but one that calls out to Java. A static method calls a private constructor. The code looks like:
private MyConfigurator(String zkConnectionString){
solrZkClient = new SolrZkClient(zkConnectionString, 30000, 30000,
new OnReconnect() {
#Override
public void command() { . . . }
});
}
"SolrZkClient" is from third party (Apache) Java library. Since it tries to connect to ZooKeeper, I would like to mock that out for this Unit Test (rather than running one internally as part of the unit test).
My test gets to the constructor without difficulty, but I can't get past that ctor:
def 'my test'() {
when:
MyConfigurator.staticMethodName('hostName:2181')
then:
// assertions
}
Is there anyway to do this?
Since the class under test is written in Groovy, you should be able to mock the constructor call by way of a global Groovy Mock/Stub/Spy (see Mocking Constructors in the Spock Reference Documentation). However, a better solution is to decouple the implementation of the MyConfigurator class, in order to make it more testable. For example, you could add a second constructor and/or static method that allows to pass an instance of SolrZkClient (or a base interface, if there is one). Then you can easily pass in a mock.
You can use GroovySpy for mocking constructors in Spock
For example:
def 'my test'() {
given:
def solrZkClient = GroovySpy(SolrZkClient.class,global: true);
when:
MyConfigurator.staticMethodName('hostName:2181')
then:
// assertions
}
def anySubscriber = GroovySpy(RealSubscriber, global: true)
1 * new RealSubscriber("Fred")
put that into def setup(){ } block or inside of given: label block
I have an application which should produce cars and operate with them. Car object creation is a complex process so I need a factory for each type of car. Also I want users to be able to provide their own type of cars and factories which produce them. These car types and factories should be plugged to my application as jars (probably there is a better way than jars but I don't see it).
I've come to an idea of making a common CarFactory which accepts the name of the car ("mercedes", "bmw", "nissan", etc) as an argument. CarFactory has a map where each name is mapped to its own factory class. The code looks something like this (sorry I can't provide a working copy because I'm still evaluating it and don't have a version which compiles without errors)
import scala.collection.mutable.Map
class CarFactory {
var knownCarTypes = Map[String, Class[Factory]]()
def create(carType: String) = knownCarTypes.get(carType) match {
case Some(factoryClass) => Some(factoryClass.getMethod("create").invoke(null).asInstanceOf[Car])
case None => None
}
}
}
The knownCarTypes is mutable because I want user factories to register on this map providing what type of car they are responsible for and what is the name of the factory class. So from a user class it looks like this
class Mercedes extends Car
object MercedesFactory extends Factory {
def register() {
CarFactory.knownCarTypes("mercedes") = getClass
}
def create() = new Mercedes()
}
And here is my question. I don't know how to trigger the register() method of a user factory. Is it possible? Is there a better solution than my approach?
I thought about making common trait for factories, find all loaded classes implementing the trait and trigger method via reflection. But it looks quite complex. I hope some design pattern or OOP trick can be used here. What do you think?
Thanks!
If I understand your question correctly, all you have to do is call register from the object's "body":
object MercedesFactory extends Factory {
def register() {
CarFactory.knownCarTypes("mercedes") = getClass
}
register
def create() = new Mercedes()
}
Finally I got it working via reflection. I iterated over all jars on specified path, found all classes implementing my com.example.Factory trait and triggered their register() method. For jars inspection I used Clapper ClassFinder and for invoking object method I followed Thomas Jung advice. Here is the final code
import org.apache.commons.io.FileUtils
import org.clapper.classutil.{ClassFinder, ClassInfo}
import scala.collection.JavaConverters._
def triggerFactories() {
val jars = FileUtils.iterateFiles(new File("lib"), Array[String]("jar"), true).asScala.toList
val classes = ClassFinder(jars).getClasses
val factories = ClassFinder.concreteSubclasses("com.example.Factory", classes)
factories.foreach { (factory: ClassInfo) =>
companion[Factory](factory.name).register()
}
}
def companion[T](name: String)(implicit man: Manifest[T]): T =
Class.forName(name).getField("MODULE$").get(man.erasure).asInstanceOf[T]
It's worked for me. It looks tricky but I hope it won't break anything in my application in future. Please post if there is better approach, I'll reaccept the answer.
I am writing client code in Scala that needs to interface with a framework in Java. The framework is responsible for creating object instances of classes specified via an API, which it does using reflection. For example:
public class ReflectionUtil {
public static <T> T newInstance(Class<T> aClass) {
T result;
try {
Constructor<T> meth = aClass.getDeclaredConstructor(new Class[]{});
meth.setAccessible(true);
result = meth.newInstance();
} catch (Exception e) {
throw new RuntimeException(e);
}
return result;
}
}
The classes of the object instances I want to create are implemented in Scala and are paramerterised on a type that has a context bound on it. For example:
class OrderedValue[A](var value: A)(implicit ord: Ordering[A]) {
def get: A = value
def set(x: A) = { value = x }
def cmp(that: OrderedValue[A]): Int = ord.compare(this.value, that.value)
}
I run into a problem when I pass this class to the Java framework to construct new instances as the framework makes the assumption that the class will have a zero-argument constructor available. For example, the following code will result in a NoSuchMethodException from within newInstance:
def main(args: Array[String]) {
val a: OrderedValue[Int] = ReflectionUtil.newInstance(classOf[OrderedValue[Int]])
val b: OrderedValue[Int] = ReflectionUtil.newInstance(classOf[OrderedValue[Int]])
a.set(3)
b.set(5)
println(a.cmp(b))
}
An attempt at resolving this issue is to add a zero-argument constructor to OrderedValue however there is no reasonable value for the implicit parameter ord. Setting it to null will result in a NullPointerException within cmp:
def this() = this(null.asInstanceOf[A])(null.asInstanceOf[Ordering[A]])
Another approach is to subclass a particular concrete value of OrderedValue. For example:
class OrderedIntValue(val v: Int) extends OrderedValue[Int](v) {
def this() = this(null.asInstanceOf[Int])
}
val a: OrderedValue[Int] = ReflectionUtil.newInstance(classOf[OrderedValue[Int]])
This will work but is not ideal as it is not always convenient or possible to know the concrete type of OrderedValue. For example, newInstance may be called within a scope that is also parameterised on a type (i.e. we don't know that it's specifically an Int).
So my question is: given that context bounds (i.e. type classes) are a very useful, and now commonly used, feature within Scala, and given I can not change the internals of the Java framework that I am interfacing with, has anyone encountered or developed an approach that can make this all work?
Implicit arguments are filled in by the Scala compiler at the compile time. If you want to instantiate classes using reflection you will have to specify those arguments manually. There is just no way around it. So you can either have context bounds or no-argument constructors.
I found that the easiest way of calling Scala code from Java is to write an intermediate Scala layer with the Scala equivalent of POJOs. No implicits, no closures in signatures, no Companion object, no complicated type inference, etc. (of course, you can use these internally). I also try to replace most scala Collections types in signatures with java.util Collections.
Yes, it is ugly, tedious and not very flexible, but at least it removes a lot of syntax nightmare from the Java side.