I am using Flink v.1.4.0.
I have implemented a module, as part of a package I am developing, whose role is to deduplicate a stream. The module is quite simple:
public class RemoveDuplicateFilter<T> extends RichFlatMapFunction<T, T> {
static final ValueStateDescriptor<Boolean> SEEN_DESCRIPTOR = new ValueStateDescriptor<>("seen", Boolean.class);
private ValueState<Boolean> seen;
#Override
public void open(Configuration configuration) {
RuntimeContext runtimeContext = this.getRuntimeContext();
seen = runtimeContext.getState(SEEN_DESCRIPTOR);
}
#Override
public void flatMap(T value, Collector<T> out) throws Exception {
Boolean hasBeenSeen = seen.value();
if(hasBeenSeen == null || !hasBeenSeen) {
out.collect(value);
seen.update(true);
}
}
The question is: how do I test this code without having to instantiate an actual Flink ValueState? i.e. using Mockito?
I have tried a number of things but, essentially, when it comes down to calling:
RuntimeContext runtimeContext = Mockito.mock(RuntimeContext.class);
...
when(runtimeContext.getState(SEEN_DESCRIPTOR)).thenReturn(seen);
The call always fails. I have tried replacing the SEEN_DESCRIPTOR with Matchers.any() but still no luck.
Any suggestions?
You can use flinkspector to do unit-testing of functions.
Related
I am new to the DataStage world and I am trying to start the process() method by myself.
"Why do you want to do that?"
Sadly, I have not the hands on DataStage directly, I am "just" the Java developer in charge of creating the Java classes that will be used by DataStage. My goal is to know what my classes do exactly without the DataStage treatment in the picture to be able to debug and explain the process.
"What have you tried?"
My approach so far was to "simulate" the Configuration interface in order to call the validateConfiguration() method to have the InputLink set up before calling the process() method. But I don't know how to "simulate" a Configuration object with my data.
I have considered use a mock of my Configuration or of my InputLink objects but I don't know how to do that either.
It's time to show you some code:
First, the class that extends com.ibm.is.cc.javastage.api.Processor class:
public class DownloadExportOutputDatastage extends Processor {
#Override
public boolean validateConfiguration(Configuration configuration, boolean b) throws Exception
{
this.m_inputLink = configuration.getInputLink(0);
this.m_outputLink = configuration.getOutputLink(0);
return true;
}
#Override
public Capabilities getCapabilities() {
Capabilities capabilities = new Capabilities();
// ...
return capabilities;
}
#Override
public void process() throws Exception {
InputRecord inputRecord;
while ((inputRecord = this.m_inputLink.readRecord()) != null) { // How to use a custom inputLink with my data in it
DownloadExportOutput objDownloadExportOutput = new DownloadExportOutput();
objDownloadExportOutput.setRequestId((String) inputRecord.getValue("idrequest")); // The only value that I need in my Record
// My custom process here
OutputRecord outputRecord = this.m_outputLink.getOutputRecord();
outputRecord.setValue("body", soapContent);
this.m_outputLink.writeRecord(outputRecord);
}
}
}
Finally, what I have tried:
public class main {
public static void main(String[] args) throws Exception {
StandaloneConfiguration standaloneConfiguration = new StandaloneConfiguration(); // A custom implementation of the DataStage Configuration interface
DownloadExportOutputDatastage sample = new DownloadExportOutputDatastage(); // The class that you can see above
sample.validateConfiguration(standaloneConfiguration, true); // I try to load an InputLink but don't know what to put in it
sample.getCapabilities();
sample.process(); // Get a NPE because of my empty InputLink
}
}
I've read a lot of documentation, examples on the net, and still have no idea of how to do it. (if it could be done).
I need to test a method in a object called by other thread and it has a requirement of time. So my original idea was creating a spy, override the method and do the task, but the problem is, I can't access test method variables from the spy. So I can't get the value returned from System.currentTimeMillis() from the test method and proccess it to get the difference within times.
I will write some abstract code describing the situation so you could understand better the situation.
#Test
public void test()
{
Objectspied spy = Mockito.spy(new Objectspied(Parameters p));
long[] timesRetrieved = new long[numberOfCallsIExpect];
//here goes the tricky part
Mockito.doAnswer(new Answer<void>(){
#override
public methodCalledFromAnotherThread(int index)
{
//what i wish to do but can't do it
timesRetrieved[i] = System.currentTimeMilis();
}
}
).when(spy).methodCalledFromAnotherThread(anyInt);
//here initialize the thread who call it
//then proccess al time diferences and get sure they are what i've expected
}
public class AnotherThread
{
ObjectSpied reference;
public void run()
{
for (int i=0;i<repeat;i++)
{
reference.methodCalledFromAnotherThread(i);
Thread.sleep(1000);
}
}
}
I'm new to Mockito so i know syntax is totally wrong but i can fix that by myself, all i need to know is:
Is there any way to do it with Mockito?
If so, can you point me to the right direction?
You should be fine this way. Just make sure:
1) You mark the array as final
2) Properly implement the answer interface method
final long[] timesRetrieved = new long[size];
Mockito.doAnswer(new Answer<Void>() {
#Override
public Void answer(InvocationOnMock invocationOnMock) throws Throwable {
int index = invocationOnMock.getArgumentAt(0, Integer.class);
timesRetrieved[index] = System.currentTimeMillis();;
return null;
}
}
).when(spy).methodCalledFromAnotherThread(Mockito.anyInt());
I am writing test cases for this code.
But I am not able to write junit test cases for this. I am having problem to write junit test cases for this.
Please help to write mockito or junit test case for this.
public class BasicCacheManager implements CacheManager {
private ConcurrentHashMap<String, BasicCache> cacheMap;
private String defaultCache;
public BasicCacheManager() {
this.cacheMap = new ConcurrentHashMap<String, BasicCache>();
}
public BasicCacheManager(List<String> cacheNames) {
this.cacheMap = new ConcurrentHashMap<String, BasicCache>();
addCache(cacheNames);
}
#Override
public BasicCache getCache(){
return getCache(defaultCache);
}
#Override
public BasicCache getCache(String name) {
return cacheMap.get(name);
};
#Override
public void addCache(BasicCache cache) {
cacheMap.put(cache.getName(), cache);
};
#Override
public void removeCache(String name) {
cacheMap.remove(name);
};
#Override
public void removeAll() {
cacheMap = new ConcurrentHashMap<String, BasicCache>();
};
#Override
public boolean cacheExists(String name) {
return cacheMap.containsKey(name);
};
#Override
public void removeElement(String cacheName, String element){
BasicCache cache = getCache(cacheName);
if(cache != null){
cache.remove(element);
}
}
// added final to avoid any child classes from overriding this method
private final void addCache(List<String> cacheNames) {
for (String cacheName : cacheNames) {
BasicCache cache = new BasicCache(cacheName);
addCache(cache);
}
}
}
You don't need mocking here.
Your CUT (class under test) has very clear semantics; and interfaces that basically guide you into writing testcases. What you have to do: create objects of your cache class; and then use its methods to modify it; and asserts to verify.
Example:
#Test
public void testNewCacheIsEmpty() {
BasicCache underTest = new BasicCache();
assertThat(underTest.isEmpty(), is(true));
}
(in case you dont have an isEmpty() yet, you could add that one as package-protected method; just to allow for such tests).
Then you write other tests; for example that make sure that adding a certain cache-key ... actually does what you expect that to do.
So, the primary thing to understand here: you should design your whole cache so that you can test it ... without knowing what exactly it is doing.
Of course, you could create a mocked ConcurrentHashMap object and pass that into your class; to verify that exactly those calls that you expect for some operation to take place really happen. But that means: testing implementation details. And you want to avoid that if possible.
And just for the record: your field defaultCache doesn't make any sense in the code you are showing. It is null initially, there is no setter for it; so it has absolutely no purpose.
I am writing a JUnit test for code submitted to a competition. The rules of the competition require that certain methods not be called from other methods. (I unfortunately can not change the rules.)
The contestants are all implementing an interface we supplied which includes an add(K key, V value) method and a delete(K key) method. We need to test that entries do not implement delete by adding every other element to a new object and return that object.
We are also trying to avoid adding dependencies outside of the Java core since we are using a lot of automated tools (like the Marmoset Project) to test the hundreds of submissions.
I read through the documentation for Java Reflection and Instrumentation and nothing jumped out at me.
We are using Java 8 if it makes a difference.
AspectJ compile time weaving will probably be your best bet.
You will need to recompile the code with aspectj compiler and add advice to intercept the call.
If you give me more details I can show some example code.
You probably want a mocking library, and to use a "spy" test object. Using Mockito it might look something like this.
eg.
import static org.mockito.Mockito.*;
public class Test {
#Spy
ClassUnderTest classUnderTest;
#Before
public void init() {
MockitoAnnotations.initMocks(this);
}
#Test
public void deleteNeverCalled() {
// given
String key = randomString();
String value = randomString();
// when
classUnderTest.add(key, value)
// then
verify(classUnderTest, never()).delete(any());
}
}
This was my solution in the end. It looks like in the original question, I did not mention that this was a binary tree, so the compareTo function would be used constantly.
I created an Exception we could throw in our test framework and then detect.
public static class NotAllowedException extends RuntimeException
I created a new type that would have a flag that could be set to true by the testing framework before calling delete.
/**
* This class uses reflection to check whether {#link compareTo()} is being
* called inside the add method after the test decides it is done with the
* add method.
* It will throw a {#link NotAllowedException}.
*
* #author yakatz <email#domain.com>
*/
private class MyIntWrapper {
private boolean doneAdding = false;
public void doneAdding() {
this.doneAdding(true);
}
public void doneAdding(boolean b) {
this.doneAdding = b;
}
private class MyInteger implements Comparable<MyInteger> {
private Integer value;
public MyInteger(int value) {
this.value = value;
}
#Override
public int compareTo(MyInteger o) {
if (MyIntWrapper.this.doneAdding) {
StackTraceElement[] causes = Thread.currentThread().getStackTrace();
for (StackTraceElement cause : causes) {
if (cause.getClassName().equals("tree.Node") && cause.getMethodName().equals("add")) {
throw new NotAllowedException();
}
}
}
return this.value.compareTo(o.value);
}
}
}
I can then use the class in tests like this:
MyIntWrapper mir = new MyIntWrapper();
Tree<MyIntWrapper.MyInteger, String> tree = new Tree();
// Add stuff to the tree
mir.doneAdding();
MyIntWrapper.MyInteger mi = mir.new MyInteger(1);
tree = tree.delete(mi); // Will throw NotAllowedException if add() is called
I need to test handleIn() method using Mockito.
However the code need to call this legacy code Util.getContextPDO which is a static method.
Note that in testing environment this Util.getContextPDO is always returns Exception, and I intend to bypass this Util.getContextPDO() by always return a dummy IPDO.
public class MyClass {
public IPDO getIPDO()
{
return Util.getContextPDO(); // note that Util.getContextPDO() is a static, not mockable.
}
public String handleIn(Object input) throws Throwable
{
String result = "";
IPDO pdo = getIPDO();
// some important business logic.
return result;
}
}
Initially I thought this achieveable by using spy() of the class "MyClass", so I can mock the return value of getIPDO(). Below is my initial effort using spy ()
#Test
public void testHandleIn() throws Exception
{
IPDO pdo = new PDODummy();
MyClass handler = new MyClass ();
MyClass handler2 = spy(handler);
when(handler2.getIPDO()).thenReturn(pdo);
PDOUtil.setPDO(pdo, LogicalFieldEnum.P_TX_CTGY, "test123");
IPDO pdoNew = handler2.getIPDO();
Assert.assertEquals("test123,(PDOUtil.getValueAsString(pdoNew, LogicalFieldEnum.P_TX_CTGY)));
}
However the when(handler2.getIPDO()).thenReturn(pdo); is throwing the Exception that I want to avoid ( because handler2.getIPDO() ) seems to call the real method.
Any idea on how to test this part of code?
A good technique for getting rid of static calls on 3rd party API is hiding the static call behind an interface.
Let's say you make this interface :
interface IPDOFacade {
IPDO getContextPDO();
}
and have a default implementation that simply calls the static method on the 3rd party API :
class IPDOFacadeImpl implements IPDOFacade {
#Override
public IPDO getContextPDO() {
return Util.getContextPDO();
}
}
Then it is simply a matter of injecting a dependency on the interface into MyClass and using the interface, rather than the 3rd party API directly :
public class MyClass {
private final IPDOFacade ipdoFacade;
public MyClass(IPDOFacade ipdoFacade) {
this.ipdoFacade = ipdoFacade;
}
private IPDO getIPDO() {
return ipdoFacade.getContextPDO();
}
public String handleIn(Object input) throws Throwable
{
String result = "";
IPDO pdo = getIPDO();
someImportantBusinessLogic(pdo);
return result;
}
...
}
In your unit test, you can then easily mock your own interface, stub it any way you like and inject it into the unit under test.
This
avoids the need to make private methods package private.
makes your tests more readable by avoiding partial mocking.
applies inversion of control.
decouples your application from a specific 3rd party library.
Changed my testing to :
#Test
public void testHandleIn() throws Exception
{
IPDO pdo = new PDODummy();
MyClass handler = new MyClass ();
MyClass handler2 = spy(handler);
doReturn(pdo ).when( handler2 ).getIPDO();
PDOUtil.setPDO(pdo, LogicalFieldEnum.P_TX_CTGY, "test123");
IPDO pdoNew = handler2.getIPDO();
Assert.assertEquals("test123,(PDOUtil.getValueAsString(pdoNew, LogicalFieldEnum.P_TX_CTGY)));
}
Solved after reading Effective Mockito.
when(handler2.getIPDO()).thenReturn(pdo);
Will actually call the method and then return pdo regardless.
Whereas:
doReturn(pdo).when(handler2).getIPDO();
Will return pdo without calling the getIPDO() method.