I have two simple interfaces called Processor and SeedPreProcessor and they are defined like:
Processor:
public interface Processor<I, O> {
Mono<O> process(I input);
}
SeedPreProcessor:
public interface SeedPreProcessor<D> extends Processor<D, D> {
/**
* Specify the location of this processor in the pipeline.
*
* #return the order
*/
Integer order();
String name();
}
and a PipeLine defined like:
public class PipeLine {
private final List<SeedPreProcessor<PreProcessorDocument>> allProcessors;
public PipeLine(List<SeedPreProcessor<PreProcessorDocument>> allProcessors) {
this.allProcessors = new ArrayList<>(allProcessors);
this.allProcessors.sort(comparingInt(SeedPreProcessor::order));
}
public Mono<PreProcessorDocument> execute(String url) {
log.info("Start processing URL = {}", url);
var initial = new PreProcessorDocument(url);
return Flux
.fromIterable(allProcessors)
.map(proc -> proc.process(initial).t) // my problem is here
}
}
I want to for a inital PreProcessorDocument to execute all the SeedPreProcessor in the
list allProcessors one by one.
How can I achieve this?
Simple foreach loop and flatMap operator will do the trick :
public Mono<PreProcessorDocument> execute(String url) {
var initial = new PreProcessorDocument(url);
Mono<PreProcessorDocument> m = Mono.just(initial);
for (SeedPreProcessor<PreProcessorDocument> p : allProcessors) {
m = m.flatMap(p::process);
}
return m;
}
Related
I have an object in use throughout my codebase, UnsecureObject. This object is auto-generated with no getters/setters, and all member fields are public. So editing is done by doing something like the following:
unsecureObjInstance.firstName = "Jane";
This is not desirable for numerous reasons that I probably don't have to explain here. But using this generated class is required for some other technical details with our messaging pipeline that I won't go into.
I have a desire is to leverage a mapping utility written by someone else on my team to convert this UnsecureObject to a pojo that I am writing.
An example of the mapper in action (with two normal classes w/ getters/setters) would be something like:
new MapperBuilder<>(PojoOne.class, PojoTwo.class)
.from(PojoOne::getName).to(PojoTwo::getFirstName)
.build();
This will map the PojoOne#name field to the PojoTwo#firstName field.
Is there a way to translate this to input my UnsecureObject here? I have tried something like the following:
new MapperBuilder<>(UnsecureObject.class, SecureObject.class)
.from(u -> u.firstName).to(SecureObject::getFirstName)
.build();
But get an error here, something along the lines of 'u -> u.firstName' could not be invoked.
So the question is:
Is there a way to essentially "construct" a getter on the fly using these public members? So in the .from() method, I can construct the call to look like a standard method that will yield my u.firstName?
Thanks for the help!
EDIT:
this is approx what the MapperBuilder class looks like (attempted to edit a bit to take away project specific wrappers/simplify)
/**
* This class is used to convert between POJO getter method references to the corresponding field names.
* #param <B> type
*/
public interface PojoProxy<B> {
/**
* Invokes the given getter method and returns information about the invocation.
* #param getter the getter to invoke
* #return information about the method invoked
*/
<T> GetterInvocation<T> invokeGetter(Function<B, T> getter);
}
/**
* Stores information about a method invocation.
* #param <T> method return type
*/
public interface GetterInvocation<T> {
public Class<T> getReturnType();
public String getFieldName();
}
/**
* A builder class to create {#link Mapper} instances.
* #param <FROM> source type
* #param <TO> target type
*/
public class MapperBuilder<FROM, TO> {
private final Class<FROM> _fromClass;
private final Class<TO> _toClass;
private final PojoProxy<FROM> _fromProxy;
private final PojoProxy<TO> _toProxy;
public MapperBuilder(Class<FROM> fromClass, Class<TO> toClass) {
_fromClass = fromClass;
_toClass = toClass;
//We will pretend there is an impl that provides the proxy.
//Proxies wrap the from and to classes in order to get reflection information about their getter calls.
_fromProxy = PojoProxy.of(fromClass);
_toProxy = PojoProxy.of(toClass);
}
public <FROM_VALUE> ToFieldBuilder<FROM_VALUE> from(Function<FROM, FROM_VALUE> getter) {
GetterInvocation<FROM_VALUE> methodInvocation = _fromProxy.invokeGetter(getter);
return new ToFieldBuilder<>(methodInvocation.getFieldName(), methodInvocation.getReturnType());
}
public class ToFieldBuilder<FROM_VALUE> {
private final String _fromFieldPath;
private final Class<FROM_VALUE> _fromClass;
public ToFieldBuilder(String fromFieldPath, Class<FROM_VALUE> fromClass) {
_fromFieldPath = fromFieldPath;
_fromClass = fromClass;
}
public <TO_VALUE> FromFieldBuilder<FROM_VALUE, TO_VALUE> to(Function<TO, TO_VALUE> getter) {
//similar to above, but now using a FromFieldBuilder.
}
}
public class FromFieldBuilder<FROM_VALUE, TO_VALUE> {
//impl..
}
}
I dont see MapperBuilder.from() method details, you can try this implementation of MapperBuilder.java Function (getter) -> (BiConsumer) setter
public class MapperBuilder<S, D> {
private final S src;
private final D dest;
public MapperBuilder(S src, Class<D> dest) {
this.src = src;
try {
this.dest = dest.newInstance();
} catch (Exception e) {
throw new RuntimeException("Required default constructor for: " + dest);
}
}
//getter - function to get value from source instance
//setter - biConsumer to set value to destination instance
//example - map(SrcClass::getSrcValue, DestClass::setDestValue)
public <V> MapperBuilder<S, D> map(Function<S, V> getter, BiConsumer<D, V> setter) {
setter.accept(dest, getter.apply(src));
return this;
}
public D build() {
return dest;
}
}
SrcClass.java some source class:
public class SrcClass {
private String srcValue;
public String getSrcValue() {
return srcValue;
}
public void setSrcValue(String srcValue) {
this.srcValue = srcValue;
}
}
DestClass.java some destination class:
package com.example.demo;
public class DestClass {
private String destValue;
public String getDestValue() {
return destValue;
}
public void setDestValue(String destValue) {
this.destValue = destValue;
}
}
DemoApplication.java demo:
public class DemoApplication {
public static void main(String[] args) {
SrcClass src = new SrcClass();
src.setSrcValue("someValue");
DestClass dest = new MapperBuilder<>(src, DestClass.class)
.map(SrcClass::getSrcValue, DestClass::setDestValue)
// map another fields
.build();
// for your UnsecureObject case
UnsecureObject unsecureObject = new MapperBuilder<>(src, UnsecureObject.class)
.map(SrcClass::getSrcValue,
(unsecure, srcValue) -> unsecure.unsecureValue = srcValue)
.build();
}
}
I am new to Reactor framework and trying to utilize it in one of our existing implementations. LocationProfileService and InventoryService both return a Mono and are to executed in parallel and have no dependency on each other (from the MainService). Within LocationProfileService - there are 4 queries issued and the last 2 queries have a dependency on the first query.
What is a better way to write this? I see the calls getting executed sequentially, while some of them should be executed in parallel. What is the right way to do it?
public class LocationProfileService {
static final Cache<String, String> customerIdCache //define Cache
#Override
public Mono<LocationProfileInfo> getProfileInfoByLocationAndCustomer(String customerId, String location) {
//These 2 are not interdependent and can be executed immediately
Mono<String> customerAccountMono = getCustomerArNumber(customerId,location) LocationNumber).subscribeOn(Schedulers.parallel()).switchIfEmpty(Mono.error(new CustomerNotFoundException(location, customerId))).log();
Mono<LocationProfile> locationProfileMono = Mono.fromFuture(//location query).subscribeOn(Schedulers.parallel()).log();
//Should block be called, or is there a better way to do ?
String custAccount = customerAccountMono.block(); // This is needed to execute and the value from this is needed for the next 2 calls
Mono<Customer> customerMono = Mono.fromFuture(//query uses custAccount from earlier step).subscribeOn(Schedulers.parallel()).log();
Mono<Result<LocationPricing>> locationPricingMono = Mono.fromFuture(//query uses custAccount from earlier step).subscribeOn(Schedulers.parallel()).log();
return Mono.zip(locationProfileMono,customerMono,locationPricingMono).flatMap(tuple -> {
LocationProfileInfo locationProfileInfo = new LocationProfileInfo();
//populate values from tuple
return Mono.just(locationProfileInfo);
});
}
private Mono<String> getCustomerAccount(String conversationId, String customerId, String location) {
return CacheMono.lookup((Map)customerIdCache.asMap(),customerId).onCacheMissResume(Mono.fromFuture(//query).subscribeOn(Schedulers.parallel()).map(x -> x.getAccountNumber()));
}
}
public class InventoryService {
#Override
public Mono<InventoryInfo> getInventoryInfo(String inventoryId) {
Mono<Inventory> inventoryMono = Mono.fromFuture(//inventory query).subscribeOn(Schedulers.parallel()).log();
Mono<List<InventorySale>> isMono = Mono.fromFuture(//inventory sale query).subscribeOn(Schedulers.parallel()).log();
return Mono.zip(inventoryMono,isMono).flatMap(tuple -> {
InventoryInfo inventoryInfo = new InventoryInfo();
//populate value from tuple
return Mono.just(inventoryInfo);
});
}
}
public class MainService {
#Autowired
LocationProfileService locationProfileService;
#Autowired
InventoryService inventoryService
public void mainService(String customerId, String location, String inventoryId) {
Mono<LocationProfileInfo> locationProfileMono = locationProfileService.getProfileInfoByLocationAndCustomer(....);
Mono<InventoryInfo> inventoryMono = inventoryService.getInventoryInfo(....);
//is using block fine or is there a better way to do?
Mono.zip(locationProfileMono,inventoryMono).subscribeOn(Schedulers.parallel()).block();
}
}
You don't need to block in order to get the pass that parameter your code is very close to the solution. I wrote the code using the class names that you provided. Just replace all the Mono.just(....) with the call to the correct service.
public Mono<LocationProfileInfo> getProfileInfoByLocationAndCustomer(String customerId, String location) {
Mono<String> customerAccountMono = Mono.just("customerAccount");
Mono<LocationProfile> locationProfileMono = Mono.just(new LocationProfile());
return Mono.zip(customerAccountMono, locationProfileMono)
.flatMap(tuple -> {
Mono<Customer> customerMono = Mono.just(new Customer(tuple.getT1()));
Mono<Result<LocationPricing>> result = Mono.just(new Result<LocationPricing>());
Mono<LocationProfile> locationProfile = Mono.just(tuple.getT2());
return Mono.zip(customerMono, result, locationProfile);
})
.map(LocationProfileInfo::new)
;
}
public static class LocationProfileInfo {
public LocationProfileInfo(Tuple3<Customer, Result<LocationPricing>, LocationProfile> tuple){
//do wathever
}
}
public static class LocationProfile {}
private static class Customer {
public Customer(String cutomerAccount) {
}
}
private static class Result<T> {}
private static class LocationPricing {}
Pleas remember that the first zip is not necessary. I re write it to mach your solution. But I would solve the problem a little bit differently. It would be clearer.
public Mono<LocationProfileInfo> getProfileInfoByLocationAndCustomer(String customerId, String location) {
return Mono.just("customerAccount") //call the service
.flatMap(customerAccount -> {
//declare the call to get the customer
Mono<Customer> customerMono = Mono.just(new Customer(customerAccount));
//declare the call to get the location pricing
Mono<Result<LocationPricing>> result = Mono.just(new Result<LocationPricing>());
//declare the call to get the location profile
Mono<LocationProfile> locationProfileMono = Mono.just(new LocationProfile());
//in the zip call all the services actually are executed
return Mono.zip(customerMono, result, locationProfileMono);
})
.map(LocationProfileInfo::new)
;
}
I'm Trying to read an Avro file and perform some operations on it, everything works fine but the aggregation functions, when I use them it get the below exception :
aggregating on field positions is only possible on tuple data types
then I change my class to implement Tuple4 (as I have 4 fields) but then when I want to collect the results get AvroTypeException Unknown Type : T0
Here are my data and job classes :
public class Nation{
public Integer N_NATIONKEY;
public String N_NAME;
public Integer N_REGIONKEY;
public String N_COMMENT;
public Integer getN_NATIONKEY() {
return N_NATIONKEY;
}
public void setN_NATIONKEY(Integer n_NATIONKEY) {
N_NATIONKEY = n_NATIONKEY;
}
public String getN_NAME() {
return N_NAME;
}
public void setN_NAME(String n_NAME) {
N_NAME = n_NAME;
}
public Integer getN_REGIONKEY() {
return N_REGIONKEY;
}
public void setN_REGIONKEY(Integer n_REGIONKEY) {
N_REGIONKEY = n_REGIONKEY;
}
public String getN_COMMENT() {
return N_COMMENT;
}
public void setN_COMMENT(String n_COMMENT) {
N_COMMENT = n_COMMENT;
}
public Nation() {
}
public static void main(String[] args) throws Exception {
Configuration parameters = new Configuration();
final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
Path path2 = new Path("/Users/violet/Desktop/nation.avro");
AvroInputFormat<Nation> format = new AvroInputFormat<Nation>(path2,Nation.class);
format.configure(parameters);
DataSet<Nation> nation = env.createInput(format);
nation.aggregate(Aggregations.SUM,0);
JobExecutionResult res = env.execute();
}
and here's the tuple class and the same code for the job as above:
public class NationTuple extends Tuple4<Integer,String,Integer,String> {
Integer N_NATIONKEY(){ return this.f0;}
String N_NAME(){return this.f1;}
Integer N_REGIONKEY(){ return this.f2;}
String N_COMMENT(){ return this.f3;}
}
I tried with this class and got the TypeException (Used NationTuple everywhere instead of Nation)
I don't think having your class implementing Tuple4 is right way to go. Instead you should add to your topology a MapFunction that converts your NationTuple to Tuple4.
static Tuple4<Integer, String, Integer, String> toTuple(Nation nation) {
return Tuple4.of(nation.N_NATIONKEY, ...);
}
And then in your topology call:
inputData.map(p -> toTuple(p)).returns(new TypeHint<Tuple4<Integer, String, Integer, String>(){});
The only subtle part is that you need to provide a type hint so flink can figure out what kind of tuple your function returns.
Another solution is to use field names instead of tuple field indices when doing your aggregation. For example:
groupBy("N_NATIONKEY", "N_REGIONKEY")
This is all explained here: https://ci.apache.org/projects/flink/flink-docs-stable/dev/api_concepts.html#specifying-keys
I have the following activity:
#ActivityRegistrationOptions(defaultTaskScheduleToStartTimeoutSeconds = 300, defaultTaskStartToCloseTimeoutSeconds = 10)
#Activities(version="1.0")
public interface MyActivities {
B first(A a) throws Exception;
void second(C c) throws Exception;
}
And I have the following workflow:
public class MyWorkflowImpl implements MyWorkflow {
#Autowired
private MyActivitiesClient operations;
#Override
public void start(SomeType input) {
A a = new A(...);
Promise<B> b = operations.first(a);
Promise<C> c = ...;
/* Here, I would like to create c based on b */
operations.second(c);
}
}
Now b is not available until the first operation finishes, but the workflow continues even if b is not available.
Any idea?
Use #Asynchronous annotation of a method:
public void start(SomeType input) {
A a = new A(...);
Promise<B> b = operations.first(a);
Promise<C> c = ...;
operations.second(c);
}
#Asynchronous
private Promise<C> calculateC(Promise<B> b) {
Settable<C> result = new Settable<C>();
/* Here create c based on b */
result.set(b.get()....);
return result;
}
A method annotated as #Asynchronous is internally converted to a callback that is invoked when all its parameters of type Promise are ready. #Asynchronous implementation relies on AspecJ so make sure to follow setup instructions carefully to enable it.
Another option is to use Task:
#Override
public void start(SomeType input) {
A a = new A(...);
Promise<B> b = operations.first(a);
Promise<C> c = new Settable<C>();
new Task(b) {
protected void doExecute() {
/* Here c based on b */
c.set(b.get() ....);
}
}
operations.second(c);
}
Task doExecute method is invoked when all parameters of type Promise passed to Task constructor are ready.
I have the following db table:
id method_id
1 1
1 2
1 3
and 2 classes:
EmailController and Smscontroller
in my code, I need to iterate over the table and according to the method_id (1 or 2) to invoke the send method of either EmailController or Smscontroller.
What is the recommended design pattern for it?
EDITED
There could be 100 methods! I put only 3. This is why I do not prefer the if else.
As well, the object that I send to EmailController send method is different than the one that I send to SmsController send method.
In EmailController I need to send User object.
In SmsController I need to send Manager object
I can't think of a design pattern. But for ultimate flexibility you can have a design similar to this:
public interface Sendable /* or Sender, SendingManager, etc. */ {
public int getId();
public void send();
}
public class EmailController implements Sendable {
}
public class SmsController implements Sendable {
}
public class Sendables {
private Map<Integer, Sendable> sendables = new HashMap<Integer, Sendable>();
public void addSendable(Sendable s) {
this.sendables.put(s.getId(), s);
}
public void sendById(Integer id) {
this.sendables.get(id).send();
}
}
Then you can use it like this:
Sendables sendables = new Sendables();
sendables.add(new EmailController());
sendables.add(new SmsController());
sendables.add(new ChatController());
// etc.
Row row = table.getRow(...); // let's assume this gets a row from your table
sendables.send(row.getId());
Another solution could be to have an extra table like this:
TABLE: CLASS_NAMES
method_id class_name
1 "com.foo.SmsController"
2 "com.foo.EmailController"
And then pass class_name to Class.forName and let it instantiate the appropriate controller for you to use.
EDIT: A reflection-based version of the code as suggested by Luis. Note that for production use you should ensure that the passed parameters are valid (not null, etc.) and also handle exceptions with rigor.
TABLE: CLASS_NAMES
method_id class_name param_class_name
1 "com.foo.SmsController" "com.foo.Manager"
2 "com.foo.EmailController" "com.foo.User"
SendManager
public class SendManager {
private static final String SEND_METHOD_NAME = "send";
/* DAO for the CLASS_NAMES tables */
private ClassNameDAO classNameDao;
/**
* Gets the row corresponding to methodId, for example
* (1, "com.foo.SmsController", "com.foo.Manager") then using reflection
* instantiates an instance of SmsController and invokes its send method
* with <code>param</code> passed to it.
*/
public void send(int methodId, Object param) throws Exception {
ClassNameRow classNameRow = classNameDao.findByMethodId(methodId);
String senderParameterClassName = className.senderParameterClassName();
Class paramClass = Class.forName(senderParameterClassName);
if (!paramClass.isInstance(param)) {
throw new IllegalArgumentException("methodId and param are not compatible");
}
String senderClassName = classNameRow.getSenderClassName();
Class senderClass = Class.forName(senderClassName);
/* Your sender classes must be JavaBeans and have no-arg constructors */
Object sender = senderClass.newInstance();
Class paramClass = Class.forName(senderParameterClassName);
Method send = senderClass.getMethod(SEND_METHOD_NAME, paramClass);
send.invoke(sender, param);
}
}
Sample Usage
SendManager sendManager = new SendManager();
Manager m = ...;
sendManager.send(1, m);
User u = ...;
sendManager.send(2, u);
How about this:
abstract class Controller {
public static Controller getInstance(int methodId) {
switch (methodId) {
case 1:
return new EmailController();
case 2:
return new SmsController();
default:
return null;
}
}
public abstract void send();
}
class EmailController extends Controller {
#Override
public void send() {
System.out.println("sending email");
}
}
class SmsController extends Controller {
#Override
public void send() {
System.out.println("sending sms");
}
}
And use it like this:
Controller.getInstance(methodId).send();
I'm using the Strategy pattern and the Factory Method pattern in my solution.
Strategy Pattern
http://johnlindquist.com/2010/08/25/patterncraft-strategy-pattern/
http://en.wikipedia.org/wiki/Strategy_pattern