I have a sink which extends from RichSinkFunction that is caching some information.
When my process is finished, I want to update all that information that is cached so I want to force its invocation.
I can call that sink from an KeyedProcessAccumulatorFunction, calling it with the context of type ReadOnlyContext and it works.
public class PageAccumulateFunction implements KeyedProcessAccumulatorFunction{
public SessionAccumulator accumulate(
#NonNull Tuple2<CollectionMessage, PropertyInfo> value,
#NonNull SessionAccumulator accumulator,
#NonNull KeyedBroadcastProcessFunction.ReadOnlyContext ctx) {
....
ctx.output(outputTag, message);
}
}
But in my RichMapFunction class, I can't invoke that sink. I can get a RuntimeContext object (but not a ReadOnlyContext), but I don't know if I can use it to invoke the RichSinkFunction sink.
public class SessionMapper extends RichMapFunction<SessionAccumulator, GenericRecord>{
public GenericRecord map(SessionAccumulator sessionAccumulator) {
....
RuntimeContext ctx = getRuntimeContext();
....
}
}
Any idea?
Only process functions can use side outputs (which you write to via ctx.output).
A MapFunction automatically sends the return value of its map method downstream (toward the sink). It works this way because a map is a one-to-one mapping from inputs to outputs. Most other function types (e.g., process functions, flatmaps) are passed a Collector you can use to send events downstream.
Related
I'm trying to understand what is the right way of implementing post_construct methods in Spring Webflux.
On startup of an application I need to read data from the DB (I have an R2dbcRepository configured), and then perform some logic and save result as Bean's fields).
So I have a findAll() method returning Flux. How should it be done?
I tried using .block(), AtomicBoolean flag, none of these worked
First of all, never use block() method. Use it for tests at most, but there is a better solution out there than StepVerifier. (If you use Kotlin there are await prefixed methods that work like block but not blocking.)
If you need data at launch, that says it is bad design to me because if there is no user, what do you do with it? I think it's illogical. What happens when you use query when you need it, add to cache and reuse it when you need it again. In the case of WebFlux, you can prepare a Mono object that uses a query from the database and use .cache() end of chain. So Spring Bean can contain this Mono object that will be run when you subscribe.
Ofc below example, repo.find will never call if function of Service won't run.
https://projectreactor.io/docs/core/release/api/reactor/core/publisher/Mono.html#cache--
#Configuration
public class Config {
private R2dbcRepository repo;
public Config(R2dbcRepository repo) {
this.repo = repo;
}
#Bean
public Mono<Data> myCachedDbData() {
return repo.find(...)
.map(it -> new Data(it))
.cache()
}
}
#Service
public class Service {
private Mono<Data> data;
public Config(Mono<Data> data) {
this.data = data;
}
public Object function() {
return data.flatMap(...)
}
}
I'm a new for a project reactor, but i have task to send some information from classic spring rest controller to some service, which is interacts with different system. Whole project developed with project reactor.
Here is my rest controller:
#RestController
public class Controller {
#Autowired
Service service;
#PostMapping("/path")
public Mono<String> test(#RequestHeader Map<String, String> headers) throws Exception {
testService.saveHeader(headers.get("header"));
return service.getData();
}
And here is my service:
#Service
public class Service {
private Mono<String> monoHeader;
private InteractionService interactor;
public Mono<String> getData() {
return Mono.fromSupplier(() -> interactor.interact(monoHeader.block()));
}
public void saveHeader(String header) {
String key = "header";
monoHeader = Mono.just("")
.flatMap( s -> Mono.subscriberContext()
.map( ctx -> s + ctx.get(key)))
.subscriberContext(ctx -> ctx.put(key, header));
}
Is it acceptable solution?
Fisrt off, I don't think you need the Context here. It is useful to implicitly pass data to a Flux or a Mono that you don't create (eg. one that a database driver creates for you). But here you're in charge of creating the Mono<String>.
Does the service saveHeader really achieve something? The call seem transient in nature: you always immediately call the interactor with the last saved header. (there could be a side effect there where two parallel calls to your endpoint end up overwriting each other's headers).
If you really want to store the headers, you could add a list or map in your service, but the most logical path would be to add the header as a parameter of getData().
This eliminates monoHeader field and saveHeader method.
Then getData itself: you don't need to ever block() on a Mono if you aim at returning a Mono. Adding an input parameter would allow you to rewrite the method as:
public Mono<String> getData(String header) {
return Mono.fromSupplier(() -> interactor.interact(header));
}
Last but not least, blocking.
The interactor seems to be an external service or library that is not reactive in nature. If the operation involves some latency (which it probably does) or blocks for more than a few milliseconds, then it should run on a separate thread.
Mono.fromSupplier runs in whatever thread is subscribing to it. In this case, Spring WebFlux will subscribe to it, and it will run in the Netty eventloop thread. If you block that thread, it means no other request can be serviced in the whole application!
So you want to execute the interactor in a dedicated thread, which you can do by using subscribeOn(Schedulers.boundedElastic()).
All in all:
#RestController
public class Controller {
#Autowired
Service service;
#PostMapping("/path")
public Mono<String> test(#RequestHeader Map<String, String> headers) throws Exception {
return service.getData(headers.get("header"));
}
}
#Service
public class Service {
private InteractionService interactor;
public Mono<String> getData(String header) {
return Mono.fromSupplier(() -> interactor.interact(header))
.subscribeOn(Schedulers.boundedElastic());
}
}
How to transfer data via reactor's subscriber context?
Is it acceptable solution?
No.
Your code of saveHeader() method is an equivalent of simple
public void saveHeader(String header) {
monoHeader = Mono.just(header);
}
A subscriberContext is needed if you consume the value elsewhere - if the mono is constructed elsewhere. In your case (where you have all code before your eyes in the same method) just use the actual value.
BTW, there are many ways to implement your getData() method.
One is as suggested by Simon Baslé to get rid of a separate saveHeader() method.
One other way, if you have to keep your monoHeader field, could be
public Mono<String> getData() {
return monoHeader.publishOn(Schedulers.boundedElastic())
.map(header -> interactor.interact(header));
}
I am working on an spring 2.0.1.RELEASE application.
Brief of Application:
1. I have separate Transformer beans that transforms my DTO to Domain
and vice versa.
2. I have separate Validator beans that validate my domain object being passed.
3. I have Service classes that takes care of the applying rules and calling persistence layer.
Now, i want to build a Workflow in my application:
where i will just call the start of the workflow and below mentioned steps will be executed in order and exception handling will be done as per the step:
1.First-Transformtion - transformToDomain() method will be called for that object type.
2.Second-Validator - class valid() method will be called for that object.
3.Third-Service - class save() method will be called for that object.
4.Fourth- Transformation - transformToDTO() method will be called for that object type.
after this my workflow ends and i will return the DTO object as response of my REST API.
Exception handling part is the one, i also want to take care of, like if particular exception handler exist for that step then call it, else call global exception handler.
I designed some prototype of same, but looking for some expert advice and how this can be achieved with a better design in java.
Explanation with example considering above use case is highly appreciable.
I'm not so sure if what you are describing is a workflow system in its true sense, perhaps a Chain of Responsibility is more of what you are talking about?
Following what you described as a sequence of execution, here is a simplified example of how I would implement the chain:
Transformer.java
public interface Transformer<IN, OUT> {
OUT transformToDomain(IN dto);
IN transformToDTO(OUT domainObject);
}
Validator.java
public interface Validator<T> {
boolean isValid(T object);
}
Service.java
public interface Service {
void save(Object object);
}
And the implementation that binds everything:
ProcessChain.java
public class ProcessChain {
private Transformer transformer;
private Service service;
private Validator validator;
Object process(Object dto) throws MyValidationException {
Object domainObject = transformer.transformToDomain(dto);
boolean isValid = validator.isValid(domainObject);
if(!isValid){
throw new MyValidationException("Validation message here");
}
service.save(domainObject);
return transformer.transformToDTO(domainObject);
}
}
I haven't specified any Spring related things here because your question seems to be a design question rather than a technology questions.
Hope this helps
Brief of what i implemented in a way with not much hustle:
This is how I created flow of handlers:
Stream.<Supplier<RequestHandler>>of(
TransformToDomainRequestHandler::new,
ValidateRequestHandler::new,
PersistenceHandler::new,
TransformToDTORequestHandler::new)
.sequential()
.map(c -> c.get()) /* Create the handler instance */
.reduce((processed, unProcessed) -> { /* chains all handlers together */
RequestHandler previous = processed;
RequestHandler target = previous.getNextRequestHandler();
while (target != null && previous != null) {
previous = target;
target = target.getNextRequestHandler();
}
previous.setNextRequestHandler(unProcessed);
return processed;
}).get();
This is my Request Handler which all other handler extends
I am using java akka framework. Some of my actors will do same thing in onreceive method. Is there a way to avoid this and get them to a common location. (May be using inheritance)
Problem in detail is like below.
In some actors it send an ack message based on a condition. This ack
is send to and external system by another actor.
This is duplicated in several classes.
Because an actor's Receive block is a partial function, you can chain Receive blocks with orElse. This is helpful for sharing behavior among actors and is described in the Scala version of the documentation, but you can do the same thing in Java.
The following is a simple example that uses inheritance, in which the common acking behavior is defined in a base class. However, you don't have to use inheritance and can define Receive blocks wherever you want to:
import akka.actor.AbstractActor;
import akka.event.Logging;
import akka.event.LoggingAdapter;
public class AckingActor extends AbstractActor {
private final LoggingAdapter log = Logging.getLogger(getContext().getSystem(), this);
public static class Msg1 {}
public static class Ack {}
#Override
public Receive createReceive() {
return receiveBuilder()
.match(Msg1.class, m -> getSender().tell(new Ack(), getSelf()))
.matchAny(m -> log.info("unknown message"))
.build();
}
}
Actors that share the acking behavior can extend the base class:
import akka.actor.AbstractActor;
public class AckingProducer extends AckingActor {
private AbstractActor.Receive producerBehavior;
public AckingProducer() {
producerBehavior =
receiveBuilder()
.matchEquals("foo", m -> getSender().tell("bar", getSelf()))
.matchEquals("ping", m -> getSender().tell("pong", getSelf()))
.build();
}
#Override
public Receive createReceive() {
return producerBehavior.orElse(super.createReceive());
// ^ chain the Receive blocks with orElse
// e.g., behavior1.orElse(behavior2).orElse(behavior3)
}
}
You can chain an arbitrary number of Receive blocks. Note that if more than one partial function handles the same message, then the first one that matches (that is, the first Receive block in the orElse chain that matches) will handle the message.
I think the short answer may be no, but I'm hoping I can get alternative suggestions. Assume I have a data object and a data service. The data service is an interface and has the following method.
public Data getData();
I'm creating a proxy for the service using the following invocation handler plus Netty to do what I'd call asynchronous rpc. The proxy is on the client side.
#Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
// Convert the call into an async request that returns a ListenableFuture
APCRequest request = new APCRequest(serviceType, method, args);
ListenableFuture future = apcClient.asyncMessage(request);
// This blocks until the future finishes
return future.get();
}
This works fine. However, if my client is a UI, I end up wrapping the service call in something like a SwingWorker. I'd prefer to come up with a way of returning the ListenableFuture that I already have sitting there. Is there any way I can accomplish that without creating a separate, asynchronous service API. For example:
public ListenableFuture<Data> getData();
If I could have my InvocationHandler return the wrong type, I could use something like this.
public abstract class AsyncServiceCall<S, D> { // S = service type, D = expected doCall return type
protected final S service;
protected AsyncServiceCall(Class<S> serviceType, APCClient client) {
ProxyFactory proxyFactory = new ProxyFactory(client);
// The true tells the proxyFactory we're expecting a ListenableFuture<D>
// rather than the real return type.
service = proxyFactory.createProxy(serviceType, true);
}
// Sub-classes would make a normal method call using this. For
// example, service.getData()
public abstract Object doCall();
#SuppressWarnings("unchecked")
public ListenableFuture<D> execute() {
return (ListenableFuture<D>) doCall();
}
Is there another way of accomplishing what I want? Performance isn't an issue for me, so blocking until the proxy can get the return value from the future is still an option if there's no simple way of doing what I want. It just seems like a waste since I want an asynchronous call in the UI anyway.
Keeping my service API simple is more of a priority than anything. I want to be able to prototype using a simple service provider that instantiates service implementations directly and plug in my remoting protocol / server that's using dynamic proxies / Netty late in the development cycle.
If you want to keep your API simple then I would suggest providing only the async API in the interface - it's much easier to wrap up a synchronous implementation in an asynchronous API than vice-versa.
public interface DataService {
public ListenableFuture<Data> getData();
}
public abstract class LocalDataService implements DataService {
public ListenableFuture<Data> getData() {
SettableFuture<Data> result = SettableFuture.create();
try {
Data theData = computeData();
result.set(theData);
} catch(Throwable t) {
result.setException(e);
}
return result;
}
protected abstract Data computeData() throws Throwable;
}