I'm kind of forced to switch to reactive programming (and in a short time frame), using WebFlux and I'm having a really hard time understanding it. Maybe because the lack of examples or because I never did functional programming.
Anyways, my question is where to use Mono/Flux and where can I work with normal objects? E.g. my controller is waiting for a #Valid User object, should that be #Valid Mono or something like Mono<#Valid User>? If let's say it was just a User object, I pass it to my service layer, and I want to encode the password before saving it to the reactive MongoDb, should I write:
User.setPassword(...);
return reactiveMongoDbRepository.save(user); //returns Mono<User> which is returned by the Controller to the View
Or it should be something like
return Mono.just(user).map.flatmap(setPasswordSomewhereInThisHardToFollowChain).then.whatever.doOnSuccess(reactiveMongoDbRepository::save);
In other words, am I forced to use this pipeline thing EVERYWHERE to maintain reactiveness or doing some steps the imperative way, like unwrapping the object, working on it, and wrapping it back is OK?
I know my question seems to be silly but I don't have the big picture at all, reading books about it didn't really help yet, please be gentle on me. :)
Use pipelining when you require sequential, asynchronous and lazy execution. In all other cases (when you are using a non-blocking code) you're free to choose any approach and it's generally better to use the simplest one.
Sequential non-blocking code can be organised in functions that you can integrate with reactive pipeline using map/filter/doOnNext/... components.
For example, consider the following Order price calculation code.
class PriceCalculator {
private final Map<ProductCode, Price> prices;
PriceCalculator(Map<ProductCode, Price> prices) {
this.prices = prices;
}
PricedOrder calculatePrice(Order order) { // doesn't deal with Mono/Flux stuff
Double price = order.orderLines.stream()
.map(orderLine -> prices.get(orderLine.productCode))
.map(Price::doubleValue)
.sum();
return new PricedOrder(order, price);
}
}
class PricingController {
public Mono<PricedOrder> getPricedOrder(ServerRequest request) {
OrderId orderId = new OrderId(request.pathVariable("orderId"));
Mono<Order> order = orderRepository.get(orderId);
return order.map(priceCalculator::calculatePrice)
}
}
Related
The situation is as follows: I have two MongoDB documents: User and Reproduction.
Based on a check if the User has a Reproduction entry (document), I want to add another entity in MongoDB.
Currently I am using Spring Webflux and Spring Reactive Mongo. Please see code below.
#Autowired
ParserRepository parserRepository;
#Autowired
ReproductionRepository reproductionRepository;
#Autowired
UserRepository userRepository;
public void addParser(Parser parser, String keycloakUserId) {
Mono<User> userMono = userRepository.findByKeycloakUserId(keycloakUserId);
Mono<Reproduction> reproductionMono = reproductionRepository.findById(parser.getReproductionId());
userMono.zipWith(reproductionMono)
.filter(objects -> objects.getT2().getUserId().equals(objects.getT1().get_id()))
.then(parserRepository.save(parser))
.switchIfEmpty(Mono.error(new ParserDoesNotBelongToUserException("Unable to add, since this parser does not belong to you")));
}
My question is as follows: how can the result from the Mono be used in order to verify that the correct Mono is there, and based on that save Parser Document. Basically combining the results from both Mono streams in order to perform save of another document and doing this in a non-blocking way.
The method as above doesn't seem to work apparently. What is the best way of doing this scenario with two separate Mono's in this case? Any best practice tips are welcome.
taken from Mono#filter docs:
filter(Predicate<? super T> tester)
If this Mono is valued, test the result and replay it if predicate returns true.
So if the filter evaluates to true, it will pass through the value, if false, it will not.
The problem is that you are calling then after. Docs for Mono#then
then(Mono other)
Let this Mono complete then play another Mono.
the key word here is complete which basically means, that whatever the row before completes with, its ignored, as long as it completes. So whatever it completed with (false/true) in the row before doesn't really matter we run then anyway.
im guessing you want something like:
userMono.zipWith(reproductionMono).flatMap(objects -> {
if(objects.getT2().getUserId().equals(objects.getT1().get_id()))) {
return parserRepository.save(parser)
} else {
return Mono.error(new ParserDoesNotBelongToUserException("Unable to add, since this parser does not belong to you"));
}
}
I have this Spring WebFlux controller:
#RestController
public class Controller
{
#PostMapping("/doStuff")
public Mono<Response> doStuff(#RequestBody Mono<Request> request)
{
...
}
}
Now, say I wanted to relate separate requests coming to this controller from different clients to group processing based on some property of the Request object.
Take 1:
#PostMapping("/doStuff")
public Mono<Response> doStuff(#RequestBody Mono<Request> request)
{
return request.flux()
.groupBy(r -> r.someProperty())
.flatMap(gf -> gf.map(r -> doStuff(r)));
}
This will not work, because every call will get its own instance of the stream. The whole flux() call doesn't make sense, there will always ever be one Request object going through the stream even if there's many of those streams fired at the same time as a result of simultaneous calls coming from clients. What I need, I gather, is some part of the stream that is shared between all requests where I could do my grouping, which led me to this slightly over engineered code
Take 2:
private AtomicReference<FluxSink<Request>> sink = new AtomicReference<>();
private Flux<Response> serializingStream;
public Controller()
{
this.serializingStream =
Flux.<Request>create(fluxSink -> sink.set(fluxSink), ERROR)
.groupBy(r -> r.someProperty())
.flatMap(gf -> gf.map(r -> doStuff(r)));
.publish()
.autoConnect();
this.serializingStream.subscribe().dispose(); //dummy subscription to set the sink;
}
#PostMapping("/doStuff")
public Mono<Response> doStuff(#RequestBody Request request)
{
req.setReqId(UUID.randomUUID().toString());
return
serializingStream
.doOnSubscribe(__ -> sink.get().next(req))
.filter(resp -> resp.getReqId().equals(req.getReqId()))
.take(1)
.single();
}
And this kind of works, though it looks like I am doing things I shouldn't (or at least they don't feel right), like leaking the FluxSink and then injecting a value through it while subscribing, adding a request ID so that I can then filter the right response. Also, if error happens in the serializingStream then it breakes everything for everyone, but I guess I could try to isolate the errors to keep things going.
The question is, is there a better way of doing this that doesn't feel like an open heart surgery.
Also, related question for a similar scenario. I was thinking about using Akka Persistence to implement event sourcing and have it trigerred from inside that Reactor stream. I was reading about Akka Streams that allow to wrap an Actor and then there's some ways of converting that into something that can be hooked up with Reactor (aka Publisher or Subscriber), but then if every requests gets it's own stream, I am effectively loosing back pressure and am risking OOME because of flooding the Persistent Actor's mailbox, so I guess that problem falls in to the same category like the one I described above.
I have a request that is rather simple to formulate, but I cannot pull it of without leaking resources.
I want to return a response of type application/stream+json, featuring news events someone posted. I do not want to use Websockets, not because I don't like them, I just want to know how to do it with a stream.
For this I need to return a Flux<News> from my restcontroller, that is continuously fed with news, once someone posts any.
My attempt for this was creating a Publisher:
public class UpdatePublisher<T> implements Publisher<T> {
private List<Subscriber<? super T>> subscribers = new ArrayList<>();
#Override
public void subscribe(Subscriber<? super T> s) {
subscribers.add(s);
}
public void pushUpdate(T message) {
subscribers.forEach(s -> s.onNext(message));
}
}
And a simple News Object:
public class News {
String message;
// Constructor, getters, some properties omitted for readability...
}
And endpoints to publish news respectively get the stream of news
// ...
private UpdatePublisher<String> updatePublisher = new UpdatePublisher<>();
#GetMapping(value = "/news/ticker", produces = "application/stream+json")
public Flux<News> getUpdateStream() {
return Flux.from(updatePublisher).map(News::new);
}
#PutMapping("/news")
public void putNews(#RequestBody News news) {
updatePublisher.pushUpdate(news.getMessage());
}
This WORKS, but I cannot unsubscribe, or access any given subscription again - so once a client disconnects, the updatePublisher will just continue to push onto a growing number of dead channels - as I have no way to call the onCompleted() handler on the subscriptions.
TL;DL:
Can one push messages onto a possible endless Flux from a different thread and still terminate the Flux on demand without relying on a reset by peer exception or something along those lines?
You should never try to implement yourself the Publisher interface, as it boils down to getting the reactive streams implementation right. This is exactly the issue you're facing here.
Instead you should use one of the generator operators provided by Reactor itself (this is actually a Reactor question, nothing specific to Spring WebFlux).
In this case, Flux.create or Flux.push are probably the best candidates, given your code uses some type of event listener to push events down the stream. See the reactor project reference documentation on that.
Without more details, it's hard to give you a concrete code sample that solves your problem. Here are a few pointers though:
you might want to .share() the stream of events for all subscribers if you'd like some multicast-like communication pattern
pay attention to the push/pull/push+pull model that you'd like to have here; how is the backpressure supposed to work here? What if we produce more events that the subscribers can handle?
this model would only work on a single application instance. If you'd like this to work on multiple application instances, you might want to look into messaging patterns using a broker
First of all, sorry for that abstract title. The idea is more simple with an example. Let say I have some values in a list L. I want to build parameters Request for a service, then call this service and collect all the Responses.
Currently, I'm using this kind of code structure :
private final List<String> bodies = ImmutableList.of(
"1", "2", "3", "4"
);
#Test
public void BasicRequestResponseStreamToList() {
final List<Request> requests = bodies.stream()
.map(Request::new)
.collect(Collectors.toList());
final List<Response> responses = requests.stream()
.map(service::send)
.collect(Collectors.toList());
commonAssertions(requests, responses);
}
However, I find the need of two stream not efficient considering the last request has to be built before the first one can be sent. I would like to do something like :
#Test
public void StatefulMapperStreamRequestResponseToList() {
final List<Request> requests = new ArrayList<>();
final List<Response> responses = bodies.stream()
.map(Request::new)
.map(x -> {
requests.add(x);
return service.send(x);
})
.collect(Collectors.toList());
commonAssertions(requests, responses);
}
However, I feel guilty to use such a "Hack" to the mapping semantic. However it's the only way I've found to build 2 Correlated list with lazy loading. The first solution doesn't interest me because it has to wait to build all the request before sending them.
I would love to achieve something like a wiretap in EIP. http://camel.apache.org/wire-tap.html
I would gladly have your thoughts about a more elegant manner than modifying the semantic of the map method to achieve this.
If it helps, you can find the source here : http://tinyurl.com/hojkdzu
Using .peek() while requires less changes in your code, is actually quite dirty solution. You need it, because you have a design flaw in your original code. You have "parallel data structures" (probably the term is not very good): the first element in the requests list corresponds to the first element in the responses list, and so on. When you have such situation, consider creating a new PoJo class instead. Something like this:
public class RequestAndResponse { // You may think up a better name
public final Request req; // use getters if you don't like public final fields
public final Response resp;
public RequestAndResponse(Request req, Response resp) {
this.req = req;
this.resp = resp;
}
}
Now your problem magically disappears. You can write:
List<RequestAndResponse> reqresp = bodies.stream()
.map(Request::new)
.map(req -> new RequestAndResponse(req, service.send(req)))
.collect(Collectors.toList());
commonAssertions(reqresp);
You will need to change commonAssertions method after that, but I'm pretty sure it will become simpler. Also you may find that some methods in your code use request and response together, so it's quite natural to make them as methods in RequestAndResponse class.
Not sure what you really want to achieve here. But if what you want is to collect requests "as you go" then you can use peek():
final List<Request> requests = new ArrayList<>();
final List<Response> responses = bodies.stream()
.map(Request::new)
.peek(requests::add)
.map(service::send)
.collect(Collectors.toList());
commonAssertions(requests, responses);
.peek() is pretty much what you mean in the subject of your post; it takes a Consumer, can occur at any steps in the pipeline, and here the consumer just saves the "intermediate states" into a list.
BUT... the javadoc of peek() specifically mentions this:
For parallel stream pipelines, the action may be called at whatever time and in whatever thread the element is made available by the upstream operation. If the action modifies shared state, it is responsible for providing the required synchronization.
So, well, be careful, I guess...
I need to geocode an Address object, and then store the updated Address in a search engine. This can be simplified to taking an object, performing one long-running operation on the object, and then persisting the object. This means there is an order of operations requirement that the first operation be complete before persistence occurs.
I would like to use Akka to move this off the main thread of execution.
My initial thought was to use a pair of Futures to accomplish this, but the Futures documentation is not entirely clear on which behavior (fold, map, etc) guarantees one Future to be executed before another.
I started out by creating two functions, defferedGeocode and deferredWriteToSearchEngine which return Futures for the respective operations. I chain them together using Future<>.andThen(new OnComplete...), but this gets clunky very quickly:
Future<Address> geocodeFuture = defferedGeocode(ec, address);
geocodeFuture.andThen(new OnComplete<Address>() {
public void onComplete(Throwable failure, Address geocodedAddress) {
if (geocodedAddress != null) {
Future<Address> searchEngineFuture = deferredWriteToSearchEngine(ec, addressSearchService, geocodedAddress);
searchEngineFuture.andThen(new OnComplete<Address>() {
public void onComplete(Throwable failure, Address savedAddress) {
// process search engine results
}
});
}
}
}, ec);
And then deferredGeocode is implemented like this:
private Future<Address> defferedGeocode(
final ExecutionContext ec,
final Address address) {
return Futures.future(new Callable<Address>() {
public Address call() throws Exception {
log.debug("Geocoding Address...");
return address;
}
}, ec);
};
deferredWriteToSearchEngine is pretty similar to deferredGeocode, except it takes the search engine service as an additional final parameter.
My understand is that Futures are supposed to be used to perform calculations and should not have side effects. In this case, geocoding the address is calculation, so I think using a Future is reasonable, but writing to the search engine is definitely a side effect.
What is the best practice here for Akka? How can I avoid all the nested calls, but ensure that both the geocoding and the search engine write are done off the main thread?
Is there a more appropriate tool?
Update:
Based on Viktor's comments below, I am trying this code out now:
ExecutionContext ec;
private Future<Address> addressBackgroundProcess(Address address) {
Future<Address> geocodeFuture = addressGeocodeFutureFactory.defferedGeocode(address);
return geocodeFuture.flatMap(new Mapper<Address, Future<Address>>() {
#Override
public Future<Address> apply(Address geoAddress) {
return addressSearchEngineFutureFactory.deferredWriteToSearchEngine(geoAddress);
}
}, ec);
}
This seems to work ok except for one issue which I'm not thrilled with. We are working in a Spring IOC code base, and so I would like to inject the ExecutionContext into the FutureFactory objects, but it seems wrong for this function (in our DAO) to need to be aware of the ExecutionContext.
It seems odd to me that the flatMap() function needs an EC at all, since both futures provide one.
Is there a way to maintain the separation of concerns? Am I structuring the code badly, or is this just the way it needs to be?
I thought about creating an interface in the FutureFactory's that would allow chaining of FutureFactory's, so the flatMap() call would be encapsulated in a FutureFactory base class, but this seems like it would be deliberately subverting an intentional Akka design decision.
Warning: Pseudocode ahead.
Future<Address> myFutureResult = deferredGeocode(ec, address).flatMap(
new Mapper<Address, Future<Address>>() {
public Future<Address> apply(Address geocodedAddress) {
return deferredWriteToSearchEngine(ec, addressSearchService, geocodedAddress);
}
}, ec).map(
new Mapper<Address, SomeResult>() {
public SomeResult apply(Address savedAddress) {
// Create SomeResult after deferredWriteToSearchEngine is done
}
}, ec);
See how it is not nested. flatMap and map is used for sequencing the operations. "andThen" is useful for when you want a side-effecting-only operation to run to full completion before passing the result on. Of course, if you map twice on the SAME future-instance then there is no ordering guaranteed, but since we are flatMapping and mapping on the returned futures (new ones according to the docs), there is a clear data-flow in our program.