Develop Reference

Flux consumer doesnt stop to consume data

I have this implementation
#Override
public Flux<byte[]> translateData(final String datasetId) {
return keyVaultRepository.findByDatasetId(datasetId)
.map(keyVault -> {
try {
return translatorService.createTranslator(keyVault.getKey()); // throws CryptoException in the test
} catch (CryptoException e) {
throw Exceptions.propagate(new ApiException("Unable to provide translated file"));
}
})
.flatMapMany(translator -> storageService.getEntry(datasetId).map(translator::update));
}
and this failing test
#Test
void getTranslatedDataWithError() throws StorageException, CryptoException {
final List<byte[]> bytes = new ArrayList<>();
// exec + validate
getWebTestClient()
.get()
.uri(uriBuilder -> uriBuilder.path("/{datasetId}").build(datasetId))
.exchange()
.expectStatus().is5xxServerError()
.returnResult(byte[].class)
.getResponseBody()
.onErrorStop()
.subscribe(bytes::add);
assertThat(bytes).isEmpty();
}
the part .is5xxServerError() is succeeding but the list is not empty.
The Microservice which is calling the endpoint of translateData should not consume any data from upstream but apparently this is the case.
I've found a workaround by throwing a RuntimeException in the catch block (I could also make the CryptoException unchecked but thats not the matter of my question) and then handle the case in my ControllerAdvice/GlobalExceptionhandler and just return a ResponseEntity with an ErrorDto .
The core of my question is, how can I do this natively with Flux. So that the consumer of the endpoint notices there is an error and .subscribe(bytes::add); wont be even executed.
I have tried it already with .doOnError , .onErrorResume etc. but it always ends with a non empty list :(
By this I am afraid that the bytes will later delivered to the client (which should not of course he should get an error response)

Use flatMap after method returning Mono<Void>

I'm stuck with understanding of some aspects of Project Reactor.
I have the following list of components:
Validator of input params, returns Mono<Void> or Mono.error()
Service saving data to db, returns Mono<Item>
Logger for successful actions of an user, returns Mono<Void
A business logic is quite simple: validate params (1), save an item to db (2) and log actions (3). The problem is validator (1) returns Mono.empty() if there are no errors with input data and Mono.error() if input params contain some errors.
I would like to achieve the next things:
If validator returns Mono.empty() then continue chain
If validator returns Mono.error() then immediately stop processing and throw error which will be handled by exceptionHanlder
I have tried two options:
First with .then(Mono<Item> item) after validation. It allows me to execute saving operation after validation. Given that .then() ignores any errors, I can't rise an exception.
return inputValidator.validateFields(userId, projectId)
.then(repository.save(item))
.onErrorMap(RepoException.class, ex -> new UnexpectedError("Failed to save item", ex))
.subscribeOn(Schedulers.boundedElastic())
.doOnSuccess(n -> logService.logActivity(new Activity(adminId, n))
.subscribe());
Second with .flatMap(Function<Mono<Void>, <Mono<? extends Item> func) after validation. This approach can rise an exception from validator, but I can't execute saving operation because flatMap() doesn't trigger on empty result.
return inputValidator.validateFields(userId, projectId)
.flatMap(v -> repository.save(item))
.onErrorMap(RepoException.class, ex -> new UnexpectedError("Failed to save item", ex))
.subscribeOn(Schedulers.boundedElastic())
.doOnSuccess(n -> logService.logActivity(new Activity(adminId, n))
.subscribe());
It is also important to have access for created object after saving (step 2), because I need to pass it to logger service.

You can't use flatMap because there is no onNext signal - use then instead. Not sure what do you mean by "called" but there is a difference between Assembly and Subscription time in reactive. Publisher you specified in then will not be resolved in case inputValidator.validateFields returns onError signal.
Here is a test for failed validation and as you may see subscription was not triggered
#Test
void saveWasNotCalledIfValidationFailed() {
var saveMock = PublisherProbe.of(Mono.just("id"));
var repo = mock(Repository.class);
when(repo.save())
.thenReturn(saveMock.mono());
var res = validateFields()
.then(repo.save())
.onErrorMap(IllegalArgumentException.class,
ex -> new IllegalStateException("Failed to save item", ex)
);
StepVerifier.create(res)
.expectError(IllegalStateException.class)
.verify();
saveMock.assertWasNotSubscribed();
}
private Mono<Void> validateFields() {
return Mono.error(new IllegalArgumentException("oops"));
}
public static class Repository {
public Mono<String> save() {
return Mono.just("id");
}
}
and here is a test for passed validation
#Test
void saveIsCalledIfValidationPassed() {
var saveMock = PublisherProbe.of(Mono.just("id"));
var repo = mock(Repository.class);
when(repo.save())
.thenReturn(saveMock.mono());
var res = validateFields()
.then(repo.save())
.onErrorMap(IllegalArgumentException.class,
ex -> new IllegalStateException("Failed to save item", ex)
);
StepVerifier.create(res)
.expectNext("id")
.verifyComplete();
saveMock.assertWasSubscribed();
}
private Mono<Void> validateFields() {
return Mono.empty();
}

Exceptions thrown by Mono inside flatmap not being handled by Spring webflux reactive stream

I am working on reactive streams application using Spring webflux. I have a usecase where I do a webclient request to get some data and once I get the response data, I validate it and if the validation fails, I want to throw an exception which should be handled by the main reactive pipeline. I'm using webclient call within a flatmap to use the value in the next operator of my pipeline. I have something similar to the following code:
public class Example {
public String getData(String name) {
return Mono.just(name)
.map(name -> name.toLowerCase())
.flatMap(name ->
// Webclient Request that returns a
// Mono<String> for example
.doOnSuccess(Validator::validateData); // The webclient request is chained to this doOnSuccess
)
.doOnError(ex -> log.error("Got an error, {}", er))
.onErrorMap(ex -> new AnotherCustomException(ex.getMessage()));
}
}
public class Validator {
public void validateData(String data) {
if(data.length() < 5) throw new CustomException("Invalid data received."); // public CustomException extends RuntimeException {...}
}
}
Currently this code isn't using the doOnError() & onErrorMap() operators and I'm directly receiving the CustomException stacktrace on my console. I believe the reason being the code inside flatMap itself is a publisher Mono so it should have its own doOnError(), onErrorMap() operators. How do I make this Webclient's response i.e., Mono<String> be able to use the main pipeline that's using the WebClient?

This is what you want
SomeWebclientCall().flatMap(value -> {
Validate.validate(value);
return Mono.just(value);
});
This might look wierd and that is because writing void functions that either return void or an exception is wierd and should be avoided.
Such functions are ineffective (throwing exceptions are expensive and should not be a natural flow of the program) and hard to test.
Thats why validation functions usually returns booleans as in it passed validation yes or no, true or false. Not void or exception.
Functional programming does not like void functions, only pure functions.

Handling exception in WebClient throws io.netty.handler.timeout.ReadTimeoutException

So I am new to Reactive programming and I wrote some code that I would like to test. Those are more of a integration tests as I am live copying files and later check if they are the same. I have a MockWebServer mocking my reponse to be 4xx which is handled well in the code. Unfortunately, I am also getting io.netty.handler.timeout.ReadTimeoutException which covers up my custom WebClientResponseException so in the test I am getting the wrong exception. Basically I have two questions, why on earth am I getting this io.netty.handler.timeout.ReadTimeoutException exception? It appears only after doOnError() method for some reason and I am not sure why is it even happening.
Right now code is at it is and it's being synchronous, I am well aware of that.
Second question is, how could I handle my custom exception in the tests after a given number of retries? Right now it is 3 and only then I would like my other exception to be thrown.
Here is the code:
AsynchronousFileChannel fileChannel = AsynchronousFileChannel.open(targetPath, StandardOpenOption.WRITE);
Flux<DataBuffer> fileDataStream = Mono.just(filePath)
.map(file -> targetPath.toFile().exists() ? targetPath.toFile().length() : 0)
.map(bytes -> webClient
.get()
.uri(uri)
.accept(MediaType.APPLICATION_OCTET_STREAM)
.header("Range", String.format("bytes=%d-", bytes))
.retrieve()
.onStatus(HttpStatus::is4xxClientError, clientResponse -> Mono.error(new CustomException("4xx error")))
.onStatus(HttpStatus::is5xxServerError, clientResponse -> Mono.error(new CustomException("5xx error")))
.bodyToFlux(DataBuffer.class)
.doOnError(throwable -> log.info("fileDataStream onError", throwable))
)
.flatMapMany(Function.identity());
return DataBufferUtils
.write(fileDataStream, fileChannel)
.map(DataBufferUtils::release)
.doOnError(throwable -> {
try {
fileChannel.force(true);
} catch (IOException e) {
throw new WritingException("failed force update to file channel", e);
}
})
.retry(3)
.doOnComplete(() -> {
try {
fileChannel.force(true);
} catch (IOException e) {
log.warn("failed force update to file channel", e);
throw new WritingException("failed force update to file channel", e);
}
})
.doOnError(throwable -> targetPath.toFile().delete())
.then(Mono.just(target));
The response is Mono<Path> as I am interested only in the Path of the newly created and copied file.
Any comments regarding code are welcome.
The copying mechanism was made basing on this thread Downlolad and save file from ClientRequest using ExchangeFunction in Project Reactor

So basically the problem was in tests. I had a MockResponse enqueued to the MockWebServer only once so upon retrying in the WebClient mocked server did not have any response set (basically it behaved like it is not available at all as there was no mocked response).
To be able to handle exceptions in case of the server being completely down it is in my opinion worth adding lines that goes something like this into your flux chain:
.doOnError(ChannelException.class, e -> {
throw new YourCustomExceptionForHandlingServerIsDownSituation("Server is unreachable", e);
})
This will help you handle ReadTimeoutException from Netty (in case of server being not reachable) as it extends ChannelException class. Always handle your exceptions.

spring webflux how to manage sequential business logic code in reactive world

Is this approach is reactive friendly?
I have a reactive controller "save" method calling myService.save(request).
The service layer needs to:
jdbc save(on another scheduler because code is blocking),
generate a template string (on another scheduler),
send an email(on another scheduler),
finally return the saved entity to the controller layer
I can't chain all my calls in one pipeline or I don't know how to achieve this, because I want to send back (1) that is lost as soon as I do ....flatMap(templateService::generateStringTemplate) for example.
So instead I trigger my sub operations inside (1).
Is it how am I supposed to handle this or is there a clever way to do it in one pipeline ?
Below code to support the question. Thanks.
Service called by Controller layer
public Mono<Prospect> save(final Prospect prospect) {
return Mono.fromCallable(
() -> {
Prospect savedProspect = transactionTemplate.execute(status -> prospectRepository.save(prospect));
templateService.generateProspectSubscription(savedProspect)
.map(t ->
EmailPostRequest.builder()
...
.build())
.flatMap(emailService::send)
.subscribe();
return savedProspect;
})
.subscribeOn(jdbcScheduler);
}
TemplateService
public Mono<String> generateProspectSubscription(final Prospect prospect) {
return Mono.fromCallable(
() -> {
Map<String, Object> model = new HashMap<>();
...
Template t = freemarkerConfig.getTemplate(WELCOME_EN_FTL);
String html = FreeMarkerTemplateUtils.processTemplateIntoString(t, model);
return html;
}
).subscribeOn(freemarkerScheduler);
}
EmailService
public Mono<Void> send(final EmailPostRequest e) {
return Mono.fromCallable(
() -> {
MimeMessage message = emailSender.createMimeMessage();
MimeMessageHelper mimeHelper = new MimeMessageHelper(message,
MimeMessageHelper.MULTIPART_MODE_MIXED_RELATED,
StandardCharsets.UTF_8.name());
mimeHelper.setTo(e.getTo());
mimeHelper.setText(e.getText(), true);
mimeHelper.setSubject(e.getSubject());
mimeHelper.setFrom(new InternetAddress(e.getFrom(), e.getPersonal()));
emailSender.send(message);
return Mono.empty();
}
).subscribeOn(emailScheduler).then();
}
EDITED SERVICE
I think this version of service layer is cleaner but any comments is appreciated
public Mono<Prospect> save(final Prospect prospect) {
return Mono.fromCallable(
() -> transactionTemplate.execute(status -> prospectRepository.save(prospect)))
.subscribeOn(jdbcScheduler)
.flatMap(savedProspect -> {
templateService.generateProspectSubscription(savedProspect)
.map(t ->
EmailPostRequest.builder()
...
.build())
.flatMap(emailService::send)
.subscribe();
return Mono.just(savedProspect);
}
);
}

This approach is not reactive friendly, as you're 100% wrapping blocking libraries.
With this use case, you can't really see the benefit of a reactive runtime and chances are the performance of your application is worse than a blocking one.
If your main motivation is performance, than this is probably counter-productive.
Offloading a lot of blocking I/O work on to specialized Schedulers has a runtime cost in term of memory (creating more threads) and CPU (context switching). If performance and scalability are your primary concern, then switching to Spring MVC and leveraging the Flux/Mono support where it fits, or even calling block() operators is probably a better fit.
If your main motivation is using a specific library, like Spring Framework's WebClient with Spring MVC, then you're better off using .block() operators in selected places rather than wrapping and scheduling everything.