I have code:
SqlTemplate
.forQuery(client, "SELECT * FROM user WHERE id=#{id}")
.execute(parameters)
.onSuccess(users -> {
users.forEach(row -> {
// exception here
System.out.println(row.get(UUID.class, "id1") + " " + row.getString("title"));
});
})
What is the best way to handle exceptions in consumers?
For now if exception raises it will be swallowed...
Assuming you want to fail the flow when an exception is thrown, it is better to use compose to iterate through the users and handle .onSuccess() and .onFailure() separately.
You can use CompositeFuture to achieve this. You can have a Future list and add succeededFuture / failedFuture (in case of exception) to the list in the forEach loop.
SqlTemplate
.forQuery(client, "SELECT * FROM user WHERE id=#{id}")
.execute(parameters)
.compose(users -> {
List<Future> usersFuture = new ArrayList<>();
users.forEach(row -> {
try {
// exception here
System.out.println(row.get(UUID.class, "id1") + " " + row.getString("title"));
usersFuture.add(Future.succeededFuture());
} catch (Exception e) {
usersFuture.add(Future.failedFuture(e));
}
});
return CompositeFuture.all(usersFuture).mapEmpty();
})
.onSuccess(res -> {
// end the flow with success
})
.onFailure(e -> {
// Add error message and fail the flow
});
Also in general I'm curious about the exceptions that can be thrown here. As it is the data that you have written to your db after validations, you should be aware of the possible error scenarios and handle them accordingly without failing the flow.
Related
I need to convert a CompletionStage returned by an external library into a Mono inside my reactive pipeline. How do i handle exceptions returned from the call (I want to ignore them and continue with the sequence)? The onErrorResume / doOn* operators are not getting invoked when the external call throws an exception (possibly because the Mono is never created because of the exception).
private void example() {
Flux.range(1, 2)
.flatMap(i ->
Mono.fromCompletionStage(externalCall(i))
.doOnNext(ni -> System.out.println("onNext: " + ni))
.doOnError(err -> System.err.println("onError: " + err.getMessage()))
.onErrorResume(e -> Mono.empty())
)
.subscribe();
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
CompletionStage<String> externalCall(int i) {
if (new Random().nextBoolean()){
throw new RuntimeException("Exception in external call");
}
return Mono.just(i)
.map(e -> String.valueOf((char) (e + 64)))
.toFuture();
}
Stack trace
2021-10-11T02:16:09,944 main r.c.p.Operators - Operator called default onErrorDropped
reactor.core.Exceptions$ErrorCallbackNotImplemented: java.lang.RuntimeException: Exception in external call
Caused by: java.lang.RuntimeException: Exception in external call
at lrn.chap.SubscribingInFlatMap.externalCall(SubscribingInFlatMap.java:107)
at lrn.chap.SubscribingInFlatMap.lambda$example$10(SubscribingInFlatMap.java:91)
at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:386)
at reactor.core.publisher.FluxRange$RangeSubscription.slowPath(FluxRange.java:156)
at reactor.core.publisher.FluxRange$RangeSubscription.request(FluxRange.java:111)
at reactor.core.publisher.FluxFlatMap$FlatMapMain.onSubscribe(FluxFlatMap.java:371)
at reactor.core.publisher.FluxRange.subscribe(FluxRange.java:69)
at reactor.core.publisher.Flux.subscribe(Flux.java:8468)
at reactor.core.publisher.Flux.subscribeWith(Flux.java:8641)
at reactor.core.publisher.Flux.subscribe(Flux.java:8438)
Process finished with exit code 0
You can use Mono.defer() to convert it to a lazy publisher like this:
Mono.defer(() -> Mono.fromCompletionStage(externalCall(i)))
.doOnNext(ni -> System.out.println("onNext: " + ni))
.doOnError(err -> System.err.println("onError: " + err.getMessage()))
.onErrorResume(e -> Mono.empty()))
The defer method mentioned in the other answer is a good option but there is another shorter option which is using the lambda version of the fromCompletionStage method:
Mono.fromCompletionStage(() -> externalCall(i))
.doOnNext(ni -> System.out.println("onNext: " + ni))
.doOnError(err -> System.err.println("onError: " + err.getMessage()))
.onErrorResume(e -> Mono.empty()))
Just like defer, this will make sure that the externalCall method is called on-demand and errors are handled as part of the reactive chain.
I'm using Data Movement SDK from MarkLogic Java API to transform several documents, up to now I can transform documents by using a query batcher and a transform, but i'm only able to use URIS selectors by StructuredQuery objects.
My question is: ¿How may I use a selector module from my database instead of define it into my java application?
Update:
Up to now I already have a code that looks for document's URIS and applies a transform on them. I want to change that query batcher and use a module or selector module instead of looking for all documents into a directory
public TransformExecutionResults applyTransformByModule(String transformName, String filterText, int batchSize, int threadCount, String selectorModuleName, Map<String,String> parameters ) {
final ConcurrentHashMap<String, TransformExecutionResults> transformResult = new ConcurrentHashMap<>();
try {
// Specify a server-side transformation module (stored procedure) by name
ServerTransform transform = new ServerTransform(transformName);
ApplyTransformListener transformListener = new ApplyTransformListener().withTransform(transform).withApplyResult(ApplyResult.REPLACE) // Transform in-place, i.e. rewrite
.onSuccess(batch -> {
transformResult.compute(transformName, (k, v) -> TransformExecutionResults.Success);
System.out.println("Transformation " + transformName + " executed succesfully.");
}).onSkipped(batch -> {
System.out.println("Transformation " + transformName + " skipped succesfully.");
transformResult.compute(transformName, (k, v) -> TransformExecutionResults.Skipped);
}).onFailure((batchListener, throwable) -> {
System.err.println("Transformation " + transformName + " executed with errors.");
transformResult.compute(transformName, (k, v) -> TransformExecutionResults.Failed); // failed
});
// Apply the transformation to only the documents that match a query.
QueryManager qm = DbClient.newQueryManager();
StructuredQueryBuilder sqb = qm.newStructuredQueryBuilder();
// instead of this StruturedQueryDefinition, I want to use a module to get all URIS
StructuredQueryDefinition queryBySubdirectory = sqb.directory(true, "/temp/" + filterText + "/");
final QueryBatcher batcher = DMManager.newQueryBatcher(queryBySubdirectory);
batcher.withBatchSize(batchSize);
batcher.withThreadCount(threadCount);
batcher.withConsistentSnapshot();
batcher.onUrisReady(transformListener).onQueryFailure(exception -> {
exception.printStackTrace();
System.out.println("There was an error on Transform process.");
});
final JobTicket ticket = DMManager.startJob(batcher);
batcher.awaitCompletion();
DMManager.stopJob(ticket);
} catch (Exception fault) {
transformResult.compute(transformName, (k, v) -> TransformExecutionResults.GeneralException); // general exception
}
return transformResult.get(transformName);
}
If the job is small enough, you can just implement the document rewriting within your enode code either by making a call to a resource service extension:
http://docs.marklogic.com/guide/java/resourceservices#id_27702
http://docs.marklogic.com/javadoc/client/com/marklogic/client/extensions/ResourceServices.html
or by invoking a main module:
http://docs.marklogic.com/guide/java/resourceservices#id_84134
If the job is too long to fit in a single transaction, your can create a QueryBatcher with a document URI iterator instead of with a query. See:
http://docs.marklogic.com/javadoc/client/com/marklogic/client/datamovement/DataMovementManager.html#newQueryBatcher-java.util.Iterator-
For some examples illustrating the approach, see the second half of the second example in the class description for QueryBatcher:
http://docs.marklogic.com/javadoc/client/com/marklogic/client/datamovement/QueryBatcher.html
as well as the second half of this example:
http://docs.marklogic.com/javadoc/client/com/marklogic/client/datamovement/UrisToWriterListener.html
In your case, you could implement an Iterator that calls a resource service extension or invokes a main module to get and return the URIs (preferrably with read ahead), blocking when necessary.
By returning the uris to the client, it's easy to log the uris for later audit.
Hoping that helps,
I have the following case:
I'm iterating over my Affiliate entities and for each of them I need to persist and update data in one unique transaction. So I have a service with a method annotated with Spring #Transactional annotation (where data is created and updated) but I don't know how can I see that the transaction has been rollback for an affiliate ?
I would like to know that for a special Affiliate the transaction has been rollback and retrieve a custom error code from my service..
This was my service before using Spring:
public void savePostingPaymentDetails(List<Posting> postingsToUpdate, List<PaymentPostingDetail> detailsToInsert, Payment payment) {
logger.info("DB ACCESS : INSERT PAYMENT DETAILS & UPDATE POSTINGS");
long begin = System.nanoTime();
this.em.getTransaction().begin();
try {
// TEST
// 1 - Save Payments
this.em.persist(payment);
// 2 - Save Details
for (PaymentPostingDetail ppd : detailsToInsert) {
this.em.persist(ppd);
}
// 3 - Update Postings
for (Posting p : postingsToUpdate) {
if(p.getSignature() != null)
{
p.getSignature().setModification("withholding-tax.pay", new Date());
}
else
{
logger.error("The Posting with id = " + p.getIdentifier() + " has no PersistenceSignature ?!");
}
this.em.merge(p);
}
}
catch (Exception e)
{
logger.error("Unexpected error on saving/updating the DB.", e);
this.em.getTransaction().rollback();
logger.info("RollBack done.");
e.printStackTrace();
System.exit(JobStatus.ABNORMAL_END_OF_EXECUTION_ERROR.getCode());
}
this.em.getTransaction().commit();
logger.info("Details inserted & Postings updated.");
long end = System.nanoTime();
logger.info("Execution time = " + ((end-begin) / 1000000) + " milliseconds.");
logger.info("----------------------------------------------------------");
}
Now I have this:
#Transactional
public void savePostingPaymentDetails(List<Posting> postings, List<PaymentPostingDetail> paymentDetails, Payment payment)
{
logger.info("DB ACCESS : INSERT PAYMENT DETAILS & UPDATE POSTINGS");
long begin = System.nanoTime();
this.paymentRepository.save(payment);
this.ppdRepository.save(paymentDetails);
for(Posting p : postings){
if(p.getSignature() != null)
{
p.getSignature().setModifiedAt(LocalDate.now());
p.getSignature().setModifiedBy(PayCopyrightWithholdingTaxProcess.SIGNATURE);
}
else{
p.setSignature(new PersistenceSignature(LocalDate.now(), PayCopyrightWithholdingTaxProcess.SIGNATURE));
}
this.postingRepository.save(p);
}
long end = System.nanoTime();
logger.info("Execution time = " + ((end-begin) / 1000000) + " milliseconds.");
logger.info("----------------------------------------------------------");
}
But how can I return let us say a special integer (instead of System.exit()) if the transaction has been rollback ?
There is something called User managed Transaction(UMT) and Container managed Transaction (CMT)
When you are using #Transactional you are actually delegating the transaction management to your Spring container (CMT), which is responsible for e.g opening and closing the transaction for you. It
rolls back automatically when unchecked Exception is thrown like NullPointerException, or RuntimeException ). For checked
exceptions you have to specify when the rollback is supposed to occured #Transactional(rollbackFor=myCheckedException.class).
You can also listen, observe how the transaction is doing with a TransactionalEventListener and react with some AOP listening code like shown here. But You are not ultimately managing the Transaction, Spring is doing for you. The client code can't react with some custom code, when something special happens, because the management of the transaction is delegated to Spring.
Therefore you have to fall back on the User managed Transaction, where you open your transaction, commit it and react in case of a rollback. That is exactly the purpose of UMT: giving total control of your transaction.
from your old code you may get something like:
public int savePostingPaymentDetails(List<Posting> postingsToUpdate, List<PaymentPostingDetail> detailsToInsert, Payment payment) {
int returnCode = 1 // 1 -> "success" , 0 -> "failure"
logger.info("DB ACCESS : INSERT PAYMENT DETAILS & UPDATE POSTINGS");
long begin = System.nanoTime();
long end = 0;
this.em.getTransaction().begin();
try {
// TEST
// 1 - Save Payments
this.em.persist(payment);
// 2 - Save Details
for (PaymentPostingDetail ppd : detailsToInsert) {
this.em.persist(ppd);
}
// 3 - Update Postings
for (Posting p : postingsToUpdate) {
if(p.getSignature() != null)
{
p.getSignature().setModification("withholding-tax.pay", new Date());
}
else
{
logger.error("The Posting with id = " + p.getIdentifier() + " has no PersistenceSignature ?!");
}
this.em.merge(p);
}
this.em.getTransaction().commit();
end = System.nanoTime();
}
catch (Exception e)
{
returnCode = 0;
logger.error("Unexpected error on saving/updating the DB.", e);
this.em.getTransaction().rollback();
logger.info("RollBack done.");
// e.printStackTrace();
System.exit(JobStatus.ABNORMAL_END_OF_EXECUTION_ERROR.getCode());
return returnCode;
}
//this.em.getTransaction().commit();
logger.info("Details inserted & Postings updated.");
//long end = System.nanoTime();
logger.info("Execution time = " + ((end-begin) / 1000000) + " milliseconds.");
logger.info("----------------------------------------------------------");
return returnCode = 1;
}
PS: on a side note, best practice would have you to throw an Exception when your commit fails, instead of special code.
your new method signature could be:
public void savePostingPaymentDetails(List<Posting> postingsToUpdate, List<PaymentPostingDetail> detailsToInsert, Payment payment)
throws MyFailedDbOperationException, OtherException {
}
and Throw the exception on your catch block
catch (Exception e)
{
logger.error("Unexpected error on saving/updating the DB.", e);
this.em.getTransaction().rollback();
logger.info("RollBack done.");
throw MyFailedDbOperationException("my db operation failed");
}
You can use a listener (#TransactionalEventListener) to be informed of a rolled back transaction (the listener can be bound to the different phases of a transaction). See section 16.8 of https://docs.spring.io/spring/docs/4.2.x/spring-framework-reference/html/transaction.html for more information (requires Spring >= 4.2)
I have a use case where I need to aggregate the finished thread responses from multiple Observable objects and return back to the client. My question is how to achieve it with using the rX Java. Here I have written a code snippet but the issue of this one is that this won't return anything after the timeout.
Observable<AggregateResponse> aggregateResponse = Observable.
zip(callServiceA(endpoint), callServiceB(endpoint), callServiceC(endpoint),
(Mashup resultA, Mashup resultB, Mashup resultC) -> {
AggregateResponse result = new AggregateResponse();
result.setResult(resultA.getName() + " " + resultB.getName() + " " + resultC.getName());
return result;
}).timeout(5, TimeUnit.SECONDS);
Subscriber
aggregateResponse.subscribe(new Subscriber<AggregateResponse>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable throwable) {
//Timeout execute this rather than aggregating the finished tasks
System.out.println(throwable.getMessage());
System.out.println(throwable.getClass());
}
#Override
public void onNext(AggregateResponse response) {
asyncResponse.resume(response);
}
});
You need to put the timeout operator on each Observable, zip will wait for all Observables to emit a value before emitting a result, so if only one of them take longer while others already emitted, you will cut down the stream with the timeout (with onError) before the zipped Observable will have a chance to emit.
What you should do, assuming you want to ignore timed out sources while keeping the rest, is to add timeout operator to each Observable and also add error handling like onErrorReturn to each one, the error return can return some kind of 'empty' result (you can't use null in RxJava2), and when you aggregate result ignore those empty results:
Observable<AggregateResponse> aggregateResponse = Observable.
zip(callServiceA(endpoint)
.timeout(5, TimeUnit.SECONDS)
.onErrorReturn(throwable -> new Mashup()),
callServiceB(endpoint)
.timeout(5, TimeUnit.SECONDS)
.onErrorReturn(throwable -> new Mashup()),
callServiceC(endpoint)
.timeout(5, TimeUnit.SECONDS)
.onErrorReturn(throwable -> new Mashup()),
(Mashup resultA, Mashup resultB, Mashup resultC) -> {
AggregateResponse result = new AggregateResponse();
result.setResult(resultA.getName() + " " + resultB.getName() + " " + resultC.getName());
return result;
});
I am using this example as my starting point.
In that example, server responds with the modified text but in my case, server doesn't need to respond.
Looking at other similar questions, I am aware that I can either pass Empty ByteString or use filter(p -> false) to not send anything back. However, in that case, the problem is that my whenComplete block doesn't get executed. i.e. the exception gets swallowed. Is there a way to avoid this? Help appreciated on this !
connections.runForeach(connection -> {
System.out.println("New connection from: " + connection.remoteAddress());
final Flow<ByteString, ByteString, NotUsed> echo = Flow.of(ByteString.class)
.via(Framing.delimiter(ByteString.fromString("\n"), 256, FramingTruncation.DISALLOW))
.map(ByteString::utf8String)
.map(s -> s + "!!!\n")
.map(ByteString::fromString);
connection.handleWith(echo, mat);
}, mat).whenComplete((done,throwable) ->
{
//exception handling
}
);
Your analysis is correct, now you are reacting when the server shuts down rather than each connection.
Reacting on the individual connections completing would be done in the flow passed to the connection, something like this:
final Tcp tcp = Tcp.get(system);
tcp.bind("127.0.0.1", 6789).runForeach((connection) -> {
final Flow<ByteString, ByteString, NotUsed> echo = Flow.of(ByteString.class)
.via(Framing.delimiter(ByteString.fromString("\n"), 256, FramingTruncation.DISALLOW))
.map(ByteString::utf8String)
.map(s -> s + "!!!\n")
.map(ByteString::fromString)
.watchTermination((notUsed, completionStage) -> {
completionStage.whenComplete((done, exception) -> {
System.out.println("Connection from " + connection.remoteAddress() + " completed");
});
return notUsed;
});
connection.handleWith(echo, materializer);
}, materializer);