I am new to reactive programming and using helidon reactive java libraries in our code.
I am unable to achieve the below use case.
I have a scenario as below.
First I invoke a REST API and get response.From the response that contains list of countries I have to invoke another
REST api that retrieves the response for a country id and update the country object.
By the time I invoke second API and set value to country object as below the response is already returned.
I get use .get() and wait() on Single as it blocks the thread.
Please suggest how to overcome the below for loop and update the list of objects reactive way.
Single<WebClientResponse> singleWebClientResp = webClient.get("REST_URL");
Single<String> apiResponse = singleWebClientResponse.flatMapSingle(webClientResponse -> {
return webClientResponse.content().as(String.class);
});
apiResponse.flatMapSingle(fusionAPIResponseString -> {
List<Country> countries =
objectMapper.readValue(fusionAPIResponseString,new TypeReference<List<Country>>() {});
for (Country country : countries) {
getCountryByRegion(country.getRegion()).forSingle(newCountry -> {
LOGGER.log(Level.FINE, "newCountry ---> " + newCountry);
country.setRegion(country.getRegion() + "modified" + newCountry);
});
}
});
private Single<String> getCountryByRegion(String regionName) {
LOGGER.log(Level.FINE, "Entering getCountryByRegion");
Single<WebClientResponse> singleWebClientResponse2 = webClient.get().path("v3.1/region/" + regionName)
.contentType(MediaType.APPLICATION_JSON).request();
Single<String> retVal = singleWebClientResponse2.flatMapSingle(webClientResponse -> {
return webClientResponse.content().as(String.class);
});
LOGGER.log(Level.FINE, "Exiting getCountryByRegion");
return retVal;
}
Regards
// NOTE: this should be a static constant
GenericType<List<Country>> countriesType = new GenericType<>() {};
// NOTE: create the webClient only once, not for every request
WebClient webClient = WebClient.builder()
.addMediaSupport(JacksonSupport.create())
.baseUri("service-url")
.build();
// the pipeline starts with the initial countries (i.e. Single<List<Country>>)
webClient.get()
.path("/countries")
// get the countries as List<Country>
.request(countriesType)
// add each country to the reactive pipeline (i.e. Multi<Country>)
// to allow individual reactive mapping
.flatMap(Multi::just)
// map each country by creating a new country with new region
// use flatMap to inline the webClient result in the reactive pipeline
.flatMap(country ->
webClient.get()
.path("/region/" + country.getRegion())
.request(String.class)
.map(newCountry -> new Country(newCountry, country.getRegion())))
// aggregate all items (i.e. Single<List<Country>>)
.collectList()
.onError(res::send)
.forSingle(res::send))
Related
I have many reactive operations .existById() to do,
then on the collected list I want to execute once some operations.
The problem is I don't know which reactive operators I should use, or how to rewrite this to archive the desirable effect.
Every try I coded effect execution of only one part of that code.
First try where no-reactive code weren't waiting on Cassandra repo:
//First part:
signaturesFromFile.forEach(signatureFromFile -> {
SignatureKey signatureKey = new SignatureKey(signatureFromFile.getProfileId(), signatureFromFile.getUserId(), signatureFromFile.getId());
signatureRepositoryCassandra.existsById(signatureKey)
.map(exists -> {
if (!exists) {
notSavedSignatures.add(signatureFromFile);
}
return null;
}).subscribe();
});
//Second part:
String notSavedSignaturesListJson = convertSignaturesToJson(notSavedSignatures);
String pathToNotSavedSigantures = NOT_TRANSFERRED_SIGNATURES_DIRECTORY + signatureFilename;
saveIfFileNotExist(getUserIdFromFileName(signatureFilename), pathToNotSavedSigantures, notSavedSignaturesListJson);
deleteTransferredByUserIdSignaturesFile(signatureFilename);
The second idea was to close it in one reactive stream, but the problem recurred and the roles reversed - operations in the second part weren't executed
Flux.fromIterable(signaturesFromFile).map(signatureFromFile -> {
SignatureKey signatureKey = new SignatureKey(signatureFromFile.getProfileId(), signatureFromFile.getUserId(), signatureFromFile.getId());
//First part:
signatureRepositoryCassandra.existsById(signatureKey)
.map(exists -> {
if (!exists) {
notSavedSignatures.add(signatureFromFile);
}
return null;
}).subscribe();
return null;
}).mergeWith(e -> {
//Second part:
String notSavedSignaturesListJson = convertSignaturesToJson(notSavedSignatures);
String pathToNotSavedSigantures = NOT_TRANSFERRED_SIGNATURES_DIRECTORY + signatureFilename;
saveIfFileNotExist(getUserIdFromFileName(signatureFilename), pathToNotSavedSigantures, notSavedSignaturesListJson);
deleteTransferredByUserIdSignaturesFile(signatureFilename);
}).subscribe();
I have a workaround for that problem ->
.findAll() and filter data instead of many executed .existById()
but I would like to resolve it with the correct operators ;)
If it wasn't reactive I'd do something like.
/**
* Given a refresh token and the user name, create a "payload" to represent
* secret data and store it into Redis as a hash and set it to expire in 30 seconds.
*/
Map<String, String> provideAuthenticatedData(String refreshTokenMono, String username) {
var ops = redisTemplate.opsForHash();
var payload = Map.of(
"username", username,
"secret", UUID.randomUUID().toString()
);
var puts = payload.entrySet()
.stream()
.map(e->ops.putIfAbsent(refreshToken, e.key(), e.value())
.filter(success -> !success) // finds those that have failed
.toList();
if (!puts.isEmpty()) {
throw new IllegalStateException("some elements failed to save");
}
var expireCheck = redisTemplate.expireAt(refreshToken, Instant.now().plusSeconds(30));
if (!expireCheck) {
throw new IllegalStateException("unable to expire");
}
return payload;
}
Trying to do it with Reactive it looks to get a bit messier and I got stuck after a point
/**
* Given a refresh token mono and the user name, create a "payload" to represent
* secret data and store it into Redis as a hash and set it to expire in 30 seconds.
*/
Mono<Map<String, String>> provideAuthenticatedData(Mono<String> refreshTokenMono, String username) {
var ops = reactiveRedisTemplate.opsForHash();
var payload = Map.of(
"username", username,
"secret", UUID.randomUUID().toString()
);
return refreshTokenMono
.flatMapIterable(
refreshToken -> payload.entrySet()
.stream()
.map(
e -> ops.putIfAbsent(refreshToken, e.getKey(), e.getValue())
)
.toList() // can't find an operator that would take a stream
)
// at this point I have a Flux<Mono<Boolean>>
// somehow I have to find out if any of them are false then return a Mono.error()
// then once all of it is done, set the key to expire
// finally return the payload I originally created
}
Another approach I did was this but it does not do any error handling.
Mono<Map<String, String>> provideAuthenticatedDataMono(
Mono<String> refreshTokenMono, String username) {
var ops = reactiveRedisTemplate.opsForHash();
var payload = Map.of(
"username", username,
"secret", UUID.randomUUID().toString()
);
return refreshTokenMono
.doOnNext(
refreshToken ->
payload
.entrySet()
.stream()
.map(
e -> ops.putIfAbsent(
refreshToken,
e.getKey(),
e.getValue())
)
.forEach(Mono::subscribe)
)
.doOnNext(
refreshToken ->
redisTemplate
.expireAt(
refreshToken,
Instant.now().plusSeconds(30)
)
.subscribe()
)
.flatMap((x) -> just(payload));
}
}
The main idea of reactive is to work in stream so you should avoid subscribing everywhere you should just return the Flux, Mono Stream.
THe first example is not working as you dont subscribe the mono that redis gives you.
As you are using reactive why you mix it with java stream.
one solution would be like
public Mono< YOUR_OBJECT > save(Object YOUR_OBJECT) {
return template.opsForValue().set(YOUR_OBJECT.key, YOUR_OBJECT)
.filter(aBoolean -> aBoolean)
.map(aBoolean -> YOUR_OBJECT)
.switchIfEmpty(Mono.error(new RuntimeException("Could not save data to redis")));
}
And you should be continuing the stream till the end when it will be subscribed by a controller or you
I have the following response coming from a rest call and performing some logic based on what is returned.
This is what I want.
If the overall status code is NOT 200 OR
If within the list of SimpleResponse, none of the SimpleResponse objects has a 200 httpCode, throw an error.
The example below is incomplete.
Too much going on there, having an optional within another optional.
And then throwing the same error at 2 different places.
Also if the optional result is null say at
responseEntity.getBody(), would want to throw same error too.
Is there a cleaner way to write this?
These are the 2 related objects
#Getter
#Setter
public class SimpleResponses {
private List<SimpleResponse> simpleResponsesList;
}
#Getter
#Setter
public class SimpleResponse {
private String httpCode;
// ... other fields
}
Method calling rest call and throwing error if needed.
public ResponseEntity<SimpleResponses> get() {
HttpEntity<Object> httpEntity = this.getEntity();
// restTemplate is from Spring
ResponseEntity<SimpleResponses> responseEntity = restTemplate.exchange(url, HttpMethod.GET, httpEntity, SimpleResponses.class);
// START
// This is the logic to throw error depending on output as mentioned above.
// looking for a better way to write this.
// if none of the object inside the list has 200 code, throw error
Optional.ofNullable(responseEntity.getBody())
.map(SimpleResponses::getSimpleResponses)
.ifPresent(response -> {
Optional<SimpleResponse> simpleResponse = response.stream()
.filter(responseStream -> responseStream.getHttpCode().equals("200"))
.findAny();
if (!simpleResponse.isPresent()) {
throw new CustomRuntimeException("Failed ..... "); // repetitive same error being thrown again below.
}
});
// if overall code is not 200, throw error too
if (!responseEntity.getStatusCode().is2xxSuccessful()) {
throw new CustomRuntimeException("Failed ..... ");
}
// END
return responseEntity;
}
Using Optional::flatMap and Optional::ifPresentOrElse the following solution may be offered:
Optional.ofNullable(responseEntity.getBody())
.flatMap(body -> SimpleResponses.getSimpleResponses(body) // SimpleResponses
.getSimpleResponsesList()
.stream() // Stream<SimpleResponse>
.filter(sr -> "200".equals(sr.getHtppCode()))
.findAny() // Optional<SimpleResponse>
)
.ifPresentOrElse( // Java 9
System.out::println,
() -> { throw new CustomRuntimeException("Bad response"); }
);
For Java 8, Optional::orElseThrow may be used:
SimpleResponse good = Optional.ofNullable(responseEntity.getBody())
.flatMap(body -> SimpleResponses.getSimpleResponses(body) // SimpleResponses
.getSimpleResponsesList()
.stream() // Stream<SimpleResponse>
.filter(sr -> "200".equals(sr.getHtppCode()))
.findAny() // Optional<SimpleResponse>
)
.orElseThrow(() -> new CustomRuntimeException("Bad response"));
System.out.println(good);
I've got a list of futures that perform data deletion for given list of studentIds from cassandra:
val studentIds: List<String> = getStudentIds(...)
val boundStatements: List<BoundStatement> = studentIds.map(bindStudentDelete(it))
val deleteFutures = boundStatements.map { session.executeAsync(it) }
deleteFutures.forEach {
// callback that will send metrics for monitoring
Futures.addCallback(it, MyCallback(...))
}
Above I have registered a callback MyCallback(...) for each future for sending metrics. Then I do:
Futures.inCompletionOrder(deleteFutures).forEach { it.get() }
to wait for the completion of all the deletes. If for any reason that some of the futures end up failing (cancelled, something else goes wrong, etc.), I want to return the list of studentIds so that I can deal with it later.
What is the best way to achieve that?
EDIT
The callback could be a way to mutate a state to track success/failure of all the deletions.
class MyCallback(
private val statsDClient: StatsdClient,
private val tags: Array<String>,
val failures: MutableList<String>
) : FutureCallback<Any> {
override fun onSuccess(result: Any?) {
//send success metrics
...
}
override fun onFailure(t: Throwable) {
// send failure metrics
...
// do something here to get the associated studentId
val currId = ...
failures.add(currId)
}
}
Similarly, I could mutate a state in Futures.inCompletionOrder(deleteFutures).forEach block with a try/catch:
val failedDeletes = mutableListOf<String>()
Futures.inCompletionOrder(deleteFutures).forEach {
try {
it.get()
} catch (e: Exception) {
// do something to get the studentId for this future
val currId = ...
failedDeletes.add(currId)
}
}
However, there are 2 things I don't like/know about it. One is that it's mutating a state that we have to define outside. The other is that I still don't know how to get the studentId from the point of failure (in onFailure or catch block).
I have added a code snippet below in JAVA. This is blocking procedure.
ResultSet getUninterruptibly()
Waits for the query to return and return its result. This method is
usually more convenient than Future.get() because it:
Waits for the result uninterruptibly, and so doesn't throw InterruptedException.
Returns meaningful exceptions, instead of having to deal with ExecutionException.
As such, it is the preferred way to get the future result.
Check this link:
Interface ResultSetFuture
List<ResultSetFuture> futures = new ArrayList<>();
List<Long> futureStudentIds = new ArrayList<>();
// List<Long> successfullIds = new ArrayList<>();
List<Long> unsuccessfullIds = new ArrayList<>();
for (long studentid : studentids) {
futures.add(session.executeAsync(statement.deleteStudent(studentid)));
futureStudentIds.add(studentid);
}
for (int index = 0; index < futures.size(); index++) {
try {
futures.get(index).getUninterruptibly();
// successfullIds.add(futureStudentIds.get(index));
} catch (Exception e) {
unsuccessfullIds.add(futureStudentIds.get(index));
LOGGER.error("", e);
}
}
return unsuccessfullIds;
For Non-blocking you can use ListenableFuture.
Asynchronous queries with the Java driver
I am fairly new to functional programming and reactive RxJava. I want to get id and name of a device from database and store it in a Map, I am doing it in RxJava style. I am calling a function that doesn't need to return anything
.doOnNext(t -> updateAssetNameMap())
then the function looks like;
private void updateDeviceNameMap() {
LOGGER.debug("Reading device name and id from database");
Observable<SQLConnection> jdbcConnection = createJdbcConnection();
Scheduler defaultScheduler = RxHelper.scheduler(vertx);
Observable<JsonArray> res = jdbcConnection //need to return JsonArray
.flatMap(connection -> just(connection)
.flatMap(j -> runQuery(connection, "SELECT name,id FROM device")
.observeOn(defaultScheduler)
.doOnNext(m -> LOGGER.info("size: " + m.size()))
.flatMap(job -> { LOGGER.info(">>" + job.getJsonArray(0));
//or if I can extract JsonArray items here,
//I can update my Map here too.
return just(job.getJsonArray(0));
}
)
.doOnError(e -> { LOGGER.error("failed to connect to db", e);
connection.close(); })
.doOnCompleted(connection::close)
.onErrorReturn(e -> null));
//System.out.println("" + res.map(d -> LOGGER.info(d.toString())));
//get the JsonArray and update the deviceNameMap
The connection to DB is made successfully and query is also done correctly.
I can convert any Object to Observable by Observable.from(ObjectName), but can't to the opposite. An appropriate mapping needs to be done after .flatMap(job -> just(job.getJsonArray(0)) but I have no clue how. After running the Verticle, I even cannot see anything logged from line .flatMap(job -> { LOGGER.info(">>" + job.getJsonArray(0));.
Am I missing something ?
You must subscribe to your Observable<JsonArray> otherwise nothing happens.