Develop Reference

Spring Reactive programming if search is not there in database then prepare entity and save else return new instance of object

I am new to reactive world and trying to write for following logic:
find if entry is there in database corresponding to my service call. If the the data is not there in database then i have to preapre that vendorServiceAreaMapping object and then save.
VendorServiceAreaMapping vendorServiceAreaMapping = new VendorServiceAreaMapping();
vendorServiceAreaMapping.setVendorId(123);
vendorServiceAreaMapping.setClientId(456);
int count= vendorService.findCountByVendorId(vendorServiceAreaMapping.getVendorId(), vendorServiceAreaMapping.getClientId())
if(count ==0){
CityModel cityModel = cityService.getCityByName("Florida");
vendorServiceAreaMapping.setCityId(cityModel.getCityId());
vendorService.save(vendorServiceAreaMapping);
}else{
new VendorServiceAreaMapping();
}
Above code snippet is what i am trying to incorporate using spring reactive way:
public Mono<VendorServiceAreaMapping> createVendorMapping(String invitationName) {
return invitationService.getInvitationDetails(invitationName)
.flatMap(vendorServiceAreaMapping -> {
vendorService.findCountByVendorId(vendorServiceAreaMapping.getVendorId(), vendorServiceAreaMapping.getClientId())// returns integer
.doOnNext(count -> {
if(count == 0){// there is no corresponding entry in database
.flatMap(vendorServiceAreaMapping -> {
return cityService.getCityByName("Florida")
.map(cityModel -> {
vendorServiceAreaMapping.setCityId(cityModel.getCityId());
return vendorServiceAreaMapping;
});
})
.flatMap(vendorServiceAreaMapping -> {
return vendorService.save(vendorServiceAreaMapping);
})
}else{
return Mono.just(new VendorServiceAreaMapping());
}
});
}
}

You just need to create a reactive flow chaining async methods using flatMap.
public Mono<VendorServiceAreaMapping> createVendorMapping(String invitationName) {
return invitationService.getInvitationDetails(invitationName)
.flatMap(vendorServiceAreaMapping ->
vendorService.findCountByVendorId(vendorServiceAreaMapping.getVendorId(), vendorServiceAreaMapping.getClientId())
.flatMap(count -> {
if (count == 0) {
return cityService.getCityByName("Florida")
.flatMap(cityModel -> {
vendorServiceAreaMapping.setCityId(cityModel.getCityId());
return vendorService.save(vendorServiceAreaMapping);
});
} else {
return Mono.just(new VendorServiceAreaMapping());
}
})
);
}
This code assumes that all methods are reactive and return Mono<T>.

Spring WebFlux - how to determine when my client has finished working

I need to call certain API with multiple query params simultaneously, in order to do that I wanted to use reactive approach. I ended up with reactive client that is able to call endpoint based on passed SearchQuery, handle pagination of that response and call for remaining pages and returns Flux<Item>. So far it works fine, however what I need to do now is to:
Collect data for all search queries and save them as initial state
Once the initial data is collected, I need to start repeating those calls in small time intervals and validate each item against initial data. Basically, I need to find new items from here.
But I'm running out of options how to solve that, I came up with probably the dirties solution ever, but I bet there are much better ways to do that.
So first of all, this is relevant code of my client
public Flux<Item> collectData(final SearchQuery query) {
final var iteration = new int[]{0};
return invoke(query, 0).expand(res ->
this.handleResponse(res, query, iteration))
.flatMap(response -> Flux.fromIterable(response.collectItems()));
}
private Mono<ApiResponse> handleResponse(final ApiResponse response, final SearchQuery searchQuery, final int[] iteration) {
return hasNextPage(response) ? invoke(searchQuery, calculateOffset(++iteration[0])) : Mono.empty();
}
private Mono<ApiResponse> invoke(final SearchQuery query, final int offset) {
final var url = offset == 0 ? query.toUrlParams() : query.toUrlParamsWithOffset(offset);
return doInvoke(url).onErrorReturn(ApiResponse.emptyResponse());
}
private Mono<ApiResponse> doInvoke(final String endpoint) {
return webClient.get()
.uri(endpoint)
.retrieve()
.bodyToMono(ApiResponse.class);
}
And here is my service that is using this client
private final Map<String, Item> initialItems = new ConcurrentHashMap<>();
void work() {
final var executorService = Executors.newSingleThreadScheduledExecutor();
queryRepository.getSearchQueries().forEach(query -> {
reactiveClient.collectData(query).subscribe(item -> initialItems.put(item.getId(), item));
});
executorService.scheduleAtFixedRate(() -> {
if(isReady()) {
queryRepository.getSearchQueries().forEach(query -> {
reactiveClient.collectData(query).subscribe(this::process);
});
}
}, 0, 3, TimeUnit.SECONDS);
}
/**
* If after 2 second sleep size of initialItems remains the same,
* that most likely means that initial population phase is over,
* and we can proceed with further data processing
**/
private boolean isReady() {
try {
final var snapshotSize = initialItems.size();
Thread.sleep(2000);
return snapshotSize == initialItems.size();
} catch (Exception e) {
return false;
}
}
I think the code speaks for itself, I just want to finish first phase, which is initial data population and then start processing all incomming data.

RxJava valve use case

Is there an operator in RxJava, an external library or a way I'm missing to create a flowable/observable that recieves a function that controls the emission of data, like a valve?
I have a huge json file I need to process but I have to get a portion of the file, a list of entities, process it and then get another portion, I have tried using windows(), buffer() but the BiFunction I pass to Flowable.generate() keeps executing after I recieved the first list and I haven't finished processing it. I also tried FlowableTransformers.valve() from hu.akarnokd.rxjava3.operators but it just piles up the items before the flatMap() function that process the list
private Flowable<T> flowable(InputStream inputStream) {
return Flowable.generate(() -> jsonFactory.createParser(new GZIPInputStream(inputStream)), (jsonParser, emitter) -> {
final var token = jsonParser.nextToken();
if (token == null) {
emitter.onComplete();
}
if (JsonToken.START_ARRAY.equals(token) || JsonToken.END_ARRAY.equals(token)) {
return jsonParser;
}
if (JsonToken.START_OBJECT.equals(token)) {
emitter.onNext(reader.readValue(jsonParser));
}
return jsonParser;
}, JsonParser::close);
}
Edit: I need to control de emission of items to don't overload the memory and the function that process the data, because that function reads and writes to database, also the processing needs to be sequentially. The function that process the data it's not entirely mine and it's written in RxJava and it's expected that I use Rx.
I managed to solve it like this but if there is another way let me know please:
public static <T> Flowable<T> flowable(InputStream inputStream, JsonFactory jsonFactory, ObjectReader reader, Supplier<Boolean> booleanSupplier) {
return Flowable.generate(() -> jsonFactory.createParser(new GZIPInputStream(inputStream)), (jsonParser, emitter) -> {
if (booleanSupplier.get()) {
final var token = jsonParser.nextToken();
if (token == null) {
emitter.onComplete();
}
if (JsonToken.START_ARRAY.equals(token) || JsonToken.END_ARRAY.equals(token)) {
return jsonParser;
}
if (JsonToken.START_OBJECT.equals(token)) {
emitter.onNext(reader.readValue(jsonParser));
}
}
return jsonParser;
}, JsonParser::close);
}
Edit2: This is one of the ways I'm currently consuming the function
public Flowable<List<T>> paging(Function<List<T>, Single<List<T>>> function) {
final var atomicInteger = new AtomicInteger(0);
final var atomicBoolean = new AtomicBoolean(true);
return flowable(inputStream, jsonFactory, reader, atomicBoolean::get)
.buffer(pageSize)
.flatMapSingle(list -> {
final var counter = atomicInteger.addAndGet(1);
if (counter == numberOfPages) {
atomicBoolean.set(false);
}
return function.apply(list)
.doFinally(() -> {
if (atomicInteger.get() == numberOfPages) {
atomicInteger.set(0);
atomicBoolean.set(true);
}
});
});
}

Managed to solve it like this
public static Flowable<Object> flowable(JsonParser jsonParser, ObjectReader reader, PublishProcessor<Boolean> valve) {
return Flowable.defer(() -> {
final var token = jsonParser.nextToken();
if (token == null) {
return Completable.fromAction(jsonParser::close)
.doOnError(Throwable::printStackTrace)
.onErrorComplete()
.andThen(Flowable.empty());
}
if (JsonToken.START_OBJECT.equals(token)) {
final var value = reader.readValue(jsonParser);
final var just = Flowable.just(value).compose(FlowableTransformers.valve(valve, true));
return Flowable.concat(just, flowable(jsonParser, reader, valve));
}
return flowable(jsonParser, reader, valve);
});
}

Convert Observable of List to List of Observables and merge in RxJava

I'm learning Java with Android by creating Hacker News reader app.
What I'm trying to do is:
Send a request to /topstories, return Observable<List<int>>, emit when
request finishes.
Map each storyId to Observable<Story>
Merge Observables into one entity, which emits List<Story>, when all requests finishes.
And to the code:
private Observable<Story> getStoryById(int articleId) {
BehaviorSubject<Story> subject = BehaviorSubject.create();
// calls subject.onNext on success
JsonObjectRequest request = createStoryRequest(articleId, subject);
requestQueue.add(request);
return subject;
}
public Observable<ArrayList<Story>> getTopStories(int amount) {
Observable<ArrayList<Integer>> topStoryIds = (storyIdCache == null)
? fetchTopIds()
: Observable.just(storyIdCache);
return topStoryIds
.flatMap(id -> getStoryById(id))
// some magic here
}
Then we would use this like:
getTopStories(20)
.subscribe(stories -> ...)

You can try something like that
Observable<List<Integers>> ids = getIdsObservable();
Single<List<Story>> listSingle =
ids.flatMapIterable(ids -> ids)
.flatMap(id -> getStoryById(id)).toList();
Then you can subscribe to that Single to get the List<Story>

Please have a look at my solution. I changed your interface to return a Single for getStoryById(), because it should only return one value. After that, I created a for each Story a Single request and subscribed to all of them with Single.zip. Zip will execute given lambda, when all Singles are finished. On drawback is, that all requestes will be fired at once. If you do not want this, I will update my post. Please take into considerations that #elmorabea solution will also subscribe to the first 128 elements (BUFFER_SIZE = Math.max(1, Integer.getInteger("rx2.buffer-size", 128));), and to the next element when one finishes.
#Test
void name() {
Api api = mock(Api.class);
when(api.getTopStories()).thenReturn(Flowable.just(Arrays.asList(new Story(1), new Story(2))));
when(api.getStoryById(eq(1))).thenReturn(Single.just(new Story(888)));
when(api.getStoryById(eq(2))).thenReturn(Single.just(new Story(888)));
Flowable<List<Story>> listFlowable =
api.getTopStories()
.flatMapSingle(
stories -> {
List<Single<Story>> collect =
stories
.stream()
.map(story -> api.getStoryById(story.id))
.collect(Collectors.toList());
// possibly not the best idea to subscribe to all singles at the same time
Single<List<Story>> zip =
Single.zip(
collect,
objects -> {
return Arrays.stream(objects)
.map(o -> (Story) o)
.collect(Collectors.toList());
});
return zip;
});
TestSubscriber<List<Story>> listTestSubscriber =
listFlowable.test().assertComplete().assertValueCount(1).assertNoErrors();
List<List<Story>> values = listTestSubscriber.values();
List<Story> stories = values.get(0);
assertThat(stories.size()).isEqualTo(2);
assertThat(stories.get(0).id).isEqualTo(888);
assertThat(stories.get(1).id).isEqualTo(888);
}
interface Api {
Flowable<List<Story>> getTopStories();
Single<Story> getStoryById(int id);
}
static class Story {
private final int id;
Story(int id) {
this.id = id;
}
}

RxJava two request in sequence

I have two requests, second one it dependence in the First so, how to make it in sequence, because there is some check will receive null if it request in parallel
Observable<Map<Integer, SupportedVersion>> supportedVersionObservable = contentAPI
.getSupportedVersionsContent()
.compose(ReactiveUtils.applySchedulers())
.map(supportedVersionsContentContentContainer -> supportedVersionsContentContentContainer.getContent().get(0).getMessage())
.doOnNext(supportedVersionsMap -> {
Timber.i("doOnNext invoked from supported version observable");
for (Map.Entry<Integer,SupportedVersion> entry : supportedVersionsMap.entrySet())
if (Build.VERSION.SDK_INT >= entry.getKey())
model.setSupportedVersion(entry.getValue());
model.setCurrentVersionExpiryDate(model.getSupportedVersion().getCurrentVersionExpiryDate());
if (model.getSupportedVersion() != null)
model.setNewFeaturesSeen(sharedPreferencesManager.isNewFeaturesSeen(model.getSupportedVersion().getAvailableVersions().get(0)));
if (model.isNewFeaturesSeen());
//request data from here
})
.retry(1);
Observable<List<WhatsNew>> getWhatsNewFeature = contentAPI
.getWhatsNewFeature(model.getSupportedVersion().getAvailableVersions().get(0))
.compose(ReactiveUtils.applySchedulers())
.doOnNext(whatsNewList -> {
Timber.i("doOnNext invoked from supported version observable");
if (!whatsNewList.isEmpty())
model.setWhatsNews(whatsNewList);
})
.retry(1);

You can use flatMap for that:
public Observable<List<WhatsNew>> makeRequest {
return contentAPI
.getSupportedVersionsContent()
.flatMap(supportedVersionsMap -> {
//... model initialization
return contentAPI
.getWhatsNewFeature(model.getSupportedVersion().getAvailableVersions().get(0))
.compose(ReactiveUtils.applySchedulers())
.doOnNext(whatsNewList -> {
Timber.i("doOnNext invoked from supported version observable");
if (!whatsNewList.isEmpty())
model.setWhatsNews(whatsNewList);
})
.retry(1);
});

You do not need side-effects. You may hold the model-state in the operators:
#Test
void name() {
ContentApi mock = mock(ContentApi.class);
Observable<Model> modelObservable = mock.getSupportedVersionsContent()
.map(s -> {
// do Mapping
return new Model();
})
.flatMap(model -> mock.getWhatsNewFeature(model)
.map(whatsNews -> {
// Create new model with whatsNews
return new Model();
}), 1);
}
interface ContentApi {
Observable<String> getSupportedVersionsContent();
Observable<List<WhatsNew>> getWhatsNewFeature(Model model);
}
class Model {
}
class WhatsNew {
}
Please have a Look for a detail description of flatMap:
http://tomstechnicalblog.blogspot.de/2015/11/rxjava-operators-flatmap.html?m=0