I can't find a correct solution to this problem and I'm stuck. Let's say I have this method
#GET
#Path("/testingAsync")
public Uni<List<String>> testingMutiny() {
List<String> completeList = new ArrayList<>();
completeList.add("hello");
completeList.add("RestEasy");
List<String> finalList = new ArrayList<>();
completeList.forEach(e -> Uni.createFrom().item(e)
.onItem().delayIt().by(Duration.ofMillis(10000))
.map(value -> finalList.add(value.toUpperCase()))
.subscribe().asCompletionStage());
return Uni.createFrom().item(finalList);
}
As you see the method is simple it just takes the values from 1 list and adds them to the second one but what's the problem? When you add the waiting .onItem().delayIt().by(Duration.ofMillis(10000)) the program will return an empty list and after a while, it will just update the list. I created this method to simulate a request that the response that has some delay in it.
Let's say you hit 2 URLs with 2 different Unis after that you try to combine them and return it as one Uni. The problem is if one of those 2 URLs delay for some reason we will return the list empty but I don't want that to happen I either want the list to be completed 100% or return an error if it takes a while.
What is the best approach to that? I understand that if you add await() you are blocking the main thread and you lose all the value of using the reactive library but still, I can't find a way for this to work
EDIT
I have found out that the external URL I try to call takes about 5 seconds to do the job so I want my code to stop when creating the Uni and continue after I have received an answer from the server. I have seen in their docs (here) That I can also call await.indefinitely but I receive The current thread cannot be blocked: vert.x-eventloop-thread-14. How do I wait for a response from the server?
EDIT 2
I understand that with strings it doesn't make sense my question so much so let's say I have the following one
#GET
#Path("/testingAsync")
public Uni<List<Car>> testingMutiny() {
//ALL THIS IS IN A FOR EACH FOR EVERY CAR
//HIT ENDPOINT GET DOORS
Uni<List<JsonObjectCar>> carDoorsUni = getDoors(variable1,
variable2, variable3);
//HIT ENDPOINT GET WHEELS
Uni<List<JsonObjectCar>> carWheelsUni = getWheels(variable1,
variable2, variable3);
//HIT ENDPOINT GET WINDOWS
Uni<List<JsonObjectCar>> carWindowsUni = getWindows(variable1,
variable2, variable3);
Uni.combine()
.all()
.unis(carDoorsUni, carWheelsUni, carWindowsUni)
.combinedWith((carDoors, carWheels, carWindows) -> {
//Check if cardoors is present and set the doors into the car object
Optional.of(carDoors)
.ifPresent(val -> car.setDoors(val.getDoors()));
Optional.of(carWheels)
.ifPresent(val -> car.setWheels(val.getWheels()));
Optional.of(carWindows)
.ifPresent(val -> car.setWindows(val.getWindows()));
return car;
}).subscribe().with(e-> System.out.println("Okay it worked"));
//END OF FOR EACH
//Return car (Should have been returned with new doors / wheels/ windows but instead its empty)
return Uni.createFrom().item(car);
}
As you see in the above code It should have hit some endpoints for doors / wheels / windows and set them into the variable car but what happens, in reality, is that the car is empty because one of those endpoints has been delayed so i return a car without those values inside it. I want to first update the car object and then actually return it
You could rewrite the method like this:
#GET
#Path("/testingAsync")
public Uni<List<String>> testingMutiny() {
List<Uni<String>> unis = new ArrayList<>();
List.of("hello", "RestEasy").forEach( e -> {
unis.add( Uni.createFrom().item( e )
.onItem().delayIt().by( Duration.ofMillis( 10000 ) ) );
} );
return Uni.combine().all().unis( unis )
.combinedWith( list -> (List<String>) list);
}
Note that when you write reactive code, you want to avoid using .await().indefinetly. It shouldn't be needed anyway when using Quarkus, because it recognizes async API and interpret the results accordingly.
You also don't need to subscribe the Uni or Multi when using Quarkus, for the same reason.
Based on my previous example, you can rewrite your use case with endpoints as:
#GET
#Path("/testingAsync")
public Uni<Car> testingMutiny() {
Uni<List<JsonObjectCar>> carDoorsUni = getDoors(variable1, variable2, variable3);
Uni<List<JsonObjectCar>> carWheelsUni = getWheels(variable1,variable2, variable3);
Uni<List<JsonObjectCar>> carWindowsUni = getWindows(variable1,variable2, variable3);
return Uni.combine()
.all()
.unis(carDoorsUni, carWheelsUni, carWindowsUni)
.combinedWith(list -> {
// Result of carDoorsUni
List<JsonObjectCar> carDoors = list.get(0);
// Result of carWheelsUni
List<JsonObjectCar> carWheels = list.get(1);
// Result of carWindowsUni
List<JsonObjectCar> carWindows = list.get(2);
// Create a car instance with the previous results
Car car = createCar(...);
// You can also return a list of cars, but you need to change the return type of testingMutiny to Uni<List<Car>>
return car;
})
.invoke( () -> System.out.println("Okay it worked"));
}
You return a list, but the asynchronous processing on the Uni is delayed, so your list will be empty.
You should try returning a Uni from the pipeline that you create (and also see collect(), toUni() to put into lists) instead of doing a subscription, collect the results and re-wrap into a Uni.
I have a specific use case for data processing where I am returning a future of type Future<List<SamplePOJO>>. I have multiple such futures which I am adding to a List.
But CompositeFuture.join() doesn't work on this list as it is asking for a List<Future> instead of a List<Future<List<SamplePOJO>>>. Is there any workaround for this?
You can collect all those Future<List<SamplePOJO>> in the List<Future> instead of List<Future<List<SamplePOJO>>>.
That will make CompositeFuture.all method accept it.
Future<List<String>> f = getFuture();
List<Future> futures = new ArrayList<>();
futures.add(f);
CompositeFuture.all(futures);
Here's an expanded set of example code (that I mistakenly wrote for another question and moved here).
So there exists a bug in Vert.x that causes issues with CompositeFuture.all(listoffutures), etc., at least in JDK 17, if listoffutures is of type List<Future<SomeType>> (or List<Future<?>>).
This bug might get fixed in Vert.x 5.
I got some success with the code below. The contrived example here is that I want to turn a List<Future<File>> into a Future<List<File>>.
#SuppressWarnings("rawtypes")
static List<Future> ltol(List<Future<File>> sa) {
List<Future> l = new ArrayList<>();
l.addAll(sa);
return l;
}
// A contrived example of what I was doing, which uses .compose and returns
// a Future of the list of results (File objects in my case)
Future<List<File>> mymethodcall(List<Future<File>> attachments) {
return CompositeFuture.all(ltol(attachments)).compose(files -> {
// Note we're reading the result of the .all call in the compose
List<File> mb = new ArrayList<>();
files.list().stream().forEach(o -> {
// Do whatever you need to do here with the results but they'll likely
// need to be cast (to File, in this case).
mb.add((File) o);
});
return Future.succeededFuture(mb);
});
}
The important step is getting your List<Future<T> into a List<Future>, if you need to. I did it by gross brute force in the static method above.
I have some problems with using Optional.ifPresent statement. I would like to reduce number of NullPointerExceptions, so I decided to use Optional values.
Also I am trying to avoid a ladder of if statements anti-pattern.
So I implemented Optional.isPresent statement. But it's not really that what I expected.
Please look at these listings:
This is a part of my service:
if (getAllComputerProducers().isPresent()) {
if (isComputerProducerAlreadyExist(computerProducer))
return new ResponseEntity<>(HttpStatus.CONFLICT);
}
computerProducerRepository.save(computerProducer);
return new ResponseEntity<>(HttpStatus.CREATED);
getAllComputerProducers function looks like that:
private Optional<List<ComputerProducer>> getAllComputerProducers() {
return Optional.ofNullable(computerProducerRepository.findAll());
}
As you can see, this function returns Optional of List.
The isComputerProducerAlreadyExist function is implemented like that:
private boolean isComputerProducerAlreadyExist(ComputerProducer computerProducer) {
return getAllComputerProducers()
.get()
.stream()
.anyMatch(producer -> producer.getProducerName()
.equalsIgnoreCase(computerProducer.getProducerName()));
}
It's so much code and I believe that it could be made simpler.
My target is to reduce code to one line command like:
getAllCimputerProducers().ifPresent(***and-here-some-anyMatch-boolean-function***)
but I can't insert there a function which returns something. How can I do it?
Regards to everyone :)
You could try something like
private boolean isComputerProducerAlreadyExist(ComputerProducer computerProducer){
return this.getAllComputerProducers()
.map((List<ComputerProducer> computerProducers) -> computerProducers.stream()
.anyMatch(producer -> producer.getProducerName().equalsIgnoreCase(computerProducer.getProducerName())))
.orElse(Boolean.FALSE);
}
Or instead of loading all computer producers load only the ones using its name.
private boolean isComputerProducerAlreadyExist(ComputerProducer computerProducer){
return computerProducerRepository.findByName(computerProducer.getProducerName()).isEmpty();
}
And as far as I know Spring supports also "exist" methods for repositories without even the need to load the Entity.
The following should work
Predicate<ComputerProducer> cpPredicate = producer -> producer.getProducerName()
.equalsIgnoreCase(computerProducer.getProducerName());
boolean compProdExists = getAllCimputerProducers()
.map(list -> list.stream()
.filter(cpPredicate)
.findFirst()))
.isPresent();
You can pass the computerProducer.getProducerName() to repository to get the existing record. Method name will be 'findByProducerName(String producerName)', if producerName has unique constraint, return type will be Optional<ComputerProducer>, else Optional<List<ComputerProducer>>. However, JPA returns empty list instead of null, so optional on list is not required.
Scenario: I have a customerID string that is used to query multiple different backend systems: calendar, helpdesk, ERP, CRM etc. I want to compile a single report.
So I have roughly (psydocode):
Result myResult = new Result();
Observable<Cal> cal = Calbackend.get(customerid);
cal.subscribe(calentry -> myResult.addCal(calentry));
Observable<Erp> erp = ERPbackend.get(customerid);
erp.subscribe(erpentry -> myResult.addErp(erpentry));
Observable<Help> help = Helpbackend.get(customerid);
help.subscribe(helpentry -> myResult.addHelp(helpentry));
Observable<Crm> crm = CRMbackend.get(customerid);
crm.subscribe(crmentry -> myResult.addCrm(crmentry));
// Magic here?
return result;
The approach I was thinking of: using defer() to prevent the start and then additionally subscribe to count() for each. Then I could ZIP the count elements since they only will emit a single item each (while the others will have different numbers of events). However that could lead to loss of data if the myResult.add is performing slower than the count().
The other option I was thinking of, is to set an array of boolean flags for each subscription and check in each completion (and error) event if all of them are done and do a callback or use blocking for that one.
I had a look here and here but that examples deal with constant numbers or data types.
Or is there a better / recommended way?
Operator toList can be used together with zip like this:
Observable<List<Cal>> cal = Calbackend.get(customerid).toList();
Observable<List<Erp>> erp = ERPbackend.get(customerid).toList();
Observable<List<Help>> help = Helpbackend.get(customerid).toList();
Observable<List<Crm>> crm = CRMbackend.get(customerid).toList();
Observable.zip(cal, erp, help, crm,
new Func4<List<Cal>, List<Erp>, List<Help>, List<Crm>, Result>() {
#Override
public Result call(List<Cal> cals, List<Erp> erps, List<Help> helps, List<Crm> crms) {
Result myResult = new Result();
// add all cals, erps, helps and crms to result
return myResult;
}
})
.subscribe(new Subscriber<Result>() {
#Override
public void onNext(Result result) {
// do something with the result
}
...
});
Explanation: As the name suggests, the toList operator creates a list of the items emitted by the source observable (the list is emitted just once, when the source observable completes) and zip is then used to combine the results of the observables.
Edit: In case of the possibility that those Observables can emit an error, you could use onErrorReturn to keep the normal flow going:
Observable<List<Cal>> cal = Calbackend.get(customerid)
.onErrorReturn(new Func1<Throwable, Cal>() {
#Override
public Cal call(Throwable throwable) {
// Return something in the error case
return null;
}
})
.toList();
When do you use map vs flatMap in RxJava?
Say, for example, we want to map Files containing JSON into Strings that contain the JSON--
Using map, we have to deal with the Exception somehow. But how?:
Observable.from(jsonFile).map(new Func1<File, String>() {
#Override public String call(File file) {
try {
return new Gson().toJson(new FileReader(file), Object.class);
} catch (FileNotFoundException e) {
// So Exception. What to do ?
}
return null; // Not good :(
}
});
Using flatMap, it's much more verbose, but we can forward the problem down the chain of Observables and handle the error if we choose somewhere else and even retry:
Observable.from(jsonFile).flatMap(new Func1<File, Observable<String>>() {
#Override public Observable<String> call(final File file) {
return Observable.create(new Observable.OnSubscribe<String>() {
#Override public void call(Subscriber<? super String> subscriber) {
try {
String json = new Gson().toJson(new FileReader(file), Object.class);
subscriber.onNext(json);
subscriber.onCompleted();
} catch (FileNotFoundException e) {
subscriber.onError(e);
}
}
});
}
});
I like the simplicity of the map, but the error handling of flatmap (not the verbosity). I haven't seen any best practices on this floating around and I'm curious how this is being used in practice.
map transform one event to another.
flatMap transform one event to zero or more event. (this is taken from IntroToRx)
As you want to transform your json to an object, using map should be enough.
Dealing with the FileNotFoundException is another problem (using map or flatmap wouldn't solve this issue).
To solve your Exception problem, just throw it with a Non checked exception : RX will call the onError handler for you.
Observable.from(jsonFile).map(new Func1<File, String>() {
#Override public String call(File file) {
try {
return new Gson().toJson(new FileReader(file), Object.class);
} catch (FileNotFoundException e) {
// this exception is a part of rx-java
throw OnErrorThrowable.addValueAsLastCause(e, file);
}
}
});
the exact same version with flatmap :
Observable.from(jsonFile).flatMap(new Func1<File, Observable<String>>() {
#Override public Observable<String> call(File file) {
try {
return Observable.just(new Gson().toJson(new FileReader(file), Object.class));
} catch (FileNotFoundException e) {
// this static method is a part of rx-java. It will return an exception which is associated to the value.
throw OnErrorThrowable.addValueAsLastCause(e, file);
// alternatively, you can return Obersable.empty(); instead of throwing exception
}
}
});
You can return too, in the flatMap version a new Observable that is just an error.
Observable.from(jsonFile).flatMap(new Func1<File, Observable<String>>() {
#Override public Observable<String> call(File file) {
try {
return Observable.just(new Gson().toJson(new FileReader(file), Object.class));
} catch (FileNotFoundException e) {
return Observable.error(OnErrorThrowable.addValueAsLastCause(e, file));
}
}
});
FlatMap behaves very much like map, the difference is that the function it applies returns an observable itself, so it's perfectly suited to map over asynchronous operations.
In the practical sense, the function Map applies just makes a transformation over the chained response (not returning an Observable); while the function FlatMap applies returns an Observable<T>, that is why FlatMap is recommended if you plan to make an asynchronous call inside the method.
Summary:
Map returns an object of type T
FlatMap returns an Observable.
A clear example can be seen here: http://blog.couchbase.com/why-couchbase-chose-rxjava-new-java-sdk .
Couchbase Java 2.X Client uses Rx to provide asynchronous calls in a convenient way. Since it uses Rx, it has the methods map and FlatMap, the explanation in their documentation might be helpful to understand the general concept.
To handle errors, override onError on your susbcriber.
Subscriber<String> mySubscriber = new Subscriber<String>() {
#Override
public void onNext(String s) { System.out.println(s); }
#Override
public void onCompleted() { }
#Override
public void onError(Throwable e) { }
};
It might help to look at this document: http://blog.danlew.net/2014/09/15/grokking-rxjava-part-1/
A good source about how to manage errors with RX can be found at: https://gist.github.com/daschl/db9fcc9d2b932115b679
In your case you need map, since there is only 1 input and 1 output.
map - supplied function simply accepts an item and returns an item which will be emitted further (only once) down.
flatMap - supplied function accepts an item then returns an "Observable", meaning each item of the new "Observable" will be emitted separately further down.
May be code will clear things up for you:
Observable.just("item1").map( str -> {
System.out.println("inside the map " + str);
return str;
}).subscribe(System.out::println);
Observable.just("item2").flatMap( str -> {
System.out.println("inside the flatMap " + str);
return Observable.just(str + "+", str + "++" , str + "+++");
}).subscribe(System.out::println);
Output:
inside the map item1
item1
inside the flatMap item2
item2+
item2++
item2+++
The question is When do you use map vs flatMap in RxJava?. And I think a simple demo is more specific.
When you want to convert item emitted to another type , in your case converting file to String, map and flatMap can both work. But I prefer map operator because it's more clearly.
However in some place, flatMap can do magic work but map can't. For example, I want to get a user's info but I have to first get his id when user login in. Obviously I need two requests and they are in order.
Let's begin.
Observable<LoginResponse> login(String email, String password);
Observable<UserInfo> fetchUserInfo(String userId);
Here are two methods, one for login returned Response, and another for fetching user info.
login(email, password)
.flatMap(response ->
fetchUserInfo(response.id))
.subscribe(userInfo -> {
// get user info and you update ui now
});
As you see, in function flatMap applies, at first I get user id from Response then fetch user info. When two requests are finished, we can do our job such as updating UI or save data into database.
However if you use map you can't write such nice code. In a word, flatMap can help us serialize requests.
The way I think about it is that you use flatMap when the function you wanted to put inside of map() returns an Observable. In which case you might still try to use map() but it would be unpractical. Let me try to explain why.
If in such case you decided to stick with map, you would get an Observable<Observable<Something>>. For example in your case, if we used an imaginary RxGson library, that returned an Observable<String> from it's toJson() method (instead of simply returning a String) it would look like this:
Observable.from(jsonFile).map(new Func1<File, Observable<String>>() {
#Override public Observable<String>> call(File file) {
return new RxGson().toJson(new FileReader(file), Object.class);
}
}); // you get Observable<Observable<String>> here
At this point it would be pretty tricky to subscribe() to such an observable. Inside of it you would get an Observable<String> to which you would again need to subscribe() to get the value. Which is not practical or nice to look at.
So to make it useful one idea is to "flatten" this observable of observables (you might start to see where the name _flat_Map comes from). RxJava provides a few ways to flatten observables and for sake of simplicity lets assume merge is what we want. Merge basically takes a bunch of observables and emits whenever any of them emits. (Lots of people would argue switch would be a better default. But if you're emitting just one value, it doesn't matter anyway.)
So amending our previous snippet we would get:
Observable.from(jsonFile).map(new Func1<File, Observable<String>>() {
#Override public Observable<String>> call(File file) {
return new RxGson().toJson(new FileReader(file), Object.class);
}
}).merge(); // you get Observable<String> here
This is a lot more useful, because subscribing to that (or mapping, or filtering, or...) you just get the String value. (Also, mind you, such variant of merge() does not exist in RxJava, but if you understand the idea of merge then I hope you also understand how that would work.)
So basically because such merge() should probably only ever be useful when it succeeds a map() returning an observable and so you don't have to type this over and over again, flatMap() was created as a shorthand. It applies the mapping function just as a normal map() would, but later instead of emitting the returned values it also "flattens" (or merges) them.
That's the general use case. It is most useful in a codebase that uses Rx allover the place and you've got many methods returning observables, which you want to chain with other methods returning observables.
In your use case it happens to be useful as well, because map() can only transform one value emitted in onNext() into another value emitted in onNext(). But it cannot transform it into multiple values, no value at all or an error. And as akarnokd wrote in his answer (and mind you he's much smarter than me, probably in general, but at least when it comes to RxJava) you shouldn't throw exceptions from your map(). So instead you can use flatMap() and
return Observable.just(value);
when all goes well, but
return Observable.error(exception);
when something fails.
See his answer for a complete snippet: https://stackoverflow.com/a/30330772/1402641
Here is a simple thumb-rule that I use help me decide as when to use flatMap() over map() in Rx's Observable.
Once you come to a decision that you're going to employ a map transformation, you'd write your transformation code to return some Object right?
If what you're returning as end result of your transformation is:
a non-observable object then you'd use just map(). And map() wraps that object in an Observable and emits it.
an Observable object, then you'd use flatMap(). And flatMap() unwraps the Observable, picks the returned object, wraps it with its own Observable and emits it.
Say for example we've a method titleCase(String inputParam) that returns Titled Cased String object of the input param. The return type of this method can be String or Observable<String>.
If the return type of titleCase(..) were to be mere String, then you'd use map(s -> titleCase(s))
If the return type of titleCase(..) were to be Observable<String>, then you'd use flatMap(s -> titleCase(s))
Hope that clarifies.
I just wanted to add that with flatMap, you don't really need to use your own custom Observable inside the function and you can rely on standard factory methods/operators:
Observable.from(jsonFile).flatMap(new Func1<File, Observable<String>>() {
#Override public Observable<String> call(final File file) {
try {
String json = new Gson().toJson(new FileReader(file), Object.class);
return Observable.just(json);
} catch (FileNotFoundException ex) {
return Observable.<String>error(ex);
}
}
});
Generally, you should avoid throwing (Runtime-) exceptions from onXXX methods and callbacks if possible, even though we placed as many safeguards as we could in RxJava.
In that scenario use map, you don't need a new Observable for it.
you should use Exceptions.propagate, which is a wrapper so you can send those checked exceptions to the rx mechanism
Observable<String> obs = Observable.from(jsonFile).map(new Func1<File, String>() {
#Override public String call(File file) {
try {
return new Gson().toJson(new FileReader(file), Object.class);
} catch (FileNotFoundException e) {
throw Exceptions.propagate(t); /will propagate it as error
}
}
});
You then should handle this error in the subscriber
obs.subscribe(new Subscriber<String>() {
#Override
public void onNext(String s) { //valid result }
#Override
public void onCompleted() { }
#Override
public void onError(Throwable e) { //e might be the FileNotFoundException you got }
};);
There is an excellent post for it: http://blog.danlew.net/2015/12/08/error-handling-in-rxjava/
RxJava Map vs FlatMap
They both are Transforming operators but map has 1-1 relation and flatMap has 1-0 or many relation.
map and flatmap emits stream with
map- only 1 element
flatmap - 0/many elements
map emits single element and flatmap emits a stream of elements
Map operator
map(new Function<A, B>() {
#Override
public B apply(A a) throws Exception {
B b = new B(a);
return b;
}
})
FlatMap operator
flatMap(new Function<A, ObservableSource<B>>() {
#Override
public ObservableSource<B> apply(A a) throws Exception {
return foo(a);
}
})
[flatMap vs concatMap]
[Swift map vs flatMap]
In some cases you might end up having chain of observables, wherein your observable would return another observable. 'flatmap' kind of unwraps the second observable which is buried in the first one and let you directly access the data second observable is spitting out while subscribing.
Flatmap maps observables to observables.
Map maps items to items.
Flatmap is more flexible but Map is more lightweight and direct, so it kind of depends on your usecase.
If you are doing ANYTHING async (including switching threads), you should be using Flatmap, as Map will not check if the consumer is disposed (part of the lightweight-ness)