Java version : 11
I have a List, which contains many sublist and for each sublist I want to perform certain transformation/operations.
I want to perform this operation in non-blocking asynchronous fashion, so I am using CompletableFuture.
This is my operation:
public static List<String> convertBusinessObjectJson(List<BusinessObject> businessObjList) {
List<Either> eitherValueOrException = {//omitted logic to convert to json}
return eitherValueOrException;
}
It returns a List of Either Objects, where Either holds, either runtime exception thrown by conversion logic or String result when conversion is successful.
This is my caller code:
mainList.forEach(sublist -> {
CompletableFuture<List<Either>> listCompletableFuture = CompletableFuture.supplyAsync(() -> FutureImpl.convertBusinessObjectJson(sublist));
});
Once the CompletableFuture<List<Either>> listCompletableFuture is received, I want to chain the operation,
As in
take CompletableFuture<List<Either>> listCompletableFuture, take exceptions only from list and, perform certain operation
take CompletableFuture<List<Either>> listCompletableFuture, take results only from list and, perform certain operation
Something like this (pseudo code):
mainList.forEach(sublist -> {
CompletableFuture<List<Either>> listCompletableFuture = CompletableFuture.supplyAsync(() -> FutureImpl.convertDSRowToJson(subDSRowList));
listCompletableFuture.thenApply(//function which pushes exception to say kafka)
listCompletableFuture.thenApply(//function which pushes result to say database)
});
Can it be done?
Any help is much appreciated :)
You could try smth like this:
var futureList = mainList.stream()
.map(sublist -> CompletableFuture.supplyAsync(() -> FutureImpl.convertBusinessObjectJson(sublist)))
.collect(Collectors.toList());
The above would collect a list of CompletableFutures. Now what needs to happen is we need to wait for the completion of all those futures. We do this by:
var joinedFutureList = futureList.stream()
.map(objectCompletableFuture -> {
try {
return objectCompletableFuture.get();
} catch (InterruptedException | ExecutionException e) {
throw new RuntimeException(e);
}
}).collect(Collectors.toList());
});
After that the separation would look smth like this:
var exceptionList = joinedFutureList.stream()
.filter(obj -> obj instanceof Exception)
.peek(System.out::println)
.collect(Collectors.toList());
var successList = joinedFutureList.stream()
.filter(obj -> obj instanceof String)
.peek(System.out::println)
.collect(Collectors.toList());
Related
We have an async method:
public CompletableFuture<OlderCat> asyncGetOlderCat(String catName)
Given a list of Cats:
List<Cat> cats;
We like to create a bulk operation that will result in a map between the cat name and its async result:
public CompletableFuture<Map<String, OlderCat>>
We also like that if an exception was thrown from the asyncGetOlderCat, the cat will not be added to the map.
We were following this post and also this one and we came up with this code:
List<Cat> cats = ...
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(Collectors.toMap(Cat::getName,
c -> asynceGetOlderCat(c.getName())
.exceptionally( ex -> /* null?? */ ))
));
CompletableFuture<Void> allFutures = CompletableFuture
.allOf(completableFutures.values().toArray(new CompletableFuture[completableFutures.size()]));
return allFutures.thenApply(future -> completableFutures.keySet().stream()
.map(CompletableFuture::join) ???
.collect(Collectors.toMap(????)));
But it is not clear how in the allFutureswe can get access to the cat name and how to match between the OlderCat & the catName.
Can it be achieved?
You are almost there. You don't need to put an exceptionally() on the initial futures, but you should use handle() instead of thenApply() after the allOf(), because if any future fails, the allOf() will fail as well.
When processing the futures, you can then just filter out the failing ones from the result, and rebuild the expected map:
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(toMap(Cat::getName, c -> asyncGetOlderCat(c.getName())));
CompletableFuture<Void> allFutures = CompletableFuture
.allOf(completableFutures.values().toArray(new CompletableFuture[0]));
return allFutures.handle((dummy, ex) ->
completableFutures.entrySet().stream()
.filter(entry -> !entry.getValue().isCompletedExceptionally())
.collect(toMap(Map.Entry::getKey, e -> e.getValue().join())));
Note that the calls to join() are guaranteed to be non-blocking since the thenApply() will only be executed after all futures are completed.
As I get it, what you need is CompletableFuture with all results, the code below does exactly what you need
public CompletableFuture<Map<String, OlderCat>> getOlderCats(List<Cat> cats) {
return CompletableFuture.supplyAsync(
() -> {
Map<String, CompletableFuture<OlderCat>> completableFutures = cats
.stream()
.collect(Collectors.toMap(Cat::getName,
c -> asyncGetOlderCat(c.getName())
.exceptionally(ex -> {
ex.printStackTrace();
// if exception happens - return null
// if you don't want null - save failed ones to separate list and process them separately
return null;
}))
);
return completableFutures
.entrySet()
.stream()
.collect(Collectors.toMap(
Map.Entry::getKey,
e -> e.getValue().join()
));
}
);
}
What it does here - returns future, which creates more completable future inside and waits at the end.
I'm using Java Reactor Core, and I have a reactive Flux of objects. For each object of the Flux I need to do an external query that will return one different object for each input. The newly generated Flux needs then to be zipped with the original one - so the items of the 2 Flux must be synchronized and generated in the same order.
I'm just re-using the same flow twice, like this:
Flux<MyObj> aTest = Flux.fromIterable(aListOfObj);
Flux<String> myObjLists = aTest.map(o -> MyRepository.findById(o.name)).map(o -> {
if (!o.isPresent()) {
System.out.println("Fallback to empty-object");
return "";
}
List<String> l = o.get();
if (l.size() > 1) {
System.out.println("that's bad");
}
return l.get(0);
});
Flux.zip(aTest, myObjLists, (a, b) -> doSomethingWith(a,b))
Is it the right way to do it? If the myObjLists emits an error, how do I prevent the zip phase to skip the failing iteration?
I've finally opted for using Tuples and Optionals (to prevent null-items that would break the flux), so that I don't need to re-use the initial Flux:
Flux<Tuple<MyObj, Optional<String>>> myObjLists = Flux.fromIterable(aListOfObj)
.map(o -> Tuples.of(o, Optional.ofNullable(MyRepository.findById(o.name))
.flatMap(t -> {
if (!t.getT2().isPresent()) {
System.out.println("Discarding this item");
return Flux.empty();
}
List<String> l = t.getT2().get();
if (l.size() > 1) {
System.out.println("that's bad");
}
return Tuples.of(t.getT1(), l.get(0));
})
.map(t -> doSomethingWith(t.getT1(),t.getT2()))
Note that the flatMap could be replaced with a .map().filter(), removing tuples with missing Optional items
Given I subscribe to 2 different Observables and I want get both of them on onNext after do some operations to them
let's say I have 2 Observables
Observable<List<String>> childName = Observable.from(children)... some operations
Observable<List<String>> teacherName = Observable.from(teachers)... some operations
how do I get both of them on my subscribe?
subscribe(
onNext(List<String> childName, List<String> className)
so that I can pass both of them in my listener in that manner.
I don't want to combine them, I just want when after the operations are finished, get both of them and pass it on my subscriptions
You can zip their values into a Pair:
Observable.zip(childName, className,
(a, b) -> Pair.of(a, b))
.subscribe((Pair<List<String>, List<String>> pair) -> {
// do something with pair.first and pair.second
}, Throwable::printStackTrace);
hacky but simple
Observable.zip(childName, teacherName, (childList, teachersList) -> {
// handle childList & teachersList
return Observable.empty();
}).subscribe(o -> {}, error -> {
//handle errors
});
The code I'm having problems with is:
Executor executor = (Executor) callList;
List<ProgState> newProgList = executor.invokeAll(callList).stream()
.map(future -> {try {return future.get();} catch(Exception e){e.printStackTrace();}})
.filter(p -> p!=null).collect(Collectors.toList());
The method invokeAll(List>) is undefined for the type Executor
I am told I should use an executor like the one in the code snippet.
The Callables are defined within the following code:
List<Callable<ProgState>> callList = (List<Callable<ProgState>>) lst.stream()
.map(p -> ((Callable<ProgState>)(() -> {return p.oneStep();})))
.collect(Collectors.toList());
Here is the teacher's code:
//prepare the list of callables
List<Callable<PrgState>> callList = prgList.stream().map(p -> (() -> {return p.oneStep();})).collect(Collectors.toList());
//start the execution of the callables
//it returns the list of new created threads
List<PrgState> newPrgList = executor.invokeAll(callList).stream()
.map(future -> { try {
return future.get();
}
catch(Exception e) {
//here you can treat the possible
// exceptions thrown by statements
// execution
}
})
.filter(p -> p!=null).collect(Collectors.toList());
//add the new created threads to the list of existing threads
prgList.addAll(newPrgList);
If you can use stream(), why not parallelStream() as it would be much simpler.
List<PrgState> prgStates = prgList.parallelStream()
.map(p -> p.oneStep())
.collect(Collectors.toList());
This way you have no thread pool to configure, start or stop when finished.
Some might suggest that parallelStream() was the main reason for adding Stream and lambdas to Java 8 in the first place. ;)
You can't cast list of Callables with ExecutorService. You need to define ExecutorService which will inturn pick up callables and execute them in one or multiple threads in parallel.
This is what i think you are after:
ExecutorService executor = Executors.newCachedThreadPool();//change executor type as per your need.
List<ProgState> newProgList = executor.invokeAll(callList).stream().map(future -> {...
I want convert a piece of code from a Connection Pool project i have been working on to use streams
the original code is
for (Map.Entry<JdbConnection,Instant> entry : borrowed.entrySet()) {
Instant leaseTime = entry.getValue();
JdbConnection jdbConnection = entry.getKey();
Duration timeElapsed = Duration.between(leaseTime, Instant.now());
if (timeElapsed.toMillis() > leaseTimeInMillis) {
//expired, let's close it and remove it from the map
jdbConnection.close();
borrowed.remove(jdbConnection);
//create a new one, mark it as borrowed and give it to the client
JdbConnection newJdbConnection = factory.create();
borrowed.put(newJdbConnection,Instant.now());
return newJdbConnection;
}
}
throw new ConnectionPoolException("No connections available");
I have got to the point of this
borrowed.entrySet().stream()
.filter(entry -> Duration.between(entry.getValue(), Instant.now()).toMillis() > leaseTimeInMillis)
.findFirst()
.ifPresent(entry -> {
entry.getKey().close();
borrowed.remove(entry.getKey());
});
JdbConnection newJdbConnection = factory.create();
borrowed.put(newJdbConnection,Instant.now());
return newJdbConnection;
The above can compile but the moment i add orElseThrow after IfPresent i am getting the following
/home/prakashs/connection_pool/src/main/java/com/spakai/ConnectionPool.java:83: error: void cannot be dereferenced
.orElseThrow(ConnectionPoolException::new);
That's because ifPresent returns void. It can't be chained. You could do something like:
Entry<JdbConnection, Instant> entry =
borrowed.entrySet().stream()
.filter(entry -> Duration.between(entry.getValue(), Instant.now())
.toMillis() > leaseTimeInMillis)
.findFirst()
.orElseThrow(ConnectionPoolException::new));
entry.getKey().close();
borrowed.remove(entry.getKey());
What you were looking for would read well:
.findFirst().ifPresent(value -> use(value)).orElseThrow(Exception::new);
But for it to work, ifPresent would have to return the Optional, which would be a little odd. It would mean you could chain one ifPresent after another, doing multiple operations on the value. That might have been a good design, but it isn't the one the creators of Optional went with.
Use map instead of isPresent, and return with an Optional instead of an exception.
borrowed.entrySet().stream()
.filter(entry -> Duration.between(entry.getValue(), Instant.now()).toMillis() > leaseTimeInMillis)
.findFirst()
.map(entry -> {
entry.getKey().close();
borrowed.remove(entry.getKey());
JdbConnection newJdbConnection = factory.create();
borrowed.put(newJdbConnection,Instant.now());
return newJdbConnection;
})