class MyItem {
private param1, param2, param3;
}
MyItem item = new MyItem();
computeParam1(item);
computeParam2(item);
computeParam3(item);
waitForAllParamsToBeSet();
Each of the steps is independent from each other, and each step write the paramter into the object as final result.
The methods are completely different from their logic, no recursion.
How could I parallelize those steps, if possible at all?
Start Futures and then wait for results before assigning.
Future<Type1> item1 = ComputeParam1();
Future<Type2> item2 = ComputeParam2();
Future<Type3> item2 = ComputeParam3();
MyItem item = new MyItem();
assignParam1(item1.get());
assignParam2(item2.get());
assignParam3(item3.get());
As all computeParamX() accept one MyItem argument and have void return, they have a signature of Consumer<MyItem>. So you can parallelize their execution calling them in .forEach() of parallel stream, as follows:
final MyItem item = new MyItem();
Stream.<Consumer<MyItem>>of(this::computeParam1, this::computeParam2, this::computeParam3)
.parallel()
.forEach(c -> c.accept(item));
As .forEach() is terminal operation, it will block until all operations complete, so you can safely use item object after it returns.
In Java 8 you could simply create your collection of tasks as next:
Collection<Runnable> tasks = Arrays.asList(
() -> System.out.println("Compute param1"),
() -> System.out.println("Compute param2"),
() -> System.out.println("Compute param3")
);
Then launch the tasks in parallel
tasks.parallelStream().forEach(Runnable::run);
Related
Suppose Supplier<Optional<Item>> is the source and Supplier<Item> is what we want: the same items without Optional.Empty cases.
Inspired by Boris and khelwood:
Supplier<Item> filtered = Stream.generate(optSupplier)
.flatMap(Optional::stream)
.iterator()::next;
If you're pre-Java 9, replace flatMap with .filter(Optional::isPresent).map(Optional::get).
Given Supplier<Optional<Item>> optSupplier:
Supplier<Item> itemSupplier = () -> {
Optional<Item> opt = optSupplier.get();
while (!opt.isPresent()) {
opt = optSupplier.get();
}
return opt.get();
};
When you try to get from the item supplier, it will keep pulling from the optional supplier until it gets a result that isn't empty, and then return the item.
(Caveat: If the optional supplier always returns an empty optional, then the item supplier will be stuck in an infinite loop.)
I'm wondering if there is a way I can update two times an object in a Stream lambda code, I need to update two properties of a class, I need to update the value and the recordsCount properties
Object:
public class HistoricDataModelParsed {
private Date startDate;
private Date endDate;
private Double value;
private int recordsCount;
}
I tried doing something like this:
val existingRecord = response.stream()
.filter(dateTime ->fromDate.equals(dateTime.getStartDate()))
.findAny()
.orElse(null);
response.stream()
.filter(dateTime ->fromDate.equals(dateTime.getStartDate()))
.findAny()
.orElse(existingRecord)
.setValue(valueAdded)
.setRecordsCount(amount);
But I got this error: "Cannot invoke setRecordsCount(int) on the primitive type void"
So I ended up doing the stream two times to update each of the two fields I needed
response.stream()
.filter(dateTime ->fromDate.equals(dateTime.getStartDate()))
.findAny()
.orElse(existingRecord)
.setValue(valueAdded);
response.stream()
.filter(dateTime ->fromDate.equals(dateTime.getStartDate()))
.findAny()
.orElse(existingRecord)
.setRecordsCount(amount);
Is there a way I can achieve what I need without the need to stream two times the list?
The return type of setValue is void and not HistoricDataModelParsed. So you cannot invoke the method setRecordsCount which is in HistoricDataModelParsed class.
You could have added a method in HistoricDataModelParsed which takes two parameters for value and recordsCount:
public void setValueAndCount(Double value, int count) {
this.value = value;
this.recordsCount = count;
}
Then call this method after orElse:
response.stream()
.filter(dateTime ->fromDate.equals(dateTime.getStartDate()))
.findAny()
.orElse(existingRecord)
.setValueAndCount(valueAdded, amount);
The state of an object should not change within a stream. It can lead to inconsistent results. But you can create new instances of the objects and pass new values via the constructor. Here is a simple record that demonstrates the method. Records are basically immutable classes that have no setters. The getters are the names of the variables. A class would also work in this example.
record Temp(int getA, int getB) {
#Override
public String toString() {
return "[" + getA + ", " + getB +"]";
}
}
Some data
List<Temp> list = List.of(new Temp(10, 20), new Temp(50, 200),
new Temp(100, 200));
And the transformation. A new instance of Temp with new values is created along with the old ones to completely populate the constructor. Otherwise, the existing object is passed along.
List<Temp> result = list.stream().map(
t -> t.getA() == 50 ? new Temp(2000, t.getB()) : t)
.toList();
System.out.println(result);
Prints
[[10, 20], [2000, 200], [100, 200]]
To answer the void error you got it's because a stream expects values to continue thru out the stream so if a method is void, it isn't returning anything so you would have to return it. Here is an example:
stream.map(t->{voidReturnMethod(t); return t;}).toList();
The return ensures the pipeline continues.
Simply store the result of orElse and then call your methods on it.
HistoricDataModelParsed record =
response.stream()
.filter(dateTime -> fromDate.equals(dateTime.getStartDate()))
.findAny()
.orElse(existingRecord);
record.setValue(valueAdded)
record.setRecordsCount(amount);
So, I have sorted by condition list of objects
private Observable<CallServiceCode> getUnansweredQuestionList() {
return Observable.fromIterable(getServiceCodeArrayList())
.subscribeOn(Schedulers.computation())
.filter(iServiceCode -> iServiceCode.getServiceCodeFormStatus().isUnanswered());
}
and now what I need to do:
Every object has list servicePartList , I need to filter this list by condition and eventually if final size of this filtered list >0, so I need to add object that contains this list CallServiceCode object as a key and this filtered list as a value.
So it should be like this:
private Map<CallServiceCode, ArrayList<CallServicePart>> getSortedMap() {
Map<CallServiceCode, ArrayList<CallServicePart>> result = new HashMap<>();
getUnansweredQuestionList()
.filter(callServiceCode -> Observable.fromIterable(callServiceCode.getCallServicePartList()) //
.filter(servicePart -> servicePart.getServicePartFormStatus().isUnanswered())//
.isNotEmpty())
.subscribe(callServiceCode -> result.put(callServiceCode, Observable.fromIterable(callServiceCode.getCallServicePartList()) //
.filter(servicePart -> servicePart.getServicePartFormStatus().isUnanswered()));
return result;
}
But there is no such method isNotEmpty() in RxJava2 and also it is not right to add key like this:
Observable.fromIterable(callServiceCode.getCallServicePartList())
.filter(servicePart -> servicePart.getServicePartFormStatus().isUnanswered())
So question is how to make it properly?
One solution could be to use collect to create the Map directly from the observable:
return getUnansweredQuestionList()
.collect(HashMap<CallServiceCode, List<CallServicePart>>::new,(hashMap, callServiceCode) -> {
List<CallServicePart> callServiceParts = Observable.fromIterable(callServiceCode.getServicePartList())
.filter(s -> !s.getServicePartFormStatus().isUnanswered())
.toList().blockingGet();
if (!callServiceParts.isEmpty())
hashMap.put(callServiceCode, callServiceParts);
}).blockingGet();
If you extract filtering into a method (could be also member of CallServiceCode) then the code is much cleaner:
return getUnansweredQuestionList()
.collect(HashMap<CallServiceCode, List<CallServicePart>>::new, (hashMap, callServiceCode) -> {
List<CallServicePart> filteredParts = getFilteredServiceParts(callServiceCode.getServicePartList());
if (!filteredParts .isEmpty())
hashMap.put(callServiceCode, filteredParts);
}).blockingGet();
I have an infinite observable (db observer) that emits finite sequences of items, that need to be processed and then emitted back as a group.
The problem here is, how to write it so that toList doesn't wait for the original source to finish, but for the flatMapIterable generated sequence instead;
DbObservable.map(new Func1<Query, List<Item>>() {
#Override
public List<Item> call(Query query) {
return query.items; //get items from db query
}
})
.flatMapIterable(new Func1<List<Item>, Iterable<Item>>() {
#Override
public Iterable<Item> call(List<GeoStop> geoStops) {
return geoStops;
}
})
.flatMap(/*process*/)
.toList() // regenerate List that was passed in to flatMapIterable
//subscribe and emit list of items
Nothing reaches subscribers due to toList being stuck waiting for DbObservable's onComplete.
toList() waits for onCompleted() event but the flatMap(/*process*/) doesn't propagate complete.
So, you need to call them inside a new flatMap()
db.map(q -> q.items)
.flatMap(items -> Observable.from(items)
.flatMapIterable(items)
.flatMap(/*process*/)
.toList()
)
.subscribe(...)
You could process the List inside the first flatMap with the flatMap+toList:
db.map(q -> q.items)
.flatMap(items -> Observable.from(items).flatMap(...).toList())
.subscribe(...)
I want to perform a stream where the output from the stream is then used as the source for the same stream, in the same operation.
I currently perform this sort of operation using a queue; I remove an item, process it, and add any results that need further processing back to the queue. Here are two examples of this sort of thing:
Queue<WorkItem> workQueue = new Queue<>(workToDo);
while(!workQueue.isEmpty()){
WorkItem item = workQueue.remove();
item.doOneWorkUnit();
if(!item.isDone()) workQueue.add(item);
}
Queue<Node> nodes = new Queue<>(rootNodes);
while(!nodesLeft.isEmpty()){
Node node = nodes.remove();
process(node);
nodes.addAll(node.children());
}
I would imagine that the first could be performed concurrently like this:
try {
LinkedBlockingQueue<WorkItem> workQueue = new LinkedBlockingQueue<>();
Stream<WorkItem> reprocess = Stream.generate(() -> workQueue.remove()).parallel();
Stream.concat(workToDo.parallelstream(), reprocess)
.filter(item -> {item.doOneWorkUnit(); return !item.isDone();})
.collect(Collectors.toCollection(() -> workQueue));
} catch (NoSuchElementException e){}
And the second as:
try {
LinkedBlockingQueue<Node> reprocessQueue = new LinkedBlockingQueue<>();
Stream<WorkItem> reprocess = Stream.generate(() -> nodes.remove()).parallel();
Stream.concat(rootNodes.parallelStream(), reprocess)
.filter(item -> {process(item); return true;})
.flatMap(node -> node.children().parallelStream())
.collect(Collectors.toCollection(() -> reprocessQueue));
} catch (NoSuchElementException e){}
However, these feel like kludgy workarounds, and I dislike having to resort to using exceptions. Does anyone have a better way to do this sort of thing?
To make work parallel, I would use standard java.util.concurrent.Executor. To return the task back to working queue, in the end of the code of each task, add executor.execute(this).