I'm using project reactor to load data from a web service using rest. This is done in parallel with multiple threads. I'm starting to hit rate limits on the web service, so I would like to send at most 10 requests per second to avoid getting these errors. How would I do that using reactor?
Using zipWith(Mono.delayMillis(100))? Or is there some better way?
Thank you
You can use delayElements instead of the whole zipwith.
One could use Flux.delayElements to process a 10 requests batch at every 1s; be aware though that if the processing takes longer than 1s the next batch will still be started in parallel hence being processed together with the previous one (and potentially many other previous ones)!
That's why I propose another solution where a 10 requests batch is still processed at every 1s but, if its processing takes longer than 1s, the next batch will fail (see overflow IllegalStateException); one could deal with that failure such that to continue the overall processing but I won't show that here because I want to keep the example simple; see onErrorResume useful to handle overflow IllegalStateException.
The code below will do a GET on https://www.google.com/ at a rate of 10 requests per second. You'll have to do additional changes in order to support the situation where your server is not able to process in 1s all your 10 requests; you could just skip sending requests when those asked at previous second are still processed by your server.
#Test
void parallelHttpRequests() {
// this is just for limiting the test running period otherwise you don't need it
int COUNT = 2;
// use whatever (blocking) http client you desire;
// when using e.g. WebClient (Spring, non blocking client)
// the example will slightly change for no longer use
// subscribeOn(Schedulers.elastic())
RestTemplate client = new RestTemplate();
// exit, lock, condition are provided to allow one to run
// all this code in a #Test, otherwise they won't be needed
var exit = new AtomicBoolean(false);
var lock = new ReentrantLock();
var condition = lock.newCondition();
MessageFormat message = new MessageFormat("#batch: {0}, #req: {1}, resultLength: {2}");
Flux.interval(Duration.ofSeconds(1L))
.take(COUNT) // this is just for limiting the test running period otherwise you don't need it
.doOnNext(batch -> debug("#batch", batch)) // just for debugging
.flatMap(batch -> Flux.range(1, 10) // 10 requests per 1 second
.flatMap(i -> Mono.fromSupplier(() ->
client.getForEntity("https://www.google.com/", String.class).getBody()) // your request goes here (1 of 10)
.map(s -> message.format(new Object[]{batch, i, s.length()})) // here the request's result will be the output of message.format(...)
.doOnSubscribe(s -> debug("doOnSubscribe: #batch = " + batch + ", i = " + i)) // just for debugging
.subscribeOn(Schedulers.elastic()) // one I/O thread per request
)
)
// consider using onErrorResume to handle overflow IllegalStateException
.subscribe(
s -> debug("received", s) // do something with the above request's result
e -> {
// pay special attention to overflow IllegalStateException
debug("error", e.getMessage());
signalAll(exit, condition, lock);
},
() -> {
debug("done");
signalAll(exit, condition, lock);
}
);
await(exit, condition, lock);
}
// you won't need the "await" and "signalAll" methods below which
// I created only to be easier for one to run this in a test class
private void await(AtomicBoolean exit, Condition condition, Lock lock) {
lock.lock();
while (!exit.get()) {
try {
condition.await();
} catch (InterruptedException e) {
// maybe spurious wakeup
e.printStackTrace();
}
}
lock.unlock();
debug("exit");
}
private void signalAll(AtomicBoolean exit, Condition condition, Lock lock) {
exit.set(true);
try {
lock.lock();
condition.signalAll();
} finally {
lock.unlock();
}
}
Related
I need to post-process result of CompletableFuture.supplyAsync execution to get intermediate result.
My code looks following
var executor = new ThreadPoolExecutor(
Runtime.getRuntime().availableProcessors(),
Integer.MAX_VALUE,
2L,
TimeUnit.SECONDS,
// size of queue has to be restricted since Java Heap Space could appear;
// default size of queue is Integer.MAX_VALUE
new LinkedBlockingQueue<>(10_000_000));
var resultOfBatch = new ResultOfBatch();
var lock = new ReentrantLock();
// usually `settings.getRuns()` could be up to 1_000_000_000 runs
LongStream.range(0, settings.getRuns())
.forEach(l -> {
CompletableFuture.supplyAsync(task, executor)
// collecting result per run to resultOfBatch (mainly simple operations like adding values to primitives)
.thenApply(resultPerRun -> {
lock.lock();
return resultOfBatch.addResultPerBatch(resultPerRun);
})
// the idea in logging partial result - ex.,every 10K passes
.thenAccept(resultPerBatch -> {
if (resultPerBatch.getRuns() % 10_000 == 0) {
// log intermediate result of execution
resultOfBatch.reset();
}
lock.unlock();
});
});
In a result I'm facing with java.util.concurrent.CompletionException: java.lang.IllegalMonitorStateException on .thenAccept(resultPerBatch -> { line
Seems like I'm using lock in wrong way but I cannot figure out how to avoid this kind of exception.
There's no guarantee that the Function passed to thenApply and the Consumer passed to thenAccept will execute on the same thread.
In this case, there is no need to separate them into separate steps:
CompletableFuture.supplyAsync(() -> l, executor)
.thenAcceptAsync(resultPerRun -> {
lock.lock();
try {
var resultPerBatch = resultOfBatch.addResultPerBatch(resultPerRun);
if (resultPerBatch.getRuns() % 10_000 == 0) {
System.out.println(resultPerBatch.getRuns());
resultOfBatch.reset();
}
} finally {
lock.unlock();
}
}, executor);
However, it is probably a better idea to process this data in batches rather than trying to create so many threads. This will ultimately either run out of memory, out of available native threads, or reject work because the queue is full.
I want to make web calls to 2 different services simultaneously. At the end, I zip the 2 Response objects into one stream. I'm using a Callable, but I'm not sure I'm going about this in the correct way. It seems as though I'm still going to be blocked by the first get() call to the Future, right? Can someone tell me if I'm on the right track? This is what I have so far:
// submit the 2 calls to the thread pool
ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
Future<Mono<Response<ProcessInstance>>> processFuture =
executorService.submit(() -> getProcessInstances(processDefinitionKey, encryptedIacToken));
Future<Mono<Response<Task>>> taskFuture =
executorService.submit(() -> getTaskResponses(processDefinitionKey, encryptedIacToken, 100, 0));
// get the result of the 2 calls
Optional<Tuple2<Response<ProcessInstance>, Response<Task>>> tuple;
try {
Mono<Response<ProcessInstance>> processInstances = processFuture.get();
Mono<Response<Task>> userTasks = taskFuture.get();
tuple = processInstances.zipWith(userTasks).blockOptional();
} catch (InterruptedException e) {
log.error("Exception while processing response", e);
// Restore interrupted state...
Thread.currentThread().interrupt();
return emptyProcessResponseList;
} catch (ExecutionException e) {
log.error("Exception while processing response", e);
return emptyProcessResponseList;
}
Given: You need to wait until both tasks are complete.
If processFuture ends first, you'll immediately fall through and wait until taskFuture ends. If taskFuture ends first you'll block until processFuture ends, but the taskFuture.get() call will return instantly since that task is done. In either case the result is the same.
You could use CompletableFutures instead and then CompletableFuture.allOf() but for something this simple what you have works fine. See also Waiting on a list of Future
Your code will block until the processFuture is finished, then it will block until the taskFuture is finished.
The callables will be processed in parallel, so here you are saving time (assuming thread pool size >= 2).
I have an infinite hot flux of data. I am about to engage in carrying out an operation on each element in the stream, each of which returns a Mono which will complete (one way or another) after some finite time.
There is the possibility of an error being thrown from these operations. If so, I want to resubscribe to the hot flux without missing anything, retrying elements that were in the middle of being processed when the error was thrown (i.e. anything that did not complete successfully).
What do I do here? I can tolerate repeated operations on the same elements, but not losing elements entirely from the stream.
I've attempted to use a ReplayProcessor to handle this, but I can't see a way of making it work without repeating a lot of operations that might well have succeeded (using a very conservative timeout), or losing elements due to new elements overriding old ones in the buffer (as below).
Test case:
#Test
public void fluxTest() {
List<String> strings = new ArrayList<>();
strings.add("one");
strings.add("two");
strings.add("three");
strings.add("four");
ConnectableFlux<String> flux = Flux.fromIterable(strings).publish();
//Goes boom after three uses of its method, otherwise
//returns a mono. completing after a little time
DangerousClass dangerousClass = new DangerousClass(3);
ReplayProcessor<String> replay = ReplayProcessor.create(3);
flux.subscribe(replay);
replay.flatMap(dangerousClass::doThis)
.retry(1)
.doOnNext(s -> LOG.info("Completed {}", s))
.subscribe();
flux.connect();
flux.blockLast();
}
public class DangerousClass {
Logger LOG = LoggerFactory.getLogger(DangerousClass.class);
private int boomCount;
private AtomicInteger count;
public DangerousClass(int boomCount) {
this.boomCount = boomCount;
this.count = new AtomicInteger(0);
}
public Mono<String> doThis(String s) {
return Mono.fromSupplier(() -> {
LOG.info("doing dangerous {}", s);
if (count.getAndIncrement() == boomCount) {
LOG.error("Throwing exception from {}", s);
throw new RuntimeException("Boom!");
}
return s;
}).delayElement(Duration.ofMillis(600));
}
}
This prints:
doing dangerous one
doing dangerous two
doing dangerous three
doing dangerous four
Throwing exception from four
doing dangerous two
doing dangerous three
doing dangerous four
Completed four
Completed two
Completed three
One is never completed.
The error (at least in the above example) can only occur in the flatMap(dangerousClass::doThis) call - so resubscribing to the root Flux and replaying elements when this one flatMap() call has failed seems a bit odd, and (probably) isn't what you want to do.
Instead, I'd recommend ditching the ReplayProcessor and just calling retry on the inner flatMap() call instead, so you end up with something like:
ConnectableFlux<String> flux = Flux.range(1, 10).map(n -> "Entry " + n).publish();
DangerousClass dangerousClass = new DangerousClass(3);
flux.flatMap(x -> dangerousClass.doThis(x).retry(1))
.doOnNext(s -> System.out.println("Completed " + s))
.subscribe();
flux.connect();
This will give you something like the following, with all entries completed and no retries:
doing dangerous Entry 1
doing dangerous Entry 2
doing dangerous Entry 3
doing dangerous Entry 4
Throwing exception from Entry 4
doing dangerous Entry 4
Completed Entry 2
Completed Entry 1
Completed Entry 3
Completed Entry 4
I am using a proprietary, 3rd party framework in my Android app -- EMDK from Zebra, to be specific -- and two of their exposed methods:
.read() and .cancelRead() are asynchronous, each taking anywhere from a split second to a 5 whole seconds to complete. I need to be able to spam them without crashing my application and ensure that each one isn't called twice in a row. How can I go about doing this? I don't have any access to the methods themselves and a decompiler will only give me runtime stubs.
Edit: I also have no idea when each of these two calls ever completes.
Changing asynchronous programs into blocking ones is a more general requirement to this problem.
In Java, we can do this with CountDownLatch (as well as Phaser), or LockSupport + Atomic.
For example, if it is required to change an asynchronous call asyncDoSomethingAwesome(param, callback) into a blocking one, we could write a "wrapper" method like this:
ResultType doSomethingAwesome(ParamType param) {
AtomicReference<ResultType> resultContainer = new AtomicReference<>();
Thread callingThread = Thread.currentThread();
asyncDoSomethingAwesome(param, result -> {
resultContainer.set(result);
LockSupport.unpark(callingThread);
});
ResultType result;
while ((result = resultContainer.get()) == null) {
LockSupport.park();
}
return result;
}
I think this will be enough to solve your problem. However, when we are writing blocking programs, we usually want a "timeout" to keep the system stable even when an underlying interface is not working properly, for example:
ResultType doSomethingAwesome(ParamType param, Duration timeout) throws TimeoutException {
AtomicReference<ResultType> resultContainer = new AtomicReference<>();
Thread callingThread = Thread.currentThread();
asyncDoSomethingAwesome(param, result -> {
resultContainer.set(result);
LockSupport.unpark(callingThread);
});
ResultType result;
long deadline = Instant.now().plus(timeout).toEpochMilli();
while ((result = resultContainer.get()) == null) {
if (System.currentTimeMillis() >= deadline) {
throw new TimeoutException();
}
LockSupport.parkUntil(deadline);
}
return result;
}
Sometimes we need more refined management to the signal among threads, especially when writing concurrency libries. For example, when we need to know whether the blocking thread received the signal from another thread calling LockSupport.unpark, or whether that thread successfully notified the blocking thread, it is usually not easy to implement with Java standard library. Thus I designed another library with more complete mechanism to solve this issue:
https://github.com/wmx16835/experimental_java_common/blob/master/alpha/src/main/java/mingxin/wang/common/concurrent/DisposableBlocker.java
With the support of DisposableBlocker, life will become much easier :)
ResultType doSomethingAwesome(ParamType param, Duration timeout) throws TimeoutException {
// We can use org.apache.commons.lang3.mutable.MutableObject instead of AtomicReference,
// because this object will never be accessed concurrently
MutableObject<ResultType> resultContainer = new MutableObject<>();
DisposableBlocker blocker = new DisposableBlocker();
asyncDoSomethingAwesome(param, result -> {
resultContainer.setValue(result);
blocker.unblock();
});
if (!blocker.blockFor(timeout)) {
throw new TimeoutException();
}
return resultContainer.getValue();
}
Might be off on this as I'm not 100% sure what you're trying to achieve/nor the structure, but could you wrap each in an AsyncTask? Then in a parent AsyncTask or background thread:
AsyncTask1.execute().get(); //get will block until complete
AsyncTask2.execute().get(); //get will block until complete
This is assuming there is some way of knowing the calls you're making completed.
I'm consuming an API that returns CompletableFutures for querying devices (similar to digitalpetri modbus).
I need to call this API with a couple of options to query a device and figure out what it is - this is basically trial and error until it succeeds. These are embedded device protocols that I cannot change, but you can think of the process as working similar to the following:
Are you an apple?
If not, then are you a pineapple?
If not, then are you a pen?
...
While the API uses futures, in reality, the communications are serial (going over the same physical piece of wire), so they will never be executed synchronously. Once I know what it is, I want to be able to stop trying and let the caller know what it is.
I already know that I can get the result of only one of the futures with any (see below), but that may result in additional attempts that should be avoided.
Is there a pattern for chaining futures where you stop once one of them succeeds?
Similar, but is wasteful of very limited resources.
List<CompletableFuture<String>> futures = Arrays.asList(
CompletableFuture.supplyAsync(() -> "attempt 1"),
CompletableFuture.supplyAsync(() -> "attempt 2"),
CompletableFuture.supplyAsync(() -> "attempt 3"));
CompletableFuture<String>[] futuresArray = (CompletableFuture<String>[]) futures.toArray();
CompletableFuture<Object> c = CompletableFuture.anyOf(futuresArray);
Suppose that you have a method that is "pseudo-asynchronous" as you describe, i.e. it has an asynchronous API but requires some locking to perform:
private final static Object lock = new Object();
private static CompletableFuture<Boolean> pseudoAsyncCall(int input) {
return CompletableFuture.supplyAsync(() -> {
synchronized (lock) {
System.out.println("Executing for " + input);
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
return input > 3;
}
});
}
And a List<Integer> of inputs that you want to check against this method, you can check each of them in sequence with recursive composition:
public static CompletableFuture<Integer> findMatch(List<Integer> inputs) {
return findMatch(inputs, 0);
}
private static CompletableFuture<Integer> findMatch(List<Integer> inputs, int startIndex) {
if (startIndex >= inputs.size()) {
// no match found -- an exception could be thrown here if preferred
return CompletableFuture.completedFuture(null);
}
return pseudoAsyncCall(inputs.get(startIndex))
.thenCompose(result -> {
if (result) {
return CompletableFuture.completedFuture(inputs.get(startIndex));
} else {
return findMatch(inputs, startIndex + 1);
}
});
}
This would be used like this:
public static void main(String[] args) {
List<Integer> inputs = Arrays.asList(0, 1, 2, 3, 4, 5);
CompletableFuture<Integer> matching = findMatch(inputs);
System.out.println("Found match: " + matching.join());
}
Output:
Executing for 0
Executing for 1
Executing for 2
Executing for 3
Executing for 4
Found match: 4
As you can see, it is not called for input 5, while your API (findMatch()) remains asynchronous.
I think the best you can do is, after your retrieval of the result,
futures.forEach(f -> f.cancel(true));
This will not affect the one having produced the result, and tries its best to stop the others. Since IIUC you get them from an outside source, there's no guarantee it will actually interrupt their work.
However, since
this class has no direct control over the computation that causes it to be completed, cancellation is treated as just another form of exceptional completion
(from CompletableFuture doc), I doubt it will do what you actually want.