I have following proactive method.
void acceptSome(Consumer<? super String> consumer) {
consumer.accept("A");
consumer.accept("B");
}
And I wrote following method for reactor.
void emitAll(Sinks.Many<String> sink) {
try {
acceptSome(e -> {
sink.emitNext...(e);
});
sink.emitComplete...
} catch (Exception e) {
sink.emitError...
}
}
And I tested with following two methods, one with full buffer and the other with single buffer using thread.
#Test
void emitAll__onBackpressureBufferAll() {
final var sink = Sinks.many().unicast().<String>onBackpressureBuffer();
TheClass.emitAll(sink);
sink.asFlux()
.doOnNext(e -> {
log.debug("next: {}", e);
})
.blockLast();
}
#Test
void emitAll__onBackpressureBufferOne() {
var sink = Sinks.many().unicast().<String>onBackpressureBuffer(new ArrayBlockingQueue<>(1));
new Thread(() -> {
sink.asFlux()
.doOnNext(e -> {
log.debug("next: {}", e);
})
.blockLast();
}).start();
TheClass.emitAll(sink);
}
Now, How can I (Can I do that?) implement a method accepts a sink and returns a Flux or CompletableFuture<Flux> so that caller simply subscribe to the result without thread, with minimum buffer?
CompletableFuture<Flux<String>> emitAllAsync(Sinks.Many<String> sink) {
}
Thank you.
I tried some and it works yet doesn't seem righteous.
void emitAll(Sinks.Many<String> sink, Semaphore semaphore) {
try {
acceptSome(v -> {
try {
semaphore.acquire();
} catch (final InterruptedException ie) {
Thread.currentThread().interrupt();
throw new RuntimeException(ie);
}
log.debug("emitting {}", v);
sink.tryEmitNext(v).orThrow();
});
sink.tryEmitComplete().orThrow();
} catch (final IOException ioe) {
log.error("failed to emit to {}", sink, ioe);
sink.tryEmitError(ioe).orThrow();
}
}
CompletableFuture<Flux<String>> emitAllAsync() {
var sink = Sinks.many().unicast().<String>onBackpressureBuffer(new ArrayBlockingQueue<>(1));
var semaphore = new Semaphore(1);
CompletableFuture
.runAsync(() -> emitAll(sink, semaphore));
return CompletableFuture.completedFuture(sink.asFlux().doOnNext(v -> semaphore.release()));
}
Related
The code I want to achieve is as below:
StreamSupport.stream(jsonArray.spliterator(), true).forEach(s ->{
try {
//invoke other api and set timeout for its execution
}
catch(TimeoutException e) {
s.getAsJsonObject().addProperty("processStatus", "Failure");
}
});
Can anyone help me in achieving "invoke other api and set timeout for it's execution" case in the above snippet?
I don't think you can do that inside a stream, but you can wrap the execution in a Callable like so to achieve the same result:
public static void main(String[] args) {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future = executor.submit(new Task());
try {
System.out.println(future.get(1, TimeUnit.SECONDS));
}catch (Exception e) {
future.cancel(true);
e.printStackTrace();
} finally {
executor.shutdownNow();
}
}
private static class Task implements Callable<String> {
#Override
public String call(){
IntStream.of(1,2,3,4,5,6,7,8,9).parallel().forEach(t -> {
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
});
return "ok";
}
}
I have a list of Consumers which I want to run in multiple threads. I have an aspect for logging and handling all the exceptions. Below is the code snippet.
The problem is that the aspect works fine till the point multiple threads are not created. After that in case of exception (#LogFailure) code doesn't enter the advice.
#Override
public void consumerExecutor(Map<Consumer<IProcessDTO>, IProcessDTO> map, IContext context) {
List<CompletableFuture<Void>> completableFutures = new ArrayList<>();
Iterator consumerIterator = map.keySet().iterator();
while (consumerIterator.hasNext()) {
Consumer<IProcessDTO> consumer = (Consumer<IProcessDTO>) consumerIterator.next();
CompletableFuture<Void> future = CompletableFuture.runAsync(() -> {
executeSingleConsumer(0, consumer,map.get(consumer));
});
completableFutures.add(future);
}
try {
CompletableFuture<Void> allFuturesResult = CompletableFuture.allOf(completableFutures.toArray(new CompletableFuture[completableFutures.size()]));
allFuturesResult.thenApply(v ->
completableFutures.stream().
map(future -> future.join()).
collect(Collectors.toList())
);
} catch (CancellationException | CompletionException exception) {
}
}
#LogFailure
#LogSuccess
private void executeSingleConsumer(int tries, Consumer<IProcessDTO> consumer,IProcessDTO processDto) {
try {
consumer.accept(null);
} catch (Exception ex) {
if (tries < maxAttempts) {
executeSingleConsumer(tries + 1, consumer,processDto);
} else if (tries == maxAttempts) {
throw new ProcessException();
} else {
throw ex;
}
}
}
#AfterThrowing(pointcut = "execution(#com.LogFailure * *.*(..))", argNames = "jp,ex", throwing = "ex")
public void handleLogFailure(JoinPoint jp, Exception ex) {
// exceptions handled
}
Any idea what I am doing wrong?
I am trying to write a simple function that long-polls multiple messages tothe downstream dependency without exhausting it and only exist when all messages succeeded.
I came up with a way to wrap each message polling into a callable and use a ExecutorService to submit a list of callables.
public void poll(final List<Long> messageIdList) {
ExecutorService executorService = Executors.newFixedThreadPool(messageIdList.size());
List<MessageStatusCallable> callables = messageIdList.stream()
.map(messageId -> new MessageStatusCallable(messageId)).collect(Collectors.toList());
boolean allSuccess = false;
try {
allSuccess = executorService.invokeAll(callables).stream().allMatch(success -> {
try {
return success.get().equals(Boolean.TRUE);
} catch (InterruptedException e) {
e.printStackTrace();
return false;
} catch (ExecutionException e) {
e.printStackTrace();
return false;
}
});
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private class MessageStatusCallable implements Callable<Boolean> {
private Long messageId;
public MessageStatusCallable(Long messageId) {
this.messageId = messageId;
}
/**
* Computes a result, or throws an exception if unable to do so.
*
* #return computed result
* #throws Exception if unable to compute a result
*/
#Override
public Boolean call() throws Exception {
String messageStatus = downstreamService.getMessageStatus(messageId);
while(messageStatus == null || !messageStatus.equals( STATUS_VALUE_SUCCEEDED) {
messageStatus = messageLogToControlServer.getMessageStatus(messageId);
Thread.sleep(TimeUnit.MICROSECONDS.toMillis(100));
}
LOG.info("Message: " + messageId + " Succeded");
return true;
}
}
I wonder if there is a better way to achieve this since Thread.sleep is blocking and ugly.
I'm not sure this is the best solution but it occurred to me you could use a CountDownLatch and ScheduledExecutorService.
public void poll(final List<Long> messageIdList) throws InterruptedException {
CountDownLatch latch = new CountDownLatch(messageIdList.size());
ScheduledExecutorService executorService = Executors.newScheduledThreadPool(POOL_SIZE);
try {
for (Long messageId : messageIdList) {
MessageStatusCallable callable = new MessageStatusCallable(messageId, latch);
executorService.scheduleWithFixedDelay(
() -> {
String messageStatus = downstreamService.getMessageStatus(messageId);
if (STATUS_VALUE_SUCCEEDED.equals(messageStatus)) {
latch.countDown();
throw new CompletionException("Success - killing the task", null);
}
},
0, 100, TimeUnit.MILLISECONDS);
}
latch.await();
} finally {
executorService.shutdown();
}
}
I probably also wouldn't have the Runnable as a lambda other than for brevity in the answer.
I have this code where I execute sets of callables, I need one set to finish all it's work before triggering the next set. This code seems to work fine but sometimes next set would start running before time. What is wrong here?
private void executeSubGraph(QuestExecutionContext ctx, Set<Activity> subGraph, int progressAfterRan) {
ExecutorService pool = Executors.newFixedThreadPool(16);
subGraph.forEach(a -> {
ActivityRunner<? extends Activity> runner = activityRunnerFactory.getRunner(ctx, a);
if (runner != null) {
Callable<List<PortValuePart>> runnerCallable = () -> {
try {
LOG.info("Running {} in {}", a, a.getClass() );
List<PortValuePart> result = runner.call();
LOG.info("Result of {} in {} is {}", a, a.getClass(), result);
if (result != null) {
result.forEach(r -> resultProcessor.processResult(new PortValuePartEnvelope(r)));
}
return result;
} catch (Exception e) {
LOG.warn("Exception for {} in {}", a, runner.getClass(), e);
resultProcessor.processResult(Progress.failed(ctx.getId(), e));
throw new RuntimeException(e);
}
};
Future<List<PortValuePart>> p = pool.submit(runnerCallable);
} else {
LOG.warn("No runner found for activity {}", a);
resultProcessor.processResult(Progress.failed(ctx.getId(), new RuntimeException("No runner found for activity " + a)));
throw new RuntimeException("No runner found for activity " + a);
}
});
pool.shutdown();
try {
pool.awaitTermination(WAIT_TIME_MILLIS, TimeUnit.MILLISECONDS);
resultProcessor.processResult(Progress.running(ctx.getId(), progressAfterRan));
} catch (InterruptedException e) {
throw new PlatformException("Execution interrupted.");
}
}
Note that ExecutorService.awaitTermination doesn't throw an exception if it times out; it just returns false. If you want to make sure that the next calls don't run concurrently with these ones, you should probably use the return value, and maybe throw an exception (and kill the tasks) if it's taking way too long.
I'm trying to convert this RxJava1 code to RxJava2
public static Observable<Path> listFolder(Path dir, String glob) {
return Observable.<Path>create(subscriber -> {
try {
DirectoryStream<Path> stream =
Files.newDirectoryStream(dir, glob);
subscriber.add(Subscriptions.create(() -> {
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}));
Observable.<Path>from(stream).subscribe(subscriber);
} catch (DirectoryIteratorException ex) {
subscriber.onError(ex);
} catch (IOException ioe) {
subscriber.onError(ioe);
}
});
}
The thing is that in Rxjava2 I don't get a subscriber to add a new subscription to it.
Enjoy RxJava 2 conciseness (Flowable is the backpressure supporting class now):
public static Flowable<Path> listFolder(Path dir, String glob) {
return Flowable.using(
() -> Files.newDirectoryStream(dir, glob),
stream -> Flowable.fromIterable(stream),
stream -> stream.close());
}
If you don't want backpressure then replace Flowable with Observable.