Develop Reference

How to retry an operation when it fails

I have singleton client with the below contract
public interface MQPublisher {
void publish(String message) throws ClientConnectionException, ClientErrorException;
void start() throws ClientException;
void stop();
}
The class which is using this publisher is as below :
public class MessagePublisher {
#Autowired
private MQPublisher publisher;
private AtomicBoolean isPublisherRunning;
public void startPublisher() {
if (!isPublisherRunning.get()) {
publisher.start();
isPublisherRunning.compareAndSet(false, true);
}
}
#Retry(RETRY_MSG_UPLOAD)
public void sendMessage(String msg) {
try {
startPublisher();
publisher.publish(msg); // when multiple requests fail with the same exception, what will happen??
} catch (Exception e) {
log.error("Exception while publishing message : {}", msg, e);
publisher.stop();
isPublisherRunning.compareAndSet(true, false);
throw e;
}
}
We are using resilience4j retry functionality to retry the sendMessage method. This works fine in case of a single request. Consider a case when multiple requests are processed parallely and all of them fails with an exception. In this case, these requests will be retried and there is a chance that one thread will start the publisher while the other will stop it and it will throw exceptions again. How to handle this scenario in a cleaner way?

It isn't clear why the whole publisher should be stopped in case of failure. Nevertheless, if there are real reasons for that, I would change the stop method to use an atomic timer that will restart on each message sending and stop the publisher only after at least 5 seconds (or the time needed for a message to be successfully sent) have passed from the message sending.
Something like that:
#Slf4j
public class MessagePublisher {
private static final int RETRY_MSG_UPLOAD = 10;
#Autowired
private MQPublisher publisher;
private AtomicBoolean isPublisherRunning;
private AtomicLong publishStart;
public void startPublisher() {
if (!isPublisherRunning.get()) {
publisher.start();
isPublisherRunning.compareAndSet(false, true);
}
}
#Retryable(maxAttempts = RETRY_MSG_UPLOAD)
public void sendMessage(String msg) throws InterruptedException {
try {
startPublisher();
publishStart.set(System.nanoTime());
publisher.publish(msg); // when multiple requests fail with the same exception, what will happen??
} catch (Exception e) {
log.error("Exception while publishing message : {}", msg, e);
while (System.nanoTime() < publishStart.get() + 5000000000L) {
Thread.sleep(1000);
}
publisher.stop();
isPublisherRunning.compareAndSet(true, false);
throw e;
}
}
}
I think it is important to mention (as you just did) that this is a terrible design, and that such calculations should be done by the publisher implementer and not by the caller.

Cannot receive another websocket message after interruption of thread with websocket session

I have problem with receiving new messages from my JS client after I use the first received message in new thread and cancel the thread. I use spring boot for the back end. The interesting thing is that the session is not closed but I just cannot receive any more messages after the first one when I interrupt my thread.
Here is my websocket config:
#Configuration
#EnableWebSocket
public class WebSocketConfiguration implements WebSocketConfigurer {
#Override
public void registerWebSocketHandlers(WebSocketHandlerRegistry registry) {
registry.addHandler(new WebSocketHandler(), "/socket1").setAllowedOrigins("*");
}
}
Here is my handler with the executor service:
public class WebSocketHandler extends AbstractWebSocketHandler {
Logger logger = LoggerFactory.getLogger(WebSocketHandler.class);
Test test = new Test();
#Override
protected void handleTextMessage(WebSocketSession session, TextMessage message) throws Exception {
logger.info("START");
List<UrlWithPageNumber> listings = new ArrayList<>();
listings.add(new UrlWithPageNumber( "www.somesite.com", 1));
listings.add(new UrlWithPageNumber( "www.anothersite.com", 1));
listings.add(new UrlWithPageNumber( "www.thirdsite.com", 1));
checkItemsAsync(listings, session);
logger.info("DONE");
session.sendMessage(new TextMessage("DONE"));
}
public void checkItemsAsync(List<UrlWithPageNumber> listings, WebSocketSession session) {
ExecutorService executorService = Executors.newFixedThreadPool(1);
final CountDownLatch latch = new CountDownLatch(listings.size());
for (UrlWithPageNumber listing : listings) {
executorService.submit(() -> {
if(Test.stop) {
return;
}
ListingInfo listingInfo = test.itemPage(listing.getLink(), 1, 1);
logger.info(listingInfo.toString());
synchronized(session) {
try {
session.sendMessage(new TextMessage(listingInfo.toString()));
} catch (IOException e) {
e.printStackTrace();
}
}
latch.countDown();
});
}
try {
latch.await();
} catch (InterruptedException ie) {
throw new RuntimeException(ie);
}
}
}
Now the problem is the following: I call my handleTextMessage method from JS client and the execution is starting then I press another button in my page and it is changing Test.stop boolean flag to true and in that way i stop the remaining threads to be executed. After that if I call handleTextMessage from the js like the first time it is not called. I checked if I close the WebSocketSession and then try to call the backend and the result is similar but in my case the session is not closed for sure ! The question is how to use the WebSocketSession many times and why the session is become broken when I use it in my executor service and stop the thread? Should I do something with the session if I shutdown threads or this is just some spring boot bug ?

The problem is in the latch it wait more than expected because I skip some threads..

PublishOn Reactor Asynchronous Flux and thread locking

I'm trying to build a scenario which follows theses steps:
(on Init) Flux publisher created
(on Init) Subscribers subscribe
(on user action) publisher start streaming/publishing events
Web controller subscriber consumes and caches last BUFFER_SIZE events
Based on http://projectreactor.io/docs/core/release/reference/#advanced-parallelizing-parralelflux and https://www.baeldung.com/reactor-core I'm trying to use create and publish to do this, and the issue I'm having is that the thread that calls flux.connect is trapped in the while loop inside the publisher.
Here is a minimal working example using spring-boot-starter-webflux:
private ConnectableFlux<Integer> flux;
private Scheduler scheduler;
private int nextRead = 0;
private static final int BUFFERSIZE = 100;
private List<Integer> sink = new LinkedList<Integer>() ;
#PostConstruct
public void Init() {
this.scheduler = Schedulers.newSingle("Streamer");
flux = Flux.<Integer>create(fluxSink -> {
while (true) {
fluxSink.next(nextRead++);
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).publishOn(scheduler).publish();
}
#GetMapping("/subscribe")
public void subscribe(){
this.flux.subscribeOn(scheduler,false).subscribe(new CoreSubscriber<Integer>() {
#Override
public Context currentContext() {
return null;
}
#Override
public void onSubscribe(Subscription subscription) {
subscription.request(Long.MAX_VALUE);
}
#Override
public void onNext(Integer e) {
while (sink.size() >= BUFFERSIZE) sink.remove(0);
sink.add(e);
logger.debug("sink event: " + e);
}
#Override
public void onError(Throwable t) {}
#Override
public void onComplete() {}
});
}
#GetMapping("/start")
public void startStream(){
logger.debug("EventStreamSimulator startStream before connect");
this.flux.connect();
logger.debug("EventStreamSimulator startStream after connect");
}
#GetMapping("/values")
public Flux<Integer> getEvents(){
return Flux.fromIterable(sink);
}
Based on this code, the web request on /start will start the streaming but the http thread will be stuck in the emitter infinite loop. requests on /values and logging shows that it is working fine though (but the original http request to /start never finishes/returns)
Sample logs:
2018-10-09 18:12:54.798 DEBUG 6024 --- [ctor-http-nio-2] com.example.FluxPocController : emmit event: 0
2018-10-09 18:12:54.798 DEBUG 6024 --- [ Streamer-1] com.example.FluxPocController : sink event: 0
Then, here is the question: is the publishOn directive supported for these async way of using Flux.create? if so, how to use it?

How to make JUnit4 "Wait" for asynchronous job to finish before running tests

I am trying to write a test for my android app that communicates with a cloud service.
Theoretically the flow for the test is supposed to be this:
Send request to the server in a worker thread
Wait for the response from the server
Check the response returned by the server
I am trying to use Espresso's IdlingResource class to accomplish that but it is not working as expected. Here's what I have so far
My Test:
#RunWith(AndroidJUnit4.class)
public class CloudManagerTest {
FirebaseOperationIdlingResource mIdlingResource;
#Before
public void setup() {
mIdlingResource = new FirebaseOperationIdlingResource();
Espresso.registerIdlingResources(mIdlingResource);
}
#Test
public void testAsyncOperation() {
Cloud.CLOUD_MANAGER.getDatabase().getCategories(new OperationResult<List<Category>>() {
#Override
public void onResult(boolean success, List<Category> result) {
mIdlingResource.onOperationEnded();
assertTrue(success);
assertNotNull(result);
}
});
mIdlingResource.onOperationStarted();
}
}
The FirebaseOperationIdlingResource
public class FirebaseOperationIdlingResource implements IdlingResource {
private boolean idleNow = true;
private ResourceCallback callback;
#Override
public String getName() {
return String.valueOf(System.currentTimeMillis());
}
public void onOperationStarted() {
idleNow = false;
}
public void onOperationEnded() {
idleNow = true;
if (callback != null) {
callback.onTransitionToIdle();
}
}
#Override
public boolean isIdleNow() {
synchronized (this) {
return idleNow;
}
}
#Override
public void registerIdleTransitionCallback(ResourceCallback callback) {
this.callback = callback;
}}
When used with Espresso's view matchers the test is executed properly, the activity waits and then check the result.
However plain JUNIT4 assert methods are ignored and JUnit is not waiting for my cloud operation to complete.
Is is possible that IdlingResource only work with Espresso methods ? Or am I doing something wrong ?

I use Awaitility for something like that.
It has a very good guide, here is the basic idea:
Wherever you need to wait:
await().until(newUserIsAdded());
elsewhere:
private Callable<Boolean> newUserIsAdded() {
return new Callable<Boolean>() {
public Boolean call() throws Exception {
return userRepository.size() == 1; // The condition that must be fulfilled
}
};
}
I think this example is pretty similar to what you're doing, so save the result of your asynchronous operation to a field, and check it in the call() method.

Junit will not wait for async tasks to complete. You can use CountDownLatch to block the thread, until you receive response from server or timeout.
Countdown latch is a simple yet elegant solution and does NOT need an external library. It also helps you focus on the actual logic to be tested rather than over-engineering the async wait or waiting for a response
void testBackgroundJob() {
Latch latch = new CountDownLatch(1);
//Do your async job
Service.doSomething(new Callback() {
#Override
public void onResponse(){
ACTUAL_RESULT = SUCCESS;
latch.countDown(); // notify the count down latch
// assertEquals(..
}
});
//Wait for api response async
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
assertEquals(expectedResult, ACTUAL_RESULT);
}

Implementing keep-alive messages in Netty using WriteTimeoutHandler

I am using Netty 3.2.7. I am trying to write functionality in my client such that if no messages are written after a certain amount of time (say, 30 seconds), a "keep-alive" message is sent to the server.
After some digging, I found that WriteTimeoutHandler should enable me to do this. I found this explanation here: https://issues.jboss.org/browse/NETTY-79.
The example given in the Netty documentation is:
public ChannelPipeline getPipeline() {
// An example configuration that implements 30-second write timeout:
return Channels.pipeline(
new WriteTimeoutHandler(timer, 30), // timer must be shared.
new MyHandler());
}
In my test client, I have done just this. In MyHandler, I also overrided the exceptionCaught() method:
public void exceptionCaught(ChannelHandlerContext ctx, ExceptionEvent e) {
if (e.getCause() instanceof WriteTimeoutException) {
log.info("Client sending keep alive!");
ChannelBuffer keepAlive = ChannelBuffers.buffer(KEEP_ALIVE_MSG_STR.length());
keepAlive.writeBytes(KEEP_ALIVE_MSG_STR.getBytes());
Channels.write(ctx, Channels.future(e.getChannel()), keepAlive);
}
}
No matter what duration the client does not write anything to the channel, the exceptionCaught() method I have overridden is never called.
Looking at the source of WriteTimeoutHandler, its writeRequested() implementation is:
public void writeRequested(ChannelHandlerContext ctx, MessageEvent e)
throws Exception {
long timeoutMillis = getTimeoutMillis(e);
if (timeoutMillis > 0) {
// Set timeout only when getTimeoutMillis() returns a positive value.
ChannelFuture future = e.getFuture();
final Timeout timeout = timer.newTimeout(
new WriteTimeoutTask(ctx, future),
timeoutMillis, TimeUnit.MILLISECONDS);
future.addListener(new TimeoutCanceller(timeout));
}
super.writeRequested(ctx, e);
}
Here, it seems that this implementation says, "When a write is requested, make a new timeout. When the write succeeds, cancel the timeout."
Using a debugger, it does seem that this is what is happening. As soon as the write completes, the timeout is cancelled. This is not the behavior I want. The behavior I want is: "If the client has not written any information to the channel for 30 seconds, throw a WriteTimeoutException."
So, is this not what WriteTimeoutHandler is for? This is how I interpreted it from what I've read online, but the implementation does not seem to work this way. Am I using it wrong? Should I use something else? In our Mina version of the same client I am trying to rewrite, I see that the sessionIdle() method is overridden to achieve the behavior I want, but this method is not available in Netty.

For Netty 4.0 and newer, you should extend ChannelDuplexHandler like in example from IdleStateHandler documentation :
// An example that sends a ping message when there is no outbound traffic
// for 30 seconds. The connection is closed when there is no inbound traffic
// for 60 seconds.
public class MyChannelInitializer extends ChannelInitializer<Channel> {
#Override
public void initChannel(Channel channel) {
channel.pipeline().addLast("idleStateHandler", new IdleStateHandler(60, 30, 0));
channel.pipeline().addLast("myHandler", new MyHandler());
}
}
// Handler should handle the IdleStateEvent triggered by IdleStateHandler.
public class MyHandler extends ChannelDuplexHandler {
#Override
public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception {
if (evt instanceof IdleStateEvent) {
IdleStateEvent e = (IdleStateEvent) evt;
if (e.state() == IdleState.READER_IDLE) {
ctx.close();
} else if (e.state() == IdleState.WRITER_IDLE) {
ctx.writeAndFlush(new PingMessage());
}
}
}
}

I would suggest to add the IdleStateHandler and then add your custom implementation of IdleStateAwareUpstreamHandler which can react on the idle state. This works out very well for me on many different projects.
The javadocs list the following example, that you could use as the base of your implementation:
public class MyPipelineFactory implements ChannelPipelineFactory {
private final Timer timer;
private final ChannelHandler idleStateHandler;
public MyPipelineFactory(Timer timer) {
this.timer = timer;
this.idleStateHandler = new IdleStateHandler(timer, 60, 30, 0);
// timer must be shared.
}
public ChannelPipeline getPipeline() {
return Channels.pipeline(
idleStateHandler,
new MyHandler());
}
}
// Handler should handle the IdleStateEvent triggered by IdleStateHandler.
public class MyHandler extends IdleStateAwareChannelHandler {
#Override
public void channelIdle(ChannelHandlerContext ctx, IdleStateEvent e) {
if (e.getState() == IdleState.READER_IDLE) {
e.getChannel().close();
} else if (e.getState() == IdleState.WRITER_IDLE) {
e.getChannel().write(new PingMessage());
}
}
}
ServerBootstrap bootstrap = ...;
Timer timer = new HashedWheelTimer();
...
bootstrap.setPipelineFactory(new MyPipelineFactory(timer));
...