I am writing unit test for the following code which uses AsyncHttpClient. The use of CountDownLatch and decrementing of the CountDownLatch within the implementation of FutureCallback is causing my JUnit testcase to hang waiting for countdown latch to be decremented. Within the JUnit test I am capturing the FutureCallback using ArgumentCaptor and then using thenAnswer I am calling the completed method to decrement the count down latch. But it doesn't seems to work, any ideas will be helpful.
public List<QueryResponse> execute(Query query) {
List<QueryResponse> res = new ArrayList<QueryResponse>();
try {
List<HttpRequestBase> requests = query.generateHttpRequests();
List<Future<HttpResponse>> futures = new ArrayList<Future<HttpResponse>>();
final CountDownLatch requestCompletionCDLatch = new CountDownLatch(requests.size());
for (HttpRequestBase request : requests) {
futures.add(httpClient.execute(request, new FutureCallback<HttpResponse>() {
#Override
public void failed(Exception e) {
logger.error("Error while executing: " + request.toString(), e);
requestCompletionCDLatch.countDown();
}
#Override
public void completed(HttpResponse result) {
requestCompletionCDLatch.countDown();
}
#Override
public void cancelled() {
logger.info("Request cancelled while executing: " + request.toString());
requestCompletionCDLatch.countDown();
}
}));
}
requestCompletionCDLatch.await();
for (Future<HttpResponse> future : futures) {
HttpResponse response = future.get(rcaRequestTimeout, TimeUnit.SECONDS);
int status = response.getStatusLine().getStatusCode();
if (status != HttpStatus.SC_OK) {
logger.warn("Query with non-success status " + status);
} else {
res.add(query.parseResponse(response.getEntity().getContent()));
}
}
} catch (IOException | InterruptedException | ExecutionException | TimeoutException e) {
logger.error("Error while querying", e);
} catch (URISyntaxException e) {
logger.error("Error while generating the query", e);
}
return res;
}
My unit test is as follows:
#Test
public void testHttpError() throws InterruptedException, ExecutionException, TimeoutException {
StatusLine status = Mockito.mock(StatusLine.class);
when(status.getStatusCode()).thenReturn(400);
HttpResponse response = Mockito.mock(HttpResponse.class);
when(response.getStatusLine()).thenReturn(status);
Future<HttpResponse> future = Mockito.mock(Future.class);
when(future.get(anyLong(), any())).thenReturn(response);
CloseableHttpAsyncClient httpClient = Mockito.mock(CloseableHttpAsyncClient.class);
ArgumentCaptor<HttpUriRequest> requestCaptor = ArgumentCaptor.forClass(HttpUriRequest.class);
ArgumentCaptor<FutureCallback<HttpResponse>> futureCallbackCaptor = ArgumentCaptor.forClass((Class)FutureCallback.class);
when(httpClient.execute(any(), any())).thenReturn(future).thenAnswer(new Answer() {
#Override
public Object answer(InvocationOnMock invocation) throws Throwable {
verify(httpClient).execute(requestCaptor.capture(), futureCallbackCaptor.capture());
futureCallbackCaptor.getValue().completed(response);
return null;
}
});
StubbedRcaClient rcaClient = new StubbedRcaClient(httpClient);
Query query = new Query("abc", "xyz", RcaHttpRequestType.GET, 1000);
List<QueryResponse> res = rcaClient.execute(query);
assertEquals(0, res.size());
IOUtils.closeQuietly(rcaClient);
}
I made this work by updating my JUnit as follows:
when(httpClient.execute(any(), any())).thenAnswer(new Answer<Future<HttpResponse>>() {
#Override
public Future<HttpResponse> answer(InvocationOnMock invocation) throws Throwable {
verify(httpClient).execute(requestCaptor.capture(), futureCallbackCaptor.capture());
futureCallbackCaptor.getValue().completed(response);
return future;
}
});
Related
CloseableHttpAsyncClient execute() method is working with FutureCallback,is there any way to write unit testing for that response, I have to increase the coverage but I am unable to cover for FutureCallback completed, failed and canceled api in below code.
#Service
public class SplunkLogger {
private Logger logger = LoggerFactory.getLogger(SplunkLogger.class);
#Autowired
private Environment environment;
#Autowired
CloseableHttpAsyncClient httpClient;
private void log(SplunkMessage splunkMessage) {
String messageStr = new Gson
().toJson(splunkMessage);
logger.info(messageStr);
if(this.environment.getActiveProfiles()[0].contains("local")){
return;
}
HttpPost post = new HttpPost(url);
post.addHeader(Constants.AUTHORIZATION_HEADER, Constants.SPLUNK_TOKEN_PREFIX + token);
post.addHeader(Constants.CONTENT_TYPE_HEADER, Constants.CONTENT_TYPE_VALUE_HEADER);
SplunkRequest request = SplunkRequest.builder().event(messageStr).build();
try {
post.setEntity(new StringEntity(new Gson().toJson(request)));
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
httpClient.start();
httpClient.execute(post, new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse httpResponse) {
try {
System.out.println("Request completed: " + IOUtils.toString(httpResponse.getEntity().getContent()));
} catch (IOException ignored) {
}
}
#Override
public void failed(Exception e) {
System.out.println("Request failed: " + e);
}
#Override
public void cancelled() {
System.out.println("Request failed: ");
}
});
}
}
How can I exhaust retries after a number of failed async request calls?
I am using AsyncHttpClient to send requests to our server. In case of request timeouts, connection exceptions, etc., I would like the client to retry N times and throw a custom exception. The calling method should receive this exception or it can be left unhandled.
// calls post
public void call(String data) throws CustomException {
asyncHttpClient.post(data, 10);
}
// posts data to http endpoint
public void post(String data, int retries) throw CustomException {
// if retries are exhausted, throw CustomException to call()
if (retry <= 0) {
throw new CustomException("exc");
}
BoundRequest request = httpClient.preparePost("http_endpoint");
ListenableFuture<Response> responseFuture = httpClient.post(request);
responseFuture.addListener(() -> {
Response response = null;
int status = 0;
try {
response = responseFuture.get();
status = response.getStatusCode();
// HTTP_ACCEPTED = {200, 201, 202}
if (ArrayUtils.contains(HTTP_ACCEPTED, status)) {
// ok
} else {
sleep(10);
post(data, retry - 1);
}
} catch (InterruptedException e) {
sleep(10);
post(data, retry - 1);
} catch (ExecutionException e) {
// ConnectionException
// RequestTimeoutException
sleep(10); // 10 seconds
post(data, retry - 1);
} catch (Exception e) {
sleep(10); // 10 seconds
post(data, retry - 1 );
} finally {
responseFuture.done();
}
}, Runnable::run);
}
This approach has a few problems:
uses recursive calls to retry.
the CustomException seems like it is never thrown and after retries == 0, the control goes back to the finally block.
...
} catch (ExecutionException e) {
// ConnectionException
// RequestTimeoutException
sleep(10); // 10 seconds
try {
post(data, retry - 1);
} catch (CustomException e) {
}
}
...
There is a predefined function in the AsyncHttpClient to handle MaxRetries,
The code below shows a simple implementation
AsyncHttpClientConfig cf = new DefaultAsyncHttpClientConfig.Builder().setMaxRequestRetry(5).setKeepAlive(true).build()
final AsyncHttpClient asyncHttpClient = new DefaultAsyncHttpClient(cf);
You can remove the retry logic of yours and let AsyncHttpClient to handle the same.
Alright, so tried to reproduce what you were trying to achieve with your code, but immediately realized that your CustomException only works if it is of type RuntimeException. The reason why is that you want to throw the exception during runtime and in another thread.
The code below shows a simple implementation of the Exception. Keep in mind that not all RuntimeExceptions stop the program. This is explained in this thread. So if you want to terminate the program, you have to manually stop it.
public class CustomException extends RuntimeException {
public CustomException(String msg) {
super(msg);
// print your exception to the console
// optional: exit the program
System.exit(0);
}
}
I changed the remaining of your implementation, so that you don't have to make recursive calls anymore. I removed the callback method and instead call the get() method, which waits for the request to finish. But since I am executing all of this in a separate thread it should be running in the background and not main thread.
public class Main {
private final AsyncHttpClient httpClient;
private final int[] HTTP_ACCEPTED = new int[]{200, 201, 202};
private final static String ENDPOINT = "https://postman-echo.com/post";
public static void main(String[] args) {
String data = "{message: 'Hello World'}";
Main m = new Main();
m.post(data, 10);
}
public Main() {
httpClient = asyncHttpClient();
}
public void post(final String data, final int retry) {
Runnable runnable = () -> {
int retries = retry;
for (int i = 0; i < retry; i++) {
Request request = httpClient.preparePost(ENDPOINT)
.addHeader("Content-Type", "application/json")
.setBody(data)
.build();
ListenableFuture<Response> responseFuture = httpClient.executeRequest(request);
try {
Response response = responseFuture.get();
int status = response.getStatusCode();
if (ArrayUtils.contains(HTTP_ACCEPTED, status)) {
System.out.println("Successful! Breaking Loop");
break;
} else {
Thread.sleep(10);
}
} catch (InterruptedException | ExecutionException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
retries--;
}
System.out.println("Remaining retries: " + retries);
if (retries <= 0) {
throw new CustomException("exc");
}
};
ExecutorService executor = Executors.newSingleThreadExecutor();
executor.submit(runnable);
}
}
Alternative
You could use the same Runnable to make asynchronous calls, without having to wait for future.get(). Each listener will conveniently be called in the same thread, which makes it more efficient for your use case.
public void post2(final String data, final int retry) {
Request request = httpClient.preparePost(ENDPOINT)
.addHeader("Content-Type", "application/json")
.setBody(data)
.build();
ListenableFuture<Response> future = httpClient.executeRequest(request);
MyRunnable runnable = new MyRunnable(retry, future, request);
future.addListener(runnable, null);
}
public class MyRunnable implements Runnable {
private int retries;
private ListenableFuture<Response> responseFuture;
private final Request request;
public MyRunnable(int retries, ListenableFuture<Response> future, Request request) {
this.retries = retries;
this.responseFuture = future;
this.request = request;
}
#Override
public void run() {
System.out.println("Remaining retries: " + this.retries);
System.out.println("Thread ID: " + Thread.currentThread().getId());
try {
Response response = responseFuture.get();
int status = response.getStatusCode();
if (ArrayUtils.contains(HTTP_ACCEPTED, status)) {
System.out.println("Success!");
//do something here
} else if (this.retries > 0) {
Thread.sleep(10);
this.execute();
} else {
throw new CustomException("Exception!");
}
} catch (InterruptedException | ExecutionException e) {
this.execute();
}
}
private void execute() {
this.retries -= 1;
this.responseFuture = httpClient.executeRequest(this.request);
this.responseFuture.addListener(this, null);
}
}
I ping services nodes and check if it is alive or not. My class, which ping all services is below:
public abstract class PingCallable implements Callable<ResponseEntity<String>> {
#Override
public ResponseEntity call() throws Exception {
try {
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headers = new HttpHeaders();
HttpEntity<String> entity = new HttpEntity<>("", headers);
return restTemplate.exchange(new URI(getPingUrl()), HttpMethod.GET, entity, String.class);
} catch (Exception ex) {
ex.printStackTrace();
return new ResponseEntity("Exception occured while ping. Exception message = " + ExceptionUtil.getFullCauseMessage(ex), HttpStatus.INTERNAL_SERVER_ERROR);
}
}
protected abstract String getPingUrl();
}
public class ServicePingCallable extends PingCallable {
private Node node;
public ServicePingCallable(Node node) {
this.node = node;
}
#Override
protected String getPingUrl() {
return "//" + node.getIp() + "/api/" + node.getService().getServicePath() + "/pingService";
}
public Node getNode(){
return this.node;
}
}
Method which pings service nodes is here:
#Scheduled(fixedDelayString = "${config.ping.fixedDelay}")
public void pingServices() throws InterruptedException {
List<Node> nodeList = serviceRepository.findAll();
List<ServicePingCallable> callableList = new ArrayList<>(nodeList.size());
ExecutorService executor = Executors.newFixedThreadPool(10);
try {
for (Node node : nodeList) {
ServicePingCallable pingCallable = new ServicePingCallable(node);
callableList.add(pingCallable);
}
List<Future<ResponseEntity<String>>> responses = executor.invokeAll(callableList, 10, TimeUnit.SECONDS);
for(Future<ResponseEntity<String>> response: responses){
// here I should check response HttpStatus and response text and save node ping state.
// But how I can know here for which Node is response?
//It whould be great to have such method:
//ServicePingCallable callable = (ServicePingCallable)response.getCallable();
//Node = callable.getNode();
}
} catch (Exception ex) {
//catch and log here
} finally {
if (!executor.isShutdown()) {
try {
executor.shutdown();
} catch (Exception ex1) {
}
}
}
I can add to PingCallable private filed ResponseEntity and return from call() the PingCallable but this approach is like workaround and not clear.
I think this is common task and should be some pattern how to get the callable from the future result. If you know it please share.
I have a java SDK,which use OkHttp client(4.0.0) to get token from IAM server and return token to application.The relation may like this：Applicaiton Sync call SDK，SDK Async call IAM.Refer to this answerJava - Retrieving Result from OkHttp Asynchronous GET,the code like:
The Async Class:
class BaseAsyncResult<T> {
private final CompletableFuture<T> future = new CompletableFuture<>();
T getResult() {
try {
return future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
return null;
}
void onFailure(IOException e) {
future.completeExceptionally(e);
}
void onResponse(Response response) throws IOException {
String bodyString = Objects.requireNonNull(response.body()).string();
future.complete(IasClientJsonUtil.json2Pojo(bodyString, new TypeReference<T>() {}));
}
}
Okhttp call like this:
public void invoke(Request request, BaseAsyncResult result) {
okHttpClient.newCall(request).enqueue(new Callback() {
#Override
public void onFailure(#NotNull Call call, #NotNull IOException e) {
result.onFailure(e);
}
#Override
public void onResponse(#NotNull Call call, #NotNull Response response) throws IOException {
result.onResponse(response);
}
});
}
The application use sdk code like,iasClient is a wrapper of okhttp client :
BaseAsyncResult<AuthenticationResponse> iasAsyncResult = new BaseAsyncResult();
iasClient.invoke(request, iasAsyncResult);
AuthenticationResponse result = iasAsyncResult.getResult();
The erroe message:
java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to x.x.x.AuthenticationResponse
What have I missed?
You need to make sure jackson knows which class to deserialize the value to . In this case, you are asking Jackson to deserialize the response to a TypeReference , which will resolve to a Map by default unless you specify the class (in this case, AuthenticationResponse ) . The Future resolves to a linkedHashMap due to this and causes the class cast.
try replacing the below line .
future.complete(IasClientJsonUtil.json2Pojo(bodyString, new TypeReference<T>() {}));
with
future.complete(IasClientJsonUtil.json2Pojo(bodyString, new TypeReference<AuthenticationResponse>() {}));
One method from #Arpan Kanthal is add a private Class type variable to BaseAsyncResult and then use that class in your json2Pojo function,then the BaseAsyncResult may like this:
public class BaseAsyncResult<T> {
private final CompletableFuture<T> future = new CompletableFuture<>();
private Class<T> classType;
public BaseAsyncResult(Class<T> classType) {
this.classType = classType;
}
public T getResult() {
try {
return future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
return null;
}
void onFailure(IOException e) {
future.completeExceptionally(e);
}
void onResponse(Response response) throws IOException {
future.complete(JacksonUtil.json2Pojo(response.body().string(), classType));
}
}
I have an onClickListener that uses Okhttp to asynchronously get some stuff in the background. Here's the OnClickListener:
mGetChartButton.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View v) {
String companyName = mSymbolValue.getText().toString();
getRequest(companyName, "chart");
Log.i(TAG, mChartProfile.getSizeDates()+""); // Null exception happens here
}
});
And here is the Okhttp snippet:
OkHttpClient client = new OkHttpClient();
Request request = new Request
.Builder()
.url(completeUrl)
.build();
Call call = client.newCall(request);
call.enqueue(new Callback() {
#Override
public void onFailure(Call call, IOException e) {
}
#Override
public void onResponse(Call call, Response response) throws IOException {
try {
String jsonData = response.body().string();
Log.v(TAG, jsonData);
if (response.isSuccessful()) {
if (requestType.equals("quote")) {
isValidSearch = getQuote(jsonData);
runOnUiThread(new Runnable() {
#Override
public void run() {
if (isValidSearch) {
updateDisplay();
}
toggleFacts(isValidSearch);
}
});
}
else{
getChartInfo(jsonData);
}
} else {
alertUserAboutError();
}
} catch (IOException e) {
Log.e(TAG, "Exception caught: ", e);
} catch (JSONException e) {
Log.e(TAG, "JSONException caught: ", e);
Toast.makeText(BuyActivity.this, "oops!", Toast.LENGTH_LONG).show();
}catch (ParseException e){
Log.e(TAG, "Unable to parse", e);
}
}
});
// How do I do a thread.join() here?
private void getChartInfo(String jsonData) throws JSONException, ParseException{
JSONObject wholeChartData = new JSONObject(jsonData);
JSONArray dates = wholeChartData.getJSONArray("Dates");
mChartProfile = new ChartProfile();
// ChartProfile contains ArrayList of ChartDate and ArrayList of ChartValue
for (int i = 0; i < dates.length(); i++){
ChartDate chartDate = new ChartDate(dates.getString(i));
mChartProfile.addToDates(chartDate);
}
JSONArray values = close.getJSONArray("values");
for (int i = 0; i < values.length(); i++){
ChartValue chartValue = new ChartValue(values.getDouble(i));
mChartProfile.addToValues(chartValue);
}
}
Right now, I'm getting an error of thread exiting with uncaught exception. And this is caused by a null exception because when calling mChartProfile.getSizeDates(), the values haven't been written in yet. My intuition is that the call to getChartInfo(jsonData) doesn't finish and the main UI thread is already returning from the getRequest() function. Hence, it will continue next line, and try to access an empty array that has not been initialized. Hence, I get a null exception. My solution is to have the main thread wait on the worker thread by calling thread.join() but I am not sure of how to do this through this Okhttp interface. Any help is deeply appreciated.