Develop Reference

Return OkHttp Asynchronous result has some problem

I have a java SDK,which use OkHttp client(4.0.0) to get token from IAM server and return token to application.The relation may like this：Applicaiton Sync call SDK，SDK Async call IAM.Refer to this answerJava - Retrieving Result from OkHttp Asynchronous GET,the code like:
The Async Class:
class BaseAsyncResult<T> {
private final CompletableFuture<T> future = new CompletableFuture<>();
T getResult() {
try {
return future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
return null;
}
void onFailure(IOException e) {
future.completeExceptionally(e);
}
void onResponse(Response response) throws IOException {
String bodyString = Objects.requireNonNull(response.body()).string();
future.complete(IasClientJsonUtil.json2Pojo(bodyString, new TypeReference<T>() {}));
}
}
Okhttp call like this:
public void invoke(Request request, BaseAsyncResult result) {
okHttpClient.newCall(request).enqueue(new Callback() {
#Override
public void onFailure(#NotNull Call call, #NotNull IOException e) {
result.onFailure(e);
}
#Override
public void onResponse(#NotNull Call call, #NotNull Response response) throws IOException {
result.onResponse(response);
}
});
}
The application use sdk code like,iasClient is a wrapper of okhttp client :
BaseAsyncResult<AuthenticationResponse> iasAsyncResult = new BaseAsyncResult();
iasClient.invoke(request, iasAsyncResult);
AuthenticationResponse result = iasAsyncResult.getResult();
The erroe message:
java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to x.x.x.AuthenticationResponse
What have I missed?

You need to make sure jackson knows which class to deserialize the value to . In this case, you are asking Jackson to deserialize the response to a TypeReference , which will resolve to a Map by default unless you specify the class (in this case, AuthenticationResponse ) . The Future resolves to a linkedHashMap due to this and causes the class cast.
try replacing the below line .
future.complete(IasClientJsonUtil.json2Pojo(bodyString, new TypeReference<T>() {}));
with
future.complete(IasClientJsonUtil.json2Pojo(bodyString, new TypeReference<AuthenticationResponse>() {}));

One method from #Arpan Kanthal is add a private Class type variable to BaseAsyncResult and then use that class in your json2Pojo function,then the BaseAsyncResult may like this:
public class BaseAsyncResult<T> {
private final CompletableFuture<T> future = new CompletableFuture<>();
private Class<T> classType;
public BaseAsyncResult(Class<T> classType) {
this.classType = classType;
}
public T getResult() {
try {
return future.get();
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
return null;
}
void onFailure(IOException e) {
future.completeExceptionally(e);
}
void onResponse(Response response) throws IOException {
future.complete(JacksonUtil.json2Pojo(response.body().string(), classType));
}
}

Avoid repetition of code in java using generic method

In my Android application I perform some actions related with my Room Database. These actions have to be done in background, this is why I use a threadExecutor. As you can see the code for both methods is almost the same and I was wondering if it would be possible to construct something generic to avoid this code repetition.
public void addOperation(Operation operation, AddOperationInteractor.CallBack callback)
{
Interactor interactor = new AbstractInteractor(ThreadExecutor.getInstance())
{
#Override
public void run()
{
try
{
operationRepository.addNewOperation(operation);
callback.onAddOperationSuccess();
}
catch (Exception ex)
{
callback.onAddOperationSuccess();
}
}
};
interactor.execute();
}
public void deleteOperation(Operation operation, RemoveOperationInteractor.CallBack callback)
{
Interactor interactor = new AbstractInteractor(ThreadExecutor.getInstance())
{
#Override
public void run()
{
try
{
operationRepository.removeOperation(operation);
callback.onRemoveOperationSuccess();
}
catch (Exception ex)
{
callback.onRemoveOperationSuccess();
}
}
};
interactor.execute();

I see no repetition in your code. To reduce boilerplate code, try a lambda:
public void addOperation(Operation operation, AddOperationInteractor.CallBack callback) {
ThreadExecutor.getInstance().execute(() -> {
try {
operationRepository.addNewOperation(operation);
}
finally {
callback.onAddOperationSuccess();
}
});
}
public void deleteOperation(Operation operation, RemoveOperationInteractor.CallBack callback) {
ThreadExecutor.getInstance().execute(() -> {
try {
operationRepository.removeOperation(operation);
}
finally {
callback.onRemoveOperationSuccess();
}
});
}
Now there is only 1 repeating line, to invoke the ThreadExecutor.
Alternatively pass callbacks to a helper method:
public void addOperation(Operation operation, AddOperationInteractor.CallBack callback) {
execute(()-> operationRepository.addNewOperation(operation),
()-> callback.onAddOperationSuccess());
}
public void deleteOperation(Operation operation, RemoveOperationInteractor.CallBack callback) {
execute(()-> operationRepository.removeOperation(operation),
()-> callback.onRemoveOperationSuccess());
}
private void execute(Runnable action, Runnable onSuccess) {
ThreadExecutor.getInstance().execute(() -> {
try {
action.run();
onSuccess.run();
} catch (Exception e) {
LOG.warn(e);
onSuccess.run();
}
}
}

Java Threading - Wait for data to be returned, but don't block other threads

I have 7 threads running in an ExecutorPool that process data and occasionally need data from a listener instance running on another thread. The listener sends a request over a socket to a server and a while later, when the result is returned, the listener will return the data to the worker thread that called it. I want to block the worker thread until the requested data is returned, but I don't want to block the listener from making other requests from the other worker threads. How do I do that?

If one thread hands off work to another thread, and then subsequently simply waits for the result, you don't need another thread to do the work. You may need a class that does the work, but which is called on the same thread. And if the same instance is used by multiple threads some synchronization may be needed. But the bottom line is this :
You don't need the listener thread. Replace it with a component that handles a request, and call it synchronously.
Edit
Given your own answer, your problem is a bit clearer. As #JimN suggests you probably want to hand out a Future to the worker thread, and make it a CompletableFuture the Listener keeps in a Map keyed by request id until the response returns.
Sample code :
public class WorkUnitProcessor implements Runnable {
// ...
#Override
public void run() {
while(true) {
WorkUnit work = master.getNextWorkUnit();
if(work == null) return;
doWork(work);
}
}
public void doWork(WorkUnit work) {
//Do some work...
try {
DataRequest dataRequest = createRequest(work);
Future<Response> future = server.getData(dataRequest);
Response response = future.get(); // this call blocks until the Response is available.
//finish doing work
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (ExecutionException e) {
// handle e.getCause()
}
}
// ...
}
public class Server implements DataSourceDrivenCallback {
private final DataSource dataSource;
private Map<Integer, CompletableFuture<Response>> openRequests = new ConcurrentHashMap<>();
public Server(DataSource dataSource) {
this.dataSource = dataSource;
}
#Override
public void incomingDataCallback(int requestId, ChunkOfData requestedData) {
CompletableFuture<Response> responseHolder = openRequests.remove(requestId); // get the responseHolder
if (responseHolder != null) {
responseHolder.complete(toResponse(requestedData)); // make the response available.
}
}
public Future<Response> getData(DataRequest datarequest) {
int requestId = dataSource.submitRequest(serializeAndTranslateRequest(datarequest));
CompletableFuture<Response> future = new CompletableFuture<>();
openRequests.put(requestId, future);
return future;
}
// ...
}

I think this might work. What I was looking for is described here:
https://docs.oracle.com/javase/tutorial/essential/concurrency/guardmeth.html
It's the ability to make a thread sleep until it is notified by the thread that it is waiting on. Seems easy to use.
public class DataProcessor {
private List<WorkUnit> work;
private Server server;
public DataProcessor(List<WorkUnit> work, int numprocessors) {
this.work = work;
setupProcessors(numprocessors);
Server server = new Server();
}
private void setupProcessors(int numprocessors) {
for(int i = 0; i < numprocessors; i++) {
WorkUnitProcessor worker = new WorkUnitProcessor(this, server);
worker.start();
}
}
public synchronized WorkUnit getNextWorkUnit() {
if(work.isEmpty()) return null;
return work.remove(0);
}
}
public class WorkUnitProcessor(Server server) {
private DataProcessor master;
private Server server;
public WorkUnitProcessor(DataProcessor master) {
this.master = master;
}
#Override
public void run() {
while(true) {
WorkUnit work = master.getNextWorkUnit();
if(work == null) return;
doWork(work);
}
}
public void doWork(WorkUnit work) {
//Do some work...
server.getData(datarequest, this);
while(!datarequest.filled) {
try {
wait();
} catch (InterruptedException e) {}
}
//finish doing work
}
}
public class Server implements DataSourceDrivenCallback {
private DataSource ds;
private Map<Integer, OpenRequest> openrequests;
public Server() {
//setup socket and establish communication with server through DataSource object
DataSource ds = new DataSource(<ID>, <Socket>);
}
public synchronized void getData(DataRequest datarequest, WorkUnitProcessor workerthread) {
int requestid = ds.submitRequest(serializeAndTranslateRequest(datarequest));
openrequests.add(new OpenRequest(workerthread, datarequest));
}
#Override
public void incomingDataCallback(int requestid, ChunkOfData requesteddata) {
OpenRequest request = openrequests.get(requestid);
request.datarequest.storeData(requesteddata);
request.workerthread.notify();
}
}
public class OpenRequest {
private WorkUnitProcessor workerthread;
private DataRequest datarequest;
//other details about request
}

Apache's HttpAsyncClient unreliable with multiple requests

I'm currently in a project where I have to do multiple, concurrent http requests to a rest service which returns a JSON response. This is a batch operation and the number of requests at any time could range from several hunderd to several thousend.
That's why I thought it would be a good idea to have an async http client so I could have concurrent requests, which dramatically could speed up the process. I first tried ning's async-http-client. Maybe I was doing something wrong, because it was kind of slow for me. About 10 seconds for 1000 requests.
After which I tried Apache's implementation which was much faster at about 4 seconds for 1000 requests. But I can't seem to get the requests to get stable. Most of the time I will get a List with a 1000 responses (like I expect), but sometimes I am just missing a few responses, like 1 or 2.
This is currently my code:
public class AsyncServiceTest {
public AsyncServiceTest(String serviceURI) {
this.httpClient = HttpAsyncClients.custom().setMaxConnPerRoute(100).setMaxConnTotal(20)
.setDefaultRequestConfig(RequestConfig.custom().build()).build();
this.objectMapper = new ObjectMapper();
this.serviceURI = serviceURI;
}
private List<Object> getResults(List<String> queryStrings) throws Exception {
try {
httpClient.start();
final List<HttpGet> requests = new ArrayList<>(addresses.size());
for (String str : queryStrings) {
requests.add(new HttpGet(buildUri(str))); // In this method we build the absolute request uri.
}
final CountDownLatch latch = new CountDownLatch(requests.size());
final List<Object> responses = new ArrayList<>(requests.size());
final List<String> stringResponses = new ArrayList<>(requests.size());
for (final HttpGet request : requests) {
httpClient.execute(request, new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse response) {
try {
stringResponses.add(IOUtils.toString(response.getEntity().getContent(), "UTF-8"));
latch.countDown();
} catch (IOException e) {
e.printStackTrace();
}
}
#Override
public void failed(Exception e) {
latch.countDown();
}
#Override
public void cancelled() {
latch.countDown();
}
});
}
latch.await();
for (String r : stringResponses) {
responses.add(mapToLocation(r)); // Mapping some Strings to JSON in this method.
}
return responses;
} finally {
httpClient.close();
}
}
}
So, in essence, I am wondering if there is something wrong with my code (probably) or is it just because of the way the library works? Because the CountDownLatch is at zero all the time. Or does anyone have a pointer in the right direction (maybe with another library)?

It seemed to be a concurrency problem (thanks to #vanOekel) in my code. The answer is to replace the ArrayList<E> with a Vector<E>, which is in fact thread-safe. Example code:
public class AsyncServiceTest {
public AsyncServiceTest(String serviceURI) {
this.httpClient = HttpAsyncClients.custom().setMaxConnPerRoute(100).setMaxConnTotal(20)
.setDefaultRequestConfig(RequestConfig.custom().build()).build();
this.objectMapper = new ObjectMapper();
this.serviceURI = serviceURI;
}
private List<Object> getResults(List<String> queryStrings) throws Exception {
try {
httpClient.start();
final CountDownLatch latch = new CountDownLatch(queryStrings.size());
final Vector<Object> responses = new Vector<>(queryStrings.size());
for (String str : queryStrings) {
// buildUri: In this method we build the absolute request uri.
httpClient.execute(new HttpGet(buildUri(str)), new FutureCallback<HttpResponse>() {
#Override
public void completed(HttpResponse response) {
try {
// mapToLocation: Mapping some Strings to JSON in this method.
responses.add(mapToLocation(IOUtils.toString(response.getEntity().getContent(), "UTF-8")));
latch.countDown();
} catch (IOException e) {
failed(e);
}
}
#Override
public void failed(Exception e) {
logger.error(e.getLocalizedMessage(), e);
latch.countDown();
}
#Override
public void cancelled() {
logger.error("Request cancelled.");
latch.countDown();
}
});
}
latch.await();
return responses;
} finally {
httpClient.close();
}
}
}
Thanks for all the helpful responses. If anyone has any suggestions regarding optimization of the above code, I will be glad to hear so.

Retrying Method calls in generic way

My Java application requires a retry logic on remote calls failures.
These remote calls are:
scattered all over the application
pertain to different Remote Service classes.
Also, the retry logic may have varying retry interval and varying retry attempts.
I need a generic retry() implementation which can make appropriate method calls depending on from where it is called. Below is a simple code illustration of I am looking for. I know we can attempt to do this using java reflection, but, is there a framework or an open source available somewhere which is read-to-use?
try {
ClassA objA = remoteServiceA.call(paramA1, paramA2, ...);
} catch (Exception e){
ClassA objA = (ClassA)retry(remoteService, listOfParams, ..); // generic method call
}
..
try {
ClassB objB = remoteServiceB.call(paramB1, paramB2, ...);
} catch (Exception e){
ClassA objB = (ClassB)retry(remoteService, listOfParams, ..); // generic method call
}

As already suggested, you should use AOP and Java annotations. I would recommend a read-made mechanism from jcabi-aspects (I'm a developer):
#RetryOnFailure(attempts = 3, delay = 5)
public String load(URL url) {
return url.openConnection().getContent();
}
Read also this blog post: http://www.yegor256.com/2014/08/15/retry-java-method-on-exception.html
Update: Check RetryFunc from Cactoos.

This is a book example of where aspectj (or aop in general) can be used, see 8.2.7 Example in Spring documentation and 5 Reasons Java Developers Should Learn and Use AspectJ.
Basically an aspect intercepts all calls to given methods (specified using annotation, naming convention, whatever) and retries.

Assume you have a method, that need to retied at every 500ms and upto 5 times.
Current class:
public class RemoteCaller{
Service serviceCaller;
public void remoteCall(String message) {
serviceCaller.updateDetails( this.message);
return null;
}
}
Modified approach:
public class RetriableHelper<T> implements Callable<T> {
private Callable<T> task;
private int numberOfRetries;
private int numberOfTriesLeft;
private long timeToWait;
public RetriableHelper(int numberOfRetries, long timeToWait, Callable<T> task) {
this.numberOfRetries = numberOfRetries;
numberOfTriesLeft = numberOfRetries;
this.timeToWait = timeToWait;
this.task = task;
}
public T call() throws Exception {
while (true) {
try {
return task.call();
} catch (InterruptedException e) {
throw e;
} catch (CancellationException e) {
throw e;
} catch (Exception e) {
numberOfTriesLeft--;
if (numberOfTriesLeft == 0) {
throw e;
}
Thread.sleep(timeToWait);
}
}
}
}
Backend system/remote call class:
public class RemoteCaller{
Service serviceCaller;
public void remoteCall(String message) {
class RemoteCallable implements Callable<Void> {
String message;
public RemoteCallable( String message)
{
this.message = message;
}
public Void call() throws Exception{
serviceCaller.updateDetails( this.message);
return null;
}
}
RetriableHelper<Void> retriableHelper = new RetriableHelper<Void>(5, 500, new RemoteCallable( message));
try {
retriableHelper.call();
} catch (Exception e) {
throw e;
}
}
}

enter link description here Spring has a retry annotation which servers the purpose
Step 1: Add following dependency to your POM
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>1.1.5.RELEASE</version>
</dependency>
Step 2: Enabling Spring Retry
To enable Spring Retry in an application, we need to add the #EnableRetry annotation to our #Configuration class:
Ex:
#Configuration
#EnableRetry
public class AppConfig { ... }
Step 3: To add retry functionality to methods, #Retryable can be used:
Ex:
#Service
public interface MyService {
#Retryable(
value = { SQLException.class },
maxAttempts = 2,
backoff = #Backoff(delay = 5000))
void retryService(String sql) throws SQLException;
...
}
Step 4.The #Recover annotation is used to define a separate recovery method when a #Retryable method fails with a specified exception:
Ex:
#Service
public interface MyService {
...
#Recover
void recover(SQLException e, String sql);
}
See Url for more details : http://www.baeldung.com/spring-retry

where do you get the services from? use a factory to Proxy the service you get from the original factory. The proxy can then implement the retry transparently. See the java Proxy/ProxyGenerators in reflection.

If you are using spring , then better go with Aspects.
Otherwise, below sample solution can work:
public class Test
{
public static void main(String[] args) throws Exception
{
Test test = new Test();
test.toRunFirst("Hello! This is normal invocation");
runWithRetry(test, "toRunFirst", "Hello! This is First, called with retry");
runWithRetry(test, "toRunSecond", "Hello! This is Second, called with retry");
}
public void toRunFirst(String s) {
System.out.println(s);
}
public void toRunSecond(String s) {
System.out.println(s);
}
public static Object runWithRetry(Object obj, String methodName, Object... args) throws Exception
{
Class<?>[] paramClass = new Class<?>[args.length];
for(int i=0; i< args.length; i++) {
paramClass[i] = args[i].getClass();
}
Method method = obj.getClass().getDeclaredMethod(methodName, paramClass);
int retryCount = 2;
for(int i=0; i< retryCount; i++) {
try {
return method.invoke(obj, args);
}
catch (Exception e) {
e.printStackTrace();
}
}
return null;
}
}

I did not find what I needed so there is mine.
The main feature is that it throws the type of Exception you need when maxRetries is reached so you can catch it in the call.
import org.apache.log4j.Logger;
public class TaskUtils {
public static <E extends Throwable> void retry(int maxRetries, Task<E> task) throws E {
retry(maxRetries, 0, null, task);
}
public static <E extends Throwable> void retry(int maxRetries, long waitTimeMs, Logger logger, Task<E> task) throws E {
while (maxRetries > 0) {
maxRetries--;
try {
task.run();
} catch (Exception e) {
if (maxRetries == 0) {
try {
throw e;
} catch (Exception ignored) { // can't happen but just in case we wrap it in
throw new RuntimeException(e);
}
}
if (logger != null)
logger.warn("Attempt " + maxRetries + " failed", e);
try {
Thread.sleep(waitTimeMs);
} catch (InterruptedException ignored) {
}
}
}
}
public interface Task<E extends Throwable> {
void run() throws E;
}
}
Usage :
TaskUtils.retry(3, 500, LOGGER, () -> stmClickhouse.execute(
"ALTER TABLE `" + database + "`.`" + table.getName() + "` ON CLUSTER " + clusterName + allColumnsSql
));

add it into pom.xml
<dependency>
<groupId>org.deking.utils</groupId>
<artifactId>retry</artifactId>
<version>0.0.2-SNAPSHOT</version>
</dependency>
new Retry<String>()
.maxOperationWaitTime(30_000)//Max operation wait time during a single operation
.retryIntervalTime(1_000)//Interval time between two operations
.maxRetryTimes(3)//Retry times when operation failed(or timeout) at the first time
.operation(() -> {
//your operation
return "success!";
})
.judgement(t -> (t == null || t.isEmpty()))//add your judgement whether the operation should be retry(Operation should return a value)
.execute();
If you want add retry config annotation on method,and call it:
class RetryTests{
#RetryConfig( maxRetryTimes=1)
public static String TestAnnotation() {
return "aaa";
}
public static void main(String[] args) {
try {
new Retry<String>()
.of(RetryTest.class.getMethod("TestAnnotation"),null)
.judgement(r -> r.equals("aaa"))
.execute();
} catch (NoSuchMethodException | SecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}