Develop Reference

Threads Error Memcached cloud Heroku Java

Background
We are developing a Java service in Heroku with 1 dyno, which is using Memcached cloud.
Issue
Meanwhile we were developing and testing it, It was working fine. However, when we decided to test it in a real environment, It started to return the following error:
java.lang.OutOfMemoryError: unable to create new native thread
java.lang.Thread.start0(Native Method)
java.lang.Thread.start(Thread.java:717)
net.spy.memcached.MemcachedConnection.<init>(MemcachedConnection.java:306) net.spy.memcached.DefaultConnectionFactory.createConnection(DefaultConnectionFactory.java:209)
net.spy.memcached.MemcachedClient.<init>(MemcachedClient.java:209)
Memcached.<init>(Memcached.java:34)
Main.lambda$main$1(Main.java:101)
spark.SparkBase$1.handle(SparkBase.java:311)
spark.webserver.MatcherFilter.doFilter(MatcherFilter.java:159)
spark.webserver.JettyHandler.doHandle(JettyHandler.java:60)
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:179)
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136)
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
org.eclipse.jetty.server.Server.handle(Server.java:451)
org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:252)
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:266)
org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:240)
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:596)
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:527)
java.lang.Thread.run(Thread.java:748)
We didn't have so much clients to arrive at 256 process or threads (limit of 1 dyno)
Workaround
We don't know it yet
Code
<!--Main-->
//...
try {
memcached = new Memcached ();
}
catch (Exception e) {
memcached = null;
}
//...
<!--Memcached-->
public class Memcached {
private MemcachedClient memcachedClient;
private boolean connected;
public Memcached () throws Exception {
setConnected(true);
try {
AuthDescriptor ad = new AuthDescriptor(new String[] { "PLAIN" },
new PlainCallbackHandler(System.getenv("MEMCACHEDCLOUD_USERNAME"),
System.getenv("MEMCACHEDCLOUD_PASSWORD")
)
);
setMemcachedClient(new MemcachedClient(
new ConnectionFactoryBuilder()
.setDaemon(true)
.setFailureMode(FailureMode.Retry)
.setProtocol(ConnectionFactoryBuilder.Protocol.BINARY) //this is the line 34
.setAuthDescriptor(ad).build(),
AddrUtil.getAddresses(System.getenv("MEMCACHEDCLOUD_SERVERS"))
));
} catch (Exception ex) {
// the Memcached client could not be initialized
setConnected(false);
throw new Exception ("{\"ErrorCode\":20,\"Portal\":\"Memcached\",\"ResponseCode\":\"\",\"Message\":\""
+ ex.getMessage() + "\"}");
}
}
public String get (String service, String fromCurrency, String toCurrency) {
if (!isConnected()) {
return null;
}
try {
return (String) getMemcachedClient().get(service + ";" + fromCurrency + ";" + toCurrency);
}
catch (Exception e) {
return null;
}
}
public boolean set (String service, String fromCurrency, String toCurrency, String value) {
if (!isConnected()) {
return false;
}
try {
double hour = Double.parseDouble(System.getenv("TIME_CACHE_HOUR"));
getMemcachedClient().set(service + ";" + fromCurrency + ";" + toCurrency, (int)(60 * 60 * hour), value);
return true;
}catch (Exception e) {
return false;
}
}
public boolean isConnected () {
return this.connected;
}
private void setConnected (boolean connected) {
this.connected = connected;
}
private MemcachedClient getMemcachedClient () {
return this.memcachedClient;
}
private void setMemcachedClient (MemcachedClient memcachedClient) {
this.memcachedClient = memcachedClient;
}
Anyone knows how can we fix this error?
EDIT: I have read the post Java: Unable to create new native thread, but It isn't our problem. We checked the attributes that they said there, but we have good values. However, I could see that everytime that memcached is initialized, It create a new thread, but that thread isn't deleted when the service stop. There's the problem.

how to make a callable implementing class multithread fast with parallel execution?

i am creating a class(CheckCon.java) that implements callable interface which basically returns the no. of devices Connected to the network. the problem is that i donot know how to execute it correctly because the results returned are really slow as compared to traditional multithreading. and i need to return values to a class in (NetScan.java). Kindly Help me execute it properly.
code for CheckCon.java (callable implementing class):
public class CheckCon implements Callable<Object>{
int startindex,endindex;
ArrayList<Object> list;
byte[] ip;
public CheckCon(int startindex, int endindex, byte[] ip) {
this.startindex = startindex;
this.endindex = endindex;
this.ip = ip;
list = new ArrayList<>();
}
#Override
public ArrayList<Object> call() throws Exception {
for(int i =startindex;i<endindex;i++){
try {
ip[3] = (byte)i;
InetAddress address = InetAddress.getByAddress(ip);
if (address.isReachable(1000))
{
System.out.println("Name is......"+address.getHostName()+"\tIP is......."+address.getHostAddress());
}
else if (!address.getHostAddress().equals(address.getHostName()))
{
String host = address.getCanonicalHostName();
String ipaddress =address.toString().replace(host+"/", "");
Object[] data = {host,ipaddress};
list.add(data);
System.out.println("Name is......"+address.getHostName()+"\tIP is......."+address.getHostAddress());
}
else
{
System.out.println("nothing");
// the host address and host name are equal, meaning the host name could not be resolved
} } catch (UnknownHostException ex) {
Logger.getLogger(NetScan.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(NetScan.class.getName()).log(Level.SEVERE, null, ex);
}
}
return list;
}
}
and the class Calling NetScan.java
private void getDataForTable() throws InterruptedException, ExecutionException {
ExecutorService executor =Executors.newCachedThreadPool();
for(int i =0; i<26 ; i++){
if(s == 250){
Future<Object> f =executor.submit(new CheckCon(s,s+5,ip));
list.add(f);
break;
}else{
Future<Object> f =executor.submit(new CheckCon(s,s+10,ip));
list.add(f);
s= s+10;
}
}
dm = new DefaultTableModel(ColumnName,0){
#Override
public boolean isCellEditable(int row, int column) {
return false;
}
};
for(Future<Object> f : list){
dm.addRow((Object[]) f.get());
}}
I am creating 25 threads checking 10 IP's in each thread.

I removed the CheckCon.java and edited the NetScan to get what i Want thanks to #amit-bera . i Created a thread and asked the completablefuture to runasync with executor defining the no.of threads i require.
ExecutorService e =Executors.newFixedThreadPool(25);
for(int i =0;i<255;i++){
final int j =i;
f = CompletableFuture.runAsync(new Runnable() {
#Override
public void run() {
try {
ip[3] = (byte)j;
InetAddress address = InetAddress.getByAddress(ip);
if (address.isReachable(1000))
{
System.out.println("Name is......"+address.getHostName()+"\tIP is......."+address.getHostAddress()+j);
}
else if (!address.getHostAddress().equals(address.getHostName()))
{
String host = address.getCanonicalHostName();
String ipaddress =address.toString().replace(host+"/", "");
Object[] data = {host,ipaddress};
dm.addRow(data);
System.out.println("Name is......"+address.getHostName()+"\tIP is......."+address.getHostAddress()+j);
}
else
{
System.out.println("nothing"+j);
// the host address and host name are equal, meaning the host name could not be resolved
} } catch (UnknownHostException ex) {
Logger.getLogger(NetScan.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(NetScan.class.getName()).log(Level.SEVERE, null, ex);
}
}
},e);
}
added this content to a method.

As per my obervationa please used singolton java pettern.example i am giving below.singolton alwaus retun instance so you save request response time on thread.
Thread is light-wight process but still initilised thread at whenever need.please make sure that thread work based on your server configrationa.
if you are using linux system
used TOP command to usage of your system perfomance and process execution on threadiwse.
i try and its work fine on my system.
class Singleton
{
private static Singleton single_instance = null;
public String s;
private Singleton()
{
s = "Hello I am a string part of Singleton class";
}
public static Singleton getInstance()
{
if (single_instance == null)
single_instance = new Singleton();
return single_instance;
}
}

How to limit the number of records while reading from mysql table using multithreading

I have 1.5 million records in my mysql table. I'm trying to read all the records in a batch process i.e,planning to read 1000 records in a batch and print those records in console.
For this I'm planning to implement multithreading concept using java. How can I implement this?

In MySQL you get all records at once or you get them one by one in a streaming fashion (see this answer). Alternatively, you can use the limit keyword for chunking (see this answer).
Whether you use streaming results or chunking, you can use multi-threading to process (or print) data while you read data. This is typically done using a producer-consumer pattern where, in this case, the producer retrieves data from the database, puts it on a queue and the consumer takes the data from the queue and processes it (e.g. print to the console).
There is a bit of administration overhead though: both producer and consumer can freeze or trip over an error and both need to be aware of this so that they do not hang forever (potentially freezing your application). This is where "reasonable" timeouts come in ("reasonable" depends entirely on what is appropriate in your situation).
I have tried to put this in a minimal running example, but it is still a lot of code (see below). There are two commented lines that can be used to test the timeout-case. There is also a refreshTestData variable that can be used to re-use inserted records (inserting records can take a long time).
To keep it clean, a lot of keywords like private/public are omitted (i.e. these need to be added in non-demo code).
import java.sql.*;
import java.util.*;
import java.util.concurrent.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class FetchRows {
private static final Logger log = LoggerFactory.getLogger(FetchRows.class);
public static void main(String[] args) {
try {
new FetchRows().print();
} catch (Exception e) {
e.printStackTrace();
}
}
void print() throws Exception {
Class.forName("com.mysql.jdbc.Driver").newInstance();
Properties dbProps = new Properties();
dbProps.setProperty("user", "test");
dbProps.setProperty("password", "test");
try (Connection conn = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", dbProps)) {
try (Statement st = conn.createStatement()) {
prepareTestData(st);
}
// https://stackoverflow.com/a/2448019/3080094
try (Statement st = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
java.sql.ResultSet.CONCUR_READ_ONLY)) {
st.setFetchSize(Integer.MIN_VALUE);
fetchAndPrintTestData(st);
}
}
}
boolean refreshTestData = true;
int maxRecords = 5_555;
void prepareTestData(Statement st) throws SQLException {
int recordCount = 0;
if (refreshTestData) {
st.execute("drop table if exists fetchrecords");
st.execute("create table fetchrecords (id mediumint not null auto_increment primary key, created timestamp default current_timestamp)");
for (int i = 0; i < maxRecords; i++) {
st.addBatch("insert into fetchrecords () values ()");
if (i % 500 == 0) {
st.executeBatch();
log.debug("{} records available.", i);
}
}
st.executeBatch();
recordCount = maxRecords;
} else {
try (ResultSet rs = st.executeQuery("select count(*) from fetchrecords")) {
rs.next();
recordCount = rs.getInt(1);
}
}
log.info("{} records available for testing.", recordCount);
}
int batchSize = 1_000;
int maxBatchesInMem = 3;
int printFinishTimeoutS = 5;
void fetchAndPrintTestData(Statement st) throws SQLException, InterruptedException {
final BlockingQueue<List<FetchRecordBean>> printQueue = new LinkedBlockingQueue<List<FetchRecordBean>>(maxBatchesInMem);
final PrintToConsole printTask = new PrintToConsole(printQueue);
new Thread(printTask).start();
try (ResultSet rs = st.executeQuery("select * from fetchrecords")) {
List<FetchRecordBean> l = new LinkedList<>();
while (rs.next()) {
FetchRecordBean bean = new FetchRecordBean();
bean.setId(rs.getInt("id"));
bean.setCreated(new java.util.Date(rs.getTimestamp("created").getTime()));
l.add(bean);
if (l.size() % batchSize == 0) {
/*
* The printTask can stop itself when this producer is too slow to put records on the print-queue.
* Therefor, also check printTask.isStopping() to break the while-loop.
*/
if (printTask.isStopping()) {
throw new TimeoutException("Print task has stopped.");
}
enqueue(printQueue, l);
l = new LinkedList<>();
}
}
if (l.size() > 0) {
enqueue(printQueue, l);
}
} catch (TimeoutException | InterruptedException e) {
log.error("Unable to finish printing records to console: {}", e.getMessage());
printTask.stop();
} finally {
log.info("Reading records finished.");
if (!printTask.isStopping()) {
try {
enqueue(printQueue, Collections.<FetchRecordBean> emptyList());
} catch (Exception e) {
log.error("Unable to signal last record to print.", e);
printTask.stop();
}
}
if (!printTask.await(printFinishTimeoutS, TimeUnit.SECONDS)) {
log.error("Print to console task did not finish.");
}
}
}
int enqueueTimeoutS = 5;
// To test a slow printer, see also Thread.sleep statement in PrintToConsole.print.
// int enqueueTimeoutS = 1;
void enqueue(BlockingQueue<List<FetchRecordBean>> printQueue, List<FetchRecordBean> l) throws InterruptedException, TimeoutException {
log.debug("Adding {} records to print-queue.", l.size());
if (!printQueue.offer(l, enqueueTimeoutS, TimeUnit.SECONDS)) {
throw new TimeoutException("Unable to put print data on queue within " + enqueueTimeoutS + " seconds.");
}
}
int dequeueTimeoutS = 5;
class PrintToConsole implements Runnable {
private final BlockingQueue<List<FetchRecordBean>> q;
private final CountDownLatch finishedLock = new CountDownLatch(1);
private volatile boolean stop;
public PrintToConsole(BlockingQueue<List<FetchRecordBean>> q) {
this.q = q;
}
#Override
public void run() {
try {
while (!stop) {
List<FetchRecordBean> l = q.poll(dequeueTimeoutS, TimeUnit.SECONDS);
if (l == null) {
log.error("Unable to get print data from queue within {} seconds.", dequeueTimeoutS);
break;
}
if (l.isEmpty()) {
break;
}
print(l);
}
if (stop) {
log.error("Printing to console was stopped.");
}
} catch (Exception e) {
log.error("Unable to print records to console.", e);
} finally {
if (!stop) {
stop = true;
log.info("Printing to console finished.");
}
finishedLock.countDown();
}
}
void print(List<FetchRecordBean> l) {
log.info("Got list with {} records from print-queue.", l.size());
// To test a slow printer, see also enqueueTimeoutS.
// try { Thread.sleep(1500L); } catch (Exception ignored) {}
}
public void stop() {
stop = true;
}
public boolean isStopping() {
return stop;
}
public void await() throws InterruptedException {
finishedLock.await();
}
public boolean await(long timeout, TimeUnit tunit) throws InterruptedException {
return finishedLock.await(timeout, tunit);
}
}
class FetchRecordBean {
private int id;
private java.util.Date created;
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public java.util.Date getCreated() {
return created;
}
public void setCreated(java.util.Date created) {
this.created = created;
}
}
}
Dependencies:
mysql:mysql-connector-java:5.1.38
org.slf4j:slf4j-api:1.7.20 (and to get logging shown in console: ch.qos.logback:logback-classic:1.1.7 with ch.qos.logback:logback-core:1.1.7)

Couchbase: Initialization from within static code block takes longer

I put my couchbase initialization code inside a static code block:
static {
initCluster();
bucket = initBucket("graph");
metaBucket = initBucket("meta");
BLACKLIST = new SetObservingCache<String>(() -> getBlackList(), BLACKLIST_REFRESH_INTERVAL_SEC * 1000);
}
I know it's not a good practice but it was very convenient and served its purpose, as I need this code to run exactly once in a multi-threaded environment and block all subsequent calls from other threads until it's finished (blacklist has been initialized).
To my surprise, the call to getBlacklist() timed out and couldn't be completed.
However, when calling it again after 2 minutes (that's what the ObservingCache does), it completed in less than a second.
In order to solve this, I refactored my code and made the blacklist acquisition lazy:
public boolean isBlacklisted(String key) {
// BLACKLIST variable should NEVER be touched outside of this context.
assureBlacklistIsPopulated();
return BLACKLIST != null ? BLACKLIST.getItems().contains(key) : false;
}
private void assureBlacklistIsPopulated() {
if (!ENABLE_BLACKLIST) {
return;
}
if (BLACKLIST == null) {
synchronized (CouchConnectionManager.class) {
if (BLACKLIST == null) {
BLACKLIST = new SetObservingCache<String>(() -> getBlackList(), BLACKLIST_REFRESH_INTERVAL_SEC * 1000);
}
}
}
}
The call to isBlacklisted() blocks all other threads that attempt to check if an entry is blacklisted until blacklist is initialized.
I'm not a big fan of this solution because it's very verbose and error prone - one might try to read from BLACKLIST without calling assureBlacklistIsPopulated() beforehand.
The static (and non final) fields within the class are as follows:
private static CouchbaseCluster cluster;
private static Bucket bucket;
private static Bucket metaBucket;
private static SetObservingCache<String> BLACKLIST;
I can't figure out why the call succeeded when it wasn't a part of the static initialization block. Is there any known performance-related vulnerability of the static initialization block that I'm not aware of?
EDIT: Added initialization code per request
private Bucket initBucket(String bucketName) {
while(true) {
Throwable t = null;
try {
ReportableThread.updateStatus("Initializing bucket " + bucketName);
return cluster.openBucket(bucketName);
} catch(Throwable t1) {
t1.printStackTrace();
t = t1;
}
try {
ReportableThread.updateStatus(String.format("Failed to open bucket: %s reason: %s", bucketName, t));
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private void initCluster() {
CouchbaseEnvironment env = DefaultCouchbaseEnvironment
.builder()
.kvTimeout(MINUTE)
.connectTimeout(MINUTE)
.retryStrategy(FailFastRetryStrategy.INSTANCE)
.requestBufferSize(16384 * 2)
.responseBufferSize(16384 * 2)
.build();
while(true) {
ReportableThread.updateStatus("Initializing couchbase cluster");
Throwable t = null;
try {
cluster = CouchbaseCluster.create(env, getServerNodes());
if(cluster != null) {
return;
}
} catch(Throwable t1) {
t1.printStackTrace();
t = t1;
}
try {
ReportableThread.updateStatus(String.format("Failed to create connection to couch %s", t));
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public Set<String> getBlackList() {
ReportableThread.updateStatus("Getting black list");
AbstractDocument<?> abstractDoc = get("blacklist", metaBucket, JsonArrayDocument.class);
JsonArrayDocument doc = null;
if (abstractDoc != null && abstractDoc instanceof JsonArrayDocument) {
doc = (JsonArrayDocument)abstractDoc;
} else {
return new HashSet<String>();
}
ReportableThread.updateStatus(String.format("%s: Got %d items | sorting items", new Date(System.currentTimeMillis()).toString(), doc.content().size()));
HashSet<String> ret = new HashSet<String>();
for (Object string : doc.content()) {
if (string != null) {
ret.add(string.toString());
}
}
return ret;
}

1st: you are doing the double-check idiom. That's always bad.
Put only one if(BLACKLIST==null) and it must be inside the synchronized.
2nd: the lazy init is fine, but do it in a static getInstance() and NEVER expose the BLACKLIST field.

Design pattern for "retrying" logic that failed?

I'm writing some reconnect logic to periodically attempt to establish a connection to a remote endpoint which went down. Essentially, the code looks like this:
public void establishConnection() {
try {
this.connection = newConnection();
} catch (IOException e) {
// connection failed, try again.
try { Thread.sleep(1000); } catch (InterruptedException e) {};
establishConnection();
}
}
I've solved this general problem with code similar to the above on many occasions, but I feel largely unsatisfied with the result. Is there a design pattern designed for dealing with this issue?

Shameless plug: I have implemented some classes to allow retrying operations. The library is not made available yet, but you may fork it on github.
And a fork exists.
It allows building a Retryer with various flexible strategies. For example:
Retryer retryer =
RetryerBuilder.newBuilder()
.withWaitStrategy(WaitStrategies.fixedWait(1, TimeUnit.SECOND))
.withStopStrategy(StopStrategies.stopAfterAttempt(3))
.retryIfExceptionOfType(IOException.class)
.build();
And you can then execute a callable (or several ones) with the Retryer:
retryer.call(new Callable<Void>() {
public Void call() throws IOException {
connection = newConnection();
return null;
}
}

You could try the Idempotent Retry Pattern.

I really like this Java 8 code from this blog and you don't need any extra library on your classpath.
You only need to pass a function to the retry class.
#Slf4j
public class RetryCommand<T> {
private int maxRetries;
RetryCommand(int maxRetries)
{
this.maxRetries = maxRetries;
}
// Takes a function and executes it, if fails, passes the function to the retry command
public T run(Supplier<T> function) {
try {
return function.get();
} catch (Exception e) {
log.error("FAILED - Command failed, will be retried " + maxRetries + " times.");
return retry(function);
}
}
private T retry(Supplier<T> function) throws RuntimeException {
int retryCounter = 0;
while (retryCounter < maxRetries) {
try {
return function.get();
} catch (Exception ex) {
retryCounter++;
log.error("FAILED - Command failed on retry " + retryCounter + " of " + maxRetries, ex);
if (retryCounter >= maxRetries) {
log.error("Max retries exceeded.");
break;
}
}
}
throw new RuntimeException("Command failed on all of " + maxRetries + " retries");
}
}
And to use it:
new RetryCommand<>(5).run(() -> client.getThatThing(id));

Using Failsafe (author here):
RetryPolicy retryPolicy = new RetryPolicy()
.retryOn(IOException.class)
.withMaxRetries(5)
.withDelay(1, TimeUnit.SECONDS);
Failsafe.with(retryPolicy).run(() -> newConnection());
No annotations, no magic, doesn't need to be a Spring app, etc. Just straightforward and simple.

I'm using AOP and Java annotations. There is a ready-made mechanism in jcabi-aspects (I'm a developer):
#RetryOnFailure(attempts = 3, delay = 1, unit = TimeUnit.SECONDS)
public void establishConnection() {
this.connection = newConnection();
}
ps. You can also try RetryScalar from Cactoos.

You can try spring-retry, it has a clean interface and it's easy to use.
Example:
#Retryable(maxAttempts = 4, backoff = #Backoff(delay = 500))
public void establishConnection() {
this.connection = newConnection();
}
In case of exception, it will retry (call) up to 4 times the method establishConnection() with a backoff policy of 500ms

You can also create a wrapper function that just does a loop over the intended operation and when is done just break out of the loop.
public static void main(String[] args) {
retryMySpecialOperation(7);
}
private static void retryMySpecialOperation(int retries) {
for (int i = 1; i <= retries; i++) {
try {
specialOperation();
break;
}
catch (Exception e) {
System.out.println(String.format("Failed operation. Retry %d", i));
}
}
}
private static void specialOperation() throws Exception {
if ((int) (Math.random()*100) % 2 == 0) {
throw new Exception("Operation failed");
}
System.out.println("Operation successful");
}

If you are using java 8, this may helps.
import java.util.function.Supplier;
public class Retrier {
public static <T> Object retry(Supplier<T> function, int retryCount) throws Exception {
while (0<retryCount) {
try {
return function.get();
} catch (Exception e) {
retryCount--;
if(retryCount == 0) {
throw e;
}
}
}
return null;
}
public static void main(String[] args) {
try {
retry(()-> {
System.out.println(5/0);
return null;
}, 5);
} catch (Exception e) {
System.out.println("Exception : " + e.getMessage());
}
}
}
Thanks,
Praveen R.

I'm using retry4j library. Test code example:
public static void main(String[] args) {
Callable<Object> callable = () -> {
doSomething();
return null;
};
RetryConfig config = new RetryConfigBuilder()
.retryOnAnyException()
.withMaxNumberOfTries(3)
.withDelayBetweenTries(5, ChronoUnit.SECONDS)
.withExponentialBackoff()
.build();
new CallExecutorBuilder<>().config(config).build().execute(callable);
}
public static void doSomething() {
System.out.println("Trying to connect");
// some logic
throw new RuntimeException("Disconnected"); // init error
// some logic
}

Here's a another approach to perform the retry. No libraries, no annotations, no extra implementations. Import java.util.concurrent.TimeUnit;
public static void myTestFunc() {
boolean retry = true;
int maxRetries = 5; //max no. of retries to be made
int retries = 1;
int delayBetweenRetries = 5; // duration between each retry (in seconds)
int wait = 1;
do {
try {
this.connection = newConnection();
break;
}
catch (Exception e) {
wait = retries * delayBetweenRetries;
pause(wait);
retries += 1;
if (retries > maxRetries) {
retry = false;
log.error("Task failed on all of " + maxRetries + " retries");
}
}
} while (retry);
}
public static void pause(int seconds) {
long secondsVal = TimeUnit.MILLISECONDS.toMillis(seconds);
try {
Thread.sleep(secondsVal);
}
catch (InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
}

there is nothing special in retrying at all - take this class as example http://www.docjar.com/html/api/org/springframework/jms/listener/DefaultMessageListenerContainer.java.html
As you can see even spring developers still writing code for retry-ing - line 791...
there is no such special pattern AFAIK..
What i can advice to deal with resources is to take apache commons pool library - check this http://commons.apache.org/pool/apidocs/org/apache/commons/pool/impl/GenericObjectPool.html and visit http://commons.apache.org/pool

I have wrote my custom annotation. Maybe you can use this annotation.
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
public #interface RetryOperation {
int retryCount();
int waitSeconds();
}
#Slf4j
#Aspect
#Component
public class RetryOperationAspect {
#Around(value = "#annotation(com.demo.infra.annotation.RetryOperation)")
public Object retryOperation(ProceedingJoinPoint joinPoint) throws Throwable {
Object response = null;
Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
RetryOperation annotation = method.getAnnotation(RetryOperation.class);
int retryCount = annotation.retryCount();
int waitSeconds = annotation.waitSeconds();
boolean successful = false;
do {
try {
response = joinPoint.proceed();
successful = true;
} catch (Exception e) {
log.error("Operation failed, retries remaining: {}", retryCount);
retryCount--;
if (retryCount < 0) {
throw e;
}
if (waitSeconds > 0) {
log.warn("Waiting for {} second(s) before next retry", waitSeconds);
Thread.sleep(waitSeconds * 1000L);
}
}
} while (!successful);
return response;
}
}
#RetryOperation(retryCount = 5, waitSeconds = 1)
public void method() {
}