Develop Reference

Couchbase: Initialization from within static code block takes longer

I put my couchbase initialization code inside a static code block:
static {
initCluster();
bucket = initBucket("graph");
metaBucket = initBucket("meta");
BLACKLIST = new SetObservingCache<String>(() -> getBlackList(), BLACKLIST_REFRESH_INTERVAL_SEC * 1000);
}
I know it's not a good practice but it was very convenient and served its purpose, as I need this code to run exactly once in a multi-threaded environment and block all subsequent calls from other threads until it's finished (blacklist has been initialized).
To my surprise, the call to getBlacklist() timed out and couldn't be completed.
However, when calling it again after 2 minutes (that's what the ObservingCache does), it completed in less than a second.
In order to solve this, I refactored my code and made the blacklist acquisition lazy:
public boolean isBlacklisted(String key) {
// BLACKLIST variable should NEVER be touched outside of this context.
assureBlacklistIsPopulated();
return BLACKLIST != null ? BLACKLIST.getItems().contains(key) : false;
}
private void assureBlacklistIsPopulated() {
if (!ENABLE_BLACKLIST) {
return;
}
if (BLACKLIST == null) {
synchronized (CouchConnectionManager.class) {
if (BLACKLIST == null) {
BLACKLIST = new SetObservingCache<String>(() -> getBlackList(), BLACKLIST_REFRESH_INTERVAL_SEC * 1000);
}
}
}
}
The call to isBlacklisted() blocks all other threads that attempt to check if an entry is blacklisted until blacklist is initialized.
I'm not a big fan of this solution because it's very verbose and error prone - one might try to read from BLACKLIST without calling assureBlacklistIsPopulated() beforehand.
The static (and non final) fields within the class are as follows:
private static CouchbaseCluster cluster;
private static Bucket bucket;
private static Bucket metaBucket;
private static SetObservingCache<String> BLACKLIST;
I can't figure out why the call succeeded when it wasn't a part of the static initialization block. Is there any known performance-related vulnerability of the static initialization block that I'm not aware of?
EDIT: Added initialization code per request
private Bucket initBucket(String bucketName) {
while(true) {
Throwable t = null;
try {
ReportableThread.updateStatus("Initializing bucket " + bucketName);
return cluster.openBucket(bucketName);
} catch(Throwable t1) {
t1.printStackTrace();
t = t1;
}
try {
ReportableThread.updateStatus(String.format("Failed to open bucket: %s reason: %s", bucketName, t));
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
private void initCluster() {
CouchbaseEnvironment env = DefaultCouchbaseEnvironment
.builder()
.kvTimeout(MINUTE)
.connectTimeout(MINUTE)
.retryStrategy(FailFastRetryStrategy.INSTANCE)
.requestBufferSize(16384 * 2)
.responseBufferSize(16384 * 2)
.build();
while(true) {
ReportableThread.updateStatus("Initializing couchbase cluster");
Throwable t = null;
try {
cluster = CouchbaseCluster.create(env, getServerNodes());
if(cluster != null) {
return;
}
} catch(Throwable t1) {
t1.printStackTrace();
t = t1;
}
try {
ReportableThread.updateStatus(String.format("Failed to create connection to couch %s", t));
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public Set<String> getBlackList() {
ReportableThread.updateStatus("Getting black list");
AbstractDocument<?> abstractDoc = get("blacklist", metaBucket, JsonArrayDocument.class);
JsonArrayDocument doc = null;
if (abstractDoc != null && abstractDoc instanceof JsonArrayDocument) {
doc = (JsonArrayDocument)abstractDoc;
} else {
return new HashSet<String>();
}
ReportableThread.updateStatus(String.format("%s: Got %d items | sorting items", new Date(System.currentTimeMillis()).toString(), doc.content().size()));
HashSet<String> ret = new HashSet<String>();
for (Object string : doc.content()) {
if (string != null) {
ret.add(string.toString());
}
}
return ret;
}

1st: you are doing the double-check idiom. That's always bad.
Put only one if(BLACKLIST==null) and it must be inside the synchronized.
2nd: the lazy init is fine, but do it in a static getInstance() and NEVER expose the BLACKLIST field.

How to get the object reference of a field of an object using reflection?

Note: Every time I go to type up this question, I think I see something, but it never pans out, so for the third or fourth time.
Synopsis: I am trying to serialize an object that inherits from the base Response class using the Response class, of which the subclass may have non-primitive field types.
The code is as such (warning: large and not elegant), ordered from the specific class (SpecificResponse), as extended from a common class (CommonResponse), which is a concrete implementation of the abstract class (Response), driven by a test program (Program).
// SpecificResponse.java
package com.jdgj.thinking;
public final class SpecificResponse extends CommonResponse {
public String hell;
public int trike;
public short tail;
public SpecificResponse() {
super();
}
}
SpecifcResponse extends CommonResponse:
// CommonResponse.java
package com.jdgj.thinking;
public class CommonResponse extends Response {
public int thing2;
public Value value;
#Override
protected void initialize() {
System.out.println("hello!");
value = new Value();
}
}
For testing purposes, I just made a simple Value object:
// Value.java
package com.jdgj.thinking;
public class Value {
public int five;
}
And, that which does a lot of work, and also the foundation for CommonResponse, the Response class:
// Response.java
package com.jdgj.thinking;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.List;
public abstract class Response {
public static Class<?> fromClassSignature(List<String> signature) throws IllegalArgumentException, IllegalStateException, ClassNotFoundException {
if (signature == null || signature.size() == 0) {
throw new IllegalArgumentException("Null or empty Response class signature.");
}
String lastClassName = null;
for (String line : signature) {
if (line.startsWith("class:")) {
lastClassName = line.split(":")[1];
}
}
if (lastClassName == null) {
throw new IllegalStateException("Could not find the Response class name.");
}
Class<?> c = Class.forName(lastClassName);
lastClassName = null;
Class<?> sc = c.getSuperclass();
while (sc != null && !sc.equals(Response.class)) {
sc = sc.getSuperclass();
}
if (sc != null) {
sc = null;
return c;
} else {
return null;
}
}
protected abstract void initialize();
private String getFieldSignature(Field field) {
return "field:" + field.getName() + "|" + field.getType().getCanonicalName();
}
private List<String> serializeObject(Class<?> c, Object o) {
List<String> serialization = new ArrayList<String>(0);
serialization.add("class:" + c.getName());
for (Field f : c.getDeclaredFields()) {
if (!f.isSynthetic() && Modifier.isPublic(f.getModifiers())) {
StringBuilder sb = new StringBuilder(getFieldSignature(f));
Class<?> t = f.getType();
Object value = null;
try {
value = f.get(o);
System.out.println(f.getName() + "=" + value);
if (t.isPrimitive() || t.equals(String.class)) {
sb.append("`" + value.toString() + "");
}
} catch (NullPointerException e) {
if (t.isPrimitive() || t.equals(String.class)) {
sb.append("`");
} else {
System.out.println("UNEXPECTED NULL POINTER EXCEPTION");
}
} catch (IllegalArgumentException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IllegalAccessException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} finally {
serialization.add(sb.toString());
if (value != null) {
if (!(t.isPrimitive() || t.equals(String.class))) {
serialization.addAll(serializeObject(t, value));
}
}
}
sb = null;
t = null;
}
}
return serialization;
}
private List<String> describeClass(Class<?> c) {
List<String> description = new ArrayList<String>(0);
description.add("class:" + c.getName());
for (Field f : c.getDeclaredFields()) {
if (!f.isSynthetic() && Modifier.isPublic(f.getModifiers())) {
description.add(getFieldSignature(f));
Class<?> t = f.getType();
if (!(t.isPrimitive() || t.equals(String.class))) {
description.addAll(describeClass(t));
}
t = null;
}
}
return description;
}
public final List<String> getSerializedObject() {
Class<?> c = getClass();
List<String> object = new ArrayList<String>(0);
while (c != null && !c.equals(Response.class)) {
object.addAll(0, serializeObject(c, this));
c = c.getSuperclass();
}
c = null;
return object;
}
public final List<String> getClassSignature() {
Class<?> c = getClass();
List<String> signature = new ArrayList<String>(0);
while (c != null && !c.equals(Response.class)) {
signature.addAll(0, describeClass(c));
c = c.getSuperclass();
}
c = null;
return signature;
}
}
These classes are driven by a 'dev-test' program for now:
// Program.java
import com.jdgj.thinking.Response;
import com.jdgj.thinking.SpecificResponse;
public class Program {
private static void printClassSignature(Response response) {
for (String line : response.getClassSignature()) {
System.out.println(line);
}
}
private static void printSerializedObject(Response response) {
for (String line : response.getSerializedObject()) {
System.out.println(line);
}
}
public static void main(String[] args) {
String CN_SPECRSP = "com.jdgj.thinking.SpecificResponse";
Class<?> response = null;
try {
response = Response.fromClassSignature(new SpecificResponse().getClassSignature());
} catch (IllegalArgumentException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IllegalStateException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (ClassNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} finally {
if (response != null) {
System.out.println("Expected: " + CN_SPECRSP + "; Actually: " + response.getCanonicalName());
Response rsp = null;
try {
rsp = (Response)response.newInstance();
} catch (InstantiationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (rsp != null) {
//printClassSignature(rsp);
printSerializedObject(rsp);
rsp = null;
}
}
response = null;
}
}
}
}
Here's the output:
Expected: com.jdgj.thinking.SpecificResponse; Actually: com.jdgj.thinking.SpecificResponse
hell=null
trike=0
tail=0
thing2=0
value=null
class:com.jdgj.thinking.CommonResponse
field:thing2|int`0
field:value|com.jdgj.thinking.Value
class:com.jdgj.thinking.SpecificResponse
field:hell|java.lang.String`
field:trike|int`0
field:tail|short`0
Why does value report null?
In both not having and having a constructor defined in CommonResponse to initialize the Value instance, it's still shown as null. If I uncomment the Program.getClassSignature method, the code delves into the Value object to get the five field:
Expected: com.jdgj.thinking.SpecificResponse; Actually: com.jdgj.thinking.SpecificResponse
class:com.jdgj.thinking.CommonResponse
field:thing2|int
field:value|com.jdgj.thinking.Value
class:com.jdgj.thinking.Value
field:five|int
class:com.jdgj.thinking.SpecificResponse
field:hell|java.lang.String
field:trike|int
field:tail|short
hell=null
trike=0
tail=0
thing2=0
value=null
class:com.jdgj.thinking.CommonResponse
field:thing2|int`0
field:value|com.jdgj.thinking.Value
class:com.jdgj.thinking.SpecificResponse
field:hell|java.lang.String`
field:trike|int`0
field:tail|short`0
I feel as if I've exhausted my Google-fu, and I feel like I'm missing something just.. blatantly obvious, but I cannot think of the reason, or the right query to ask Google. Everyone keeps providing ways to get primitive fields, but that is not what I am seeking. I therefore submit to the guidance of SO.

As Holger said you never initialize that field. You have a initialize() method to do it, but never invoke it.
Of course if you call getClassSignature() you get info about your field, actually it's there; you got a field named value of type com.jdgj.thinking.Value in your class, but it never has been instantiated, so that field value is null.

Retrying Method calls in generic way

My Java application requires a retry logic on remote calls failures.
These remote calls are:
scattered all over the application
pertain to different Remote Service classes.
Also, the retry logic may have varying retry interval and varying retry attempts.
I need a generic retry() implementation which can make appropriate method calls depending on from where it is called. Below is a simple code illustration of I am looking for. I know we can attempt to do this using java reflection, but, is there a framework or an open source available somewhere which is read-to-use?
try {
ClassA objA = remoteServiceA.call(paramA1, paramA2, ...);
} catch (Exception e){
ClassA objA = (ClassA)retry(remoteService, listOfParams, ..); // generic method call
}
..
try {
ClassB objB = remoteServiceB.call(paramB1, paramB2, ...);
} catch (Exception e){
ClassA objB = (ClassB)retry(remoteService, listOfParams, ..); // generic method call
}

As already suggested, you should use AOP and Java annotations. I would recommend a read-made mechanism from jcabi-aspects (I'm a developer):
#RetryOnFailure(attempts = 3, delay = 5)
public String load(URL url) {
return url.openConnection().getContent();
}
Read also this blog post: http://www.yegor256.com/2014/08/15/retry-java-method-on-exception.html
Update: Check RetryFunc from Cactoos.

This is a book example of where aspectj (or aop in general) can be used, see 8.2.7 Example in Spring documentation and 5 Reasons Java Developers Should Learn and Use AspectJ.
Basically an aspect intercepts all calls to given methods (specified using annotation, naming convention, whatever) and retries.

Assume you have a method, that need to retied at every 500ms and upto 5 times.
Current class:
public class RemoteCaller{
Service serviceCaller;
public void remoteCall(String message) {
serviceCaller.updateDetails( this.message);
return null;
}
}
Modified approach:
public class RetriableHelper<T> implements Callable<T> {
private Callable<T> task;
private int numberOfRetries;
private int numberOfTriesLeft;
private long timeToWait;
public RetriableHelper(int numberOfRetries, long timeToWait, Callable<T> task) {
this.numberOfRetries = numberOfRetries;
numberOfTriesLeft = numberOfRetries;
this.timeToWait = timeToWait;
this.task = task;
}
public T call() throws Exception {
while (true) {
try {
return task.call();
} catch (InterruptedException e) {
throw e;
} catch (CancellationException e) {
throw e;
} catch (Exception e) {
numberOfTriesLeft--;
if (numberOfTriesLeft == 0) {
throw e;
}
Thread.sleep(timeToWait);
}
}
}
}
Backend system/remote call class:
public class RemoteCaller{
Service serviceCaller;
public void remoteCall(String message) {
class RemoteCallable implements Callable<Void> {
String message;
public RemoteCallable( String message)
{
this.message = message;
}
public Void call() throws Exception{
serviceCaller.updateDetails( this.message);
return null;
}
}
RetriableHelper<Void> retriableHelper = new RetriableHelper<Void>(5, 500, new RemoteCallable( message));
try {
retriableHelper.call();
} catch (Exception e) {
throw e;
}
}
}

enter link description here Spring has a retry annotation which servers the purpose
Step 1: Add following dependency to your POM
<dependency>
<groupId>org.springframework.retry</groupId>
<artifactId>spring-retry</artifactId>
<version>1.1.5.RELEASE</version>
</dependency>
Step 2: Enabling Spring Retry
To enable Spring Retry in an application, we need to add the #EnableRetry annotation to our #Configuration class:
Ex:
#Configuration
#EnableRetry
public class AppConfig { ... }
Step 3: To add retry functionality to methods, #Retryable can be used:
Ex:
#Service
public interface MyService {
#Retryable(
value = { SQLException.class },
maxAttempts = 2,
backoff = #Backoff(delay = 5000))
void retryService(String sql) throws SQLException;
...
}
Step 4.The #Recover annotation is used to define a separate recovery method when a #Retryable method fails with a specified exception:
Ex:
#Service
public interface MyService {
...
#Recover
void recover(SQLException e, String sql);
}
See Url for more details : http://www.baeldung.com/spring-retry

where do you get the services from? use a factory to Proxy the service you get from the original factory. The proxy can then implement the retry transparently. See the java Proxy/ProxyGenerators in reflection.

If you are using spring , then better go with Aspects.
Otherwise, below sample solution can work:
public class Test
{
public static void main(String[] args) throws Exception
{
Test test = new Test();
test.toRunFirst("Hello! This is normal invocation");
runWithRetry(test, "toRunFirst", "Hello! This is First, called with retry");
runWithRetry(test, "toRunSecond", "Hello! This is Second, called with retry");
}
public void toRunFirst(String s) {
System.out.println(s);
}
public void toRunSecond(String s) {
System.out.println(s);
}
public static Object runWithRetry(Object obj, String methodName, Object... args) throws Exception
{
Class<?>[] paramClass = new Class<?>[args.length];
for(int i=0; i< args.length; i++) {
paramClass[i] = args[i].getClass();
}
Method method = obj.getClass().getDeclaredMethod(methodName, paramClass);
int retryCount = 2;
for(int i=0; i< retryCount; i++) {
try {
return method.invoke(obj, args);
}
catch (Exception e) {
e.printStackTrace();
}
}
return null;
}
}

I did not find what I needed so there is mine.
The main feature is that it throws the type of Exception you need when maxRetries is reached so you can catch it in the call.
import org.apache.log4j.Logger;
public class TaskUtils {
public static <E extends Throwable> void retry(int maxRetries, Task<E> task) throws E {
retry(maxRetries, 0, null, task);
}
public static <E extends Throwable> void retry(int maxRetries, long waitTimeMs, Logger logger, Task<E> task) throws E {
while (maxRetries > 0) {
maxRetries--;
try {
task.run();
} catch (Exception e) {
if (maxRetries == 0) {
try {
throw e;
} catch (Exception ignored) { // can't happen but just in case we wrap it in
throw new RuntimeException(e);
}
}
if (logger != null)
logger.warn("Attempt " + maxRetries + " failed", e);
try {
Thread.sleep(waitTimeMs);
} catch (InterruptedException ignored) {
}
}
}
}
public interface Task<E extends Throwable> {
void run() throws E;
}
}
Usage :
TaskUtils.retry(3, 500, LOGGER, () -> stmClickhouse.execute(
"ALTER TABLE `" + database + "`.`" + table.getName() + "` ON CLUSTER " + clusterName + allColumnsSql
));

add it into pom.xml
<dependency>
<groupId>org.deking.utils</groupId>
<artifactId>retry</artifactId>
<version>0.0.2-SNAPSHOT</version>
</dependency>
new Retry<String>()
.maxOperationWaitTime(30_000)//Max operation wait time during a single operation
.retryIntervalTime(1_000)//Interval time between two operations
.maxRetryTimes(3)//Retry times when operation failed(or timeout) at the first time
.operation(() -> {
//your operation
return "success!";
})
.judgement(t -> (t == null || t.isEmpty()))//add your judgement whether the operation should be retry(Operation should return a value)
.execute();
If you want add retry config annotation on method,and call it:
class RetryTests{
#RetryConfig( maxRetryTimes=1)
public static String TestAnnotation() {
return "aaa";
}
public static void main(String[] args) {
try {
new Retry<String>()
.of(RetryTest.class.getMethod("TestAnnotation"),null)
.judgement(r -> r.equals("aaa"))
.execute();
} catch (NoSuchMethodException | SecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

Apply timeout control around Java operation

I'm using a third party Java library to interact with a REST API. The REST API can sometimes take a long time to respond, eventually resulting in a java.net.ConnectException being thrown.
I'd like to shorten the timeout period but have no means of modifying the third party library.
I'd like to apply some form of timeout control around the calling of a Java method so that I can determine at what point to give up waiting.
This doesn't relate directly to network timeouts. I'd like to be able to try and perform an operation and be able to give up after a specified wait time.
The following is by no means valid Java but does conceptually demonstrate what I'd like to achieve:
try {
Entity entity = new Entity();
entity.methodThatMakesUseOfRestApi();
} catch (<it's been ages now, I don't want to wait any longer>) {
throw TimeoutException();
}

I recommend TimeLimiter from Google Guava library.

This is probably the current way how this should be done with plain Java:
public String getResult(final RESTService restService, String url) throws TimeoutException {
// should be a field, not a local variable
ExecutorService threadPool = Executors.newCachedThreadPool();
// Java 8:
Callable<String> callable = () -> restService.getResult(url);
// Java 7:
// Callable<String> callable = new Callable<String>() {
// #Override
// public String call() throws Exception {
// return restService.getResult(url);
// }
// };
Future<String> future = threadPool.submit(callable);
try {
// throws a TimeoutException after 1000 ms
return future.get(1000, TimeUnit.MILLISECONDS);
} catch (ExecutionException e) {
throw new RuntimeException(e.getCause());
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new TimeoutException();
}
}

There is no general timeout mechanism valid for arbitrary operations.
While... there is one... by using Thread.stop(Throwable). It works and it's thread safe, but your personal safety is in danger when the angry mob confronts you.
// realizable
try
{
setTimeout(1s); // 1
... any code // 2
cancelTimeout(); // 3
}
catch(TimeoutException te)
{
// if (3) isn't executed within 1s after (1)
// we'll get this exception
}

Now we have our nice CompletableFuture , here an application to achieve what was asked.
CompletableFuture.supplyAsync(this::foo).get(15, TimeUnit.SECONDS)

You could use a Timer and a TimerTask.

Here's a utility class I wrote, which should do the trick unless I've missed something. Unfortunately it can only return generic Objects and throw generic Exceptions. Others may have better ideas on how to achieve this.
public abstract class TimeoutOperation {
long timeOut = -1;
String name = "Timeout Operation";
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public long getTimeOut() {
return timeOut;
}
public void setTimeOut(long timeOut) {
this.timeOut = timeOut;
}
public TimeoutOperation (String name, long timeout) {
this.timeOut = timeout;
}
private Throwable throwable;
private Object result;
private long startTime;
public Object run () throws TimeoutException, Exception {
Thread operationThread = new Thread (getName()) {
public void run () {
try {
result = doOperation();
} catch (Exception ex) {
throwable = ex;
} catch (Throwable uncaught) {
throwable = uncaught;
}
synchronized (TimeoutOperation.this) {
TimeoutOperation.this.notifyAll();
}
}
public synchronized void start() {
super.start();
}
};
operationThread.start();
startTime = System.currentTimeMillis();
synchronized (this) {
while (operationThread.isAlive() && (getTimeOut() == -1 || System.currentTimeMillis() < startTime + getTimeOut())) {
try {
wait (1000L);
} catch (InterruptedException ex) {}
}
}
if (throwable != null) {
if (throwable instanceof Exception) {
throw (Exception) throwable;
} else if (throwable instanceof Error) {
throw (Error) throwable;
}
}
if (result != null) {
return result;
}
if (System.currentTimeMillis() > startTime + getTimeOut()) {
throw new TimeoutException("Operation '"+getName()+"' timed out after "+getTimeOut()+" ms");
} else {
throw new Exception ("No result, no exception, and no timeout!");
}
}
public abstract Object doOperation () throws Exception;
public static void main (String [] args) throws Throwable {
Object o = new TimeoutOperation("Test timeout", 4900) {
public Object doOperation() throws Exception {
try {
Thread.sleep (5000L);
} catch (InterruptedException ex) {}
return "OK";
}
}.run();
System.out.println(o);
}
}

static final int NUM_TRIES =4;
int tried =0;
boolean result =false;
while (tried < NUM_TRIES && !result)
{
try {
Entity entity = new Entity();
result = entity.methodThatMakesUseOfRestApi();
}
catch (<it's been ages now, I don't want to wait any longer>) {
if ( tried == NUM_TRIES)
{
throw new TimeoutException();
}
}
tried++;
Thread.sleep(4000);
}

Assign final variable in a try block

Very short question: Is there a more elegant way to do this:
Object tmp;
try {
tmp = somethingThatCanFail();
} catch (Fail f) {
tmp = null;
}
final Object myObject = tmp;
// now I have a final myObject, which can be used in anonymous classes

You could extract the creation of the value in its own method:
final Object myObject = getObjectOrNull();
public Object getObjectOrNull() {
try{
return somethingThatCanFail();
} catch (Fail f) {
return null;
}
}
It's longer, but depending on your definition of "elegant" it might be more elegant.

Depends what you mean by "this" (and "more elegant")
I'm not sure why you think you need tmp AND myObject, but there's no way to avoid having one of those declarations outside the try block IF you want to access it in the catch block.
What's wrong with
Object myObject = null;
try {
myObject = somethingThatCanFail();
} catch (Fail f) {
// do nothing because we can deal with myObject being null just fine
}

These days I tend to do it like this
final Thingy zeFing; {
Thingy t = null;
try {
t = somethingThatCanFail();
} catch (CurveBall f) {
// log...
}
zeFing = t;
}