I'm trying to understand how incremental backoff works in the Java Couchbase API. The following code snippet is from the Couchbase Java Tutorial (I have added a few comments).
public OperationFuture<Boolean> contSet(String key,
int exp,
Object value,
int tries) {
OperationFuture<Boolean> result = null;
OperationStatus status;
int backoffexp = 0;
try {
do {
if (backoffexp > tries) {
throw new RuntimeException("Could not perform a set after "
+ tries + " tries.");
}
result = cbc.set(key, exp, value);
status = result.getStatus(); // Is this a blocking call?
if (status.isSuccess()) {
break;
}
if (backoffexp > 0) {
double backoffMillis = Math.pow(2, backoffexp);
backoffMillis = Math.min(1000, backoffMillis); // 1 sec max
Thread.sleep((int) backoffMillis);
System.err.println("Backing off, tries so far: " + backoffexp);
}
backoffexp++;
// Why are we checking again if the operation previously failed
if (!status.isSuccess()) {
System.err.println("Failed with status: " + status.getMessage());
}
// If we break on success, why not do while(true)?
} while (status.getMessage().equals("Temporary failure"));
} catch (InterruptedException ex) {
System.err.println("Interrupted while trying to set. Exception:"
+ ex.getMessage());
}
if (result == null) {
throw new RuntimeException("Could not carry out operation.");
}
return result;
}
Do calls to getStatus() return only when the operation has either succeeded or failed? (i.e. synchronous). The Java Tutorial seem to say it is blocking, but the Java API says:
Get the current status of this operation. Note that the operation status may change as the operation is tried and potentially retried against the servers specified by the NodeLocator.
Why do we need to check status.isSuccess() multiple times? If it was successful we would have broken out of the loop, and we can assume it has failed?
If there any reason to do while (status.getMessage().equals("Temporary failure")) instead of while(true), since we call break when the status is successful?
Thanks
getStatus() forces the Future to complete (i.e. internally calls get(), so the status of the operation can be known. To put it another way, there is no such thing as a pending status in the APU, so you must force the Future to complete to be able to determine is status.
See the source code of Operation_Future:getStatus for details.
Related
I have the following code running in my project:
HashMap<String, DeviceData> deviceMap = getAllDevices();
int status = 0;
DeviceHandle devHandle = null;
for (LicenseData licenseData:listLicenses) {
Map<String, String> licenseMap = licenseData.getLicenseKeyValues();
if ((licenseMap != null && !licenseMap.isEmpty())) {
String keyDecrypt = licenseMap.get("key");
Date expiryDate = new Date(Long.parseLong(licenseMap.get("expiryDate")));
boolean allowForeign = Boolean.parseBoolean(licenseMap.get("allowForeign"));
String ipDecrypt = licenseMap.get("ipAddress");
if (expiryDate.compareTo(new Date()) > 0 || keyDecrypt.equals(licenseData.getKey().getCurrentValueAsString()))
{
try {
DeviceData device = deviceMap.get(ipDecrypt);
devHandle = (DeviceHandle)device.getHandle();
if(device != null && devHandle != null) {
deviceMap.remove(ipDecrypt, device);
System.out.println("After deletion device map.");
System.out.println(deviceMap);
createUser(devHandle);
try {
if (allowForeign) {
Process pr = Runtime.getRuntime().exec(SomeOperation);
status = pr.waitFor();
if (status == 0)
//Debug Statement
else
//Error Debug Statemnt
deleteUser(devHandle);
}
else {
Process pr = Runtime.getRuntime().exec(SomeOperation);
status = pr.waitFor();
if (status == 0)
//Debug Statement
else
//Error Debug Statement
deleteUser(devHandle);
}
} catch(Exception e) {
//Exception statement
deleteUser(devHandle);
}
} catch(Exception e) {
e.printStackTrace();
}
}
}
}
Explanation: I have a list of licenses for my application in listLicenses. All the devices present in the server are in deviceMap. For each license, I am decrypting it and getting the values. If license for a device is present, I get a handle on that device and doing some operations.
The issue is:
If I am not able to get a handle on device(getHandle()), or if I am not able to create a user after getting the device handle(createUser()), an exception is thrown. These methods are very hierarchical, i.e I am calling them from here, they are in another class throwing own exceptions and for their operation, they call other methods.
If there are three devices in the map, and three licenses, and if for the first one I am not able to get a handle or create a user, device is removed from deviceMap but no further execution happens i.e. for the next two devices.
If exception occurs for on device, I want to continue the exception for other two devices. I tried using return but couldn't get it to work.
Please help.Also, please forgive for the syntax and if any mismatch is there in the code.
Make use of first try's catch block.
This is how I handled when I faced same kind of situation.
catch (Exception exp) {
if (exp instanceof NullPointerException) {
log.info"Invalid/ Inactive ");
} else if (exp instanceof NonUniqueResultException) {
log.info("Multiple records existed");
} else {
exp.printStackTrace();
errorMsgs.append("Unexpected Error Occured. Please contact Admin.");
}
}
I currently have a loop that goes through a total of 16 website URL's, each time checking for specific text within each home page. There's sometimes a case when a few of the websites takes longer than the specified amount of time to load, thereby failing and stopping execution. What I would like is for the loop to continue until completion, then Fail the entire test if there's at least one failure that occurred during execution and pass the entire test if there were no failures.
Below is the actual code being used to set and check the load time against. Please advise how to modify the below code such that I can get the desired result above?
public static Boolean isTextPresentAfterWaitOnServer(final String strStringToAppear, RemoteWebDriver rwd, String strLoc){
try{
Wait<WebDriver> wait = new FluentWait<WebDriver>(rwd)
.withTimeout(30, TimeUnit.SECONDS)
.pollingEvery(2, TimeUnit.SECONDS)
.ignoring(NoSuchElementException.class);
Boolean foo = wait.until(new ExpectedCondition<Boolean>() {
public Boolean apply(final WebDriver webDriver) {
return (webDriver.getPageSource().indexOf(strStringToAppear) >= 0);
}
});
return foo;
}
catch(TimeoutException e){
throw new RuntimeException("Could not find text " + strStringToAppear +" on server "+strLoc +" - " + e);
}
};
I haven't used FluentWait so not sure if you can ignore the timeout exception as well like you are doing for nosuchelementexception. If it has then i guess you can ignore that as well. OR instead of raising the runtimeexception, create an errorCounter, keep incrementing it in the catch block and in finally block raise an exception based on its value.
I think in your case simple Asser.fail(String message); is very helpful. Please try this code:
public static Boolean isTextPresentAfterWaitOnServer(final String strStringToAppear, RemoteWebDriver rwd, String strLoc){
(...)
}
catch(TimeoutException e){
throw new RuntimeException("Could not find text " + strStringToAppear +" on server "+strLoc +" - " + e);
}
};
and catch this RuntimeExecption in test method:
#Test
public void someTestMethod(){
try{
(...)
isTextPresentAfterWaitOnServer("strStringToAppear", rwd, "strLoc");
}catch(RuntimeException re){
Assert.fail(re.getMessage());
} //you can specify another exceptions
}
I use fail in JUnit and works fine (all test cases running). May be has same behavior in testNG.
i am currently learning to use the the concurrent features of Java provided by the package java.util.concurrent. As an exercise i tried to write a little program that could be used to performance test a HTTP API. But somehow my program is not terminating correctly very often. It even crashes my OS.
Following is the pseudo code of my program:
Instantiate Request Objects, that query an HTTP API (In the example i just query one random site).
Instantiate multiple Callables, where each one represents a represents an Http Call.
Iterate over the Callables and schedule them via a ScheduledExecutorService (how many requests should be performed per second can be configured at the begin of the code).
After scheduling all Callables, i am beginning to iterate over the Futures. If a futures is done, retrieve the response. Do this every second. If no new Future was finished, quit the loop.
What problems am i experiencing in detail?
Lots of times, the program is not finishing correctly. I see all log prints in the console, as if the program is finishing correctly. But actually i am seeing that stop button in eclipse still remains active . If i click it, it says that the program could not be terminated correctly. It does not finish no matter how i long i wait (NOTE: I am starting the program inside eclipse).
I can provoke the error easily if i am increasing the number of Requests. If am turning up to 2000, this will happen for sure. If it happens my OS even crashes, i can still use eclipse, but other apps do not work anymore.
My Environment is Eclipse 3.7 on Mac OS X 10.7 with Java 1.6 and Apache httpclient 4.2.2
Do you spot any major erros in my code? Before i have never had such issues in a java program with crashing my OS and seeing no exceptions at all.
The code:
public class ConcurrentHttpRequestsTest {
/**
* #param args
*/
public static void main(String[] args) {
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(25);
Integer standardTimeout = 5000;
Float numberOfRequestsPerSecond = 50.0f;
Integer numberOfRequests = 500;
Integer durationBetweenRequests = Math.round(1000 / numberOfRequestsPerSecond);
// build Http Request
HttpGet request = null;
request = new HttpGet("http://www.spiegel.de");
// request.addHeader("Accept", "application/json");
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, standardTimeout);
HttpConnectionParams.setSoTimeout(params, standardTimeout);
request.setParams(params);
// setup concurrency logic
Collection<Callable<Long>> callables = new LinkedList<Callable<Long>>();
for (int i = 1; i <= numberOfRequests; i++) {
HttpClient client = new DefaultHttpClient();
callables.add(new UriCallable(request, client));
}
// start performing requests
int i = 1;
Collection<Future<Long>> futures = new LinkedList<Future<Long>>();
for (Callable<Long> callable : callables) {
ScheduledFuture<Long> future = scheduledExecutorService.schedule(callable, i * durationBetweenRequests, TimeUnit.MILLISECONDS);
futures.add(future);
i++;
}
// process futures (check wether they are ready yet)
Integer maximumNoChangeCount = 5;
boolean futuresAreReady = false;
int noChangeCount = 0;
int errorCount = 0;
List<Long> responses = new LinkedList<Long>();
while (!futuresAreReady) {
boolean allFuturesAreDone = true;
boolean atLeast1FutureIsDone = false;
Iterator<Future<Long>> iterator = futures.iterator();
while (iterator.hasNext()) {
Future<Long> future = iterator.next();
allFuturesAreDone = allFuturesAreDone && (future.isDone());
if (future.isDone()) {
try {
atLeast1FutureIsDone = true;
responses.add(future.get());
iterator.remove();
} catch (Exception e) {
// remove failed futures (e.g. timeout)
// System.out.println("Reached catch of future.get()" +
// e.getClass() + " " + e.getCause().getClass() + " " +
// e.getMessage());
iterator.remove();
errorCount++;
}
}
if (future.isCancelled()) {
// this code is never reached. Just here to make sure that
// this is not the cause of problems.
System.out.println("Found a cancelled future. Will remove it.");
iterator.remove();
}
}
if (!atLeast1FutureIsDone) {
System.out.println("At least 1 future was not done. Current noChangeCount:" + noChangeCount);
noChangeCount++;
} else {
// reset noChangeCount
noChangeCount = 0;
}
futuresAreReady = allFuturesAreDone;
// log the current state of responses, errors and remaining futures
System.out.println("Size of responses :" + responses.size() + "; Size of futures:" + futures.size() + " Errors:" + errorCount);
if (noChangeCount >= maximumNoChangeCount) {
System.out.println("Breaking while loop becauce no new future finished in the last " + maximumNoChangeCount + " iterations");
break;
}
// check every second
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
for (Long responsetime : responses) {
// analyze responsetimes or whatever
}
// clean up
// .shutdown() made even more problems than shutdownNow()
scheduledExecutorService.shutdownNow();
System.out.println("Executors have been shutdown - Main Method finished. Will exit System.");
System.out.flush();
System.exit(0);
}
private static class UriCallable implements Callable<Long> {
private HttpUriRequest request;
private HttpClient client;
public UriCallable(HttpUriRequest request, HttpClient client) {
super();
this.request = request;
this.client = client;
}
public Long call() throws Exception {
Long start = System.currentTimeMillis();
HttpResponse httpResponse = client.execute(request);
Long end = System.currentTimeMillis();
return end - start;
}
}
}
Never do this in a loop:
} catch (InterruptedException e) {
e.printStackTrace();
}
It might cause problems on shutdown.
Also, most of your code could be replaced by a single call to ExecutorService.invokeAll(), so try that and see if you have more luck.
Lastly, when you don't know what your Java application is doing, run jconsole, attach to the application, and look at the thread stacks to see what code is currently in progress.
Please find below my requirement.
Requirement : Poll a web service. Two crucial parameters of polling max_timeout, polling_interval are configured in properties file. Overall objective is to spend for a time of overall in obtaining the response. If we get the response with in max_timeout, we can return the response to client. Otherwise we will throw an error saying that the operation is unsuccessful.
Below is the code snippet I have written.
int maxTimeOut = 10;
int interval = 2;
int iterations = maxTimeOut/interval;
boolean success = false;
for (int i = 0; i < iterations; i++)
{
System.out.println("Number of iteration = " + i);
try
{
Thread.sleep(interval * 1000);
System.out.println("Waited for " + interval + " seconds");
success = getWSResponse(i);
System.out.println("CALL" + ((success) ? "SUCCESSFUL" : "FAIL"));
if(success) break;
}catch (InterruptedException ie)
{
System.out.println(ie.getMessage());
}
}
//Send the success flag to client
Could you correct me if this is the proper implementation of polling. I am little bit concerned that this code assumes that webservice call returns in no time. If this takes 2-3 seconds (usually it does ), then we would be spending more than max_timeout overall for POLLING alone. How could we fix this. Is there any better approach than this.
you can combine the use of a ScheduledExecutorService with the HttpURLConnection-Timeout to poll in a given delay - and abort the task if it takes any longer.
If polling just means that the webservice is up and running, in your poll code you can try to open a connection to the webservice (with connection timeout) .If you are successfully able to connect, this means the webservice is up.
HttpURLConnection connection = null;
URL url = new URL("URL");
connection = (HttpURLConnection) url.openConnection();
connection .setConnectTimeout(timeout);//specify the timeout and catch the IOexception
connection.connect();
EDIT
Alternatively, you can call the webservice using an executor (see java.util.concurrent.ExecutorService) in a task with timeout and can decide accordingly. Sample :
// Make the ws work a time-boxed task
final Future<Boolean> future= executor.submit(new Callable<Boolean>() {
#Override
public Boolean call() throws Exception {
// get ws result
return getWSResponse();
}
});
try {
boolean result = future.get(max_wait_time, TimeUnit.SECONDS);
} catch (TimeoutException te) {
throw e;
}
Through my java code i m connecting to multiple databases using connection pooling.if my database goes down i need handle the retry logic to get connection until its return a connection object.
If your db connection throws some sort of an Exception then you can just sleep for a bit and retry the operation again.
In the example below worker is an object that does some work such as connecting to a db, etc. It's pretty generic so you can retry any sort of an operation such as reading from a file, etc.
Note that catching Throwable might not necessarily be a great idea.
boolean success = false;
int i = 0;
long delay = retryDelay;
LOGGER.info("Starting operation");
/*
* Loop until you cannot retry anymore or the operation completed successfully
* The catch block has a nested try catch to ensure that nothing goes wrong
* while trying to sleep
*
* In case of failure the last retry exception is propagated up to the calling
* class.
*/
while (i++ < retryMax && !success)
{
try
{
worker.work();
success = true;
}
catch (Throwable t)
{
try
{
LOGGER.warn("Caught throwable", t);
if (i == retryMax)
{
LOGGER.warn("Retry maximum reached, propagating error");
throw t;
}
if (retryPolicy == RetryPolicy.ESCALATING)
{
delay *= 2;
}
LOGGER.info("Sleeping for " + delay + " milliseconds");
Thread.sleep(delay);
}
catch (Throwable tt)
{
/*
* Quick check to see if the maximum has been hit, so we don't log twice
*
* t is the original error, and tt is the error we got while retrying
* tt would most likely be a InterruptedException or something
*/
if (i == retryMax)
{
throw t;
}
LOGGER.warn("Error while retrying, propagating original error up", tt);
throw t;
}
}
} // end retry loop