Develop Reference

I have tried to optimize (memory) my program, but GC is still making it lag

I have written a piece of software in Java that checks if proxies are working by sending a HTTP request using the proxy.
It takes around 30,000 proxies from a database, then attempts to check if they are operational. The proxies received from the database used to be returned as an ArrayList<String>, but have been changed to Deque<String> for reasons stated below.
The way the program works is there is a ProxyRequest object that stores the IP & Port as a String and int respectively. The ProxyRequest object has a method isWorkingProxy() which attempts to send a request using a proxy and returns a boolean on whether it was successful.
This ProxyRequest object is wrapped around by a RunnableProxyRequest object that calls super.isWorkingProxy() in the overrided run() method. Based on the response from super.isWorkingProxy(), the RunnableProxyRequest object updates a MySQL database.
Do note that the updating of the MySQL database is synchronized().
It runs on 750 threads using a FixedThreadPool (on a VPS), but towards
the end, it becomes very slow (stuck on ~50 threads), which obviously
implies the garbage collector is working. This is the problem.
I have attempted the following to improve the lag, it does not seem to work:
1) Using a Deque<String> proxies and using Deque.pop() to obtain the String in which the proxy is. This (I believe), continuously makes the Deque<String> smaller, which should improve lag caused by the GC.
2) Set the con.setConnectTimeout(this.timeout);, where this.timeout = 5000; This way, the connection should return a result in 5 seconds. If not, the thread is completed and should no longer be active in the threadpool.
Besides this, I don't know any other way I can improve performance.
Can anyone recommend a way for me to improve performance to avoid / stop lagging towards the end of the threads by the GC? I know there is a Stackoverflow question about this (Java threads slow down towards the end of processing), but I have tried everything in the answer and it has not worked for me.
Thank you for your time.
Code snippets:
Loop adding threads to the FixedThreadPool:
//This code is executed recursively (at the end, main(args) is called again)
//Create the threadpool for requests
//Threads is an argument that is set to 750.
ThreadPoolExecutor executor = (ThreadPoolExecutor)Executors.newFixedThreadPool(threads);
Deque<String> proxies = DB.getProxiesToCheck();
while(proxies.isEmpty() == false) {
try {
String[] split = proxies.pop().split(":");
Runnable[] checks = new Runnable[] {
//HTTP check
new RunnableProxyRequest(split[0], split[1], Proxy.Type.HTTP, false),
//SSL check
new RunnableProxyRequest(split[0], split[1], Proxy.Type.HTTP, true),
//SOCKS check
new RunnableProxyRequest(split[0], split[1], Proxy.Type.SOCKS, false)
//Add more checks to this list as time goes...
};
for(Runnable check : checks) {
executor.submit(check);
}
} catch(IndexOutOfBoundsException e) {
continue;
}
}
ProxyRequest class:
//Proxy details
private String proxyIp;
private int proxyPort;
private Proxy.Type testingType;
//Request details
private boolean useSsl;
public ProxyRequest(String proxyIp, String proxyPort, Proxy.Type testingType, boolean useSsl) {
this.proxyIp = proxyIp;
try {
this.proxyPort = Integer.parseInt(proxyPort);
} catch(NumberFormatException e) {
this.proxyPort = -1;
}
this.testingType = testingType;
this.useSsl = useSsl;
}
public boolean isWorkingProxy() {
//Case of an invalid proxy
if(proxyPort == -1) {
return false;
}
HttpURLConnection con = null;
//Perform checks on URL
//IF any exception occurs here, the proxy is obviously bad.
try {
URL url = new URL(this.getTestingUrl());
//Create proxy
Proxy p = new Proxy(this.testingType, new InetSocketAddress(this.proxyIp, this.proxyPort));
//No redirect
HttpURLConnection.setFollowRedirects(false);
//Open connection with proxy
con = (HttpURLConnection)url.openConnection(p);
//Set the request method
con.setRequestMethod("GET");
//Set max timeout for a request.
con.setConnectTimeout(this.timeout);
} catch(MalformedURLException e) {
System.out.println("The testing URL is bad. Please fix this.");
return false;
} catch(Exception e) {
return false;
}
try(
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
) {
String inputLine = null; StringBuilder response = new StringBuilder();
while((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
//A valid proxy!
return con.getResponseCode() > 0;
} catch(Exception e) {
return false;
}
}
RunnableProxyRequest class:
public class RunnableProxyRequest extends ProxyRequest implements Runnable {
public RunnableProxyRequest(String proxyIp, String proxyPort, Proxy.Type testingType, boolean useSsl) {
super(proxyIp, proxyPort, testingType, useSsl);
}
#Override
public void run() {
String test = super.getTest();
if(super.isWorkingProxy()) {
System.out.println("-- Working proxy: " + super.getProxy() + " | Test: " + test);
this.updateDB(true, test);
} else {
System.out.println("-- Not working: " + super.getProxy() + " | Test: " + test);
this.updateDB(false, test);
}
}
private void updateDB(boolean success, String testingType) {
switch(testingType) {
case "SSL":
DB.updateSsl(super.getProxyIp(), super.getProxyPort(), success);
break;
case "HTTP":
DB.updateHttp(super.getProxyIp(), super.getProxyPort(), success);
break;
case "SOCKS":
DB.updateSocks(super.getProxyIp(), super.getProxyPort(), success);
break;
default:
break;
}
}
}
DB class:
//Locker for async
private static Object locker = new Object();
private static void executeUpdateQuery(String query, String proxy, int port, boolean toSet) {
synchronized(locker) {
//Some prepared statements here.
}
}

Thanks to Peter Lawrey for guiding me to the solution! :)
His comment:
#ILoveKali I have found network libraries are not aggressive enough in
shutting down a connection when things go really wrong. Timeouts tend
to work best when the connection is fine. YMMV
So I did some research, and found that I had to also use the method setReadTimeout(this.timeout);. Previously, I was only using setConnectTimeout(this.timeout);!
Thanks to this post (HttpURLConnection timeout defaults) that explained the following:
Unfortunately, in my experience, it appears using these defaults can
lead to an unstable state, depending on what happens with your
connection to the server. If you use an HttpURLConnection and don't
explicitly set (at least read) timeouts, your connection can get into
a permanent stale state. By default. So always set setReadTimeout to
"something" or you might orphan connections (and possibly threads
depending on how your app runs).
So the final answer is: The GC was doing just fine, it was not responsible for the lag. The threads were simply stuck FOREVER at a single number because I did not set the read timeout, and so the isWorkingProxy() method never got a result and kept reading.

Multithreading java optimization

In my program I try tro grasp how to use ExecutorService to optimize my program.
For some reason, It gets stuck a little on two Urls. The http://sjsu.edu/ and https://paypal.com. When it sits on these two, it does not continue executing other URLS.
Should the other 3 threads available not continue even though the two domains aren't responsing fast enough?
How is this fixed in the best possible manner?
public class SequentialPinger {
public static void main(String args[]) throws Exception {
String[] hostList = {"http://crunchify.com", "http://yahoo.com",
"http://www.ebay.com", "http://google.com",
"http://www.example.co", "https://paypal.com",
"http://bing.com/", "http://techcrunch.com/",
"http://mashable.com/", "http://thenextweb.com/",
"http://wordpress.com/", "http://cphbusiness.dk/",
"http://example.com/", "http://sjsu.edu/",
"http://ebay.co.uk/", "http://google.co.uk/",
"http://www.wikipedia.org/",
"http://dr.dk", "http://pol.dk", "https://www.google.dk",
"http://phoronix.com", "http://www.webupd8.org/",
"https://studypoint-plaul.rhcloud.com/", "http://stackoverflow.com",
"http://docs.oracle.com", "https://fronter.com",
"http://imgur.com/", "http://www.imagemagick.org"
};
List<CallableImpl> callList = new ArrayList();
ExecutorService es = Executors.newFixedThreadPool(4);
for (String url : hostList) {
CallableImpl callable = new CallableImpl(url);
callList.add(callable);
}
for (CallableImpl callableImpl : callList) {
System.out.println("Trying to connect to: " + callableImpl.getUrl());
Future<String> lol = es.submit(callableImpl);
System.out.println("status: " + lol.get());
}
es.shutdown();
}
}
My Callable implementation
public class CallableImpl implements Callable<String> {
private final String url;
public CallableImpl(String url) {
this.url = url;
}
public String getUrl() {
return url;
}
#Override
public String call() {
String result = "Error";
try {
URL siteURL = new URL(url);
HttpURLConnection connection = (HttpURLConnection) siteURL
.openConnection();
connection.setRequestMethod("GET");
connection.connect();
int code = connection.getResponseCode();
if (code == 200) {
result = "Green";
}
if (code == 301) {
result = "Redirect";
}
} catch (IOException e) {
result = "->Red<-";
}
return result;
}
}

In your code you submit Callable to ExecutorService one by one and immediately call Future.get() which will block until result is ready (or exception is thrown at runtime).
You'd better wrap ExecutorService with CompletionSerivce which provides results as soon as they are ready. And split for-loop into two loops: one to submit all Callables and second to check results.
ExecutorService es = Executors.newFixedThreadPool(4);
ExecutorCompletionService<String> completionService = new ExecutorCompletionService<>(es);
for (CallableImpl callableImpl : callList) {
System.out.println("Trying to connect to: " + callableImpl.getUrl());
completionService.submit(callableImpl);
}
for (int i = 0; i < callList.size(); ++i) {
completionService.take().get(); //fetch next finished Future and check its result
}

Problem
You call get() on the Future directly after creating it, blocking the main thread. Thus you don't have any parallel calls at all, and making the ExecutorService essentially useless. Your code is equivalent to simply calling callableImpl.call() yourself.
Solution
Don't call get() if you want to continue execution and have each CallableImpl run in parallel. Instead you can call es.awaitTermination() after es.shutdown().

I suggest using a CompletableFuture added in Java 8 and add a callback to it.
CompletableFuture.supplyAsync(myCallable::call, es)
.thenAccept(result -> {
something(result);
});
I would suggest making your Callable be a Supplier to make this simpler.

You wrote: "it does not continue executing other URLS" - I believe it does, but your log messages are misleading because are not tightly connected to the actual execution. To fix this, do the following:
Move System.out.println("Trying to connect to: ") and System.out.println("status: ") into the CallableImpl.call() method.
Do not call to lol.get() at all.
This way you will see actual sequence of the start and the end of handling each URL.

Loop On Exception

I have a custom built API for interacting with their messaging system. But this API doesn't give me any way to confirm that I have established a connection aside from when it is unable to connect an exception will be thrown.
When I receive a exception while connected, I have an exception listener that attempts to reconnect to the server. I'd like this to loop on exception to retry the connection. Doing an infinite loop until I am able to connect, or until the program is closed. I attempted to do this with break labels like so:
reconnect: try{
attemptReconnection();
}catch(Exception e){
log.error(e);
break reconnect;
}
but that was unable to find the reconnect label for me, and is a bit to close to using a GOTO statement than I would be comfortable putting into production.

Proceed this way:
do { // optional loop choice
try{
attemptReconnection();
break; // Connection was successful, break out of the loop
} catch(Exception e){
// Exception thrown, do nothing and move on to the next connection attempt (iteration)
log.error(e);
}
}while(true);
If the execution flow reaches the break; instruction then that means that you successfully connected. Otherwise, it will keep moving on to the next iteration. (Note that the loop choice is optional, you can use pretty much any loop you want)

Can't say I have experience with APIs, but I would think something like this would achieve the result you're after.
boolean success = false;
while (!success){
try{
attemptReconnection();
success = true;
}
catch(Exception e){
log.error(e);
}
}
Once attemptReconnection() executes without errors, success would be set to true and terminate the loop.

Have attemptReconnection return true when connection succeds, false otherwise.
The method attemptReconnection should also catch and log the Exception.
Then :
while(!attemptReconnection()){
log.error("Connection failure");
}

I would suggest controlling the reconnection attempts not with a while loop but with a scheduled event. This you can easily initiate multiple connections and implement a back off mechanism not to over-consume resources while trying to reconnect
private ScheduledExecutorService scheduler;
...
public void connect() {
for (int i = 0; i < numberOfConnections; i++) {
final Runnable r = new Runnable() {
int j = 1;
public void run() {
try {
final Connection connection = createNewConnection();
} catch (IOException e) {
//here we do a back off mechanism every 1,2,4,8,16... 512 seconds
final long sleep = j * 1000L;
if (j < 512) {
j *= 2;
} else {
j = 1;
}
LOGGER.error("Failed connect to host:port: {}:{}. Retrying... in {} millis",
host, port, sleep);
LOGGER.debug("{}", e);
scheduler.schedule(this, sleep, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
Thread.currentThread.interrupt();
}
}
};
scheduler.schedule(r, 1, TimeUnit.SECONDS);
}
}
Do not forget to do a scheduler.shutdownNow() if you want to close the application so as to avoid the treadpool being leaking.
You can even implement a reconnect mechanism once you have been connected and you are disconnected by having the listener execute the connect method in case of a status change on the connection.

Couchbase backoff with Java API

I'm trying to understand how incremental backoff works in the Java Couchbase API. The following code snippet is from the Couchbase Java Tutorial (I have added a few comments).
public OperationFuture<Boolean> contSet(String key,
int exp,
Object value,
int tries) {
OperationFuture<Boolean> result = null;
OperationStatus status;
int backoffexp = 0;
try {
do {
if (backoffexp > tries) {
throw new RuntimeException("Could not perform a set after "
+ tries + " tries.");
}
result = cbc.set(key, exp, value);
status = result.getStatus(); // Is this a blocking call?
if (status.isSuccess()) {
break;
}
if (backoffexp > 0) {
double backoffMillis = Math.pow(2, backoffexp);
backoffMillis = Math.min(1000, backoffMillis); // 1 sec max
Thread.sleep((int) backoffMillis);
System.err.println("Backing off, tries so far: " + backoffexp);
}
backoffexp++;
// Why are we checking again if the operation previously failed
if (!status.isSuccess()) {
System.err.println("Failed with status: " + status.getMessage());
}
// If we break on success, why not do while(true)?
} while (status.getMessage().equals("Temporary failure"));
} catch (InterruptedException ex) {
System.err.println("Interrupted while trying to set. Exception:"
+ ex.getMessage());
}
if (result == null) {
throw new RuntimeException("Could not carry out operation.");
}
return result;
}
Do calls to getStatus() return only when the operation has either succeeded or failed? (i.e. synchronous). The Java Tutorial seem to say it is blocking, but the Java API says:
Get the current status of this operation. Note that the operation status may change as the operation is tried and potentially retried against the servers specified by the NodeLocator.
Why do we need to check status.isSuccess() multiple times? If it was successful we would have broken out of the loop, and we can assume it has failed?
If there any reason to do while (status.getMessage().equals("Temporary failure")) instead of while(true), since we call break when the status is successful?
Thanks

getStatus() forces the Future to complete (i.e. internally calls get(), so the status of the operation can be known. To put it another way, there is no such thing as a pending status in the APU, so you must force the Future to complete to be able to determine is status.
See the source code of Operation_Future:getStatus for details.

Concurrent requests via Apache Http Client (and java.util.concurrent) seem to finish correctly but crash my OS

i am currently learning to use the the concurrent features of Java provided by the package java.util.concurrent. As an exercise i tried to write a little program that could be used to performance test a HTTP API. But somehow my program is not terminating correctly very often. It even crashes my OS.
Following is the pseudo code of my program:
Instantiate Request Objects, that query an HTTP API (In the example i just query one random site).
Instantiate multiple Callables, where each one represents a represents an Http Call.
Iterate over the Callables and schedule them via a ScheduledExecutorService (how many requests should be performed per second can be configured at the begin of the code).
After scheduling all Callables, i am beginning to iterate over the Futures. If a futures is done, retrieve the response. Do this every second. If no new Future was finished, quit the loop.
What problems am i experiencing in detail?
Lots of times, the program is not finishing correctly. I see all log prints in the console, as if the program is finishing correctly. But actually i am seeing that stop button in eclipse still remains active . If i click it, it says that the program could not be terminated correctly. It does not finish no matter how i long i wait (NOTE: I am starting the program inside eclipse).
I can provoke the error easily if i am increasing the number of Requests. If am turning up to 2000, this will happen for sure. If it happens my OS even crashes, i can still use eclipse, but other apps do not work anymore.
My Environment is Eclipse 3.7 on Mac OS X 10.7 with Java 1.6 and Apache httpclient 4.2.2
Do you spot any major erros in my code? Before i have never had such issues in a java program with crashing my OS and seeing no exceptions at all.
The code:
public class ConcurrentHttpRequestsTest {
/**
* #param args
*/
public static void main(String[] args) {
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(25);
Integer standardTimeout = 5000;
Float numberOfRequestsPerSecond = 50.0f;
Integer numberOfRequests = 500;
Integer durationBetweenRequests = Math.round(1000 / numberOfRequestsPerSecond);
// build Http Request
HttpGet request = null;
request = new HttpGet("http://www.spiegel.de");
// request.addHeader("Accept", "application/json");
HttpParams params = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(params, standardTimeout);
HttpConnectionParams.setSoTimeout(params, standardTimeout);
request.setParams(params);
// setup concurrency logic
Collection<Callable<Long>> callables = new LinkedList<Callable<Long>>();
for (int i = 1; i <= numberOfRequests; i++) {
HttpClient client = new DefaultHttpClient();
callables.add(new UriCallable(request, client));
}
// start performing requests
int i = 1;
Collection<Future<Long>> futures = new LinkedList<Future<Long>>();
for (Callable<Long> callable : callables) {
ScheduledFuture<Long> future = scheduledExecutorService.schedule(callable, i * durationBetweenRequests, TimeUnit.MILLISECONDS);
futures.add(future);
i++;
}
// process futures (check wether they are ready yet)
Integer maximumNoChangeCount = 5;
boolean futuresAreReady = false;
int noChangeCount = 0;
int errorCount = 0;
List<Long> responses = new LinkedList<Long>();
while (!futuresAreReady) {
boolean allFuturesAreDone = true;
boolean atLeast1FutureIsDone = false;
Iterator<Future<Long>> iterator = futures.iterator();
while (iterator.hasNext()) {
Future<Long> future = iterator.next();
allFuturesAreDone = allFuturesAreDone && (future.isDone());
if (future.isDone()) {
try {
atLeast1FutureIsDone = true;
responses.add(future.get());
iterator.remove();
} catch (Exception e) {
// remove failed futures (e.g. timeout)
// System.out.println("Reached catch of future.get()" +
// e.getClass() + " " + e.getCause().getClass() + " " +
// e.getMessage());
iterator.remove();
errorCount++;
}
}
if (future.isCancelled()) {
// this code is never reached. Just here to make sure that
// this is not the cause of problems.
System.out.println("Found a cancelled future. Will remove it.");
iterator.remove();
}
}
if (!atLeast1FutureIsDone) {
System.out.println("At least 1 future was not done. Current noChangeCount:" + noChangeCount);
noChangeCount++;
} else {
// reset noChangeCount
noChangeCount = 0;
}
futuresAreReady = allFuturesAreDone;
// log the current state of responses, errors and remaining futures
System.out.println("Size of responses :" + responses.size() + "; Size of futures:" + futures.size() + " Errors:" + errorCount);
if (noChangeCount >= maximumNoChangeCount) {
System.out.println("Breaking while loop becauce no new future finished in the last " + maximumNoChangeCount + " iterations");
break;
}
// check every second
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
for (Long responsetime : responses) {
// analyze responsetimes or whatever
}
// clean up
// .shutdown() made even more problems than shutdownNow()
scheduledExecutorService.shutdownNow();
System.out.println("Executors have been shutdown - Main Method finished. Will exit System.");
System.out.flush();
System.exit(0);
}
private static class UriCallable implements Callable<Long> {
private HttpUriRequest request;
private HttpClient client;
public UriCallable(HttpUriRequest request, HttpClient client) {
super();
this.request = request;
this.client = client;
}
public Long call() throws Exception {
Long start = System.currentTimeMillis();
HttpResponse httpResponse = client.execute(request);
Long end = System.currentTimeMillis();
return end - start;
}
}
}

Never do this in a loop:
} catch (InterruptedException e) {
e.printStackTrace();
}
It might cause problems on shutdown.
Also, most of your code could be replaced by a single call to ExecutorService.invokeAll(), so try that and see if you have more luck.
Lastly, when you don't know what your Java application is doing, run jconsole, attach to the application, and look at the thread stacks to see what code is currently in progress.