Develop Reference

How is Apache NIO HttpAsyncClient performing non-blocking HTTP Client

How is Apache NIO HttpAsyncClient able to wait for a remote response without blocking any thread? Does it have a way to setup a callback with the OS (I doubt so?). Otherwise does it perform some sort of polling?

EDIT - THIS ANSWER IS WRONG. PLEASE IGNORE AS IT IS INCORRECT.
You did not specify a version, so I can not point you to source code. But to answer your question, the way that Apache does it is by returning a Future<T>.
Take a look at this link -- https://hc.apache.org/httpcomponents-asyncclient-4.1.x/current/httpasyncclient/apidocs/org/apache/http/nio/client/HttpAsyncClient.html
Notice how the link says nio in the package. That stands for "non-blocking IO". And 9 times out of 10, that is done by doing some work with a new thread.
This operates almost exactly like a CompletableFuture<T> from your first question. Long story short, the library kicks off the process in a new thread (just like CompletableFuture<T>), stores that thread into the Future<T>, then allows you to use that Future<T> to manage that newly created thread containing your non-blocking task. By doing this, you get to decide exactly when and where the code blocks, potentially giving you the chance to make some significant performance optimizations.
To be more explicit, let's give a pseudocode example. Let's say I have a method attached to an endpoint. Whenever the endpoint is hit, the method is executed. The method takes in a single parameter --- userID. I then use that userID to perform 2 operations --- fetch the user's personal info, and fetch the user's suggested content. I need both pieces, and neither request needs to wait for the other to finish before starting. So, what I do is something like the following.
public StoreFrontPage visitStorePage(int userID)
{
final Future<UserInfo> userInfoFuture = this.fetchUserInfo(userID);
final Future<PageSuggestion> recommendedContentFuture = this.fetchRecommendedContent(userId);
final UserInfo userInfo = userInfoFuture.get();
final PageSuggestion recommendedContent = recommendedContentFuture.get();
return new StoreFrontPage(userInfo, recommendedContent);
}
When I call this.fetchUserInfo(userID), my code creates a new thread, starts fetching user info on that new thread, but let's my main thread continue and kick off this.fetchRecommendedContent(userID) in the meantime. The 2 fetches are occurring in parallel.
However, I need both results in order to create my StoreFrontPage. So, when I decided that I cannot continue any further until I have the results from both fetches, I call Future::get on each of my fetches. What this method does is merge the new thread back into my original one. In short, it says "wait for that one thread you created to finish doing what it was doing, then output the result as a return value".
And to more explicitly answer your question, no, this tool does not require you to do anything involving callbacks or polling. All it does is give you a Future<T> and lets you decide when you need to block the thread to wait on that Future<T> to finish.
EDIT - THIS ANSWER IS WRONG. PLEASE IGNORE AS IT IS INCORRECT.

Polling for a job status java

I'm looking for a way to check a job status over a period of time:
If during the polling I get a result that the job was completed, I return it, otherwise I keep polling until the period I set is over and return a failure result.
I know how to do this using a timer and a while loop.
Is there a better way of doing it?
Thanks in advance

Better way to pool for the result should be using CompletionService.
Since you are already using asynchronous way of communication, make your thread implement Callable and implement call method similar to what you would do in a run method.
Now when you use completion service and to your executor service just say submit. With completion Service you get a blocking queue on which as and when you get the result completion service will put the result onto Queue and you can then do whatever you want with the result.

Return a method when time limit is up in - java

i just started to learn programming (2 weeks ago), and i am trying to make a bot for a game. In the main class of the bot, there are 3 methods that needs to be returned within 2second, or it will return null. I want to avoid returning null and return what it has calculate during 2sec instead.
public ArrayList<PlaceArmiesMove> getPlaceArmiesMoves(BotState state, Long timeOut){
ArrayList<PlaceArmiesMove> placeArmiesMoves = new ArrayList<PlaceArmiesMove>();
// caculations filling the ArrayList
return placeArmiesMoves;
}
what i want to do is after 2 second, returning placeArmiesMoves, wether the method finished running or not. I have read about guava SimpleTimeLimiter and callWithTimeout() but i am totally lost about how to use it (i read something about multithreading but i just don't understand what this is)
i would be incredibly grateful if someone could help me! thanks

Given a function like getPlaceArmiesMove, there are several techniques you might use to bound its execution time.
Trust the function to keep track of time itself
If the function runs a loop, it can check on every iteration whether the time has expired.
long startTime = System.currentTimeMillis()
for (;;) {
// do some work
long elapsed = System.currentTimeMillis() - startTime;
if (elapsed >= timeOut) {
break;
}
}
This technique is simple, but there is no guarantee it will complete before the timeout; it depends on the function and how granular you can make the work (of course, if it's too granular, you'll be spending more time testing if the timeout has expired than actually doing work).
Run the function in a thread, and ask it to stop
I'm not familiar with Guava, but this seems to be what SimpleTimeLimiter is doing. In Java, it isn't generally possible to forcibly stop a thread, though it is possible to ignore the thread after a timeout (the function will run to completion, but you've already used its partial result, and ignore the complete result that comes in too late). Guava says that it interrupts the thread if it has not returned before the timeout. This works only if your function is testing to see if it has been interrupted, much like the "trust your function" technique.
See this answer for an example on how to test if your thread has been interrupted. Note that some Java methods (like Thread.sleep) may throw InterruptedException if the thread is interrupted.
In the end, sprinkling checks for isInterrupted() all over your function won't be much different than sprinkling manual checks for the timeout. So running in a thread, you still must trust your function, but there may be nicer helpers available for that sort of thing (e.g. Guava).
Run the function in a separate process, and kill it
An example of how to do this is left as an exercise, but if you run your function in a separate process (or a thread in languages that support forcibly stopping threads, e.g. Erlang, Ruby, others), then you can use the operating system facilities to kill the process if it does not complete after a timeout.
Having that process return a partial result will be challenging. It could periodically send "work-in-progress" to the calling process over a pipe, or periodically save work to a file.

Use Java's Timer package , however this will require you to understand concepts such as threads and method overriding. Nevertheless, if this is what you require, the answer is quite similar to this question How to set a timer in java

constantly check database [duplicate]

I'm using JDBC, need to constantly check the database against changing values.
What I have currently is an infinite loop running, inner loop iterating over a changing values, and each iteration checking against the database.
public void runInBG() { //this method called from another thread
while(true) {
while(els.hasElements()) {
Test el = (Test)els.next();
String sql = "SELECT * FROM Test WHERE id = '" + el.getId() + "'";
Record r = db.getTestRecord(sql);//this function makes connection, executeQuery etc...and return Record object with values
if(r != null) {
//do something
}
}
}
}
I'm think this isn't the best way.
The other way I'm thinking is the reverse, to keep iterating over the database.
UPDATE
Thank you for the feedback regarding timers, but I don't think it will solve my problem.
Once a change occurs in the database I need to process the results almost instantaneously against the changing values ("els" from the example code).
Even if the database does not change it still has to check constantly against the changing values.
UPDATE 2
OK, to anyone interested in the answer I believe I have the solution now. Basically the solution is NOT to use the database for this. Load in, update, add, etc... only whats needed from the database to memory.
That way you don't have to open and close the database constantly, you only deal with the database when you make a change to it, and reflect those changes back into memory and only deal with whatever is in memory at the time.
Sure this is more memory intensive but performance is absolute key here.
As to the periodic "timer" answers, I'm sorry but this is not right at all. Nobody has responded with a reason how the use of timers would solve this particular situation.
But thank you again for the feedback, it was still helpful nevertheless.

Another possibility would be using ScheduledThreadPoolExecutor.
You could implement a Runnable containing your logic and register it to the ScheduledExecutorService as follows:
ScheduledThreadPoolExecutor executor = new ScheduledThreadPoolExecutor(10);
executor.scheduleAtFixedRate(myRunnable, 0, 5, TimeUnit.SECONDS);
The code above, creates a ScheduledThreadPoolExecutor with 10 Threads in its pool, and would have a Runnable registered to it that will run in a 5 seconds period starting immediately.
To schedule your runnable you could use:
scheduleAtFixedRate
Creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given period; that is executions will commence after initialDelay then initialDelay+period, then initialDelay + 2 * period, and so on.
scheduleWithFixedDelay
Creates and executes a periodic action that becomes enabled first after the given initial delay, and subsequently with the given delay between the termination of one execution and the commencement of the next.
And here you can see the advantages of ThreadPoolExecutor, in order to see if it fits to your requirements. I advise this question: Java Timer vs ExecutorService? too in order to make a good decision.

Keeping the while(true) in the runInBG() is a bad idea. You better remove that. Instead you can have a Scheduler/Timer(use Timer & TimerTask) which would call the runInBG() periodically and check for the updates in the DB.

u could use a timer--->
Timer timer = new Timer("runInBG");
//Taking an instance of class contains your repeated method.
MyClass t = new MyClass();
timer.schedule(t, 0, 2000);

As you said in the comment above, if application controls the updates and inserts then you can create a framework which notifies for 'BG' thread or process about change in database. Notification can be over network via JMS or intra VM using observer pattern or both local and remote notifications.
You can have generic notification message like (it can be class for local notification or text message for remote notifications)
<Notification>
<Type>update/insert</Type>
<Entity>
<Name>Account/Customer</Name>
<Id>id</Id>
<Entity>
</Notification>

To avoid a 'busy loop', I would try to use triggers. H2 also supports a DatabaseEventListener API, that way you wouldn't have to create a trigger for each table.
This may not always work, for example if you use a remote connection.

UPDATE 2
OK, to anyone interested in the answer I believe I have the solution now. Basically the solution is NOT to use the database for this. Load in, update, add, etc... only whats needed from the database to memory. That way you don't have to open and close the database constantly, you only deal with the database when you make a change to it, and reflect those changes back into memory and only deal with whatever is in memory at the time. Sure this is more memory intensive but performance is absolute key here.

Reporting exceptions in a ThreadPool and shutdown

So I have this Java piece of code where I need to do some work on a bunch of items. I decided to parallelize this to get some extra boost and I though to use a ThreadPoolExecutor. The problem is that the work I need to do can throw an exception...
Now I would like to shutdown the entire job to stop as soon as an error is encountered and report it back so I can handle it. Looking online, I found that the normal way this should be done is via ExecutorCompletionService and analyzing the Future results. However, this won't let me shut everything down when the first error comes by, as there is no way to loop over based on which task finishes first...
So I did something that I thought was rather hacky and I'm curious if there is a better way to handle this. What I did was:
1) Have each Runnable that I will execute have a field for the Throwable that it might execute.
2) Override the TPE's "afterExecute" method and check if any checked exception got thrown (which gets recorded in the Runnable) or any unchecked one gets thrown (which should be reported in the second parameter of this method). If any did, then I issue a shutdownNow() on the TPE.
Again, this seems a bit hacky and I am wondering if there is something I am missing. Thanks in advance!

Look at ExecutorService.invokeAny:
Executes the given tasks, returning the result of one that has completed successfully (i.e., without throwing an exception), if any do. Upon normal or exceptional return, tasks that have not completed are cancelled. The results of this method are undefined if the given collection is modified while this operation is in progress.
It looks it does the exact same thing you are trying to do... if I understood your problem correctly.
However for cancel to do anything, you have to make your Callable tasks interrupt aware. But that applies no matter how you try to cancel your tasks.
EDIT:
This is not what you need; I misread the javadoc. Here's another solution: you could put all your Future's in a list and then have a semi-busy while loop where you check periodically on each futureList.get(i).get(100, TimeUnits.MILLISECONDS) and you can catch an exception and act accordingly. However, this no more "elegant" than your solution. It seems that afterExecute was made to do what you want anyway.