I am using Executor framework in my java code. I am facing an issue and i need clarification regarding the same.
Below is my java code,
ExecutorService executorObj = Executors.newFixedThreadPool(10);
String name = "default";
Future<String> futRes = executorObj.submit(new Callable<String>() {
#Override
public String call() {
computePropertyPage("");
return "Hello";
}
});
try {
System.out.println("waiting for name for 5 seconds maximum...");
return futRes.get(5,TimeUnit.SECONDS);
} catch (Exception e) {
System.out.println("Exception occurred : " + e);
return name;
}
In the above code, computePropertyPage() is a native method. Its properly linked with the java code. But the call to the function is not getting completed. Its stuck indefinitely. If the call is stuck for more than 5 seconds, i am expecting TimeOutException after 5 seconds. But i am not recieving it.
Instead of native method call, if i just add a sleep of 10 seconds as below,
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I get TimeOutException.
I just want to know if its the limitation from the java side that it dont have control on the native methods and thats the reason its not able to throw TimeOutException for futRes.get(5,TimeUnit.SECONDS);
Your method computePropertyPage completes in less than 5 seconds and return response. Since you aren't calling shutdown on ExecutorService it isn't terminating. Try calling executorObj.shutdown();
Related
I have a message stream, where messages comes which I need to process and then store them in database. In Java, I've written polling code which polls stream and consumes messages every 20 seconds.
This is done inside an infinite for-loop, like below:
for (;;) {
try{
//1. Logic for polling.
//2. Logic for processing the message.
//3. Logic for storing the message in database.
Thread.sleep(20000 - <time taken for above 3 steps >);
} catch(Exception E){
//4. Exception handling.
}
}
This logic runs as expected and the stream is polled, but once in a while it hits an exception or something goes wrong and polling stops.
I want to have a mechanism, that as soon as polling stopped, let's say this for loop is not running for 60 seconds, I should receive a mail or ping.
What is the best way to invoke a method if this for loop is not running for 60 seconds?
I am thinking like, each for-loop execution will ping a heartbeat, and when that heartbeat pinging not received from for-loop then a mail sending is invoked.
There are two different reasons why polling stops making progress, and each needs a different approach:
If the logic throws a Throwable other than an Exception, for instance an Error, the catch does not match, and execution will leave the for-loop, and likely reach the thread's UncaughtExceptionHandler, the default implementation of which logs the exception to System.err and terminates the thread. To prevent this, you should catch Throwable rather than Exception.
The second possibility is that some step in your logic doesn't terminate, for instance due to an infinite loop, a deadlock, waiting for I/O operations, or whatever. In this case, you'll want to take a thread dump to see where the thread is stuck. You can automate this as follows:
class Watchdog {
final Duration gracePeriod;
final Thread watchedThread;
volatile Instant lastProgress;
public Watchdog(Duration gracePeriod) {
this.gracePeriod = gracePeriod;
watchedThread = Thread.currentThread();
everythingIsFine();
var t = new Thread(this::keepWatch);
t.setDaemon(true);
t.start();
}
public void everythingIsFine() {
lastProgress = Instant.now();
}
void keepWatch() {
while (true) {
var silence = Duration.between(lastProgress, Instant.now());
if (silence.compareTo(gracePeriod) > 0) {
System.err.println("Watchdog hasn't seen any progress for " + silence.toSeconds() + " seconds. The watched thread is currently at:");
for (var element : watchedThread.getStackTrace()) {
System.err.println("\tat " + element);
}
}
try {
Thread.sleep(gracePeriod);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
while you can use as follows:
public class Test {
void step() throws Exception {
System.in.read();
}
void job() {
var snoopy = new Watchdog(Duration.ofSeconds(2));
for (;;) {
try {
step();
snoopy.everythingIsFine();
Thread.sleep(1000);
} catch (Throwable t) {
System.err.println(t);
}
}
}
public static void main(String[] args) throws Exception {
new Test().job();
}
}
once the grace period elapses, the WatchDog will print something like:
Watchdog hasn't seen any progress for 2 seconds. The watched thread is currently at:
at java.base/java.io.FileInputStream.readBytes(Native Method)
at java.base/java.io.FileInputStream.read(FileInputStream.java:293)
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:255)
at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:289)
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:276)
at stackoverflow.Test.step(Test.java:48)
at stackoverflow.Test.job(Test.java:55)
at stackoverflow.Test.main(Test.java:65)
So I have a function which looks like this
ExecutorService executorService = Executors.newFixedThreadPool(2000);
Boolean getMore = true;
try{
While (getMore) {
JSONObject response = getPaginatedResponse();
int[] ar = response.get("something");
if (ar.length > 0) {
// loop through the array and invoke executorService.submit() for each
}
else { getMore = false; }
}
executorService.shutdown();
try {
System.out.println("waiting for tasks to complete, termination starting at : "+java.time.LocalDateTime.now());
executorService.awaitTermination(15, TimeUnit.MINUTES);
} catch (InterruptedException e) {
throw new Exception("loading was interrupted... thread pool timed out!");
}
} catch (Exception) {
System.out.println("Fatal error");
}
My issue is that the each of these threads invoke x number of threads, which in turn each call an API and processes its response, the implementation stops execution after all the "First-level" threads gets fired, but not necessarily all the second level ones, which is crucial for my program, how or where can I invoke the executerService.shutdown() to make sure that all the threads were called.
you can put executorService.shutdown(); inside finally block of exception
I have multi-threaded Spring Boot application in which I am reading data from table in batches (the table contains around 1 million records).
I am getting into Java heap memory issues, and I am unable to find a workaround. Below is the code sample.
I call the Spring Boot REST API which then calls this code. Here I am reading from db in the main thread in batches, then passing the batches to thread pool executorService and then finally processing the result in another thread pool resultProcessor.
The Worker class implements Callable<WorkerResult>
ExecutorService executorService = Executors.newFixedThreadPool(15);
Long workerCount = 0L;
ExecutorService resultProcessor = Executors.newFixedThreadPool(10);
List<CompletableFuture<WorkerResult>> futures = new ArrayList<>();
while (workerCount < totalData) {
List<Model> dbRecords = repo.getData(workerCount,workerCount+rp,date);
workerCount += rp + 1;
try {
futures.add(CompletableFuture.supplyAsync(() -> {
try {
return new Worker(dbRecords).call(); // Here for each record third party api is called
} catch (Exception ex) {
throw new CompletionException(ex);
}
// Or return default value
}, executorService).thenApplyAsync(result -> {
service.resultReceived(result); // update the results into db
return result;
}, resultProcessor));
} catch (RejectedExecutionException e) {
logData("Can't submit anymore tasks %s ", e.getMessage());
}
}
}
Outside the while loop once I have read all data from DB, then I call the CompletableFuture.allOf method to finish any remaining tasks.
Below is the code for that:
try {
CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).join();
executorService.shutdown();
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
resultProcessor.shutdown();
resultProcessor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
Here, if I do not add the CompletableFuture.allOf, the result is returned from this method without completing all tasks in the queues.
Instead of calling CompletableFuture.allOf, I have tried futures.foreach(CompletableFuture::join) but my issue didn't resolve that way either.
Currently, I have assigned 1GB ram to the Tomcat server, therefore I face heap space error after some 100 thousand records are processed successfully.
What can I do here to get rid of this error and improve code efficiency as well, also the solution should be in Java 8 and not the latest versions if possible.
I don't know how much data will be in real this is a test environment data.
I need to execute two tasks in parallel and wait for them to complete. Also I need the result from the second task, for that I am using Future.
My question is that DO I need executor.awaitTermination to join the tasks or Future.get() will take care of it. Also is there a better way to achieve this with Java 8?
public class Test {
public static void main(String[] args) {
test();
System.out.println("Exiting Main");
}
public static void test() {
System.out.println("In Test");
ExecutorService executor = Executors.newFixedThreadPool(2);
executor.submit(() -> {
for(int i = 0 ; i< 5 ; i++) {
System.out.print("["+i+"]");
try {
Thread.sleep(1000);
} catch (Exception e) {e.printStackTrace();}
}
});
Future<String> result = executor.submit(() -> {
StringBuilder builder = new StringBuilder();
for(int i = 0 ; i< 10 ; i++) {
System.out.print("("+i+")");
try {
Thread.sleep(1000);
} catch (Exception e) {e.printStackTrace();}
builder.append(i);
}
return builder.toString();
});
System.out.println("shutdown");
executor.shutdown();
// DO I need this code : START
System.out.println("awaitTermination");
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
System.out.println("Error");
}
// DO I need this code : END
System.out.println("Getting result");
try {
System.out.println(result.get());
}
catch (InterruptedException e) {e.printStackTrace();}
catch (ExecutionException e) {e.printStackTrace();}
System.out.println("Exiting Test");
}
}
OUTPUT with awaitTermination:
In Test
[0]shutdown
(0)awaitTermination
[1](1)[2](2)[3](3)[4](4)(5)(6)(7)(8)(9)Getting result
0123456789
Exiting Test
Exiting Main
OUTPUT without awaitTermination:
In Test
[0]shutdown
Getting result
(0)[1](1)[2](2)[3](3)[4](4)(5)(6)(7)(8)(9)0123456789
Exiting Test
Exiting Main
From the get javadoc:
Waits if necessary for the computation to complete, and then retrieves its result.
get will wait for the second task only.
From the awaitTermination javadoc:
Blocks until all tasks have completed execution after a shutdown request, or the timeout occurs, or the current thread is interrupted, whichever happens first.
awaitTermination will wait for all tasks.
You should use CompletableFuture API
You can run a process async like follow:
CompletableFuture.supplyAsync( () -> { ... } );
It returns a future, and you can add a callback which will be called when process is finished and result is available.
For example:
CompletableFuture.runAsync( () -> {
// Here compute your string
return "something";
} ).thenAccept( result -> {
// Here do something with result (ie the computed string)
} );
Note that this statement uses internally the ForkJoinPool#commonPool() to execute the process async, but you can also call this statement with your own ExecutorService if you want. In both case, in order to be sure not exiting before tasks are completed, you need to call either get() (which is blocking) on each future of submitted tasks, or wait for the executor to shutdown.
I have a simple JAVA code it will just print hello after compile and Run the Program. But I want to print one message after successful completion. Is this possible? If yes than how?
Although, the following code snippet is way too overkill for your task but, expanding on my comment - you may want to submit a custom task to a class
which implements Callable.
public class Main {
public static void main(String[] args) {
final ExecutorService executorService;
final Future<Integer> future;
final int statusCode;
executorService = Executors.newFixedThreadPool(1);
future = executorService.submit(new TextMessagePrinter());
try {
statusCode = future.get();
if (statusCode == 10) { // Printed successfully
System.out.println("JOB DONE. EXITING...");
Runtime.getRuntime().exit(0); // A zero status code indicates normal termination.
} else {
System.out.println("ERR...SOMETHING WEIRD HAPPENED!");
Runtime.getRuntime().exit(statusCode); // A non-zero status code indicates abnormal termination.
}
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
} finally {
executorService.shutdownNow();
}
}
}
class TextMessagePrinter implements Callable<Integer> {
public Integer call() {
Integer STATUS_CODE;
try {
System.out.println("Printing hello..."); // Try printing something
System.out.println("Dividing 6 by 0 gives us: " + 6 / 0); // And then you try to do something knowing which will result in an exception
STATUS_CODE = 10; // Indicates success.
} catch (ArithmeticException e) {
STATUS_CODE = 20; // Indicates failure...setting status code to 20.
}
return STATUS_CODE;
}
}
Running the above code on my IDE gives me the following output:
When the exception happens
(Note the status code set in the catch block getting printed when the process finishes):
No exception happens, everything happens fine:
(Comment the following line)
System.out.println("Dividing 6 by 0 gives us: " + 6 / 0);
If you mean completion of the application's Runtime, I think you are looking for the answer in this StackOverflow question: Java Runtime Shutdown Hook.
Or if you want to do what is in the question title and do something after building, then you may consider a build automation tool, like Maven.