Develop Reference

How to check when polling stopped

I have a message stream, where messages comes which I need to process and then store them in database. In Java, I've written polling code which polls stream and consumes messages every 20 seconds.
This is done inside an infinite for-loop, like below:
for (;;) {
try{
//1. Logic for polling.
//2. Logic for processing the message.
//3. Logic for storing the message in database.
Thread.sleep(20000 - <time taken for above 3 steps >);
} catch(Exception E){
//4. Exception handling.
}
}
This logic runs as expected and the stream is polled, but once in a while it hits an exception or something goes wrong and polling stops.
I want to have a mechanism, that as soon as polling stopped, let's say this for loop is not running for 60 seconds, I should receive a mail or ping.
What is the best way to invoke a method if this for loop is not running for 60 seconds?
I am thinking like, each for-loop execution will ping a heartbeat, and when that heartbeat pinging not received from for-loop then a mail sending is invoked.

There are two different reasons why polling stops making progress, and each needs a different approach:
If the logic throws a Throwable other than an Exception, for instance an Error, the catch does not match, and execution will leave the for-loop, and likely reach the thread's UncaughtExceptionHandler, the default implementation of which logs the exception to System.err and terminates the thread. To prevent this, you should catch Throwable rather than Exception.
The second possibility is that some step in your logic doesn't terminate, for instance due to an infinite loop, a deadlock, waiting for I/O operations, or whatever. In this case, you'll want to take a thread dump to see where the thread is stuck. You can automate this as follows:
class Watchdog {
final Duration gracePeriod;
final Thread watchedThread;
volatile Instant lastProgress;
public Watchdog(Duration gracePeriod) {
this.gracePeriod = gracePeriod;
watchedThread = Thread.currentThread();
everythingIsFine();
var t = new Thread(this::keepWatch);
t.setDaemon(true);
t.start();
}
public void everythingIsFine() {
lastProgress = Instant.now();
}
void keepWatch() {
while (true) {
var silence = Duration.between(lastProgress, Instant.now());
if (silence.compareTo(gracePeriod) > 0) {
System.err.println("Watchdog hasn't seen any progress for " + silence.toSeconds() + " seconds. The watched thread is currently at:");
for (var element : watchedThread.getStackTrace()) {
System.err.println("\tat " + element);
}
}
try {
Thread.sleep(gracePeriod);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
while you can use as follows:
public class Test {
void step() throws Exception {
System.in.read();
}
void job() {
var snoopy = new Watchdog(Duration.ofSeconds(2));
for (;;) {
try {
step();
snoopy.everythingIsFine();
Thread.sleep(1000);
} catch (Throwable t) {
System.err.println(t);
}
}
}
public static void main(String[] args) throws Exception {
new Test().job();
}
}
once the grace period elapses, the WatchDog will print something like:
Watchdog hasn't seen any progress for 2 seconds. The watched thread is currently at:
at java.base/java.io.FileInputStream.readBytes(Native Method)
at java.base/java.io.FileInputStream.read(FileInputStream.java:293)
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:255)
at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:289)
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:276)
at stackoverflow.Test.step(Test.java:48)
at stackoverflow.Test.job(Test.java:55)
at stackoverflow.Test.main(Test.java:65)

Flux of strings emitted time to time

My problem: I want to create a stream of string that will be be sent from controller from time to time.
Processing started!
Step 1 completed. (This might be sent after 5 seconds or 10 minutes.)
Process completed. (This might be sent after 15 minutes.)
Here is code snippet in controller:
#GetMapping(value = "/stream1", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> streamData() {
return Flux.create(emitter -> {
emitter.next("Processing started!");
try {
TimeUnit.SECONDS.sleep(5);
emitter.next("Step 1 completed.");
TimeUnit.SECONDS.sleep(5);
emitter.next("Process completed.");
emitter.complete();
} catch (InterruptedException e) {
e.printStackTrace();
}
}, FluxSink.OverflowStrategy.LATEST);
//create.publish().connect();
//return create;
}
But it emmits data only when it is completed all processing. Means it emmits data after 10 seconds and all stream at once.
How to achieve some stream where it start sending data as soon as single data is ready?

you are using less ideal method for your task. You can use 'Flux.generate(...)'. It is in contrast to 'Flux.create(...)' used to generate single item, and it is used when subscriber requests something. So no problem with backpressure.
Sample:
#GetMapping(value = "/feapi/automation/approach1", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> streamData() {
final AtomicInteger counter = new AtomicInteger();
return Flux.generate(generator -> {
try {
TimeUnit.SECONDS.sleep(new Random().nextInt(1, 10));
generator.next("Next step (" + counter.incrementAndGet() + ") done. Going further.");
// NOTE only SINGLE item can be emmited in one generator call. You can also call complete or error.
} catch (InterruptedException e) {
e.printStackTrace();
}
});
}

Future.get(5,TimeUnit.SECONDS) doesnt timeout after 5 seconds if native methods are used in Java

I am using Executor framework in my java code. I am facing an issue and i need clarification regarding the same.
Below is my java code,
ExecutorService executorObj = Executors.newFixedThreadPool(10);
String name = "default";
Future<String> futRes = executorObj.submit(new Callable<String>() {
#Override
public String call() {
computePropertyPage("");
return "Hello";
}
});
try {
System.out.println("waiting for name for 5 seconds maximum...");
return futRes.get(5,TimeUnit.SECONDS);
} catch (Exception e) {
System.out.println("Exception occurred : " + e);
return name;
}
In the above code, computePropertyPage() is a native method. Its properly linked with the java code. But the call to the function is not getting completed. Its stuck indefinitely. If the call is stuck for more than 5 seconds, i am expecting TimeOutException after 5 seconds. But i am not recieving it.
Instead of native method call, if i just add a sleep of 10 seconds as below,
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
I get TimeOutException.
I just want to know if its the limitation from the java side that it dont have control on the native methods and thats the reason its not able to throw TimeOutException for futRes.get(5,TimeUnit.SECONDS);

Your method computePropertyPage completes in less than 5 seconds and return response. Since you aren't calling shutdown on ExecutorService it isn't terminating. Try calling executorObj.shutdown();

java thread.sleep() take longer time after display sleep on MacOS

The target is to make current thread sleep some time, range from a few hundred milliseconds to a few seconds. The program runs normally when display is on. But after display go to sleep mode ( On MacOS, display could go to sleep after some system idle time), the Thread.sleep() would take much longer time to return. I.g.,
Thread.sleep(5000); // to sleep 5 sec, but actually it take 30+ seconds to return.
It seems like the cpu frequency has been downscale, but java jre code did not follow the changes. Tested on MacOS 10.12 + Java 8.
Question: How to make the thread sleep code to overcome this issue?
Sample Code: both cannot work on display sleep case.
public static void threadsleep(long milliseconds) {
try {
Thread.sleep(milliseconds);
} catch (InterruptedException ex) {
Logger.getLogger(FetchDailyOptions.class.getName()).log(Level.SEVERE, null, ex);
}
}
public static void threadsleep(long milliseconds) {
try {
if (milliseconds < 100) {
Thread.sleep(milliseconds);
return;
}
long targetMilli = System.currentTimeMillis() + milliseconds;
for (;;) {
Thread.sleep(100); // wait 100 milliseconds
if (System.currentTimeMillis() > targetMilli) {
return;
}
}
}
catch (InterruptedException ex) {
Logger.getLogger(Utils.class.getName()).log(Level.SEVERE, null, ex);
}
}

How i can measure time of thread pool in java?

I have used two different approach to measure time of threads but result are not matching
**Public void Main()**
{
Timer timer = new Timer();
int timetotal;
timer.start();
int numberOfThreads=5;
ExecutorService pool= Executors.newFixedThreadPool(numberOfThreads);
List<Future<Boolean>> futureList = new ArrayList<Future<Boolean>>();
Set<ReadProcess_MongoDB> callList = new HashSet<ReadProcess_MongoDB>();
CompletionService<ReadProcess_MongoDB> taskCompletionService;
taskCompletionService = new ExecutorCompletionService<ReadProcess_MongoDB>(pool);
Collection<Callable<ReadProcess_MongoDB>> list;
list = new LinkedList<Callable<ReadProcess_MongoDB>>();
for(int i=0;i<numberOfThreads;i++)
list.add((Callable<ReadProcess_MongoDB>) new ReadProcess_MongoDB(i));
try {
for (Callable<ReadProcess_MongoDB> callable : list) {
taskCompletionService.submit(callable);
}
for (int i = 0; i < list.size(); i++) {
Future<ReadProcess_MongoDB> result = taskCompletionService.take();
}
} catch (InterruptedException e) {
// no real error handling. Don't do this in production!
e.printStackTrace();
} catch (ExecutionException e) {
// no real error handling. Don't do this in production!
e.printStackTrace();
}
finally {
pool.shutdown();
System.out.println("Done :)");
timer.stop();
System.out.println("Total consumed Time"+ timer.elapsed());
}
Other time I put in Call method()
**public String call()**
{
Timer timer = new Timer();
int timetotal;
timer.start();
DBCursor cursor = coll.find(whereQuery);
try {
Thread.sleep(100);
} catch (InterruptedException ex) {
Logger.getLogger(ReadProcess_MongoDB.class.getName()).log(Level.SEVERE, null, ex);
}
timer.stop();
usedTimeForQueryProcess = timer.elapsed();
System.out.println("Thread Number="+this.threadNumber+ " MongoDB_readQuery used time "+usedTimeForQueryProcess);
System.out.println("runing.....");
return Objects.toString(usedTimeForQueryProcess);
}
In call Function, system print time of every thread and in Main function only display totaltime.Here I try check manually ,but both time are not matching .But bigger problem is Main function show less time than total time of all threads(Call function).
I have also tried to return used time from Call function But it is also create problem convert to long (especially runtime problem ).
Time of Both function
Main function time =289
Call function time=510(5 thread).
Would please somebody expain why this happening and how i can make right measurement ?

The Main time is lower than the total of each of your jobs because they are running in parallel. If you reduce your thread pool size down to 1 then you will the numbers be more like what you expect.
This is one of the benefits of doing multithreaded programming, getting more work done in less time than if it were done sequentially.