I am having List of String to post through Rest api, not sure whether below code is correct performance-wise ??
ExecutorService exService=Executors.newFixedThreadPool(6);
reqList.parallelStream().forEach(reqId->exService.execute(new ReqRunnable(reqId)));
exService.shutdown();
try {
exService.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);
}catch(InterruptedException ee) {
LOGGER.warn(String.format("Thread %s has been interuptted %s ",Thread.currentThread().getName(),ee));
// Restore interrupted state...
Thread.currentThread().interrupt();
}
Rest code
public class ReqRunnable implements Runnable {
private static final Logger LOGGER=LoggerFactory.getLogger(ReqRunnable.class);
String reqId;
ReqRunnable(String reqId)
{
this.reqId=reqId;
}
#Override
public void run() {
LOGGER.info(String.format("Thread -%s and reqId - %s",Thread.currentThread().getName(),reqId));
IRSUpdateService.executeRequest(reqId);
}
}
Related
I need to collect data from a public API. I want to collect it daily or twice a day.
public class AlphavantageStockRequestDispatcher {
public static void startAlphavantageStockScraper(int timeInterval) {
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
Runnable getStockList =
new Runnable() {
#Override
public void run() {
List<AlphavantageStock> stocks = AlphavantageStockRequest.getStockPrices(); //Method contains requests
StockDao<AlphavantageStock> dao = new JpaAlphavantageStockDao();
for (AlphavantageStock stock : stocks) {
dao.save(stock);
}
}
};
scheduler.scheduleAtFixedRate(getStockList, 0, timeInterval, TimeUnit.HOURS);
}
}
The problem is when I start it from the same class (just added main method and invoked startAlphavantageStockScraper(1); it works fine. But when I want to test it via JUnit it's not working (test class is in symmetric package name but test subfolder):
public class AlphavantageStockRequestDispatcherTest {
#Test
public void startDispatcher_TwoFullCycles_WithOneHourIntervalBetween() {
AlphavantageStockRequestDispatcher.startAlphavantageStockScraper(1);
}
}
While debugging I found out that in unit test execution a program reaches public void run() line then skips it. So there's no error. Program ends up correctly but does nothing useful.
Any help will be appreciated.
This is how asynchronous programming works. In the AlphavantageStockRequestDispatcher class you've just submitted a task but you have to wait for it's completed. There are several ways to handle this situation. I prefer state notification using java.util.concurrent.CountDownLatch. So some refactoring is recommended in AlphavantageStockRequestDispatcher class like this:
public class AlphavantageStockRequestDispatcher {
public static void startAlphavantageStockScraper(int timeInterval, CountDownLatch latch) {
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
Runnable getStockList =
new Runnable() {
#Override
public void run() {
System.out.println("worker started");
try {
Thread.sleep(10_000L);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
System.out.println("worker finished");
Optional.ofNullable(latch).ifPresent(CountDownLatch::countDown);
}
}
};
scheduler.scheduleAtFixedRate(getStockList, 0, timeInterval, TimeUnit.HOURS);
}
}
Now it's possible to test that.
public class AlphavantageStockRequestDispatcherTest {
#Test
void startDispatcher_TwoFullCycles_WithOneHourIntervalBetween() throws InterruptedException {
CountDownLatch latch = new CountDownLatch(1);
AlphavantageStockRequestDispatcher.startAlphavantageStockScraper(1, latch);
latch.await(20, TimeUnit.SECONDS);
System.out.println("first finished - need some assertions");
}
}
I am trying to write a test for my android app that communicates with a cloud service.
Theoretically the flow for the test is supposed to be this:
Send request to the server in a worker thread
Wait for the response from the server
Check the response returned by the server
I am trying to use Espresso's IdlingResource class to accomplish that but it is not working as expected. Here's what I have so far
My Test:
#RunWith(AndroidJUnit4.class)
public class CloudManagerTest {
FirebaseOperationIdlingResource mIdlingResource;
#Before
public void setup() {
mIdlingResource = new FirebaseOperationIdlingResource();
Espresso.registerIdlingResources(mIdlingResource);
}
#Test
public void testAsyncOperation() {
Cloud.CLOUD_MANAGER.getDatabase().getCategories(new OperationResult<List<Category>>() {
#Override
public void onResult(boolean success, List<Category> result) {
mIdlingResource.onOperationEnded();
assertTrue(success);
assertNotNull(result);
}
});
mIdlingResource.onOperationStarted();
}
}
The FirebaseOperationIdlingResource
public class FirebaseOperationIdlingResource implements IdlingResource {
private boolean idleNow = true;
private ResourceCallback callback;
#Override
public String getName() {
return String.valueOf(System.currentTimeMillis());
}
public void onOperationStarted() {
idleNow = false;
}
public void onOperationEnded() {
idleNow = true;
if (callback != null) {
callback.onTransitionToIdle();
}
}
#Override
public boolean isIdleNow() {
synchronized (this) {
return idleNow;
}
}
#Override
public void registerIdleTransitionCallback(ResourceCallback callback) {
this.callback = callback;
}}
When used with Espresso's view matchers the test is executed properly, the activity waits and then check the result.
However plain JUNIT4 assert methods are ignored and JUnit is not waiting for my cloud operation to complete.
Is is possible that IdlingResource only work with Espresso methods ? Or am I doing something wrong ?
I use Awaitility for something like that.
It has a very good guide, here is the basic idea:
Wherever you need to wait:
await().until(newUserIsAdded());
elsewhere:
private Callable<Boolean> newUserIsAdded() {
return new Callable<Boolean>() {
public Boolean call() throws Exception {
return userRepository.size() == 1; // The condition that must be fulfilled
}
};
}
I think this example is pretty similar to what you're doing, so save the result of your asynchronous operation to a field, and check it in the call() method.
Junit will not wait for async tasks to complete. You can use CountDownLatch to block the thread, until you receive response from server or timeout.
Countdown latch is a simple yet elegant solution and does NOT need an external library. It also helps you focus on the actual logic to be tested rather than over-engineering the async wait or waiting for a response
void testBackgroundJob() {
Latch latch = new CountDownLatch(1);
//Do your async job
Service.doSomething(new Callback() {
#Override
public void onResponse(){
ACTUAL_RESULT = SUCCESS;
latch.countDown(); // notify the count down latch
// assertEquals(..
}
});
//Wait for api response async
try {
latch.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
assertEquals(expectedResult, ACTUAL_RESULT);
}
I know this question was answered many times, but I'm struggling to understand how it works.
So in my application the user must be able to select items which will be added to a queue (displayed in a ListView using an ObservableList<Task>) and each item needs to be processed sequentially by an ExecutorService.
Also that queue should be editable (change the order and remove items from the list).
private void handleItemClicked(MouseEvent event) {
if (event.getClickCount() == 2) {
File item = listView.getSelectionModel().getSelectedItem();
Task<Void> task = createTask(item);
facade.getTaskQueueList().add(task); // this list is bound to a ListView, where it can be edited
Future result = executor.submit(task);
// where executor is an ExecutorService of which type?
try {
result.get();
} catch (Exception e) {
// ...
}
}
}
Tried it with executor = Executors.newFixedThreadPool(1) but I don't have control over the queue.
I read about ThreadPoolExecutor and queues, but I'm struggling to understand it as I'm quite new to Concurrency.
I need to run that method handleItemClicked in a background thread, so that the UI does not freeze, how can I do that the best way?
Summed up: How can I implement a queue of tasks, which is editable and sequentially processed by a background thread?
Please help me figure it out
EDIT
Using the SerialTaskQueue class from vanOekel helped me, now I want to bind the List of tasks to my ListView.
ListProperty<Runnable> listProperty = new SimpleListProperty<>();
listProperty.set(taskQueue.getTaskList()); // getTaskList() returns the LinkedList from SerialTaskQueue
queueListView.itemsProperty().bind(listProperty);
Obviously this doesn't work as it's expecting an ObservableList. There is an elegant way to do it?
The simplest solution I can think of is to maintain the task-list outside of the executor and use a callback to feed the executor the next task if it is available. Unfortunately, it involves synchronization on the task-list and an AtomicBoolean to indicate a task executing.
The callback is simply a Runnable that wraps the original task to run and then "calls back" to see if there is another task to execute, and if so, executes it using the (background) executor.
The synchronization is needed to keep the task-list in order and at a known state. The task-list can be modified by two threads at the same time: via the callback running in the executor's (background) thread and via handleItemClicked method executed via the UI foreground thread. This in turn means that it is never exactly known when the task-list is empty for example. To keep the task-list in order and at a known fixed state, synchronization of the task-list is needed.
This still leaves an ambiguous moment to decide when a task is ready for execution. This is where the AtomicBoolean comes in: a value set is always immediatly availabe and read by any other thread and the compareAndSet method will always ensure only one thread gets an "OK".
Combining the synchronization and the use of the AtomicBoolean allows the creation of one method with a "critical section" that can be called by both foreground- and background-threads at the same time to trigger the execution of a new task if possible. The code below is designed and setup in such a way that one such method (runNextTask) can exist. It is good practice to make the "critical section" in concurrent code as simple and explicit as possible (which, in turn, generally leads to an efficient "critical section").
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class SerialTaskQueue {
public static void main(String[] args) {
ExecutorService executor = Executors.newSingleThreadExecutor();
// all operations on this list must be synchronized on the list itself.
SerialTaskQueue tq = new SerialTaskQueue(executor);
try {
// test running the tasks one by one
tq.add(new SleepSome(10L));
Thread.sleep(5L);
tq.add(new SleepSome(20L));
tq.add(new SleepSome(30L));
Thread.sleep(100L);
System.out.println("Queue size: " + tq.size()); // should be empty
tq.add(new SleepSome(10L));
Thread.sleep(100L);
} catch (Exception e) {
e.printStackTrace();
} finally {
executor.shutdownNow();
}
}
// all lookups and modifications to the list must be synchronized on the list.
private final List<Runnable> tasks = new LinkedList<Runnable>();
// atomic boolean used to ensure only 1 task is executed at any given time
private final AtomicBoolean executeNextTask = new AtomicBoolean(true);
private final Executor executor;
public SerialTaskQueue(Executor executor) {
this.executor = executor;
}
public void add(Runnable task) {
synchronized(tasks) { tasks.add(task); }
runNextTask();
}
private void runNextTask() {
// critical section that ensures one task is executed.
synchronized(tasks) {
if (!tasks.isEmpty()
&& executeNextTask.compareAndSet(true, false)) {
executor.execute(wrapTask(tasks.remove(0)));
}
}
}
private CallbackTask wrapTask(Runnable task) {
return new CallbackTask(task, new Runnable() {
#Override public void run() {
if (!executeNextTask.compareAndSet(false, true)) {
System.out.println("ERROR: programming error, the callback should always run in execute state.");
}
runNextTask();
}
});
}
public int size() {
synchronized(tasks) { return tasks.size(); }
}
public Runnable get(int index) {
synchronized(tasks) { return tasks.get(index); }
}
public Runnable remove(int index) {
synchronized(tasks) { return tasks.remove(index); }
}
// general callback-task, see https://stackoverflow.com/a/826283/3080094
static class CallbackTask implements Runnable {
private final Runnable task, callback;
public CallbackTask(Runnable task, Runnable callback) {
this.task = task;
this.callback = callback;
}
#Override public void run() {
try {
task.run();
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
callback.run();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
// task that just sleeps for a while
static class SleepSome implements Runnable {
static long startTime = System.currentTimeMillis();
private final long sleepTimeMs;
public SleepSome(long sleepTimeMs) {
this.sleepTimeMs = sleepTimeMs;
}
#Override public void run() {
try {
System.out.println(tdelta() + "Sleeping for " + sleepTimeMs + " ms.");
Thread.sleep(sleepTimeMs);
System.out.println(tdelta() + "Slept for " + sleepTimeMs + " ms.");
} catch (Exception e) {
e.printStackTrace();
}
}
private String tdelta() { return String.format("% 4d ", (System.currentTimeMillis() - startTime)); }
}
}
Update: if groups of tasks need to be executed serial, have a look at the adapted implementation here.
I'm using a 3rd party function (say runThird()) that has a tendency to loop indefinitely and has no timeout facility built in. However, I can kill it (killThird()). Is there a proper way to do this (i.e. some concurrency construct)?
Here's my attempt at this:
java.lang.Thread thread = new Thread(new Runnable(){
#Override
public void run(){
try {
Thread.sleep(TIMEOUT);
} catch (java.lang.InterruptedException e){
return;
}
killThird();
}
});
thread.start();
RunThirdResult rtr = runThird();
if (thread != null){
thread.interrupt();
}
But I'm not sure I like the overhead of creating a thread, using sleep and the contrivance of interrupting the thread if runThird() returns.
Let's assume runThird() retuns Integer ...
// ... in your class ...
private ExecutorService executor = Executors.newCachedThreadPool();
//... then somewhere, where you want to call runThird()
Future<Integer> handle = executor.submit( new Callable<Integer>(){
#Override Integer call(){
return runThird(); // Assume you made it available here ...
}
}
Integer result;
try{
result = handle.get(TIMEOUT,UNIT); // TIMEOUT and UNIT declared somewhere above ...
}
catch(TimeoutException ex) {
killThird();
// HANDLE result not being set!
}
// ... use result.
I would use a ScheduledExecutorService for this. Schedule it to be killed.
volatile RunThirdResult rtr;
ScheduledExecutorService service = Executors.newScheduledThreadPool(1);
service.schedule(new Runnable(){
public void run(){
if(rtr == null) killThird();
}
}, TIMEOUT_IN_MILLIS, TimeUnit.MILLISECONDS);
RunThirdResult rtr = runThird();
Something like that? The most interesting part is StoppableWrapper#stop(), cause graceful cancellation is a hard thing and there's no common approach for all cases. One time you need to clear filesystem, other time to close network connection, etc. In your sample, you just call interrupt(), so I assumed runThird() honors being interrupted and will take care to clean things behind itself.
class Sample {
final ExecutorService tasksExecutor = Executors.newCachedThreadPool();
class StoppableWrapper implements Runnable {
private final Runnable task;
private final CountDownLatch executed;
StoppableWrapper(Runnable task, CountDownLatch executed) {
this.task = task;
this.executed = executed;
}
void stop() {
// e.g. Thread.currentThread().interrupt()
}
#Override
public void run() {
task.run();
executed.countDown();
}
}
public void scheduleTimingOutTaskExecution(final long timeout) {
final CountDownLatch executed = new CountDownLatch(1);
final StoppableWrapper command = new StoppableWrapper(new RunThirdInstance(), executed);
tasksExecutor.execute(command);
tasksExecutor.execute(new Runnable() {
#Override
public void run() {
try {
if (!executed.await(timeout, TimeUnit.MILLISECONDS)) {
command.stop();
// additionally, you can make stop() return boolean after time-out as well and handle failure
}
} catch (InterruptedException e) {
// handle stopper exception here
}
}
});
}
}
I am trying to write Observe pattern in Java in multithread program to develop own logging program. My program is used in any Java class and can create the Logger class, and write a message to the logger. Logger class will call the logger manager class , which is a Singleton class and it also have the private class which is thread, on execution a scheduler runs, So I kept that execution in the private constructor.
private void LoggerManager() {
System.out.println("IN CONSTRUCTOR");
executorThread.submit(new CreateLoggerFileForSpecifiedTime());
}
static LoggerManager getInstance() {
if (LOGMANAGER == null) {
synchronized (LoggerManager.class) {
if(LOGMANAGER == null){
System.out.println("IN MANAGER");
LOGMANAGER = new LoggerManager();
}
}
}
return LOGMANAGER;
}
Next there is a method in the LoogerManager "writeMessageToLog".
void writeMessageToLog(String componantName, String message,
Calendar messageCreationTime) {
LoggerDetails logDetails = new LoggerDetails(componantName, message,
messageCreationTime);
LogInitiater logIntiater = new LogInitiater(logDetails, noticeOfLoggerChange,
noticeOfMessageAdded);
executorThread.submit(logIntiater);
}
LogIntitiater is a thread which is added the LogWriter to two Subject's, One subject is a scheduler I have pasted above i.e a Timer runs for every specified time and notfiies LogWriter the need to create a new log file while the other purpose is when a message is added in the queue. Code is not working as expected:
* scheduler code is not running, I tried with execute as well. That private constructor is not being called
* thread initiater is not always called
* observe pattern: I have used J2SE interfaces i.e Observable and Observe and they are not working as expected
Please help me.
Producer - Consumer way for Logger (INCOMPLETE AND NOT FULLY TESTED) - will leave that to you
This is just for the demonstration purpose and shows one way of doing ASYNCHRONOUS logging
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
class Logger extends Thread{
private static Logger logger = null;
private BlockingQueue<String> logRequests;
private Logger(){
logRequests = new ArrayBlockingQueue<String>(10000);
}
public static Logger getLogger(){
if(logger == null){
synchronized (Logger.class) {
logger = new Logger();
logger.start();
}
}
return logger;
}
#Override
public void run() {
System.out.println("Starting to log");
while(true){
try {
System.out.println(logRequests.take());
} catch (InterruptedException e) {
//interrupted so stop logging
System.out.println("Logger exiting");
break;
}
}
}
public void info(String logMessage){
//not taken care of the case when queue becomes full
//will leave it to you
logRequests.add(logMessage);
}
}
Main class
public static void main(String[] args) {
Logger.getLogger().info("This is info message");
System.out.println("I am free");
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//stop the log
Logger.getLogger().interrupt();
}