I often find myself using a "main" actor that creates a number of child actors for sub-tasks. When the sub-tasks are finished, the main actor should also stop himself. So I watch the child actors and stop the main actor when context.children.isEmpty.
I often use this pattern, but as I never read about this. I am unsure, if this is a good idea or if there are problems with failing actors...?
I have read about Shutdown Patterns in Akka 2, but this approach seems to be more complicated in Java than my solution?
Here is my pseudo code for a main actor with two subtasks:
class MainActor extends AbstractActor {
public MainActor() {
receive(ReceiveBuilder
.match(SubTask1Response.class, this::handleSubTask1)
.match(SubTask2Response.class, this::handleSubTask2)
.match(Terminated.class, x -> checkFinished())
.build());
}
#Override
public void preStart() throws Exception {
context().watch(context().actorOf(SubTask1Worker.props(), "subTask1"));
context().watch(context().actorOf(SubTask2Worker.props(), "subTask2"));
}
private void checkFinished() {
if(context().children().isEmpty()) {
context().stop(self());
}
}
// ...
}
(I must use Java 8 :-(, but I am also happy to read Scala code if you can provide me another solution)
So context().children().isEmpty() seems to work as expected.
But while debugging my Akka application I found another problem with this approach: It is not deterministic when the Terminated message arrives in the MainActor: Sometimes there is the Terminated message before the SubTask1Response from the example!
I have changed now my code to count the running children myself and decrement the number whenever the MainActor receives a result response SubTask[1,2]Response.
=> So I wouldn't recommend my original pattern.
Related
I am not sure if i can put my question in the clearest fashion but i will try my best.
Lets say i am retrieving some information from a third party api. The retrieved information will be huge in size. To have a performance gain, instead of retrieving all the info in one go, i will be retrieving the info in a paged fashion (the api gives me that facility, basically an iterator). The return type is basically a list of objects.
My aim here is to process the information i have in hand(that includes comparing and storing in db and many other operations) while i get paged response on the request.
My question here to the expert community is , what data structure do you prefer in such case. Also does a framework like spring batch help you in getting performance gains in such cases.
I know the question is a bit vague, but i am looking for general ideas,tips and pointers.
In these cases, the data structure for me is java.util.concurrent.CompletionService.
For purposes of example, I'm going to assume a couple of additional constraints:
You want only one outstanding request to the remote server at a time
You want to process the results in order.
Here goes:
// a class that knows how to update the DB given a page of results
class DatabaseUpdater implements Callable { ... }
// a background thread to do the work
final CompletionService<Object> exec = new ExecutorCompletionService(
Executors.newSingleThreadExecutor());
// first call
List<Object> results = ThirdPartyAPI.getPage( ... );
// Start loading those results to DB on background thread
exec.submit(new DatabaseUpdater(results));
while( you need to ) {
// Another call to remote service
List<Object> results = ThirdPartyAPI.getPage( ... );
// wait for existing work to complete
exec.take();
// send more work to background thread
exec.submit(new DatabaseUpdater(results));
}
// wait for the last task to complete
exec.take();
This just a simple two-thread design. The first thread is responsible for getting data from the remote service and the second is responsible for writing to the database.
Any exceptions thrown by DatabaseUpdater will be propagated to the main thread when the result is taken (via exec.take()).
Good luck.
In terms of doing the actual parallelism, one very useful construct in Java is the ThreadPoolExecutor. A rough sketch of what that might look like is this:
public class YourApp {
class Processor implements Runnable {
Widget toProcess;
public Processor(Widget toProcess) {
this.toProcess = toProcess;
}
public void run() {
// commit the Widget to the DB, etc
}
}
public static void main(String[] args) {
ThreadPoolExecutor executor =
new ThreadPoolExecutor(1, 10, 30,
TimeUnit.SECONDS,
new LinkedBlockingDeque());
while(thereAreStillWidgets()) {
ArrayList<Widget> widgets = doExpensiveDatabaseCall();
for(Widget widget : widgets) {
Processor procesor = new Processor(widget);
executor.execute(processor);
}
}
}
}
But as I said in a comment: calls to an external API are expensive. It's very likely that the best strategy is to pull all the Widget objects down from the API in one call, and then process them in parallel once you've got them. Doing more API calls gives you the overhead of sending the data all the way from the server to you, every time -- it's probably best to pay that cost the fewest number of times that you can.
Also, keep in mind that if you're doing DB operations, it's possible that your DB doesn't allow for parallel writes, so you might get a slowdown there.
I am having some strange trouble with the method await(Future future) of the Controller.
Whenever I add an await line anywhere in my code, some GenericModels which have nothing to do with where I placed await, start loading incorrectly and I can not access to any of their attributes.
The wierdest thing is that if I change something in another completely different java file anywhere in the project, play will try to recompile I guess and in that moment it starts working perfectly, until I clean tmp again.
When you use await in a controller it does bytecode enhancement to break a single method into two threads. This is pretty cool, but definitely one of the 'black magic' tricks of Play1. But, this is one place where Play often acts weird and requires a restart (or as you found, some code changing) - the other place it can act strange is when you change a Model class.
http://www.playframework.com/documentation/1.2.5/asynchronous#SuspendingHTTPrequests
To make it easier to deal with asynchronous code we have introduced
continuations. Continuations allow your code to be suspended and
resumed transparently. So you write your code in a very imperative
way, as:
public static void computeSomething() {
Promise delayedResult = veryLongComputation(…);
String result = await(delayedResult);
render(result); }
In fact here, your code will be executed in 2 steps, in 2 different hreads. But as you see it, it’s very
transparent for your application code.
Using await(…) and continuations, you could write a loop:
public static void loopWithoutBlocking() {
for(int i=0; i<=10; i++) {
Logger.info(i);
await("1s");
}
renderText("Loop finished"); }
And using only 1 thread (which is the default in development mode) to process requests, Play is able to
run concurrently these loops for several requests at the same time.
To respond to your comment:
public static void generatePDF(Long reportId) {
Promise<InputStream> pdf = new ReportAsPDFJob(report).now();
InputStream pdfStream = await(pdf);
renderBinary(pdfStream);
and ReportAsPDFJob is simply a play Job class with doJobWithResult overridden - so it returns the object. See http://www.playframework.com/documentation/1.2.5/jobs for more on jobs.
Calling job.now() returns a future/promise, which you can use like this: await(job.now())
I am writing a blackberry app that communicates with a simple Bluetooth peripheral using text based AT commands - similar to a modem... I can only get it working on the blackberry using an event listener. So the communication is now asynchronous.
However, since it is a simple device and I need to control concurrent access, I would prefer to just have a blocking call.
I have the following code which tries to convert the communications to blocking by using a wait/notify. But when I run it, notifyResults never runs until getStringValue completes. i.e. it will always timeout no matter what the delay.
The btCon object runs on a separate thread already.
I'm sure I am missing something obvious with threading. Could someone kindly point it out?
Thanks
I should also add the the notifyAll blows up with an IllegalMonitorStateException.
I previously tried it with a simple boolean flag and a wait loop. But the same problem existed. notifyResult never runs until after getStringValue completes.
public class BTCommand implements ResultListener{
String cmd;
private BluetoothClient btCon;
private String result;
public BTCommand (String cmd){
this.cmd=cmd;
btCon = BluetoothClient.getInstance();
btCon.addListener(this);
System.out.println("[BTCL] BTCommand init");
}
public String getStringValue(){
result = "TIMEOUT";
btCon.sendCommand(cmd);
System.out.println("[BTCL] BTCommand getStringValue sent and waiting");
synchronized (result){
try {
result.wait(5000);
} catch (InterruptedException e) {
System.out.println("[BTCL] BTCommand getStringValue interrupted");
}
}//sync
System.out.println("[BTCL] BTCommand getStringValue result="+result);
return result;
}
public void notifyResults(String cmd) {
if(cmd.equalsIgnoreCase(this.cmd)){
synchronized(result){
result = btCon.getHash(cmd);
System.out.println("[BTCL] BTCommand resultReady: "+cmd+"="+result);
result.notifyAll();
}//sync
}
}
}
Since both notifyResults and getStringValue have synchronized clauses on the same object, assuming getStringValues gets to the synchronized section first notifyResults will block at the start of the synchronized clause until getStringValues exits the synchronized area. If I understand, this is the behaviour you're seeing.
Nicholas' advice is probably good, but you may not find any of those implementations in BlackBerry APIs you're using. You may want to have a look at the produce-consumer pattern.
It may be more appropriate to use a Latch, Semaphore, or a Barrier, as recommended by Brian Goetz book Java Concurrency in Practice.
These classes will make it easier to write blocking methods, and will likely help to prevent bugs, especially if you are unfamiliar with wait() and notifyAll(). (I am not suggesting that YOU are unfamiliar, it is just a note for others...)
The code will work ok. If you will use final object instead of string variable. I'm surprised that you don't get NPE or IMSE.
Create field:
private final Object resultLock = new Object();
Change all synchronized sections to use it instead of string field result.
I don't like magic number 5 sec. I hope you treat null result as timeout in your application.
I'm currently working on a project that need to communicate with java from air without the use of a server like Tomcat. For this i found and use Flerry.
Communicating between Java and Air is no problem, as long as I try to send a message from the class that I initially instantiated from Air.
What I try to do is subscript to messages from a Message Class in java and use that class to send info and errors to Air from Java.
MessageController.java:
public class MessageController
{
public MessageController()
{
}
public static void sendErrorMessage(String errorMessage)
{
NativeObject.sendMessage(errorMessage, "error");
}
public static void sendInfoMessage(String infoMessage)
{
NativeObject.sendMessage(infoMessage, "info");
}
}
In Air I create a NativeObject on the messaging class and subscribe to the messages:
var messageController:NativeObject = new NativeObject();
messageController.source = "controller.MessageController" ;
messageController.singleton = true;
messageController.debug = false;
messageController.addEventListener(FaultEvent.FAULT, onFileControllerFault, false, 0, true);
messageController.subscribe("info", infoMessageHandler);
messageController.subscribe("error", errorMessageHandler);
As you would have guessed, this doesn't work.
It seems that I am only able to dispatch messages from the class that I subscribe to directly, for example if I do this:
messageController.start();
and in my MessageController.java i put this:
public void start()
{
NativeObject.sendMessage("test message", "info");
}
the infoMessageHandler receives an message containing test message, as it should.
How can I dispatch messages from whatever class in Java and catch them on the Air side?
I'm not sure I fully understand the issue yet, but is there a reason you can't simply send and receive all messages through a Java 'communication' class? If that's working, I'd just set up public methods in Java interfacing/communications class and be done with it.
(It's been awhile since I've dug into how Flerry works so I can't recall enough to point out the likely cause of the behavior you're seeing.) Also Flerry is open source, and not very big at all. If you really want to know why it's behaving in a certain way, I'd wager you could figure it out by looking at the source. (Won't take 6 months to learn like some Spring or Hibernate code base, or even BlazeDS, I promise.)
I do use Flerry in a small app, but now that I think about it, I only have one class which dispatches anything to Flex! But I feel like you may need to describe the problem you're facing differently, because it sounds like expected behavior to me.
This question already has answers here:
How should I unit test multithreaded code?
(29 answers)
Closed 5 years ago.
Executive Summary: When assertion errors are thrown in the threads, the unit test doesn't die. This makes sense, since one thread shouldn't be allowed to crash another thread. The question is how do I either 1) make the whole test fail when the first of the helper threads crashes or 2) loop through and determine the state of each thread after they have all completed (see code below). One way of doing the latter is by having a per thread status variable, e.g., "boolean[] statuses" and have "statuses[i] == false" mean that the thread failed (this could be extended to capture more information). However, that is not what I want: I want it to fail just like any other unit test when the assertion errors are thrown. Is this even possible? Is it desirable?
I got bored and I decided to spawn a bunch of threads in my unit test and then have them call a service method, just for the heck of it. The code looks approximately like:
Thread[] threads = new Thread[MAX_THREADS];
for( int i = 0; i < threads.length; i++ ) {
threads[i] = new Thread( new Runnable() {
private final int ID = threadIdSequenceNumber++;
public void run() {
try {
resultRefs[ID] = runTest( Integer.toString( ID ) ); // returns an object
}
catch( Throwable t ) {
// this code is EVIL - it catches even
// Errors - don't copy it - more on this below
final String message = "error testing thread with id => "
+ ID;
logger.debug( message, t );
throw new IllegalStateException( message, t );
// need to wrap throwable in a
// run time exception so it will compile
}
}
} );
}
After this, we will loop through the array of threads and start each one. After that we will wait for them all to finish. Finally, we will perform some checks on the result references.
for( Thread thread : threads )
thread.start();
logger.debug( "waiting for threads to finish ..." );
boolean done = false;
while( !done ) {
done = true;
for( Thread thread : threads )
if( thread.isAlive() )
done = false;
}
for( int i = 0; i < resultRefs.length; i++ ) {
assertTrue( "you've got the world messed, dawg!",
myCondition(resultRefs[i]) );
Here's the problem. Did you notice that nasty try-catch-throwable block? I just added that as a temporary hack so I could see what was going on. In runTest( String ) a few assertions are made, e.g., assertNotNull( null ), but since it is in a different thread, it doesn't cause the unit test to fail!!!!
My guess is that we will need to somehow iterate over the threads array, check the status of each, and manually cause an assertion error if the thread terminated in a nasty way. What's the name of the method that gives this information (the stack trace of the dead thread).
Concurrency is one of those things that are very difficult to unit test. If you are just trying to test that the code inside each thread is doing what it is supposed to test, may be you should just test this code isolated of the context.
If in this example the threads collaborate to reach a result, may be you can test that collaboration without using threads. That would be done by executing all the collaborative parts sequentially.
If you want to test for race conditions and these kind of things, unit testing is not the best way. You will get tests that sometimes fail and sometimes don´t fail.
To summarize, I think that may be your problem is that you are unit testing in a level too high.
Hope this helps
The Google Testing Blog had an excellent article on this subject that's well worth reading: http://googletesting.blogspot.com/2008/08/tott-sleeping-synchronization.html
It's written in Python, but I think the principles are directly transferable to Java.
Unit testing in a multithreaded environment is tough... so some adjustments need to be made. Unit tests must be repeatable.. deterministic. As a result anything with multiple threads fails this criteria. Tests with multiple threads also tend to be slow.
I'd either try to see if I can get by with testing on a single thread.. does the logic under test really need multiple threads.
If that doesn't work, go with the member variable approach that you can check against an expected value at the end of the test, when all the threads have finished running.
Hey seems like there's another question just like this. Check my post for a link to a longer discussion at the tdd yahoogroup
Unit testing a multithreaded application?
Your runnable wrapper should be passing the exception object back to your test class and then you can store them in a collection. When all the tests are finish you can test the collection. If it isn't empty, iterate over each of the exceptions and .printStackTrace() then fail.
Implement a UncaughtExceptionHandler that sets some flags (which the Threads peridocially check) and set it on each Thread.
Another popular option for Junit concurrent thread testing is Matthieu Carbou's method using a custom JunitRunner and a simple annotation.
See the full documentation
It is possible making the unit test to fail, by using a special synchronization object. Take a look at the following article:
Sprinkler - Advanced synchronization object
I'll try to explain the main points here.
You want to be able to externalize internal threads failures to the main thread, which, in your case is the test. So you have to use a shared object/lock that both the internal thread and the test will use to sync each other.
See the following test - it creates a thread which simulates a thrown exception by calling a shared object named Sprinkler.
The main thread (the test) is blocked on Sprinkler.getInstance().await(CONTEXT, 10000)
which, by the time release is called - will be free and catch the thrown exception.
In the catch block you can write the assert which fails the test.
#Test
public void testAwait_InnerThreadExternalizeException() {
final int CONTEXT = 1;
final String EXCEPTION_MESSAGE = "test inner thread exception message";
// release will occur sometime in the future - simulate exception in the releaser thread
ExecutorServiceFactory.getCachedThreadPoolExecutor().submit(new Callable<void>() {
#Override
public Void call() throws Exception {
Sprinkler.getInstance().release(CONTEXT, new RuntimeException(EXCEPTION_MESSAGE));
return null;
}
});
Throwable thrown = null;
try {
Sprinkler.getInstance().await(CONTEXT, 10000);
} catch (Throwable t) {
// if the releaser thread delivers exception it will be externelized to this thread
thrown = t;
}
Assert.assertTrue(thrown instanceof SprinklerException);
Assert.assertEquals(EXCEPTION_MESSAGE, thrown.getCause().getMessage());
}