My goal is to obtain a list of results from 10 (or some other arbitrary number) of asynchronous operations.
I'm using com.google.guava for the utilities in concurrency and if someone could graciously point me in the right direction I would be much appreciative.
In the example I'm trying to get a list of successfulBombs (Bomb being pretty much an empty object but has a random probability of throwing a Problem when created to simulate problems with service call execution)
ListeningExecutorService service = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));
List<ListenableFuture<Bomb>> bombs;
ListenableFuture<List<Bomb>> successfulBombs;
Edit:
This is what I've come up with so far but the list is empty even though it should have some successful elements... I can't quite discern why
ListeningExecutorService service = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));
List<ListenableFuture<Bomb>> bombs = new ArrayList<ListenableFuture<Bomb>>();
for(int i=0;i<10;i++)
{
ListenableFuture<Bomb> bomb = service.submit(new Callable<Bomb>(){
public Bomb call() throws Problem
{
return craftBomb();
}
});
}
ListenableFuture<List<Bomb>> successfulBombs = Futures.successfulAsList(bombs);
Futures.addCallback(successfulBombs, new FutureCallback<List<Bomb>>(){
public void onSuccess(List<Bomb> bombs)
{
System.out.println("My successful bombs");
for(Bomb b : bombs)
{
System.out.println(b);
}
}
public void onFailure(Throwable thrown)
{
System.err.println("There was a problem making this bomb.");
}
});
In closing what I'm looking for:
Correct pattern for starting asynchronous operations
Collecting a List for the resulting operations
Collecting a List of successful operations using the guava Framework
The list is empty because you're never adding anything to bombs. You're passing an empty list to Futures.successfulAsList.
Are you looking for ListeningExecutorService.invokeAll(List<Callable<T>>)? Perhaps combined with Futures.allAsList(List<ListenableFuture<T>>)?
The working solution was as follows
ListeningExecutorService service = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));
List<ListenableFuture<Bomb>> bombs = new ArrayList<ListenableFuture<Bomb>>();
for(int i=0;i<10;i++)
{
ListenableFuture<Bomb> bomb = service.submit(new Callable<Bomb>(){
public Bomb call() throws Problem
{
return craftBomb();
}
});
bombs.add(bomb);
}
ListenableFuture<List<Bomb>> successfulBombs = Futures.successfulAsList(bombs);
Futures.addCallback(successfulBombs, new FutureCallback<List<Bomb>>(){
public void onSuccess(List<Bomb> bombs)
{
System.out.println("My successful bombs");
for(Bomb b : bombs)
{
System.out.println(b);
}
}
public void onFailure(Throwable thrown)
{
System.err.println("There was a problem making this bomb.");
}
});
Related
I have a daemon thread running which calls a function (prepareOrder) whenever the cook is not busy and there are orders to be delivered. The prepareOrder calls the orderComplete function after a certain interval of time depending upon the time required to complete the order. Now the problem i am facing is only the last call to the prepareOrder gets displayed on sout.
The daemon
package ui;
import Model.takeOrderModel;
public class daemonThread extends Thread{
//call this method in the main method of driving fucntion
private takeOrderModel orderModel;
daemonThread(takeOrderModel orderModel){
this.orderModel = orderModel;
}
public void assignCook(){
while(true){
int toComplete = orderModel.toCompleteOrders.size();
if ( !orderModel.cookBusy && toComplete>0 ) orderModel.prepareOrder();
}
}
}
The prepare order function.
public void prepareOrder(){
// pick the last element from list
if (toCompleteOrders.size() > 0){
String nextPrepare = toCompleteOrders.get(toCompleteOrders.size()-1);
order orderToComplete = allOrdersPlaced.get(nextPrepare);
completeOrder(orderToComplete);
toCompleteOrders.remove(nextPrepare);
}
}
//Helper function to prepareOrder moves an order from toComplete to prepared order
private void completeOrder(order orderToComplete){
changeCookState();
new java.util.Timer().schedule(
new java.util.TimerTask(){
#Override
public void run() {
changeCookState();
preparedOrders.add(orderToComplete.id);
deliverOrder(orderToComplete.id);
}
}, (long) (orderToComplete.timeToComplete*60)
);
}
public void changeCookState(){
this.cookBusy = !cookBusy;
}
// MODIFIES removes a order from the prepared list and puts it in delivered list
public String deliverOrder(String completedOrder){
preparedOrders.remove(completedOrder);
deliveredOrders.add(completedOrder);
System.out.println(String.format("The order of %s is here", allOrdersPlaced.get(completedOrder).customerName));
return String.format("The order of %s is here", allOrdersPlaced.get(completedOrder).customerName);
}
The main function driving code.
orderMachine.takeNewOrder(fullMeal, "Tom");
orderMachine.takeNewOrder(halfMeal, "Bob");
daemonThread backThread = new daemonThread(orderMachine);
backThread.setDaemon(true);
backThread.assignCook();
Now for me only the last placed order("Bob") gets printed on sout. How can all calls created by Timer.schedule stay in stack.
Edits
The take new order function.
public boolean takeNewOrder(List<item> itemsInOrder, String customerName){
try {
order newOrder = new order(itemsInOrder, customerName);
allOrdersPlaced.put(newOrder.id, newOrder);
toCompleteOrders.add(newOrder.id);
return true;
} catch (Exception e) {
e.printStackTrace();
return false;
}
}
Edit 2
here is the public repo containing the complete code
https://github.com/oreanroy/Share_code_samples/tree/master/takeOrder
The problem in this code is a concurrency bug - the cookBusy variable is being written to from two different threads. To fix this, use an AtomicBoolean instead of a boolean, as this is thread safe.
AtomicBoolean cookBusy = new AtomicBoolean(false);
Use compareAndSet to ensure the shared variable is set to a known value before updating it.
public void changeCookState(boolean busy){
if (!this.cookBusy.compareAndSet(!busy, busy))
{
throw new RuntimeException("shared variable set to unexpected value");
}
}
I'm implementing a GraphQL client in a Java application using Apollo's auto generation of queries, and so far I've been able to chain calls and I also get the data I want. The issue is that Apollo makes me implement the anonymous method ApolloCall.Callback<>() which overrides void onResponse(Response response) and void onFailure(), but I'm unable to find a way to get a hold of this Response object, which I want to collect and make sure I have.
This is a Spring Boot project on Java 11, I've tried to make use of CompletableFuture but with limited knowledge of it and how to use it for this particular problem I feel out of luck. I've also tried to implement the RxJava support that Apollo is supposed to have but I couldn't resolve dependency issues with that approach.
I'm pretty sure that futures will solve it but again I don't know how.
public void getOwnerIdFromClient() {
client
.query(getOwnerDbIdQuery)
.enqueue(
new ApolloCall.Callback<>() {
#Override
public void onResponse(#Nonnull Response<Optional<GetOwnerDbIdQuery.Data>> response) {
int ownerId =
response
.data()
.get()
.entities()
.get()
.edges()
.get()
.get(0)
.node()
.get()
.ownerDbId()
.get();
System.out.println("OwnerId = " + ownerId);
}
#Override
public void onFailure(#Nonnull ApolloException e) {
logger.error("Could not retrieve response from GetOwnerDbIdQuery.", e);
}
});
}
Since I wish to work with this int ownerId outside of the onResponse this isn't a sufficient solution. I'd actually like to make this call x amount of times, and create a list of all the id's I actually got, since this might return a null id as well, which means I need some way to wait for them all to finish.
You are right, this can be done using Futures:
change return type to Future
complete the future in onResponse
Approximately:
public Future<Integer> getOwnerIdFromClient(){
Future<Integer> result=new CompletableFuture<Integer>();
client
.query(getOwnerDbIdQuery)
.enqueue(
new ApolloCall.Callback<>(){
#Override
public void onResponse(#Nonnull Response<Optional<GetOwnerDbIdQuery.Data>>response){
// get owner Id
System.out.println("OwnerId = "+ownerId);
result.complete(ownerId)
}
#Override
public void onFailure(#Nonnull ApolloException e){
logger.error("Could not retrieve response from GetOwnerDbIdQuery.",e);result.completeExceptionally(e);
}
});
return result;
}
If anyone else is coming across this, it took me quite a while to figure out the generics, but you can do this in a generic manner (to avoid the copy/paste for all your different query types) by using the following function as a separate class or wrapper:
private <D extends Operation.Data, T, V extends Operation.Variables> CompletableFuture<T> execute(Query<D, T, V> query) {
CompletableFuture<T> future = new CompletableFuture<>();
client.query(query).enqueue(new ApolloCall.Callback<>() {
#Override
public void onResponse(#NotNull Response<T> response) {
if (response.hasErrors()) {
String errors = Objects.requireNonNull(response.getErrors()).stream().map(Object::toString).collect(Collectors.joining(", "));
future.completeExceptionally(new ApolloException(errors));
return;
}
future.complete(response.getData());
}
#Override
public void onFailure(#NotNull ApolloException e) {
future.completeExceptionally(e);
}
});
return future;
}
Then it should just be a case of calling
Integer myResult = execute(getOwnerDbIdQuery).get();
I am trying to implement a CoAP client based on Californium. I make this client observing to a resource:
public static class CoapCl{
double val = 0;
CoapClient client = new CoapClient("coap://localhost/Ultrasonic");
CoapObserveRelation relation = client.observe(new CoapHandler() {
#Override public void onLoad(CoapResponse response)
{
val = Double.parseDouble(response.getResponseText());
}
#Override
public void onError() {
System.out.println("Failed");
}
});
}
I want to access the value "val" from another class. How can I do it ? I tried to call a reference from the CoapCl class like this and print the value out:
CoapCl client = new CoapCl();
while(true)
{
System.out.println("Testing: " + client.val);
}
This will print all the value I get from the CoAP client, both changed and unchanged value. What should I do if I only want to get the changed value ?
Well, the issue itself isn't related to Californium and CoAP.
Except that CoapHandler is async but this is rather a strench.
Nevertheless, I'd recommend to end up with some kind of callback:
public class CoapCl {
private final Consumer<Double> valueChangedAction;
private final CoapClient client = new CoapClient("coap://localhost/Ultrasonic");
public CoapCl(Consumer<Double> valueChangedAction) {
this.valueChangedAction = valueChangedAction;
}
public void run() {
client.observe(new CoapHandler() {
#Override
public void onLoad(CoapResponse response) {
valueChangedAction.accept(
Double.parseDouble(
response.getResponseText()
)
);
}
#Override
public void onError() {
System.out.println("Failed");
}
});
}
}
new CoapCl(val -> System.out.println("Testing: " + val)).run();
Please keep in mind you have to block the main thread someway to keep the program from immediate exit.
Before, you had blocked it with your infinite loop.
Now you'll have to use System.in.read() or Thread.sleep or something else if you have no such stuff yet in your program.
i have joined to one of those Vertx lovers , how ever the single threaded main frame may not be working for me , because in my server there might be 50 file download requests at a moment , as a work around i have created this class
public abstract T onRun() throws Exception;
public abstract void onSuccess(T result);
public abstract void onException();
private static final int poolSize = Runtime.getRuntime().availableProcessors();
private static final long maxExecuteTime = 120000;
private static WorkerExecutor mExecutor;
private static final String BG_THREAD_TAG = "BG_THREAD";
protected RoutingContext ctx;
private boolean isThreadInBackground(){
return Thread.currentThread().getName() != null && Thread.currentThread().getName().equals(BG_THREAD_TAG);
}
//on success will not be called if exception be thrown
public BackgroundExecutor(RoutingContext ctx){
this.ctx = ctx;
if(mExecutor == null){
mExecutor = MyVertxServer.vertx.createSharedWorkerExecutor("my-worker-pool",poolSize,maxExecuteTime);
}
if(!isThreadInBackground()){
/** we are unlocking the lock before res.succeeded , because it might take long and keeps any thread waiting */
mExecutor.executeBlocking(future -> {
try{
Thread.currentThread().setName(BG_THREAD_TAG);
T result = onRun();
future.complete(result);
}catch (Exception e) {
GUI.display(e);
e.printStackTrace();
onException();
future.fail(e);
}
/** false here means they should not be parallel , and will run without order multiple times on same context*/
},false, res -> {
if(res.succeeded()){
onSuccess((T)res.result());
}
});
}else{
GUI.display("AVOIDED DUPLICATE BACKGROUND THREADING");
System.out.println("AVOIDED DUPLICATE BACKGROUND THREADING");
try{
T result = onRun();
onSuccess((T)result);
}catch (Exception e) {
GUI.display(e);
e.printStackTrace();
onException();
}
}
}
allowing the handlers to extend it and use it like this
public abstract class DefaultFileHandler implements MyHttpHandler{
public abstract File getFile(String suffix);
#Override
public void Handle(RoutingContext ctx, VertxUtils utils, String suffix) {
new BackgroundExecutor<Void>(ctx) {
#Override
public Void onRun() throws Exception {
File file = getFile(URLDecoder.decode(suffix, "UTF-8"));
if(file == null || !file.exists()){
utils.sendResponseAndEnd(ctx.response(),404);
return null;
}else{
utils.sendFile(ctx, file);
}
return null;
}
#Override
public void onSuccess(Void result) {}
#Override
public void onException() {
utils.sendResponseAndEnd(ctx.response(),404);
}
};
}
and here is how i initialize my vertx server
vertx.deployVerticle(MainDeployment.class.getCanonicalName(),res -> {
if (res.succeeded()) {
GUI.display("Deployed");
} else {
res.cause().printStackTrace();
}
});
server.requestHandler(router::accept).listen(port);
and here is my MainDeployment class
public class MainDeployment extends AbstractVerticle{
#Override
public void start() throws Exception {
// Different ways of deploying verticles
// Deploy a verticle and don't wait for it to start
for(Entry<String, MyHttpHandler> entry : MyVertxServer.map.entrySet()){
MyVertxServer.router.route(entry.getKey()).handler(new Handler<RoutingContext>() {
#Override
public void handle(RoutingContext ctx) {
String[] handlerID = ctx.request().uri().split(ctx.currentRoute().getPath());
String suffix = handlerID.length > 1 ? handlerID[1] : null;
entry.getValue().Handle(ctx, new VertxUtils(), suffix);
}
});
}
}
}
this is working just fine when and where i need it , but i still wonder if is there any better way to handle concurencies like this on vertx , if so an example would be really appreciated . thanks alot
I don't fully understand your problem and reasons for your solution. Why don't you implement one verticle to handle your http uploads and deploy it multiple times? I think that handling 50 concurrent uploads should be a piece of cake for vert.x.
When deploying a verticle using a verticle name, you can specify the number of verticle instances that you want to deploy:
DeploymentOptions options = new DeploymentOptions().setInstances(16);
vertx.deployVerticle("com.mycompany.MyOrderProcessorVerticle", options);
This is useful for scaling easily across multiple cores. For example you might have a web-server verticle to deploy and multiple cores on your machine, so you want to deploy multiple instances to take utilise all the cores.
http://vertx.io/docs/vertx-core/java/#_specifying_number_of_verticle_instances
vertx is a well-designed model so that a concurrency issue does not occur.
generally, vertx does not recommend the multi-thread model.
(because, handling is not easy.)
If you select multi-thread model, you have to think about shared data..
Simply, if you just only want to split EventLoop Area,
first of all, you make sure Check your a number of CPU Cores.
and then Set up the count of Instances .
DeploymentOptions options = new DeploymentOptions().setInstances(4);
vertx.deployVerticle("com.mycompany.MyOrderProcessorVerticle", options);
But, If you have 4cores of CPU, you don't set up over 4 instances.
If you set up to number four or more, the performance won't improve.
vertx concurrency reference
http://vertx.io/docs/vertx-core/java/
I have an app. I have a big button that allows the user to sync all their data at once to the cloud. A re-sync feature that allows them to send all their data again. (300+ entries)
I am using RXjava2 and retrofit2. I have my unit test working with a single call. However I need to make N network calls.
What I want to avoid is having the observable call the next item in a queue. I am at the point where I need to implement my runnable. I have seen a bit about Maps but I have not seen anyone use it as a queue. Also I want to avoid having one item fail and it report back as ALL items fail, like the Zip feature would do. Should I just do the nasty manager class that keeps track of a queue? Or is there a cleaner way to send several hundred items?
NOTE: SOLUTION CANNOT DEPEND ON JAVA8 / LAMBDAS. That has proved to be way more work than is justified.
Note all items are the same object.
#Test
public void test_Upload() {
TestSubscriber<Record> testSubscriber = new TestSubscriber<>();
ClientSecureDataToolKit clientSecureDataToolKit = ClientSecureDataToolKit.getClientSecureDataKit();
clientSecureDataToolKit.putUserDataToSDK(mPayloadSecureDataToolKit).subscribe(testSubscriber);
testSubscriber.awaitTerminalEvent();
testSubscriber.assertNoErrors();
testSubscriber.assertValueCount(1);
testSubscriber.assertCompleted();
}
My helper to gather and send all my items
public class SecureDataToolKitHelper {
private final static String TAG = "SecureDataToolKitHelper";
private final static SimpleDateFormat timeStampSimpleDateFormat =
new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
public static void uploadAll(Context context, RuntimeExceptionDao<EventModel, UUID> eventDao) {
List<EventModel> eventModels = eventDao.queryForAll();
QueryBuilder<EventModel, UUID> eventsQuery = eventDao.queryBuilder();
String[] columns = {...};
eventsQuery.selectColumns(columns);
try {
List<EventModel> models;
models = eventsQuery.orderBy("timeStamp", false).query();
if (models == null || models.size() == 0) {
return;
}
ArrayList<PayloadSecureDataToolKit> toSendList = new ArrayList<>();
for (EventModel eventModel : models) {
try {
PayloadSecureDataToolKit payloadSecureDataToolKit = new PayloadSecureDataToolKit();
if (eventModel != null) {
// map my items ... not shown
toSendList.add(payloadSecureDataToolKit);
}
} catch (Exception e) {
Log.e(TAG, "Error adding payload! " + e + " ..... Skipping entry");
}
}
doAllNetworkCalls(toSendList);
} catch (SQLException e) {
e.printStackTrace();
}
}
my Retrofit stuff
public class ClientSecureDataToolKit {
private static ClientSecureDataToolKit mClientSecureDataToolKit;
private static Retrofit mRetrofit;
private ClientSecureDataToolKit(){
mRetrofit = new Retrofit.Builder()
.baseUrl(Utilities.getSecureDataToolkitURL())
.addCallAdapterFactory(RxJavaCallAdapterFactory.create())
.addConverterFactory(GsonConverterFactory.create())
.build();
}
public static ClientSecureDataToolKit getClientSecureDataKit(){
if(mClientSecureDataToolKit == null){
mClientSecureDataToolKit = new ClientSecureDataToolKit();
}
return mClientSecureDataToolKit;
}
public Observable<Record> putUserDataToSDK(PayloadSecureDataToolKit payloadSecureDataToolKit){
InterfaceSecureDataToolKit interfaceSecureDataToolKit = mRetrofit.create(InterfaceSecureDataToolKit.class);
Observable<Record> observable = interfaceSecureDataToolKit.putRecord(NetworkUtils.SECURE_DATA_TOOL_KIT_AUTH, payloadSecureDataToolKit);
return observable;
}
}
public interface InterfaceSecureDataToolKit {
#Headers({
"Content-Type: application/json"
})
#POST("/api/create")
Observable<Record> putRecord(#Query("api_token") String api_token, #Body PayloadSecureDataToolKit payloadSecureDataToolKit);
}
Update. I have been trying to apply this answer to not much luck. I am running out of steam for tonight. I am trying to implement this as a unit test, like I did for the original call for one item.. It looks like something is not right with use of lambda maybe..
public class RxJavaBatchTest {
Context context;
final static List<EventModel> models = new ArrayList<>();
#Before
public void before() throws Exception {
context = new MockContext();
EventModel eventModel = new EventModel();
//manually set all my eventmodel data here.. not shown
eventModel.setSampleId("SAMPLE0");
models.add(eventModel);
eventModel.setSampleId("SAMPLE1");
models.add(eventModel);
eventModel.setSampleId("SAMPLE3");
models.add(eventModel);
}
#Test
public void testSetupData() {
Assert.assertEquals(3, models.size());
}
#Test
public void testBatchSDK_Upload() {
Callable<List<EventModel> > callable = new Callable<List<EventModel> >() {
#Override
public List<EventModel> call() throws Exception {
return models;
}
};
Observable.fromCallable(callable)
.flatMapIterable(models -> models)
.flatMap(eventModel -> {
PayloadSecureDataToolKit payloadSecureDataToolKit = new PayloadSecureDataToolKit(eventModel);
return doNetworkCall(payloadSecureDataToolKit) // I assume this is just my normal network call.. I am getting incompatibility errors when I apply a testsubscriber...
.subscribeOn(Schedulers.io());
}, true, 1);
}
private Observable<Record> doNetworkCall(PayloadSecureDataToolKit payloadSecureDataToolKit) {
ClientSecureDataToolKit clientSecureDataToolKit = ClientSecureDataToolKit.getClientSecureDataKit();
Observable observable = clientSecureDataToolKit.putUserDataToSDK(payloadSecureDataToolKit);//.subscribe((Observer<? super Record>) testSubscriber);
return observable;
}
Result is..
An exception has occurred in the compiler (1.8.0_112-release). Please file a bug against the Java compiler via the Java bug reporting page (http://bugreport.java.com) after checking the Bug Database (http://bugs.java.com) for duplicates. Include your program and the following diagnostic in your report. Thank you.
com.sun.tools.javac.code.Symbol$CompletionFailure: class file for java.lang.invoke.MethodType not found
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':app:compile<MyBuildFlavorhere>UnitTestJavaWithJavac'.
> Compilation failed; see the compiler error output for details.
Edit. No longer trying Lambdas. Even after setting up the path on my mac, javahome to point to 1.8, etc. I could not get it to work. If this was a newer project I would push harder. However as this is an inherited android application written by web developers trying android, it is just not a great option. Nor is it worth the time sink to get it working. Already into the days of this assignment instead of the half day it should have taken.
I could not find a good non lambda flatmap example. I tried it myself and it was getting messy.
If I understand you correctly, you want to make your calls in parallel?
So rx-y way of doing this would be something like:
Observable.fromCallable(() -> eventsQuery.orderBy("timeStamp", false).query())
.flatMapIterable(models -> models)
.flatMap(model -> {
// map your model
//avoid throwing exceptions in a chain, just return Observable.error(e) if you really need to
//try to wrap your methods that throw exceptions in an Observable via Observable.fromCallable()
return doNetworkCall(someParameter)
.subscribeOn(Schedulers.io());
}, true /*because you don't want to terminate a stream if error occurs*/, maxConcurrent /* specify number of concurrent calls, typically available processors + 1 */)
.subscribe(result -> {/* handle result */}, error -> {/* handle error */});
In your ClientSecureDataToolKit move this part into constructor
InterfaceSecureDataToolKit interfaceSecureDataToolKit = mRetrofit.create(InterfaceSecureDataToolKit.class);