Develop Reference

How to get task count in GCP cloud task

Is there any standard way to get cloud task count for a queue?
I know we can loop through nextPageToken and achieve this.
Can this be achieved with single request?

By looking at the documentation, QueueStats seems to be the standard way to get task count.
As mentioned in #Sameer comment, cloudtask v2beta3 has a feature to request for stats of a queue:
public long geQueueDepth(String queuePath) {
long depth=-1;
try {
FieldMask.Builder fieldBuilder = FieldMask.newBuilder();
fieldBuilder.addPaths("stats");
com.google.cloud.tasks.v2beta3.GetQueueRequest queueRequest =
com.google.cloud.tasks.v2beta3.GetQueueRequest.newBuilder().setName(queuePath).setReadMask(fieldBuilder).build();;
com.google.cloud.tasks.v2beta3.Queue queue = cloudTasksClientV3.getQueue(queueRequest);
depth=queue.getStats().getTasksCount();
}catch (Exception e){
LOGGER.error("Error while getting queue depth =>{}",e.getMessage());
}
return depth;
}

How to pass own executors to the redis lettuce library?

I have below code to get the data from Redis asynchronously. By default get() call in lettuce library uses nio-event thread pool.
Code 1:
StatefulRedisConnection<String, String> connection = redisClient.connect();
RedisAsyncCommands<String, String> command = connection.async();
CompletionStage<String> result = command.get(id)
.thenAccept(code ->
logger.log(Level.INFO, "Thread Id " + Thread.currentThread().getName());
//Sample code to print thread ID
Thread Id printed is lettuce-nioEventLoop-6-2.
Code 2:
CompletionStage<String> result = command.get(id)
.thenAcceptAsync(code -> {
logger.log(Level.INFO, "Thread Id " + Thread.currentThread().getName());
//my original code
}, executors);
Thread Id printed is pool-1-thread-1.
My questions:
Is there a way to pass my executors?
Is it recommended approach to use nio-event thread pool to get(using get() call) the data from redis?
Lettuce version: 5.2.2.RELEASE
thanks,
Ashok

class io.lettuce.core.RedisClient has a creator method:
public static RedisClient create(ClientResources clientResources, String uri) {
assertNotNull(clientResources);
LettuceAssert.notEmpty(uri, "URI must not be empty");
return create(clientResources, RedisURI.create(uri));
}
You can build your ClientResources by ClientResources#builder(), and pass anything you want. Refer the JavaDoc, there is something you can customize:
EventLoopGroupProvider to obtain particular EventLoopGroups
EventExecutorGroup to perform internal computation tasks
Timer for scheduling
EventBus for client event dispatching
EventPublisherOptions
CommandLatencyCollector to collect latency details. Requires the HdrHistogram library.
DnsResolver to collect latency details. Requires the LatencyUtils library.
Reconnect Delay.
Tracing to trace Redis commands.

RxJava wait for single Event, RxJava

How do you wait for a single value to come on an observable with a timout?
I am looking for something like:
Observable<Acknowledgement> acknowledgementObservable;
port.send(new Message());
Optional<Acknowledgement> ack = acknowledgementObservable.getFirst(100, TimeUnit.MILLISECONDS);

First, convert Observable to CompletableFuture as described in Converting between Completablefuture and Observable:
Observable<Acknowledgement> acknowledgementObservable;
port.send(new Message());
CompletableFuture<T> future = new CompletableFuture<>();
acknowledgementObservable
.doOnError(future::completeExceptionally)
.single()
.forEach(future::complete);
Then, wait for the event using timeout:
Acknowledgement ack = future.get(100, TimeUnit.MILLISECONDS);
It throws TimeoutException if timeout occurs.

You can do so by adding .timeout() and .onErrorComplete() .blockingGet()
So in this example it would be:
Acknowledgement ack = acknowledgementObservable
.firstElement()
.timeout(3, TimeUnit.SECONDS)
.onErrorComplete()
.blockingGet();
If the timeout is hit, ack will be null.

Akka actor pool for blocking requests

I am trying to use a thread pool to make blocking requests.
The problem is, each request is blocking the whole pool and items are process sequentially.
Not sure if this is even possible. Somebody please help
city-dispatcher {
type = Dispatcher
executor = "thread-pool-executor"
thread-pool-executor {
fixed-pool-size = 16
}
throughput = 100
}
And Java
Props props = Props.create(CityDataProcessorActor.class, psRespHolder).withDispatcher("akka.actor.city-dispatcher");
SmallestMailboxPool pool = new SmallestMailboxPool(10);
ActorRef cityRequestActorPool = actorSystem.actorOf(pool.props(props), "city-request-route");
for (String city : citiesArray) {
Future<Object> future = Patterns.ask(cityRequestActorPool, new CityCommand(city.trim()), timeout);
Object results = Await.result(future, duration);
log.info(results.toString());
}

As #Mon Calamari mentioned
Object results = Await.result(future, duration); is a blocking call. you can try future with callback
future onComplete{
case Success()=> println(result)
case Failure()=> println("some error")
}

Mon Calamari's comment is exactly correct. Here's an implementation. It will create a List of Futures as you create them. Then it blocks on the collected Futures sequentially to log each one. The awaits should become trivial as the iteration progresses, providing later Futures have completed in similar time.
....
Array<Future<Object>> futures = new ArrayList<>();
for (String city : citiesArray) {
Future<Object> future = Patterns.ask(cityRequestActorPool, new CityCommand(city.trim()), timeout);
futures.add(future);
}
for (<Future<Object>> f :futures){
Object results = Await.result(f, duration);
log.info(results.toString());
}

Unable to store job because one already exists with this identification

I'm new with Quartz.
I succeeded to install it and run it.
But I have an error when I run it for the second time because the job already exists with this identification.
Here my code :
public void scheduleJobs() throws Exception {
try {
int i = 0;
scheduler = new StdSchedulerFactory().getScheduler();
JobKey job1Key = JobKey.jobKey("job"+i, "my-jobs"+i);
JobDetail job1 = JobBuilder
.newJob(SimpleJob.class)
.withIdentity(job1Key)
.build();
TriggerKey tk1 = TriggerKey.triggerKey("trigger"+i, "my-jobs"+i);
Trigger trigger1 = TriggerBuilder
.newTrigger()
.withIdentity(tk1)
.withSchedule(CronScheduleBuilder.dailyAtHourAndMinute(11, 25))
.build();
scheduler.start(); // start before scheduling jobs
scheduler.scheduleJob(job1, trigger1);
i++;
printJobsAndTriggers(scheduler);
} catch (SchedulerException e) {
LOG.error("Error while creating scheduler", e);
}
}
I tried to use an integer i to change the name but it does not work.
Do you have any idea how can I fix it?
Many thanks.

You can:
check if the "job key" already exists, and remove the existing job before creating a new one:
scheduler.deleteJob(job1Key);
or create a new job with another key (in your case, each time you execute scheduleJobs(), variable i has the same value (0)
or just re-use the same job (why would you create a new job if the old one is still good)
or use the RAM Job Store, which does not persist jobs in database (each time you will use your software, you will have an empty job store)
It really depends on what you want to do with your jobs!

Check for existing job before scheduling:
JobDetail job;
SimpleTrigger trigger;
//Create your trigger and job
Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler();
scheduler.start();
if (scheduler.checkExists(job.getKey())){
scheduler.deleteJob(job.getKey());
}
scheduler.scheduleJob(job, trigger);

This is not a direct answer to the specific code listed in the question, but I didn't notice it when searching elsewhere and thought this might be useful for future readers:
If you're in a situation where you have an existing Job but just want to add a new Trigger, you can call:
scheduler.ScheduleJob(trigger);
and it will add the Trigger to the Job without trying to recreate the Job. The only trick is that you have to make sure the Trigger's JobKey is correct.
My overall code for this interaction looks roughly like:
IJobDetail job; // Handed in
ITrigger trigger; // Handed in
// Keeping track of this because we need to know later whether it's new or not
var newJob = job == null;
if (newJob)
{
job = JobBuilder.Create<TargetJob>()
.WithIdentity([whatever])
[.OtherConfiguration()]
.Build();
}
var trigger = TriggerBuilder
.Create()
.WithIdentity([whatever])
// ** Gotcha #1: Make sure it's linked to the job **
.ForJob(job.Key)
[.OtherConfiguration()]
.Build();
if (newJob)
{
_scheduler.ScheduleJob(job, trigger);
}
else
{
// ** Gotcha #2: Make sure you don't reschedule the job **
_scheduler.ScheduleJob(trigger);
}

If anyone of you are facing the same issue and your solution is in C#. This is how you can fix this error.
This is where we configure the scheduler.
public async Task StartAsync(CancellationToken cancellationToken) {
try {
var scheduler = await GetScheduler();
var serviceProvider = GetConfiguredServiceProvider();
scheduler.JobFactory = new CustomJobFactory(serviceProvider);
await scheduler.Start();
await ConfigureDailyJob(scheduler);
}
catch(Exception ex) {
_logger.Error(new CustomConfigurationException(ex.Message));
}
}
This is how we can configure the Job, please be noted that we are checking whether the job is already there, and if the await scheduler.CheckExists(dailyJob.Key) returns true, we delete that job info and create a new one with the same key.
private async Task ConfigureDailyJob(IScheduler scheduler) {
var dailyJob = GetDailyJob();
if (await scheduler.CheckExists(dailyJob.Key)) {
await scheduler.DeleteJob(dailyJob.Key);
_logger.Info($ "The job key {dailyJob.Key} was already existed, thus deleted the same");
}
await scheduler.ScheduleJob(dailyJob, GetDailyJobTrigger());
}
There are the supporting private functions.
private IJobDetail GetDailyJob() {
return JobBuilder.Create < IDailyJob > ().WithIdentity("dailyjob", "dailygroup").Build();
}
private ITrigger GetDailyJobTrigger() {
return TriggerBuilder.Create().WithIdentity("dailytrigger", "dailygroup").StartNow().WithSimpleSchedule(x = >x.WithIntervalInHours(24).RepeatForever()).Build();
}
You can get the complete source code from this GitHub repository.

You can create new jobs by taking i as a static int. And instead of "job"+i it would be "job"+ Integer.toString(i) . It worked for me.