Develop Reference

Is there a way to tell what Carrier Thread a Virtual Thread is running on?

I am playing around with Project Loom for the first time and I have some code
try (var executor = Executors.newVirtualThreadExecutor()) {
IntStream.range(0, 16).forEach(i -> {
System.out.println("i = " + i + ", Thread ID = " + Thread.currentThread());
executor.submit(() -> {
System.out.println("Thread ID = " + Thread.currentThread());
});
});
}
with output like
Thread ID = VirtualThread[#37]/runnable#ForkJoinPool-1-worker-4
Thread ID = VirtualThread[#33]/runnable#ForkJoinPool-1-worker-5
i = 9, Thread ID = Thread[#1,main,5,main]
Thread ID = VirtualThread[#43]/runnable#ForkJoinPool-1-worker-9
Thread ID = VirtualThread[#46]/runnable#ForkJoinPool-1-worker-11
i = 10, Thread ID = Thread[#1,main,5,main]
i = 11, Thread ID = Thread[#1,main,5,main]
Is there a way I can tell what Carrier Thread each Virtual Thread is running on?
Does ForkJoinPool-1-worker-11 represent a particular Carrier (Platform) Thread, or does it mean something else?

Yes, this suffix is the name of the current carrier thread.
When I use the following code
public static void main(String[] args) throws InterruptedException {
Set<String> threadStrings = ConcurrentHashMap.newKeySet();
try(var executor = Executors.newVirtualThreadPerTaskExecutor()) {
executor.invokeAll(Collections.nCopies(16,
() -> threadStrings.add(Thread.currentThread().toString())));
}
System.out.println("\tSimple Run");
threadStrings.stream().sorted().forEachOrdered(System.out::println);
threadStrings.clear();
try(var executor = Executors.newVirtualThreadPerTaskExecutor()) {
executor.invokeAll(Collections.nCopies(16, () -> {
threadStrings.add(Thread.currentThread().toString());
Thread.sleep(100);
return threadStrings.add(Thread.currentThread().toString());
}));
}
System.out.println("\tWith wait");
threadStrings.stream().sorted().forEachOrdered(System.out::println);
}
It prints
Simple Run
VirtualThread[#15]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#17]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#18]/runnable#ForkJoinPool-1-worker-3
VirtualThread[#19]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#20]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#21]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#22]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#23]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#24]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#25]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#26]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#27]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#28]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#29]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#30]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#31]/runnable#ForkJoinPool-1-worker-4
With wait
VirtualThread[#36]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#37]/runnable#ForkJoinPool-1-worker-3
VirtualThread[#37]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#38]/runnable#ForkJoinPool-1-worker-4
VirtualThread[#38]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#39]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#39]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#40]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#40]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#41]/runnable#ForkJoinPool-1-worker-6
VirtualThread[#41]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#42]/runnable#ForkJoinPool-1-worker-7
VirtualThread[#42]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#43]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#43]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#44]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#44]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#45]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#45]/runnable#ForkJoinPool-1-worker-6
VirtualThread[#46]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#46]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#47]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#49]/runnable#ForkJoinPool-1-worker-1
VirtualThread[#49]/runnable#ForkJoinPool-1-worker-8
VirtualThread[#50]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#50]/runnable#ForkJoinPool-1-worker-6
VirtualThread[#51]/runnable#ForkJoinPool-1-worker-3
VirtualThread[#51]/runnable#ForkJoinPool-1-worker-5
VirtualThread[#52]/runnable#ForkJoinPool-1-worker-2
VirtualThread[#52]/runnable#ForkJoinPool-1-worker-8
(results may vary)
demonstrating how the carrier thread might change when performing a sleep. But in the current snapshot (“build 18-loom+6-282”) it’s not possible to specify your own Executor anymore and there is no method for querying the virtual thread about the carrier thread it uses (other than the implicit hint via toString()). So, the management of the underlying host threads is mostly a black box in this version.
Keep in mind that this is an ongoing development. It’s not clear whether and how this will change.

Avoid timeout in Elasticsearch re-indexing in Java

Below code returned a timeout in client (Elasticsearch Client) when number of records are higher.
CompletableFuture<BulkByScrollResponse> future = new CompletableFuture<>();
client.reindexAsync(request, RequestOptions.DEFAULT, new ActionListener<BulkByScrollResponse>() {
#Override
public void onResponse(BulkByScrollResponse bulkByScrollResponse) {
future.complete(bulkByScrollResponse);
}
#Override
public void onFailure(Exception e) {
future.completeExceptionally(e);
}
});
BulkByScrollResponse response = future.get(10, TimeUnit.MINUTES); // client timeout occured before this timeout
Below is the client config.
connectTimeout: 60000
socketTimeout: 600000
maxRetryTimeoutMillis: 600000
Is there a way to wait indefinitely until the re-indexing complete?

submit the reindex request as a task:
TaskSubmissionResponse task = esClient.submitReindexTask(reindex, RequestOptions.DEFAULT);
acquire the task id:
TaskId taskId = new TaskId(task.getTask());
then check the task status periodically:
GetTaskRequest taskQuery = new GetTaskRequest(taskId.getNodeId(), taskId.getId());
GetTaskResponse taskStatus;
do {
Thread.sleep(TimeUnit.MINUTES.toMillis(1));
taskStatus = esClient.tasks()
.get(taskQuery, RequestOptions.DEFAULT)
.orElseThrow(() -> new IllegalStateException("Reindex task not found. id=" + taskId));
} while (!taskStatus.isCompleted());
Elasticsearch java api doc about task handling just sucks.
Ref

I don't think its a better choice to wait indefinitely to complete the re-indexing process and give very high value for timeout as this is not a proper fix and will cause more harm than good.
Instead you should examine the response, add more debugging logging to find the root-cause and address them. Also please have a look at my tips to improve re-indexing speed, which should fix some of your underlying issues.

How to pass own executors to the redis lettuce library?

I have below code to get the data from Redis asynchronously. By default get() call in lettuce library uses nio-event thread pool.
Code 1:
StatefulRedisConnection<String, String> connection = redisClient.connect();
RedisAsyncCommands<String, String> command = connection.async();
CompletionStage<String> result = command.get(id)
.thenAccept(code ->
logger.log(Level.INFO, "Thread Id " + Thread.currentThread().getName());
//Sample code to print thread ID
Thread Id printed is lettuce-nioEventLoop-6-2.
Code 2:
CompletionStage<String> result = command.get(id)
.thenAcceptAsync(code -> {
logger.log(Level.INFO, "Thread Id " + Thread.currentThread().getName());
//my original code
}, executors);
Thread Id printed is pool-1-thread-1.
My questions:
Is there a way to pass my executors?
Is it recommended approach to use nio-event thread pool to get(using get() call) the data from redis?
Lettuce version: 5.2.2.RELEASE
thanks,
Ashok

class io.lettuce.core.RedisClient has a creator method:
public static RedisClient create(ClientResources clientResources, String uri) {
assertNotNull(clientResources);
LettuceAssert.notEmpty(uri, "URI must not be empty");
return create(clientResources, RedisURI.create(uri));
}
You can build your ClientResources by ClientResources#builder(), and pass anything you want. Refer the JavaDoc, there is something you can customize:
EventLoopGroupProvider to obtain particular EventLoopGroups
EventExecutorGroup to perform internal computation tasks
Timer for scheduling
EventBus for client event dispatching
EventPublisherOptions
CommandLatencyCollector to collect latency details. Requires the HdrHistogram library.
DnsResolver to collect latency details. Requires the LatencyUtils library.
Reconnect Delay.
Tracing to trace Redis commands.

How to gracefully end test client

I have a simple cumulocity test client. The code runs fine but the jvm does not stop when the code finishes.
Some threads are still running: MultiThreadedHttpConnectionManager.
How to gracefully shutdown the open connections allocated by:
InventoryApi inventory = platform.getInventoryApi() ?
_
...
platform = new PlatformImpl(App.C8Y_URL, new CumulocityCredentials(App.C8Y_USER, App.C8Y_PWD));
InventoryApi inventory = platform.getInventoryApi();
GId testId = new GId("123456");
ManagedObjectRepresentation testDevice = inventory.get(testId);
MeasurementApi mApi = platform.getMeasurementApi();
MeasurementRepresentation measurement = new MeasurementRepresentation();
measurement.setType("c8y_SampleRate");
measurement.setSource(testDevice);
DateTime time = new DateTime();
System.out.println("time " + time.toString());
measurement.setDateTime(time);
Map<String, Object> flowRateFragment = App.createFlowRateFragment(new BigDecimal(20.5));
measurement.set(flowRateFragment, "c8y_SampleRate");
MeasurementRepresentation measurementCreation = mApi.create(measurement);
...

To gracefully shutdown your Cumulocity client call platform.close().

NO you cannot exit after the code finished. Cumulocity agent is made to run continuously so, they won't stop the thread.
If you want out exit then you have use
System.exit(0); //Change the status code accordingly.

Unable to store job because one already exists with this identification

I'm new with Quartz.
I succeeded to install it and run it.
But I have an error when I run it for the second time because the job already exists with this identification.
Here my code :
public void scheduleJobs() throws Exception {
try {
int i = 0;
scheduler = new StdSchedulerFactory().getScheduler();
JobKey job1Key = JobKey.jobKey("job"+i, "my-jobs"+i);
JobDetail job1 = JobBuilder
.newJob(SimpleJob.class)
.withIdentity(job1Key)
.build();
TriggerKey tk1 = TriggerKey.triggerKey("trigger"+i, "my-jobs"+i);
Trigger trigger1 = TriggerBuilder
.newTrigger()
.withIdentity(tk1)
.withSchedule(CronScheduleBuilder.dailyAtHourAndMinute(11, 25))
.build();
scheduler.start(); // start before scheduling jobs
scheduler.scheduleJob(job1, trigger1);
i++;
printJobsAndTriggers(scheduler);
} catch (SchedulerException e) {
LOG.error("Error while creating scheduler", e);
}
}
I tried to use an integer i to change the name but it does not work.
Do you have any idea how can I fix it?
Many thanks.

You can:
check if the "job key" already exists, and remove the existing job before creating a new one:
scheduler.deleteJob(job1Key);
or create a new job with another key (in your case, each time you execute scheduleJobs(), variable i has the same value (0)
or just re-use the same job (why would you create a new job if the old one is still good)
or use the RAM Job Store, which does not persist jobs in database (each time you will use your software, you will have an empty job store)
It really depends on what you want to do with your jobs!

Check for existing job before scheduling:
JobDetail job;
SimpleTrigger trigger;
//Create your trigger and job
Scheduler scheduler = StdSchedulerFactory.getDefaultScheduler();
scheduler.start();
if (scheduler.checkExists(job.getKey())){
scheduler.deleteJob(job.getKey());
}
scheduler.scheduleJob(job, trigger);

This is not a direct answer to the specific code listed in the question, but I didn't notice it when searching elsewhere and thought this might be useful for future readers:
If you're in a situation where you have an existing Job but just want to add a new Trigger, you can call:
scheduler.ScheduleJob(trigger);
and it will add the Trigger to the Job without trying to recreate the Job. The only trick is that you have to make sure the Trigger's JobKey is correct.
My overall code for this interaction looks roughly like:
IJobDetail job; // Handed in
ITrigger trigger; // Handed in
// Keeping track of this because we need to know later whether it's new or not
var newJob = job == null;
if (newJob)
{
job = JobBuilder.Create<TargetJob>()
.WithIdentity([whatever])
[.OtherConfiguration()]
.Build();
}
var trigger = TriggerBuilder
.Create()
.WithIdentity([whatever])
// ** Gotcha #1: Make sure it's linked to the job **
.ForJob(job.Key)
[.OtherConfiguration()]
.Build();
if (newJob)
{
_scheduler.ScheduleJob(job, trigger);
}
else
{
// ** Gotcha #2: Make sure you don't reschedule the job **
_scheduler.ScheduleJob(trigger);
}

If anyone of you are facing the same issue and your solution is in C#. This is how you can fix this error.
This is where we configure the scheduler.
public async Task StartAsync(CancellationToken cancellationToken) {
try {
var scheduler = await GetScheduler();
var serviceProvider = GetConfiguredServiceProvider();
scheduler.JobFactory = new CustomJobFactory(serviceProvider);
await scheduler.Start();
await ConfigureDailyJob(scheduler);
}
catch(Exception ex) {
_logger.Error(new CustomConfigurationException(ex.Message));
}
}
This is how we can configure the Job, please be noted that we are checking whether the job is already there, and if the await scheduler.CheckExists(dailyJob.Key) returns true, we delete that job info and create a new one with the same key.
private async Task ConfigureDailyJob(IScheduler scheduler) {
var dailyJob = GetDailyJob();
if (await scheduler.CheckExists(dailyJob.Key)) {
await scheduler.DeleteJob(dailyJob.Key);
_logger.Info($ "The job key {dailyJob.Key} was already existed, thus deleted the same");
}
await scheduler.ScheduleJob(dailyJob, GetDailyJobTrigger());
}
There are the supporting private functions.
private IJobDetail GetDailyJob() {
return JobBuilder.Create < IDailyJob > ().WithIdentity("dailyjob", "dailygroup").Build();
}
private ITrigger GetDailyJobTrigger() {
return TriggerBuilder.Create().WithIdentity("dailytrigger", "dailygroup").StartNow().WithSimpleSchedule(x = >x.WithIntervalInHours(24).RepeatForever()).Build();
}
You can get the complete source code from this GitHub repository.

You can create new jobs by taking i as a static int. And instead of "job"+i it would be "job"+ Integer.toString(i) . It worked for me.