I would like to use ExecutorService in JAVA to schedule the insertion into database every 5 min.
this the task that i want to execute it:
MyClass{
a counter that count a handled data received from a Thread
I receive usually 300 data line/second
Apply a treatment on these data and keep the results in the memory
every five minutes {
update the database with the counters saved in memory*
}
}
Basically it's calling the task each time he get a data from thread running in the background.
and as i have more then 300 data/sec it's impossible to use it in this way.
So what i am trying to do id to handle the received tasks and keep a counter in the memory and update the database only each 5 min.
My question, is it possible to use these java function ScheduledExecutorService to do this, and how can we do it ( i don't want to block my JAVA application on the task for 5min, i want that that application run normally but without executing the task every time, i appreciate if you can show me an example of usage for these function ?
ScheduledExecutorService service = Executors.newScheduledThreadPool(1);
service.scheduleAtFixedRate(command, 5, 5, TimeUnit.MINUTES);
Where command is:
Runnable command = new Runnable() {
public void run() {
// update database
}
}
Related
I am trying to use fixed thread pool using executor framework. Each runnable instance submitted to the executor is worker thread which process a java result set. For every iteration of the result set I have to call rest webservice which used the oauth token. The oauth token is to be refreshed after every 50 min and need to be shared among all the runnable submitted to the executor.I am also using scheduled executor which execute after every 50 minutes but sometimes it is called correctly and some time not due to which the rest web service fails as it used the expired token in its header. I need to ensure that the scheduled service must be called after every 50 min without fail. But I am stuck on this. Also I need some mechanism that after the group of rest web service call is completed then only the new web service calls should be made while iterating the result set.
ThreadPoolExecutor executorPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(2);
WorkerThread wt1=new WorkerThread(conn,Queries.getAddressInfo("AL"),oauth_token,restTemplate);
WorkerThread wt2=new WorkerThread(conn,Queries.getAddressInfo("AK"),oauth_token,restTemplate);
executorPool.execute(wt1);
executorPool.execute(wt2);
ScheduledFuture scheduledFuture =
scheduledExecutorService.schedule(new Runnable() {
public void run() {
System.out.println("token service");
String url="";
try {
url = WebServicePropertyFileReader.getOauthUrl()+String.format(urlToGetOauthToken, WebServicePropertyFileReader.getClientId(),
WebServicePropertyFileReader.getClientSecret());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Layer7Token token=restTemplate.postForObject(url, null, Layer7Token.class);
GlobalTokenAccessor.oauth_token=token.getAccessToken();
}
},
50,
TimeUnit.MINUTES);
There are two issues :
Schedule schedules it only one time.
You should use scheduleAtFixedRate to schedule periodic tasks. There is no guarantee that the thread will get scheduled within 50 minutes. So, you may want to schedule the refresh every 49 minutes or so.
Secondly, you should control the access to the shared variable though a synchronized write and read methods. Otherwise, the read thread may read incorrect values.
I am using Aerospike AsyncClient and Aerospike UDF module to insert 10 Million record in Aerospike 2 node cluster using multiple thread. Steps that i follow are :
Created a executor service of 10 threads and all threads will use same AsyncClient
Run a loop 10 Million time and use below code to invoke execute command
for(int 1-0; i < 10000000; i++) {
final int j = i;
executorService.execute(new Runnable() {
public void run() {
put("test", "binname", "setname", j, list1, list2, aerospikeClient);
}
});
}
private void put(String namespace, String setName, String binName,
String keyVlaue,
List campaignsIdsToBeAdd, List campaignsIdToBeRemoved, AsyncClient aerospikeClient) {
Key key = new Key(namespace, setName, keyVlaue);
aerospikeClient.execute(writePolicy, new WriteHandler(), key, "aerospike_udf", "update_record", Value.get(campaignsIdsToBeAdd),
Value.get(campaignsIdToBeRemoved), Value.get(binName));
}
once executor exit i am calling close method
public void closed() {
System.out.println("=== 1");
if(aerospikeClient != null && aerospikeClient.isConnected())
aerospikeClient.close();
if(executorService != null && !executorService.isShutdown())
executorService.shutdown();
}
executor service and AsyncClient stop immediately and after that there are no insertions happen and i am loosing submitted data.
Can any one help me to solve this? or suggest me the better way to use AsyncClient along with Aerospike udf function.
I tried to do this using sync client but it's took 70 min to insert all value but i want to complete it in 5 min. And when i tried to use Sync client between multiple thread it's throwing client timeout exception
Seems you are running udfs which are slower than regular read/write operations. So, even if you wait for the async calls to finish, you may end up taking more than 5 mins. You need to size the cluster to meet your performance needs. So, you need to look at non programming alternatives too.
Having more nodes will help in executing the udfs in parallel, reducing the overall time.
See the performance if you implement the logic of udf in the application. Sever will have less work as some computation cost is moved to the clients.
I run a jar with an embedded Jetty. From time to time it happens that one request get stuck in some endless loop. Obviously fixing the endless-loop would be the best option. However, this is currently not possible.
So I am looking for an option, that checks if a request exists for more than e.g. 5 minutes, and kills the corresponding thread.
I tried the typical Jetty options:
maxIdleTime
soLingerTime
stopTimeout
None of them worked as expected. Is there another option to consider?
Do you access to the code that kicks of the code which takes too long to complete? If so you can use callable and an Executor to achieve this yourself, below is a unit test with an example:
#Test
public void timerTest() throws Exception
{
//create an executor
ExecutorService executor = Executors.newFixedThreadPool(10);
//some code to run
Callable callable = () -> {
Thread.sleep(10000); //sleep for 10 seconds
return 123;
};
//run the callable code
Future<Integer> future = (Future<Integer>) executor.submit(callable);
Integer value = future.get(5000, TimeUnit.MILLISECONDS); //this will timeout after 5 seconds
//kill the thread
future.cancel(true);
}
In jave i have a state machine which i need to migrate to automated state changes, in this i mean i need state changes to occur after certan intervals.. for instance after 5 seconds state one, after 10 seconds state 2.
I was thinking of using ;
ScheduledExecutorService executor = new ScheduledThreadPoolExecutor(1);
Schedule a task that basically calls a method, which would change state and then schedule another task to change to next state, and so on.
Any ideas?
Need code similar to this:
ScheduledThreadPoolExecutor exec = new ScheduledThreadPoolExecutor(1);
exec.scheduleWithFixedDelay(new Runnable() {
public void run() {
// TODO: do something
}
},
0, // no wait and start the 1st one
5, // delay 5 seconds and do the next one
TimeUnit.SECONDS);
You could also look into scheduleAtFixedRate().
I am using the JavaMail API , and there is a method in the Folder class called "search" that sometimes take too long to execute. What i want is to execute this method over a maximum period of time( say for example 15 seconds in maximum) , that way i am sure that this method will not run up more than 15 seconds.
Pseudo Code
messages = maximumMethod(Folder.search(),15);
Do I have to create a thread just to execute this method and in the main thread use the wait method ?
The best way to do this is create a single threaded executor which you can submit callables with. The return value is a Future<?> which you can get the results from. You can also say wait this long to get the results. Here is sample code:
ExecutorService service = Executors.newSingleThreadExecutor();
Future<Message[]> future = service.submit(new Callable<Message[]>() {
#Override
public Message[] call() throws Exception {
return Folder.search(/*...*/);
}
});
try {
Message[] messages = future.get(15, TimeUnit.SECONDS);
}
catch(TimeoutException e) {
// timeout
}
You could
mark current time
launch a thread that will search in the folder
while you get the result (still in thread) don't do anything if current time exceeds time obtained in 1 plus 15 seconds. You won't be able to stop the connection if it is pending but you could just disgard a late result.
Also, if you have access to the socket used to search the folder, you could set its timeout but I fear it's gonna be fully encapsulated by javamail.
Regards,
Stéphane
This SO question shows how to send a timeout exception to the client code: How do I call some blocking method with a timeout in Java?
You might be able to interrupt the actual search using Thread.interrupt(), but that depends on the method's implementation. You may end up completing the action only to discard the results.