Good evening Java wizards,
I am fairly new to Java, and would like to be educated on the mistake I am not able to resolve after some hours of research.
Assume a program that has the main thread with some variables that could dynamically read/lookup value in a Map of values. This Map is shared by threads and objects accross the application.
There would be one or more separate threads updating the Map with values thus the map will be synchronized - I am considering ConcurrentHashMap.
When the reading thread arrives to one of the variables with the dynamic values it will reach in to the cross-thread shared Map and retreive its latest value against its key.
I have some prototype code here - it compliles and seam to be running as expected. However it is set to work only with a String variable and values.
Here is the code:
<code>
/*
* Cross thread / cross object shared Map
* some threads update the map
* other threads read the map for most current values (updated by other threads)
* associated with the provided key
*/
public class CrossThreadUpdatableValues {
private static final Map<Integer, String> list = new ConcurrentHashMap<>();
// return the last updated value
// from the map
public static String getValue(Integer key) {
return list.get(key);
}
// update/write the passed in value in to the map
// (in to the element having the given key)
public static void setValue(Integer key, String ev) {
list.put(key, ev);
}
}
/*
* Thread 1
* 10 loops that update the cross thread / cross object shared Map
* with unique values
*/
public class Thread1 extends Thread {
private final int delay;
private final int threadNum = 1;
Thread1(int delay) {
this.delay = delay;
}
public void run(){
for (int i=0;i<10;i++) {
String v1 = threadNum + "-AAA"; Integer key1 = 1;
String v2 = threadNum + "-BBB"; Integer key2 = 2;
String v3 = threadNum + "-CCC"; Integer key3 = 3;
CrossThreadUpdatableValues.setValue(key1, v1);
CrossThreadUpdatableValues.setValue(key2, v2);
CrossThreadUpdatableValues.setValue(key3, v3);
System.out.println("Map values updated by Thread " + threadNum + ", loop " + i);
try {
Thread.sleep(this.delay);
} catch (InterruptedException ex) {
Logger.getLogger(Thread2.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
/*
* Thread 2 (similar to Thread 1)
* 10 loops that update the cross thread / cross object shared Map
* with unique values
*/
public class Thread2 extends Thread {
private final int delay;
private final int threadNum = 2;
Thread2(int delay) {
this.delay = delay;
}
public void run(){
for (int i=0;i<10;i++) {
String v1 = threadNum + "-XXX"; Integer key1 = 1;
String v2 = threadNum + "-YYY"; Integer key2 = 2;
String v3 = threadNum + "-ZZZ"; Integer key3 = 3;
CrossThreadUpdatableValues.setValue(key1, v1);
CrossThreadUpdatableValues.setValue(key2, v2);
CrossThreadUpdatableValues.setValue(key3, v3);
System.out.println("Map values updated by Thread " + threadNum + ", loop " + i);
try {
Thread.sleep(this.delay);
} catch (InterruptedException ex) {
Logger.getLogger(Thread2.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
/*
* Reading thread -
* 20 loops that read the cross thread / crooss object shared Map
* for the most current values updated by other threads in various intervals
*/
public class ThreadRead extends Thread {
private final int delay;
private final int threadNum = 0;
ThreadRead(int delay) {
this.delay = delay;
}
public void run(){
Integer key1 = 1;
Integer key2 = 2;
Integer key3 = 3;
for (int i=0;i<20;i++) {
String v1 = CrossThreadUpdatableValues.getValue(key1);
String v2 = CrossThreadUpdatableValues.getValue(key1);
String v3 = CrossThreadUpdatableValues.getValue(key1);
System.out.println(" - - - Map values read by (reading) thread " + threadNum + ", loop " + i + "; v1 = " + v2 + "; v1 = " + v2 + "; v3 = " + v3);
try {
Thread.sleep(this.delay);
} catch (InterruptedException ex) {
Logger.getLogger(Thread2.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
/**
*
* Main test class - start of test run
*/
public class Test_Main {
public static void main(String[] args) {
// start thread that populates the shared Map with unique values
// in 5 second intervals (10x)
Thread1 thread1 = new Thread1(5000);
thread1.start();
// start thread that populates the shared Map with unique values
// in 10 second intervals (10x)
Thread2 thread2 = new Thread2(10000);
thread2.start();
// start thread that reads the shared Map of unique values
// - the latest updates from any previous thread
// in 5 second intervals (20x)
ThreadRead threadRead = new ThreadRead(5000);
threadRead.start();
}
}
</code>
These dynamic variables will naturally be of different types (Integers, Strings, etc.), so I am considering using generics, BUT that gives me the uneducated headache. As the Map needs to be shared between all involved classes and threads, it needs to be declared static and Java won't permit the use of generics on this static Map.
Here is the modification of above class CrossThreadUpdatableValues using generics - that will NOT work but a hint what I am trying to achieve:
<code>
/*
* Cross thread / crooss object shared Map
* some threads update the map
* other threads read the map for most current values (udated by other threads)
* associated with the provided key
*/
public class CrossThreadUpdatableValues<K, V> {
private static final Map<K, V> list = new ConcurrentHashMap<>();
// return the last updated value
// from the map
public static V getValue(K key) {
return list.get(key);
}
// update/write the passed in value in to the map
// (in to the element having the given key)
public static void setValue(K key, V v) {
list.put(key, v);
}
}
</code>
I would appreciate your input into how to approach this in a thread save manner, allow handling various types of variables (I know Object could be used instead of V but is it the right way to go?) and perhaps point out some hints or references on a solution or a better approach.
Thank you
If you want to save different types in the Map you will need to use Object, using generics will force a design where a specific map is created for each combination of <K, V>.
If you want to store in the same map differents kind of objects, you don´t need generics, declare your map as
Map <String, Object>
Otherwise, you can use an interface like this
package a;
public interface ICrossThreadUpdatableValues<K, V> {
// return the last updated value
// from the map
V getValue(K key);
// update/write the passed in value in to the map
// (in to the element having the given key)
void setValue(K key, V v);
}
And then do a concrete implementation for the desired types like this
package a;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/*
* Cross thread / crooss object shared Map
* some threads update the map
* other threads read the map for most current values (udated by other threads)
* associated with the provided key
*/
public class IntegerCrossThreadUpdatableValues implements ICrossThreadUpdatableValues<String, Integer> {
private final Map<String, Integer> list = new ConcurrentHashMap<>();
private static IntegerCrossThreadUpdatableValues instance;
private IntegerCrossThreadUpdatableValues() {
}
// return the last updated value
// from the map
public Integer getValue(String key) {
return list.get(key);
}
// update/write the passed in value in to the map
// (in to the element having the given key)
public void setValue(String key, Integer v) {
list.put(key, v);
}
public static IntegerCrossThreadUpdatableValues getInstance() {
if (instance == null) {
instance = new IntegerCrossThreadUpdatableValues();
}
return instance;
}
}
Note that the implementation defines a singleton pattern, you can´t instantiate the class outside of it and you can get always the same object calling 'getInstance'
Related
This code is part of within a method. The code go through two lists using two for loop. I want to see whether there is a possibility of using multi thread to speed up this process for the two loops. My concern is how to make it thread safe.
EDITTED: more complete code
static class Similarity {
double similarity;
String seedWord;
String candidateWord;
public Similarity(double similarity, String seedWord, String candidateWord) {
this.similarity = similarity;
this.seedWord = seedWord;
this.candidateWord = candidateWord;
}
public double getSimilarity() {
return similarity;
}
public String getSeedWord() {
return seedWord;
}
public String getCandidateWord() {
return candidateWord;
}
}
static class SimilarityTask implements Callable<Similarity> {
Word2Vec vectors;
String seedWord;
String candidateWord;
Collection<String> label1;
Collection<String> label2;
public SimilarityTask(Word2Vec vectors, String seedWord, String candidateWord, Collection<String> label1, Collection<String> label2) {
this.vectors = vectors;
this.seedWord = seedWord;
this.candidateWord = candidateWord;
this.label1 = label1;
this.label2 = label2;
}
#Override
public Similarity call() {
double similarity = cosineSimForSentence(vectors, label1, label2);
return new Similarity(similarity, seedWord, candidateWord);
}
}
Now, is this 'compute' thread safe? There are 3 variables involved:
1) vectors;
2) toeknizerFactory;
3) similarities;
public static void compute() throws Exception {
File modelFile = new File("sim.bin");
Word2Vec vectors = WordVectorSerializer.readWord2VecModel(modelFile);
TokenizerFactory tokenizerFactory = new TokenizerFactory()
List<String> seedList = loadSeeds();
List<String> candidateList = loadCandidates();
log.info("Computing similarity: ");
ExecutorService POOL = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
List<Future<Similarity>> tasks = new ArrayList<>();
int totalCount=0;
for (String seed : seedList) {
Collection<String> label1 = getTokens(seed.trim(), tokenizerFactory);
if (label1.isEmpty()) {
continue;
}
for (String candidate : candidateList) {
Collection<String> label2 = getTokens(candidate.trim(), tokenizerFactory);
if (label2.isEmpty()) {
continue;
}
Callable<Similarity> callable = new SimilarityTask(vectors, seed, candidate, label1, label2);
tasks.add(POOL.submit(callable));
log.info("TotalCount:" + (++totalCount));
}
}
Map<String, Set<String>> similarities = new HashMap<>();
int validCount = 0;
for (Future<Similarity> task : tasks) {
Similarity simi = task.get();
Double similarity = simi.getSimilarity();
String seedWord = simi.getSeedWord();
String candidateWord = simi.getCandidateWord();
Set<String> similarityWords = similarities.get(seedWord);
if (similarity >= 0.85) {
if (similarityWords == null) {
similarityWords = new HashSet<>();
}
similarityWords.add(candidateWord);
log.info(seedWord + " " + similarity + " " + candidateWord);
log.info("ValidCount: " + (++validCount));
}
if (similarityWords != null) {
similarities.put(seedWord, similarityWords);
}
}
}
Added one more relevant method, which is used by the call() method:
public static double cosineSimForSentence(Word2Vec vectors, Collection<String> label1, Collection<String> label2) {
try {
return Transforms.cosineSim(vectors.getWordVectorsMean(label1), vector.getWordVectorsMean(label2));
} catch (Exception e) {
log.warn("OOV: " + label1.toString() + " " + label2.toString());
//e.getMessage();
//e.printStackTrace();
return 0.0;
}
}
(Answer updated for changed question.)
In general you should profile the code before attempting to optimise it, particularly if it is quite complex.
For threading you need to identify which mutable state is shared between threads. Ideally as much as that as possible before resorting to locks and concurrent data structures. Mutable state that is contained within one thread isn't a problem as such. Immutables are great.
I assume nothing passed to your task gets modified. It's tricky to tell. final on fields is a good idea. Collections can be placed in unmodifiable wrappers, though that doesn't stop them being modified via other references and does now show itself in static types.
Assuming you don't break up the inner loop, the only shared mutable state appears to be similarities and the values it contains.
You may or may not find you still end up doing too much serially and need to change similarities to become concurrent
ConcurrentMap<String, Set<String>> similarities = new ConcurrentHashMap<>();
The get and put of similarities will need to be thread-safe. I suggest always creating the Set.
Set<String> similarityWords = similarities.getOrDefault(seed, new HashSet<>());
or
Set<String> similarityWords = similarities.computeIfAbsent(seed, key -> new HashSet<>());
You could use a thread-safe Set (for instance with Collections.synchronizedSet), but I suggest holding a relevant lock for the entire inner loop.
synchronized (similarityWords) {
...
}
If you wanted to create similarityWords lazily then it would be "more fun".
I am new in Java concurrency, I have a class holding data (doubles in the sample code below) that should be accessed with a get (like a Map), but with data stored internally in a array for performance reasons.
This run in a multithreaded environment and this index must be updated sometimes.
public class ConcurrencySampleCode {
private static Object lock = new Object();
private Map<String, Integer> map = ...
private double[] array = ...
public Double get(String id) {
synchronized (lock) {
Integer i = map.get(id);
if (i == null) {
return null;
}
return array[i];
}
}
public void update() {
Map<String, Integer> tmpMap = updateMap(...);
double[] tmpArray = updateArray(...);
synchronized (lock) { // should be atomic
map = tmpMap;
array = tmpArray;
}
}
}
I am not sure whether this code is correct or not? Also, is the synchronized keyword needed in the get function ?
Is there a better way of doing this ?
Thanks for your help
There's nothing wrong with your code, but you will need to use the volatile keyword on the map and the array to ensure all threads see the updated values immediately, and I'm not sure you want the lock to be static.
As an alternative you may want to check out the java.util.concurrent.atomic package. It has some handy thread-safe variable. For example you could move your map and array into their own class, then use the AtomicReference to store the object.
public class ConcurrencySampleCode {
private AtomicReference<DoubleMap> atomicMap = new AtomicReference(new DoubleMap());
//Inner class used to hold the map and array pair
public class DoubleMap {
private Map<String, Integer> map = ...
private double[] array = ...
}
public Double get(String id) {
DoubleMap map = atomicMap.get();
...
}
public void update() {
Map<String, Integer> tmpMap = updateMap(...);
double[] tmpArray = updateArray(...);
DoubleMap newMap = new DoubleMap(tmpMap, tmpArray);
atomicMap.set(newMap);
}
}
There is a lot going on in concurrent programming, but for instance your update() method is faulty. In the current state multiple Threads can call ConcurrencySampleCode.update() and every each one of them will initiate both update calls inside the body before the synchronization kicks in. This means that after the round-robin turnover the last Thread with the update call will not have the changes from the previous update calls in the newly update map and array.
Long story, try to use and understand the ConcurrentHashMap
I have a Record class which extends a DoctorRecord class. I want threads to store these records in a hash map which I create as a singleton. My problem is every time a thread adds a record to the hash map and I try to print it's size in the main method (once all the threads have finished executing), I always get a size of 0, as if no modifications were made. However, when I print the size of the map within the method adding the records, it shows the correct size. I'm assuming it's an issue with a pass-by-value mechanic but I can't seem to correct the problem.
Hash Map singleton class:
public class MtlHashMap {
private static HashMap<Character, ArrayList<Record>> hmMtl = new HashMap<Character, ArrayList<Record>>();
public static HashMap<Character, ArrayList<Record>> getInstance(){
return hmMtl;
}
}
Thread method being called inside the overridden run() method (The hash map contains one list for every letter of the alphabet)
public synchronized void createDRecord(String firstName, String lastName, String address, long phone, String specialization, String location){
System.out.println("Create D Record");
DoctorRecord d1 = new DoctorRecord(firstName, lastName, address, phone, specialization, location);
hm = MtlHashMap.getInstance();
Character ch = lastName.toLowerCase().charAt(0);
ArrayList<Record> list = (ArrayList<Record>) hm.get(ch);
if(list == null){
System.out.println("Create list");
list = new ArrayList<Record>();
}
list.add(d1);
System.out.println("Size of list " + ch + " is " + list.size());
hm.put(ch,list);
System.out.println("Size of " + location + "hash map is " + hm.size());
}
Main
public class Main {
public static void main(String[] args) throws InterruptedException{
System.out.println("Main");
ManagerClient m1 = new ManagerClient("mtl");
ManagerClient m2 = new ManagerClient("mtl");
ManagerClient m3 = new ManagerClient("mtl");
ManagerClient m4 = new ManagerClient("mtl");
//Create Doctor Record
m1.run("Joana", "Sam", "272 Montpellier", 1231, "surgeon", "mtl");
m2.run("Joana", "Sam", "34 Lake", 1231, "surgeon", "mtl");
m4.run("Joana", "Sam", "34 Lake", 1231, "surgeon", "mtl");
m1.join();
m2.join();
m4.join();
System.out.println("Size of MTL hash map is: " + MtlHashMap.getInstance().size());
System.out.println("Size of LVL hash map is: " + LvlHashMap.getInstance().size());
System.out.println("Size of DDO hash map is: " + DdoHashMap.getInstance().size());
Manager Client Class
public class ManagerClient extends Thread{
private String location;
private String id;
private static int managerIdCounter = 1000; //Maintains unique global IDs
public ManagerClient(String location){
this.location = location.toLowerCase();
this.id = location.toLowerCase()+managerIdCounter++;
}
public String getLocation(){
return (this.location).toLowerCase();
}
public String getmId(){
return this.id;
}
//Different run method overloads for each of the four methods needed,
//with the appropriate server being called using the locateServer() method
//Default run method never used, but must be overridden anyway
public void run(){
System.out.println("This should never appear");
}
//Create Doctor Record (5 String and 1 long argument) SYNCHRONIZE THIS FOR SAFE RUNNING
#SuppressWarnings("deprecation")
public synchronized void run(String firstName, String lastName, String address, long phone, String specialization, String location){
System.out.println("Manager " + this.getmId() + " creates a D record");
try{
System.setSecurityManager(new RMISecurityManager());
String path = "rmi://localhost:2020/"+getLocation();
ClinicServerInterface server = (ClinicServerInterface)Naming.lookup(path);
server.createDRecord(firstName, lastName, address, phone, specialization, location);
}catch(Exception e){
//e.printStackTrace();
}
}
//Create Nurse Record (6 String arguments)
public synchronized void run(String firstName, String lastName, String designation, String status, String statusDate){
System.out.println("Manager " + this.getmId() + " creates a N record");
try{
System.setSecurityManager(new RMISecurityManager());
String path = "rmi://localhost:2020/"+getLocation();
ClinicServerInterface server = (ClinicServerInterface)Naming.lookup(path);
server.createNRecord(firstName, lastName, designation, status, statusDate, getLocation());
}catch(Exception e){
//e.printStackTrace();
}
}
//Get Record Counts (1 int argument)
public void run(int type){
String location = this.location;
}
//Edit Record (3 String arguments)
public void run(String recrodID, String fieldName, String newValue){
String location = this.location;
}
}
I don't know what's up with your hash map, but your program does not create any threads. Your main() routine does this:
public static void main(String[] args) throws InterruptedException {
...
ManagerClient m1 = new ManagerClient("mtl");
...
m1.run("Joana", "Sam", "272 Montpellier", 1231, "surgeon", "mtl");
...
m1.join();
...
}
That does not create a thread. That creates a ManagerClient object, and a ManagerClient object is a kind of Thread object; but a Thread is not a thread.
A thread is an operating system object that executes your code, and a Thread object is a Java object that your program uses to create an manage the life cycle of a thread. The thread will not be created until your program calls m1.start().
If you change your program to call m1.start(), here is what will happen: Your program will print This should never appear.
That's because your ManagerClient class overrides Thread.run() as follows:
//Default run method never used, but must be overridden anyway
public void run(){
System.out.println("This should never appear");
}
Your ManagerClient class also defines some other methods named "run". E.g.:
public synchronized void run(String firstName, String lastName, String address, long phone, String specialization, String location)
But that is a different method. If your program calls m1.start() to start the new thread, the new thread will call m1.run() and NOT m1.run("Joana", "Sam", "272 Montpellier", ...).
There's nothing you can do to change that. It's just how Thread.start() works.
The normal way to pass arguments to a thread that you explicitly create and start like that is to pass them through the constructor. E.g.;
ManagerClient m1 = new ManagerClient("Joana", "Sam", "272 Montpillier", ...);
m1.start();
But you might want to reconsider creating and starting threads in that way.
Your run(...) methods each just do one thing and then return. A thread dies when it's run() method returns. Creating and killing threads to perform simple tasks is a Bad Idea. Not that it will cause any harm in a toy program like your assignment, but in real-world software, the good idea that you should be using instead is called a thread pool.
You can learn about thread pools by reading up on ExecutorService, and ThreadPoolExecutor, and Executors.newFixedThreadPool(). And, by checking out the Java Concurrency Tutorial (http://docs.oracle.com/javase/tutorial/essential/concurrency/).
As it says in the Javadoc of HashMap:
Note that this implementation is not synchronized. If multiple threads access a hash map concurrently, and at least one of the threads modifies the map structurally, it must be synchronized externally.
But, he protests, the method is synchronized. Well, yes, it is, but not in a useful way.
If public synchronized void createDRecord(...) is a method inside the ManagerClient class, then the monitor on which you are synchronizing is the ManagerClient instance - and you have 4 of those. So whilst monitors are being acquired and released, nothing else is contending them, so it is like the synchronization is not there.
You should either use a synchronized block inside the method to lock on the hash map itself:
public void createDRecord(...) {
HashMap<Character, ArrayList<Record>> hm = MtlHashMap.getInstance();
synchronized (hm) {
// Mutually exclusive access to hm goes here.
}
}
or use a ConcurrentHashMap, like #MickMnemonic suggested in his comment.
You should also make the static hash map final, in order to guarantee that its initialized value is visible in all threads:
private static final HashMap<Character, ArrayList<Record>> hmMtl = new HashMap<>();
^
I'm new to spark, and was trying to run the example JavaSparkPi.java, it runs well, but because i have to use this in another java s I copy all things from main to a method in the class and try to call the method in main, it saids
org.apache.spark.SparkException: Job aborted: Task not serializable:
java.io.NotSerializableException
the code looks like this:
public class JavaSparkPi {
public void cal(){
JavaSparkContext jsc = new JavaSparkContext("local", "JavaLogQuery");
int slices = 2;
int n = 100000 * slices;
List<Integer> l = new ArrayList<Integer>(n);
for (int i = 0; i < n; i++) {
l.add(i);
}
JavaRDD<Integer> dataSet = jsc.parallelize(l, slices);
System.out.println("count is: "+ dataSet.count());
dataSet.foreach(new VoidFunction<Integer>(){
public void call(Integer i){
System.out.println(i);
}
});
int count = dataSet.map(new Function<Integer, Integer>() {
#Override
public Integer call(Integer integer) throws Exception {
double x = Math.random() * 2 - 1;
double y = Math.random() * 2 - 1;
return (x * x + y * y < 1) ? 1 : 0;
}
}).reduce(new Function2<Integer, Integer, Integer>() {
#Override
public Integer call(Integer integer, Integer integer2) throws Exception {
return integer + integer2;
}
});
System.out.println("Pi is roughly " + 4.0 * count / n);
}
public static void main(String[] args) throws Exception {
JavaSparkPi myClass = new JavaSparkPi();
myClass.cal();
}
}
anyone have idea on this? thanks!
The nested functions hold a reference to the containing object (JavaSparkPi). So this object will get serialized. For this to work, it needs to be serializable. Simple to do:
public class JavaSparkPi implements Serializable {
...
The main problem is that when you create an Anonymous Class in java it is passed a reference of the enclosing class.
This can be fixed in many ways
Declare the enclosing class Serializable
This works in your case but will fall flat in case your enclosing class has some field that is not serializable. I would also say that serializing the parent class is a total waste.
Create the Closure in a static function
Creating the closure by invoking some static function doesn't pass the reference to the closure and hence no need to make serializable this way.
This error comes because you have multiple physical CPUs in your local or cluster and spark engine try to send this function to multiple CPUs over network.
Your function
dataSet.foreach(new VoidFunction<Integer>(){
public void call(Integer i){
***System.out.println(i);***
}
});
uses println() which is not serialize. So the exception thrown by Spark Engine.
The solution is you can use below:
dataSet.collect().forEach(new VoidFunction<Integer>(){
public void call(Integer i){
System.out.println(i);
}
});
It is about very common sensor data processing problem.
To synchronize and merge sensor data from different sources, I would like to implement it in Java without too complicated 3rd libs or framework.
Say, I define an object (O) which consists of, for example, 4 attributes (A1,..A4). The 4 attributes come from different data channels, e.g. socket channel.
The 4 attributes arrive generally in a rate of 1.0 ~ 2.0 Hz and their arrivals are independent from each other.
Once there are 4 attributes (A1, ..A4) coming at the same time (within a small time window, e.g. 100ms), then I construct a new object (O) from those 4 attributes.
a descriptive scenario is as follows.
the arrival time point of A1 ~ A4 is marked with *.
Objects O1 ~ U3 are constructed on the time point of t1, t2 and t3 respectively.
Some attributes arrives between t2 and t3, but are not complete for constructing an Object, therefore they
would be dropped and ignored.
A1 * * * *
A2 * * * *
A3 * * *
A4 * * * *
--------|------------|-----------------|----------> time
t1 t2 t3
O1 O2 O3
some requirements:
identify the time point a.s.a.p. to construct a object from the last incoming 4 attributes.
FIFO, O1 must be constructed before O2, and so on.
less locking in Java
drop data eventually if they are not complete to construct a object.
Some quick idea on implementation are:
store any incoming attributes in a FIFO queue of time-discrete buckets (each bucket contains 4 different attributes).
run an endless thread concurrently to check the FIFO queue (from the head of the queue) if any bucket is already filled with 4 different attributes. If yes, then construct an object and remove the bucket from the queue. If a bucket is not complete filled within a specific time window, it will be dropped.
any suggestion and correction is welcome!
This is unlikely to solve your problem, but it might point you in the right direction.
I would use Google Guava's MapMaker for a first attempt:
ConcurrentMap<Key, Bucket> graphs = new MapMaker()
.expireAfterAccess(100, TimeUnit.MILLISECOND)
.makeComputingMap(new Function<Key, Bucket>() {
public Bucket apply(Key key) {
return new Bucket(key);
}
});
This would create a map whose entries would disappear if they had not been accessed for 100 ms, and creates a new bucket when it is asked for.
What I can't work out is exactly what the Key would be :S What you're really after is the same kind of functionality in the form of a queue.
Here's another crazy idea:
use one single LinkedBlockingQueue to write values to from all sensors A1-A4
assign this queue to AtomicReference variable
create a timer task which will switch this queue with a new one at specified intervals (100ms)
fetch all data from the old queue and see if you have all data A1-A4
if yes, then create the object, otherwise drop everything
This is another way of doing it - it's just pseudocode though, you'll need to write it yourself :)
class SlidingWindow {
AtomicReference<Object> a1;
AtomicReference<Object> a2;
AtomicReference<Object> a3;
AtomicReference<Object> a4;
Queue<Long> arrivalTimes = new Queue(4);
public Bucket setA1(Object data) {
a1.set(data);
now = System.currentTimeInMillis()
long oldestArrivalTime = arrivalTimes.pop();
arrivalTimes.push(now);
if (now - oldestArrivalTime < 100) {
return buildBucket();
}
return null;
}
public Bucket setA2(Object data) { ...
...
private Bucket buildBucket() {
Bucket b = new Bucket(a1, a2, a3, a4);
a1.clear();
a2.clear();
a3.clear();
a4.clear();
return b;
}
}
You could do something like this, the get operation is blocking till data has arrived, the add operation is not blocking. The get operation could be optimized a bit so that you keep candidates in a paralell structure so that you don't need to iterate over all candidates when filtering out old items. Iterating over 4 items should however be fast enough.
import java.util.HashMap;
import java.util.Iterator;
import java.util.concurrent.LinkedBlockingQueue;
public class Filter<V> {
private static final long MAX_AGE_IN_MS = 100;
private final int numberOfSources;
private final LinkedBlockingQueue<Item> values = new LinkedBlockingQueue<Item>();
public Filter(int numberOfSources) {
this.numberOfSources = numberOfSources;
}
public void add(String source, V data) {
values.add(new Item(source, data));
}
public void get() throws InterruptedException {
HashMap<String, Item> result = new HashMap<String, Item>();
while (true) {
while (result.size() < numberOfSources) {
Item i = values.take();
result.put(i.source, i);
if (result.size() == numberOfSources) {
break;
}
}
//We got candidates from each source now, check if some are too old.
long now = System.currentTimeMillis();
Iterator<Item> it = result.values().iterator();
while (it.hasNext()) {
Item item = it.next();
if (now - item.creationTime > MAX_AGE_IN_MS) {
it.remove();
}
}
if (result.size() == numberOfSources) {
System.out.println("Got result, create a result object and return the items " + result.values());
break;
}
}
}
private class Item {
final String source;
final V value;
final long creationTime;
public Item(String source, V value) {
this.source = source;
this.value = value;
this.creationTime = System.currentTimeMillis();
}
public String toString() {
return String.valueOf(value);
}
}
public static void main(String[] args) throws Exception {
final Filter<String> filter = new Filter<String>(4);
new Thread(new Runnable() {
public void run() {
try {
filter.get();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}).start();
filter.add("a0", "va0.1");
filter.add("a0", "va0.2");
Thread.sleep(2000);
filter.add("a0", "va0.3");
Thread.sleep(100);
filter.add("a1", "va1.1");
filter.add("a2", "va2.1");
filter.add("a0", "va0.4");
Thread.sleep(100);
filter.add("a3", "va3.1");
Thread.sleep(10);
filter.add("a1", "va1.2");
filter.add("a2", "va2.2");
filter.add("a0", "va0.5");
}
}