Develop Reference

Java <key, value> collection to retrieve minimum element in O(1) or O(log(n)) at worst

I'm iterating through a huge file reading key and value from every line. I need to obtain specific number (say 100k) of elements with highest values. To store them I figured that I need a collection that allows me to check a minimum value in O(1) or O(log(n)) and if currently read value is higher then remove element with minimum value and put new one. What collection enables me to do that? Values are not unique so BiMap is probably not adequate here.
EDIT:
Ultimate goal is to obtain best [key, value] that will be used later. Say my file looks like below (first column - key, second value):
3 6
5 9
2 7
1 6
4 5
Let's assume I'm looking for best two elements and algorithm to achieve that. I figured that I'll use a key-based collection to store best elements. First two elements (<3, 6>, <5, 9>) will be obviously added to the collection as its capacity is 2. But when I get to the third line I need to check if <2, 7> is eligible to be added to the collection (so I need to be able to check if 7 is higher than minimum value in collection (6)

It sounds like you don't actually need a structure because you are simply looking for the largest N values with their corresponding keys, and the keys are not actually used for sorting or retrieval for the purpose of this problem.
I would use the PriorityQueue, with the minimum value at the root. This allows you to retrieve the smallest element in constant time, and if your next value is larger, removal and insertion in O(log N) time.
class V{
int key;
int value;
}
class ComparatorV implements Comparator<V>{
int compare(V a, V b){
return Integer.compare(a.value, b.value);
}
}

For your specific situation, you can use a TreeSet, and to get around the uniqueness of elements in a set you can store pairs which are comparable but which never appear equal when compared. This will allow you to violate the contract with Set which specifies that the Set not contain equal values.
The documentation for TreeSet contains:
The behavior of a set is well-defined even if its ordering is
inconsistent with equals; it just fails to obey the general contract
of the Set interface
So using the TreeSet with the Comparable inconsistent with equals should be fine in this situation. If you ever need to compare your chess pairs for a different reason (perhaps some other algorithm you are also running in this app) where the comparison should be consistent with equals, then provide a Comparator for the other use. Notice that TreeSet has a constructor which takes a Comparator, so you can use that instead of having ChessPair implement Comparable.
Notice: A TreeSet provides more flexibility than a PriorityQueue in general because of all of its utility methods, but by violating the "comparable consistent with equals" contract of Set some of the functionality of the TreeSet is lost. For example, you can still remove the first element of the set using Set.pollFirst, but you cannot remove an arbitrary element using remove since that will rely on the elements being equivalent.
Per your "n or at worst log(n)" requirement, the documentation also states:
This implementation provides guaranteed log(n) time cost for the basic
operations (add, remove and contains).
Also, I provide an optimization below which reduces the minimum-value query to O(1).
Example
Set s = new TreeSet<ChessPair>();
and
public class ChessPair implements Comparable<ChessPair>
{
final int location;
final int value;
public ChessPair(final int location, final int value)
{
this.location = location;
this.value = value;
}
#Override
public int compareTo(ChessPair o)
{
if(value < o.value) return -1;
return 1;
}
}
Now you have an ordered set containing your pairs of numbers, they are ordered by your value, you can have duplicate values, and you can get the associated locations. You can also easily grab the first element (set.first), last (set.last), or get a sub-set (set.subSet(a,b)), or iterate over the first (or last, by using descendingSet) n elements. This provides everything you asked for.
Example Use
You specified wanting to keep the 100 000 best elements. So I would use one algorithm for the first 100 000 possibilities which simply adds every time.
for(int i = 0; i < 100000 && dataSource.hasNext(); i += 1)
{
ChessPair p = dataSource.next(); // or whatever you do to get the next line
set.add(p);
}
and then a different one after that
while(dataSource.hasNext())
{
ChessPair p = dataSource.next();
if(p.value > set.first().value)
{
set.remove(set.pollFirst());
set.add(p);
}
}
Optimization
In your case, you can insert an optimization into the algorithm where you compare against the lowest value. The above, simple version performs an O(log(n)) operation every time it compares against minimum-value since set.first() is O(log(n)). Instead, you can store the minimum value in a local variable.
This optimization works well for scaling this algorithm because the impact is negligible - no gain, no loss - when n is close to the total data set size (ie: you want best 100 values out of 110), but when the total data set is vastly larger than n (ie: best 100 000 out of 100 000 000 000) the query for the minimum value is going to be your most common operation and will now be constant.
So now we have (after loading the initial n values)...
int minimum = set.first().value;
while(dataSource.hasNext())
{
ChessPair p = dataSource.next();
if(p.value > minimum)
{
set.remove(set.pollFirst());
set.add(p);
minimum = set.first().value;
}
}
Now your most common operation - query minimum value - is constant time (O(1)), your second most common operation - add - is worst case log(n) time, and your least most common operation - remove - is worst case log(n) time.
For arbitrarily large data sets, each input is now processed in constant O(1) time.
See java.util.TreeSet
Previous answer (now obsolete)
Based on question edits and discussion in the question's comments, I no longer believe my original answer to be correct. I am leaving it below for reference.
If you want a Map collection which allows fast access to elements based on order, then you want an ordered Map, for which there is a sub-interface SortedMap. Fortunately for you, Java has a great implementation of SortedMap: it's TreeMap, a Map which is backed by a "red-black" tree structure which is an ordered tree.
Red-black-trees are nice since they rotate branches in order to keep the tree balanced. That is, you will not end up with a tree that branches n times in one direction, yielding n layers, just because your data may already have been sorted. You are guaranteed to have approximately log(n) layers in the tree, so it is always fast and guarantees log(n) query even for worst-case.
For your situation, try out the java.util.TreeMap. On the page linked in the previous sentence, there are links also to Map and SortedMap. You should check out the one for SortedMap too, so you can see where TreeMap gets some of the specific functionality that you are looking for. It allows you to get the first key, the last key, and a sub-map that fetches a range from within this map.
For your situation though, it is probably sufficient to just grab an iterator from the TreeMap and iterate over the first n pairs, where n is the number of lowest (or highest) values that you want.

Use a TreeSet, which offers O(log n) insertion and O(1) retrieval of either the highest or lowest scored item.
Your item class must:
Implement Comparable
Not implement equals()
To keep the top 100K items only, use this code:
Item item; // to add
if (treeSet.size() == 100_000) {
if (treeSet.first().compareTo(item) < 0) {
treeSet.remove(treeSet.first());
treeSet.add(item);
}
} else {
treeSet.add(item);
}

If you want a collection ordered by values, you can use a TreeSet which stores tuples of your keys and values. A TreeSet has O(log(n)) access times.
class KeyValuePair<Key, Value: Comparable<Value>> implements Comparable<KeyValuePair<Key, Value>> {
Key key;
Value value;
KeyValuePair(Key key, Value value) {
this.key = key;
this.value = value;
}
public int compare(KeyValuePair<Key, Value> other) {
return this.value.compare(other.value);
}
}
or instead of implementing Comparable, you can pass a Comparator to the set at creation time.
You can then retrieve the first value using treeSet.first().value.

Something like this?
entry for your data structure, that can be sorted based on the value
class Entry implements Comparable<Entry> {
public final String key;
public final long value;
public Entry(String key, long value) {
this.key = key;
this.value = value;
}
public int compareTo(Entry other) {
return this.value - other.value;
}
public int hashCode() {
//hashcode based on the same values on which equals works
}
}
actual code that works with a PriorityQueue. The sorting is based on the value, not on the key as with a TreeMap. This is because of the compareMethod defined in Entry. If the sets grows above 100000, the lowest entry (with the lowest value) is removed.
public class ProcessData {
private int maxSize;
private PriorityQueue<Entry> largestEntries = new PriorityQueue<>(maxSize);
public ProcessData(int maxSize) {
this.maxSize = maxSize;
}
public void addKeyValue(String key, long value) {
largestEntries.add(new Entry(key, value));
if (largestEntries.size() > maxSize) {
largestEntries.poll();
}
}
}

Traversing Hash Map linear using generics (Java)

I have a hash map which uses linear probing to deal with collisions. I would like to traverse it. Conceptually, this is quite easy, however, the use of generics is spinning me off.
The entries in the internal array of the hash map have their key-value pairs as generics - like this
public entry(K key, V value) {
this.key = key;
this.value = value;
}
These entries are stored in an entry array - like this
private entry[] entries;
I would like to traverse the hash map beginning at a certain key, I will reach the end of the internal array, then go back to the beginning of the array up to the key, in a circular fashion so the whole array is covered.
public V traverse(K k) {
//look from current key
for(int i = (int)k; i < entries.length; i++){
//visit node
}
//go back to start, and look up to key
for(int i = 0; i < (int)k; i++){
//visit node
}
}
I've realized that type casting the key as an integer was sort of stupid, but i'm struggling to find a working way to actually do this traversal.

If I understood you maybe you can create LinkedHashMap(HashMap)() and with it you can do it transversally.

A HashMap does not store the entries in the order you expect. The ordering is based on the hash of the key, and organized as an array of linked-lists.
If you want to have a defined ordering you should use TreeMap, which uses the natural ordering of the key. If you need a custom sorting you can provide a Comparator by constructor. Then just iterate on the map.keySet.
But I think what you need (starting the iteration somewhere in the middle) is not supported. You start iterating at the first node, and do nothing until you reach your desired key, then do your work with the rest of the entries, then start again iterating until you reach your desired key again.

How to get element position from Java Map

I have this Java Map:
Can you tell me how I can get the 6-th element of the Map?
private static final Map<String, Users> cache = new HashMap<>();
is this possible? Or I have to use another Java collection?

Though a bit late to answer. But the option is to use LinkedHashMap: this map preserves the order according to insertion of elements, as everyone has suggested. However, As a warning, it has a constructor LinkedHashMap(int initialCapacity, float loadFactor, boolean accessOrder) which will create a linked hash map whose order of iteration is the order in which its entries were last accessed. Don't use this constructor for this case.
However, if I needed such functionality, i would extend it and implement my necessary function to re-use them in OOP way.
class MyLinkedMap<K, V> extends LinkedHashMap<K, V>
{
public V getValue(int i)
{
Map.Entry<K, V>entry = this.getEntry(i);
if(entry == null) return null;
return entry.getValue();
}
public Map.Entry<K, V> getEntry(int i)
{
// check if negetive index provided
Set<Map.Entry<K,V>>entries = entrySet();
int j = 0;
for(Map.Entry<K, V>entry : entries)
if(j++ == i)return entry;
return null;
}
}
Now i can instantiate it and can get a entry and value either way i want:
MyLinkedMap<String, Integer>map = new MyLinkedMap<>();
map.put("a first", 1);
map.put("a second", 2);
map.put("a third", 3);
System.out.println(map.getValue(2));
System.out.println(map.getEntry(1));
Output:
3
a second=2

HashMap doesn't grantee the order. If you concern about order you should use LinkedHashMap
Map<String, Users> orderedMap=new LinkedHashMap<>();
Now when you put an element it will keep the order what you put.
If you want to get 6th element, now you can do it since you have your elements in order.
orderedMap.values().toArray()[5]// will give you 6th value in the map.
Example
Map<String, String> orderedMap=new LinkedHashMap<>();
orderedMap.put("a","a");
orderedMap.put("b","b");
System.out.println(orderedMap.values().toArray()[1]); // you will get b(value)
System.out.println(orderedMap.keySet().toArray()[1]); // you will get b(key)
}

A HashMap does not maintain the order of the elements inserted in it. You can used a LinkedHashMap instead which maintains the order of the elements inserted in it.
Though you need to note that even a LinkedHashMap has no such method which would give the element at a particular index. You will have to manually iterate through the entries and extract the element at the 6th iteration.

With guava's Iterables
Iterables.get(map.entrySet(),6);

The HashMap has no defined ordering of keys.It's Unordered.
You can use LinkedHashMap which will store your keys in order of insertion.You can retrieve them by calling keySet().

HashMaps do not preserve ordering:
LinkedHashMap which guarantees a predictable iteration order.
Example
public class Users
{
private String Id;
public String getId()
{
return Id;
}
public void setId(String id)
{
Id = id;
}
}
Users user;
LinkedHashMap<String,Users> linkedHashMap = new LinkedHashMap<String,Users>();
for (int i = 0; i < 3; i++)
{
user = new Users();
user.setId("value"+i);
linkedHashMap.put("key"+i,user);
}
/* Get by position */
int pos = 1;
Users value = (new ArrayList<Users>(linkedHashMap.values())).get(pos);
System.out.println(value.getId());

According to documentation, HashMap is a Hash table based implementation of the Map interface. This implementation provides all of the optional map operations, and permits null values and the null key. (The HashMap class is roughly equivalent to Hashtable, except that it is unsynchronized and permits nulls.) This class makes no guarantees as to the order of the map; in particular, it does not guarantee that the order will remain constant over time.
That's why it is not wise to use this kind of Collection.
UPDATE:
Based on #Prateek implementation of LinkedHashMap I would suggest something like:
LinkedHashMap<String,User> linkedHashMap = new LinkedHashMap<String,User>();
// or LinkedHashMap<String,User> linkedHashMap = new LinkedHashMap<>(); //for java 7+
linkedHashMap.put("1",userObj1);
linkedHashMap.put("2",userObj2);
linkedHashMap.put("3",userObj3);
/* Get by position */
int pos = 1; // Your position
User tmp= (new ArrayList<User>(linkedHashMap.values())).get(pos);
System.out.println(tmp.getName());

A HashMap doesn't have a position. You can iterate through its KeySet or EntrySet, and pick the nth element, but it's not really the same as a position. A LinkedHashMap does have a position, since it has a predictable iteration order.

You need to use a LinkedHashMap in order to be able to tell the order of the inserted elements. HashMap is not capable of doing so.

There is no Order in HashMap. You can obtain the list of may keys using map.keySet() but there's no guarantee the key set will be in the order which you add it in. Use LinkedHashMap instead of HashMap It will always return keys in same order (as insertion)

Correct!!
you will have to use other collection for getting values on index(position).
You can use ArrayList

If the ordering is to mean anything significant, you could consider using a SortedMap.
Concrete implementation: TreeMap

Use LinkedHashMap instead of HashMap It will return keys in same order (as insertion) when calling keySet().
For mare detail about LinkedHashMap see this
For example to get the element from specific index
Create a new list from your values and get the value based on index.
LinkedHashMap<String, List<String>> hMap;
List<List<String>> l = new ArrayList<List<String>>(hMap.values());
l.get(6);

Implementing search based on 2 fields in a java class

I am trying to present a simplified version of my requirement here for ease of understanding.
I have this class
public class MyClass {
private byte[] data1;
private byte[] data2;
private long hash1; // Hash value for data1
private long hash2; // Hash value for data2
// getter and setters }
Now I need to search between 2 List instances of this class, find how many hash1's match between the 2 instances and for all matches how many corresponding hash2's match. The 2 list will have about 10 million objects of MyClass.
Now I am planning to iterate over first list and search in the second one. Is there a way I can optimize the search by sorting or ordering in any particular way? Should I sort both list or only 1?

Best solution would be to iterate there is no faster solution than this. You can create Hashmap and take advantage that map does not add same key but then it has its own creation overload

sort only second, iterate over first and do binary search in second, sort O(nlogn) and binary search for n item O(nlogn)
or use hashset for second, iterate over first and search in second, O(n)

If you have to check all the elements, I think you should iterate over the first list and have a Hashmap for the second one as said AmitD.
You just have to correctly override equals and hashcode in your MyClass class. Finally, I will recomend you to use basic types as much as possible. For example, for the first list, instead of a list will be better to use a simple array.
Also, at the beginning you could select which of the two lists is the shorter one (if there's a difference in the size) and iterate over that one.

I think you should create a hashmap for one of the lists (say list1) -
Map<Long, MyClass> map = new HashMap<Long, MyClass>(list1.size());//specify the capacity
//populate map like - put(myClass.getHash1(), myClass) : for each element in the list
Now just iterate through the second list (there is no point in sorting both) -
int hash1MatchCount = 0;
int hash2MatchCount = 0;
for(MyClass myClass : list2) {
MyClass mc = map.get(myClass.getHash1());
if(mc != null) {
hash1MatchCount++;
if(myClass.getHash2() == mc.getHash2) {
hash2MatchCount++;
}
}
}
Note: Assuming that there is no problem regarding hash1 being duplicates.

Get the list of keys where the values are equal for those keys

I require to get the list of keys where the values are equal for those keys from a HashMap.
For example , my hashmap contains the below elements.
Key Value
1 a,b
2 e,c
3 a,b
4 f
5 e,c
6 c
We need to evaluate as
1,3 contains value (a,b)
2,5 contains value (e,c)
4 contains value (f)
6 contains value (c)
Thx

You could invert your hash: build a new hash with the key type being the type of your current map's value, and the value type being a list of your current map's key type.
Iterate over your current map's keys, and push them to the right slot in your new map. You'll then have exactly the mapping you are asking for.
If the values in your current map aren't directly comparable right now, you'll need to find a representation that is. This depends completely on the nature of the data.
One simple approach is to sort the list and use it's toString representation as your new key. This only works if the toString representation of the underlying objects is sane for this purpose.

You can create other map where your keys a used as values and values as keys. If for example your source map is defined as Map<Integer, String> create map Map<String, List<Integer>>. The list of integers will contain keys (from your source map) that have certain values.

Building on Mat's answer, if you need to do this operation frequently, use one of the bidirectional map classes from Guava or Apache Commons Collections; e.g. HashBiMap<K,V> or DualHashBidiMap or DualTreeBidiMap. These data structures maintain a pair of maps that represent the forward and inverse mappings.
Alternatively, for a once off computation:
Extract the Map.entries() collection into an array.
Sort the array in order of the values.
Iterate the array, and extract the entry keys for which subsequent entry values are equal.
(This should be O(NlogN) in time and require O(N) extra space ... depending on the sort algorithm used.)

The most basic method will be to:
Get the first key of the HashMap and iterate over the map checking for keys with the same value.
If found, remove that key from the map and store the key in another collection (maybe a Vector).
Then after all other keys are checked, add the current key to that collection.
If no other keys are found, add the current key to that collection.
Then add the keys in that collection to another map with the relevant value. Clear the collection.
Proceed to the next key and do the same.
After doing this, you will end up with what you want.
EDIT: The Code:
HashMap comp = new HashMap(); // Calculations Done
Vector v = new Vector(); // Temporary List To Store Keys
// Get The List Of Keys
Vector<Integer> keys = new Vector<Integer>();
Iterator<Integer> it = hm.keySet().iterator();
while(it.hasNext()) keys.add(it.next());
// For Every Key In Map...
for(int i = 0; i < hm.size(); i++) {
int key = keys.get(i);
v.add(key); // Add the Current Key To Temporary List
// Check If Others Exist
for(int j = i+1; j < hm.size(); j++) {
int nkey = keys.get(j);
if(hm.get(key).equals(hm.get(nkey))) {
v.add(nkey);
}
}
// Store The Value Of Current Key And The Keys In Temporary List In The Comp HashMap
String val = hm.get(key);
String cKey = "";
for(int x = 0; x < v.size(); x++)
cKey += v.get(x) + ",";
// Remove The Comma From Last Key, Put The Keys As Value And Value As Key
cKey = cKey.substring(0, cKey.length()-1);
comp.put(cKey, val);
// Clear The Temporary List
v.clear();
}
There is a little problem in this code: Duplicates occur and also the last duplicate seems to be correct.
The output using your example give. (You need to do a little formatting).
{3=a,b, 6=c, 5=e,c, 2,5=e,c, 4=f, 1,3=a,b}