I try use Set interface as value for hazelcast IMap instance and when I run my test I found that test hung inside ConcurrentMap#compute method.
Why do I have infinite loop when I use hazelcast IMap in this code:
import com.hazelcast.config.Config;
import com.hazelcast.config.MapConfig;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.IMap;
import java.io.Serializable;
import java.util.*;
public class Main {
public static void main(String[] args) {
IMap<String, HashSet<StringWrapper>> store = Hazelcast.newHazelcastInstance(
new Config().addMapConfig(new MapConfig("store"))
).getMap("store");
store.compute("user", (k, value) -> {
HashSet<StringWrapper> newValues = Objects.isNull(value) ? new HashSet<>() : new HashSet<>(value);
newValues.add(new StringWrapper("user"));
return newValues;
});
store.compute("user", (k, value) -> {
HashSet<StringWrapper> newValues = Objects.isNull(value) ? new HashSet<>() : new HashSet<>(value);
newValues.add(new StringWrapper("user"));
return newValues;
});
System.out.println(store.keySet());
}
// Data class
public static class StringWrapper implements Serializable {
String value;
public StringWrapper() {}
public StringWrapper(String value) {
this.value = value;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
if (!super.equals(o)) return false;
StringWrapper value = (StringWrapper) o;
return Objects.equals(this.value, value.value);
}
#Override
public int hashCode() {
return Objects.hash(super.hashCode(), value);
}
}
}
Hazelcast: 3.9.3
Java:build 1.8.0_161-b12
Operating system: macOS High Sierra 10.13.3
#Alykoff I reproduced the issue based on above example & ArrayList version, which is reported as a github issue: https://github.com/hazelcast/hazelcast/issues/12557.
There are 2 seperate problems:
1 - When using HashSet, the problem is how Java deserialize the HashSet/ArrayList (collections) & how compute method works. Inside compute method (since Hazelcast complied with Java 6 & there is no compute method to override, default implementation from ConcurrentMap called ), this block causes the infinite loop:
// replace
if (replace(key, oldValue, newValue)) {
// replaced as expected.
return newValue;
}
// some other value replaced old value. try again.
oldValue = get(key);
this replace method calls IMap replace method. IMap checks if the current value equal to the user-supplied value. But because of a Java Serialization optimization, the check fails. Please check HashSet.readObject method. You'll see that when deserializing the HashSet, since element size is known, it creates the inner HashMap with a capacity:
// Set the capacity according to the size and load factor ensuring that
// the HashMap is at least 25% full but clamping to maximum capacity.
capacity = (int) Math.min(size * Math.min(1 / loadFactor, 4.0f),
HashMap.MAXIMUM_CAPACITY);
But your HashSet, created without an initial capacity, has a default capacity of 16, while the deserialized one has the initial capacity of 1. This changes the serialization, index 51 contains the current capacity & it seems JDK re-calculate it based on size when deserializing the object to minimize the size.
Please see below example:
HazelcastInstance hz = Hazelcast.newHazelcastInstance();
IMap<String, Collection<String>> store = instance.getMap("store");
Collection<String> val = new HashSet<>();
val.add("a");
store.put("a", val);
Collection<String> oldVal = store.get("a");
byte[] dataOld = ((HazelcastInstanceProxy) hz).getSerializationService().toBytes(oldVal);
byte[] dataNew = ((HazelcastInstanceProxy) hz).getSerializationService().toBytes(val);
System.out.println(Arrays.equals(dataNew, dataOld));
This code prints false. But if you create the HashSet with the initial size 1, then both byte arrays are equal. And in your case, you won't get an infinite loop.
2 - When using ArrayList, or any other collection, there's another problem which you pointed above. Due to how compute method implemented in ConcurrentMap, when you assign the old value to the newValue & add a new element, you actually modify the oldValue thus causing replace method fail. But when you change the code to new ArrayList(value), now you're creating a new ArrayList & value collection is not modified. It's a best practice to wrap a collection before using it if you don't want to modify the original one. Same works for HashSet if you create with size 1 due to the first issue I explained.
So in your case, you should use
Collection<String> newValues = Objects.isNull(value) ? new HashSet<>(1) : new HashSet<>(value);
or
Collection<String> newValues = Objects.isNull(value) ? new ArrayList<>() : new ArrayList<>(value);
That HashSet case seems to be a JDK issue, rather than an optimization. I don't know any of these cases can be solved/fixed in Hazelcast, unless Hazalcast overrides the HashXXX collection serialization & overrides the compute method.
Related
I try to add objects of Employee class to a TreeSet. I don't implement Comparable or Comparator interface. But the add method code behaves differently in different systems. Why so? Code snippet below:-
import java.util.Set;
import java.util.TreeSet;
public class TreeSetTest {
public static void main(String[] args) {
Set<Employee> set = new TreeSet<Employee>();
set.add(new Employee());
// set.add(new Employee());
// set.add(new Employee());
}
}
On the current system (Win 10), whether I write set.add() method once or thrice. It always throws ClassCastException at runtime.
But talking of this question- Why does TreeSet throws ClassCastException
The user there has written, that he doesn't get exception when he uses add method only once.
Also, in another system (Win 7), yesterday I had tried adding object 3 times, calling set method thrice, and there is no ClassCastException!! The size of set remains 1 only, so it appeared that multiple objects are just NOT getting added to set.
So what could be the reason for different-different kind of behavior of add method?
TreeSet.add() delegates to TreeMap.put(), which has differing behavior in Java 6 and Java 8.
Java 6:
public V put(K key, V value) {
Entry<K,V> t = root;
if (t == null) {
// TBD:
// 5045147: (coll) Adding null to an empty TreeSet should
// throw NullPointerException
//
// compare(key, key); // type check
root = new Entry<K,V>(key, value, null);
size = 1;
modCount++;
return null;
}
...
Java 8:
public V put(K key, V value) {
Entry<K,V> t = root;
if (t == null) {
compare(key, key); // type (and possibly null) check
root = new Entry<>(key, value, null);
size = 1;
modCount++;
return null;
}
...
As you can see, the earlier version had the compare() line commented out for some reason, but it was added back in the later version. Hence the exception you're seeing for the first element.
See here also: Why TreeSet can be used as a key for TreeMap in jdk 1.6?
Ok, So I know this topic has been asked lots of times. All usually referring to using a set of some kind.
Now here is the problem.
I have a RateSheet (ArrayList) which has many rates (object) in it. In this rate object we have the variables:
desName
prefix
cost
Now to summarise, I want to take this RateSheet and remove all the duplicate rates based on desName.
I have used a set and collection todo this. Which works, but then I can't go in and access individual objects and then their variables etc as I get cast errors. So I am trying todo this without using sets.
I assume you are creating a set from the rate sheet and inject in that set the desName of each rate sheet.
Unfortunately, the Set api is based upon the equals method and the hashCode methods, so once you imnplemented the equals and hashCode based upon the desName attribute, your sets will contain unique rate sheet objects regarding the desName, if you want them to be unique regarding the prefix, you have to change the equals method and hashCode method.
I've done something like this in the past by plugging in an adapter inside the rate sheet that either links to the desName or to the prefix, hence the hashCode redirects to the "hashCode" provided by the current adapter.
Set<String> s = new HashSet();
for(int i=0;i<RateSheet.size;i++){
Rate rate = (Rate)RateSheet.get(i);
s.add(rate.desName);
}
Set<Rate> rs = new HashSet();
for(int i=0;i<RateSheet.size;i++){
Rate rate = (Rate) RateSheet.get(i);
Iterator it = s.iterator();
while(it.hasNext()){
if(rate.desName.equals(it.next()){
rs.add(rate);
}
}
}
List<RateSheets> dedupSheets(List<RateSheet> sheets) {
Map<String,RateSheet> map = new HashMap<>();
for(Rate r: sheets) map.put(r.getName(), r);
return new ArrayList<>(rateSheets.values());
}
Here's an example code on how to do it with a Set. You need to override hashCode() and equals() because they are used under the hood to find out what's a duplicate and what's not:
import java.util.HashSet;
public class RateSheet
{
private String desName;
public RateSheet(String name)
{
desName = name;
}
#Override
public boolean equals(Object other)
{
if (other == null) return false;
if (other == this) return true;
if (!(other instanceof RateSheet))return false;
RateSheet otherAsRateSheet = (RateSheet)other;
return desName.equals(otherAsRateSheet.desName);
}
#Override
public int hashCode()
{
return desName.hashCode();
}
public static void main(String[] args)
{
HashSet<RateSheet> sheets= new HashSet<RateSheet>();
RateSheet a = new RateSheet("a");
RateSheet b = new RateSheet("a");
RateSheet c = new RateSheet("b");
System.out.println(sheets.add(a));
System.out.println(sheets.add(b));
System.out.println(sheets.add(c));
}
}
This is based on this other answer found here:
Java Set collection - override equals method
and also this one:
Why is equals() not called while adding to HashSet and hashCode matches?
and what #GerritCap posted. (sorry, I am a little slow)
I know that Guava has a BiMultimap class internally but didn't outsource the code. I need a data structure which is bi-directional, i.e. lookup by key and by value and also accepts duplicates.
i.e. something like this: (in my case, values are unique, but two values can point to the same key)
0 <-> 5
1 <-> 10
2 <-> 7
2 <-> 8
3 <-> 11
I want to be able to get(7) -> returning 2 and get(2) returning [7, 8].
Is there another library out there which has a data structure I can make use of?
If not, what do you suggest is the better option to handle this case? Is keeping two Multimaps in memory one with and the other with a bad practice?
P.S.: I have read this question: Bidirectional multi-valued map in Java but considering it is dated in 2011, I thought I'll open a more recent question
What do you mean by
Guava has a BiMultimap class internally but didn't outsource the code
The code of an implementation is here.
I didn't check if this is a working implementation, nor if it made it into a release or if I'm just looking at some kind of snapshot. Everything is out in the open, so you should be able to get it.
From a quick glance at the source code it looks like the implementation does maintain two MultMaps, and this should be fine for the general case.
If you don't need the whole bunch of Guava HashBiMultimap functionality, but just getByKey() and getByValue(), as you specified, I can suggest the approach, where only one HashMultiMap is used as a storage.
The idea is to treat provided key and value as equilibrium objects and put both of them in the storage map as keys and values.
For example: Let we have the following multiMap.put(0, 5), so we should get the storage map containing something like this [[key:0, value:5], [key:5, value:0]].
As far as we need our BiMultiMap to be generic, we also need to provide some wrapper classes, that should be used as storage map type parameters.
Here is this wrapper class:
public class ObjectHolder {
public static ObjectHolder newLeftHolder(Object object) {
return new ObjectHolder(object, false);
}
public static ObjectHolder newRightHolder(Object object) {
return new ObjectHolder(object, true);
}
private Object object;
private boolean flag;
private ObjectHolder(Object object, boolean flag) {
this.object = object;
this.flag = flag;
}
public Object getObject() {
return object;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (!(o instanceof ObjectHolder)) return false;
ObjectHolder that = (ObjectHolder) o;
if (flag != that.flag) return false;
if (!object.equals(that.object)) return false;
return true;
}
#Override
public int hashCode() {
int result = object.hashCode();
result = 31 * result + (flag ? 1 : 0);
return result;
}
}
And here is the MultiMap:
public class BiHashMultiMap<L, R> {
private Map<ObjectHolder, Set<ObjectHolder>> storage;
public SimpleBiMultiMap() {
storage = new HashMap<ObjectHolder, Set<ObjectHolder>>();
}
public void put(L left, R right) {
ObjectHolder leftObjectHolder = ObjectHolder.newLeftHolder(left);
ObjectHolder rightObjectHolder = ObjectHolder.newRightHolder(right);
put(leftObjectHolder, rightObjectHolder);
put(rightObjectHolder, leftObjectHolder);
}
private void put(ObjectHolder key, ObjectHolder value) {
if (!storage.containsKey(key)) {
storage.put(key, new HashSet<ObjectHolder>());
}
storage.get(key).add(value);
}
public Set<R> getRight(L left) {
return this.get(ObjectHolder.newLeftHolder(left));
}
public Set<L> getLeft(R right) {
return this.get(ObjectHolder.newRightHolder(right));
}
private <V> Set<V> get(ObjectHolder key) {
Set<ObjectHolder> values = storage.get(key);
if (values == null || values.isEmpty()) {
return null;
}
Set<V> result = new HashSet<V>();
for (ObjectHolder value : values) {
result.add((V)value.getObject());
}
return result;
}
}
Thing that could seem strange is the left and right prefixed variable everywhere. You can think of them as left is the original key, that was putted to map and right is the value.
Usage example:
BiHashMultiMap<Integer, Integer> multiMap = new BiHashMultiMap<Integer, Integer>();
multiMap.put(0,5);
multiMap.put(1,10);
multiMap.put(2,7);
multiMap.put(3,7);
multiMap.put(2,8);
multiMap.put(3,11);
Set<Integer> left10 = multiMap.getLeft(10); // [1]
Set<Integer> left7 = multiMap.getLeft(7); // [2, 3]
Set<Integer> right0 = multiMap.getRight(0); // [5]
Set<Integer> right3 = multiMap.getRight(3); // [7, 11]
So to get left value we need to provide right value as key and to get right value we need to provide left as a key.
And of course to make map fully function we need to provide other methods, like remove(), contains() and so on.
I was just wondering, what would happen if key of a HashMap is mutable, test program below demonstrate that and I am unable to understand when both equals and hashCode methods returns
true and same value, why does hashmap.containsKey return false.
public class MutableKeyHashMap {
public static void main(String []a){
HashMap<Mutable, String> map = new HashMap<Mutable, String>();
Mutable m1 = new Mutable(5);
map.put(m1, "m1");
Mutable m2 = new Mutable(5);
System.out.println(map.containsKey(m2));
m2.setA(6);
m1.setA(6);
Mutable m3 = map.keySet().iterator().next();
System.out.println(map.containsKey(m2)+" "+m3.hashCode()+" "+m2.hashCode()+" "+m3.equals(m2));
}
}
class Mutable {
int a;
public Mutable(int a) {
this.a = a;
}
#Override
public boolean equals(Object obj) {
Mutable m = (Mutable) obj;
return m.a == this.a ? true : false;
}
#Override
public int hashCode(){
return a;
}
public void setA(int a) {
this.a = a;
}
public int getA() {
return a;
}
}
This the output :
true
false 6 6 true
The javadoc explains it
Note: great care must be exercised if mutable objects are used as map keys. The behavior of a map is not specified if the value of an object is changed in a manner that affects equals comparisons while the object is a key in the map.
Basically, don't use mutable objects as keys in a Map, you're going to get burnt
To extrapolate, because the docs may not appear clear, I believe the pertinent point here is `changed in a manner that affects equals', and you seem to be assuming that equals(Object) is called each time contains is invoked. The docs don't say that, the wording implies they may be allowed to cache computations.
Looking at the source, it seems that because your hashCode returns a different value (was 5, now 6), it's possible that it's being looked up in a different bucket based on implementation details.
You can think of if this way, the Map has 16 buckets. When you give it an object with A == 5, it tosses it into bucket 5. Now you can change A to 6, but it's still in bucket 5. The Map doesn't know you changed A, it doesn't rearrange things internally.
Now you come over with another object with A == 6, and you ask the Map if it has one of those. It goes and looks in bucket 6 and says "Nope, nothing there." It's not going to go and check all the other buckets for you.
Obviously how things get put into buckets is more complicated than that, but that's how it works at the core.
The HashMap puts your object at the location for hash key 5. Then you change the key to 6 and use containsKey to ask the map whether it contains the object. The map looks at position 6 and finds nothing, so it answers false.
So don't do that, then.
When you put "m1" the first time around, hashCode() was 5. Thus the HashMap used 5 to place the value into the appropriate bucket. After changing m2, the hashCode() was 6 so when you tried looking for the value you put in, it the bucket it looked in was different.
A code example to accompany ptomli's answer.
import java.util.*;
class Elem {
private int n;
public Elem(int n) {
this.n = n;
}
public void setN(int n) {
this.n = n;
}
#Override
public int hashCode() {
return n;
}
#Override
public boolean equals(Object e) {
if (this == e)
return true;
if (!(e instanceof Elem))
return false;
Elem an = (Elem) e;
return n == an.n;
}
}
public class MapTest {
public static void main (String [] args) {
Elem e1 = new Elem(1);
Elem e2 = new Elem(2);
HashMap map = new HashMap();
map.put(e1, 100);
map.put(e2, 200);
System.out.println("before modification: " + map.get(e1));
e1.setN(9);
System.out.println("after modification using updated key: " + map.get(e1));
Elem e3 = new Elem(1);
System.out.println("after modification using key which equals to the original key: " + map.get(e3));
}
}
Compiles and runs it. The result is:
before modification: 100
after modification using updated key: null
after modification using key which equals to the original key: null
I am using Java 6 on Linux.
I have created a short example of my problem. I'm creating a list of objects anonymously and adding them to an ArrayList. Once items are in the ArrayList I later come back and add more information to each object within the list. Is there a way to extract a specific object from the list if you do not know its index?
I know only the Object's 'name' but you cannot do a list.get(ObjectName) or anything. What is the recommended way to handle this? I'd rather not have to iterate through the entire list every time I want to retrieve one specific object.
public class TestCode{
public static void main (String args []) {
Cave cave = new Cave();
// Loop adds several Parties to the cave's party list
cave.parties.add(new Party("FirstParty")); // all anonymously added
cave.parties.add(new Party("SecondParty"));
cave.parties.add(new Party("ThirdParty"));
// How do I go about setting the 'index' value of SecondParty for example?
}
}
class Cave {
ArrayList<Party> parties = new ArrayList<Party>();
}
class Party extends CaveElement{
int index;
public Party(String n){
name = n;
}
// getter and setter methods
public String toString () {
return name;
}
}
class CaveElement {
String name = "";
int index = 0;
public String toString () {
return name + "" + index;
}
}
Given the use of List, there's no way to "lookup" a value without iterating through it...
For example...
Cave cave = new Cave();
// Loop adds several Parties to the cave's party list
cave.parties.add(new Party("FirstParty")); // all anonymously added
cave.parties.add(new Party("SecondParty"));
cave.parties.add(new Party("ThirdParty"));
for (Party p : cave.parties) {
if (p.name.equals("SecondParty") {
p.index = ...;
break;
}
}
Now, this will take time. If the element you are looking for is at the end of the list, you will have to iterate to the end of the list before you find a match.
It might be better to use a Map of some kind...
So, if we update Cave to look like...
class Cave {
Map<String, Party> parties = new HashMap<String, Party>(25);
}
We could do something like...
Cave cave = new Cave();
// Loop adds several Parties to the cave's party list
cave.parties.put("FirstParty", new Party("FirstParty")); // all anonymously added
cave.parties.put("SecondParty", new Party("SecondParty"));
cave.parties.put("ThirdParty", new Party("ThirdParty"));
if (cave.parties.containsKey("SecondParty")) {
cave.parties.get("SecondParty").index = ...
}
Instead...
Ultimately, this will all depend on what it is you want to achieve...
List.indexOf() will give you what you want, provided you know precisely what you're after, and provided that the equals() method for Party is well-defined.
Party searchCandidate = new Party("FirstParty");
int index = cave.parties.indexOf(searchCandidate);
This is where it gets interesting - subclasses shouldn't be examining the private properties of their parents, so we'll define equals() in the superclass.
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof CaveElement)) {
return false;
}
CaveElement that = (CaveElement) o;
if (index != that.index) {
return false;
}
if (name != null ? !name.equals(that.name) : that.name != null) {
return false;
}
return true;
}
It's also wise to override hashCode if you override equals - the general contract for hashCode mandates that, if x.equals(y), then x.hashCode() == y.hashCode().
#Override
public int hashCode() {
int result = name != null ? name.hashCode() : 0;
result = 31 * result + index;
return result;
}
If you want to lookup objects based on their String name, this is a textbook case for a Map, say a HashMap. You could use a LinkedHashMap and convert it to a List or Array later (Chris has covered this nicely in the comments below).
LinkedHashMap because it lets you access the elements in the order you insert them if you want to do so. Otherwise HashMap or TreeMap will do.
You could get this to work with List as the others are suggesting, but that feels Hacky to me.. and this will be cleaner both in short and long run.
If you MUST use a list for the object, you could still store a Map of the object name to the index in the array. This is a bit uglier, but you get almost the same performance as a plain Map.
You could use list.indexOf(Object) bug in all honesty what you're describing sounds like you'd be better off using a Map.
Try this:
Map<String, Object> mapOfObjects = new HashMap<String, Object>();
mapOfObjects.put("objectName", object);
Then later when you want to retrieve the object, use
mapOfObjects.get("objectName");
Assuming you do know the object's name as you stated, this will be both cleaner and will have faster performance besides, particularly if the map contains large numbers of objects.
If you need the objects in the Map to stay in order, you can use
Map<String, Object> mapOfObjects = new LinkedHashMap<String, Object>();
instead
As per your question requirement , I would like to suggest that Map will solve your problem very efficient and without any hassle.
In Map you can give the name as key and your original object as value.
Map<String,Cave> myMap=new HashMap<String,Cave>();
I would suggest overriding the equals(Object) of your Party class. It might look something like this:
public boolean equals(Object o){
if(o == null)
return false;
if(o instanceof String)
return name.equalsIgnoreCase((String)o);
else if(o instanceof Party)
return equals(((Party)o).name);
return false;
}
After you do that, you could use the indexOf(Object) method to retrieve the index of the party specified by its name, as shown below:
int index = cave.parties.indexOf("SecondParty");
Would return the index of the Party with the name SecondParty.
Note: This only works because you are overriding the equals(Object) method.
You could simply create a method to get the object by it's name.
public Party getPartyByName(String name) {
for(Party party : parties) {
if(name.equalsIgnoreCase(party.name)) {
return party;
}
}
return null;
}