Develop Reference

Comparing two large lists in java

I have to Array lists with 1000 objects in each of them. I need to remove all elements in Array list 1 which are there in Array list 2. Currently I am running 2 loops which is resulting in 1000 x 1000 operations in worst case.
List<DataClass> dbRows = object1.get("dbData");
List<DataClass> modifiedData = object1.get("dbData");
List<DataClass> dbRowsForLog = object2.get("dbData");
for (DataClass newDbRows : dbRows) {
boolean found=false;
for (DataClass oldDbRows : dbRowsForLog) {
if (newDbRows.equals(oldDbRows)) {
found=true;
modifiedData.remove(oldDbRows);
break;
}
}
}
public class DataClass{
private int categoryPosition;
private int subCategoryPosition;
private Timestamp lastUpdateTime;
private String lastModifiedUser;
// + so many other variables
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
DataClass dataClassRow = (DataClass) o;
return categoryPosition == dataClassRow.categoryPosition
&& subCategoryPosition == dataClassRow.subCategoryPosition && (lastUpdateTime.compareTo(dataClassRow.lastUpdateTime)==0?true:false)
&& stringComparator(lastModifiedUser,dataClassRow.lastModifiedUser);
}
public String toString(){
return "DataClass[categoryPosition="+categoryPosition+",subCategoryPosition="+subCategoryPosition
+",lastUpdateTime="+lastUpdateTime+",lastModifiedUser="+lastModifiedUser+"]";
}
public static boolean stringComparator(String str1, String str2){
return (str1 == null ? str2 == null : str1.equals(str2));
}
public int hashCode() {
int hash = 7;
hash = 31 * hash + (int) categoryPosition;
hash = 31 * hash + (int) subCategoryPosition
hash = 31 * hash + (lastModifiedUser == null ? 0 : lastModifiedUser.hashCode());
return hash;
}
}
The best work around i could think of is create 2 sets of strings by calling tostring() method of DataClass and compare string. It will result in 1000 (for making set1) + 1000 (for making set 2) + 1000 (searching in set ) = 3000 operations. I am stuck in Java 7. Is there any better way to do this? Thanks.

Let Java's builtin collections classes handle most of the optimization for you by taking advantage of a HashSet. The complexity of its contains method is O(1). I would highly recommend looking up how it achieves this because it's very interesting.
List<DataClass> a = object1.get("dbData");
HashSet<DataClass> b = new HashSet<>(object2.get("dbData"));
a.removeAll(b);
return a;
And it's all done for you.
EDIT: caveat
In order for this to work, DataClass needs to implement Object::hashCode. Otherwise, you can't use any of the hash-based collection algorithms.
EDIT 2: implementing hashCode
An object's hash code does not need to change every time an instance variable changes. The hash code only needs to reflect the instance variables that determine equality.
For example, imagine each object had a unique field private final UUID id. In this case, you could determine if two objects were the same by simply testing the id value. Fields like lastUpdateTime and lastModifiedUser would provide information about the object, but two instances with the same id would refer to the same object, even if the lastUpdateTime and lastModifiedUser of each were different.
The point is that if you really want to want to optimize this, include as few fields as possible in the hash computation. From your example, it seems like categoryPosition and subCategoryPosition might be enough.
Whatever fields you choose to include, the simplest way to compute a hash code from them is to use Objects::hash rather than running the numbers yourself.

It is a Set A-B operation(only retain elements in Set A that are not in Set B = A-B)
If using Set is fine then we can do like below. We can use ArrayList as well in place of Set but in AL case for each element to remove/retain check it needs to go through an entire other list scan.
Set<DataClass> a = new HashSet<>(object1.get("dbData"));
Set<DataClass> b = new HashSet<>(object2.get("dbData"));
a.removeAll(b);
If ordering is needed, use TreeSet.
Try to return a set from object1.get("dbData") and object2.get("dbData") that skips one more intermediate collection creation.

Get the only element in a Set

I need to know the "best" and safest way to get a value held within a Set if there is only one entry. methodToGetValues() is used extensively to read config files and return a list of values given a specific key, in this case "enabled". For the enabled key, there should only be one entry returned in the Set, obviously "true" or "false" but, mistakes happen. I have the following which seems a little convoluted:
Set<String> enabled = methodToGetValues("enabled");
if (!enabled.isEmpty() && enabled.size() < 2 && "true".equals(enabled.iterator().next())) {
...
}
Can anyone suggest a simpler yet still robust way of checking this?

Your question asks to get something from the Set. But your example just needs a check.
If you know what to expect in the Set, this works fine.
if (enabled != null && enabled.size() == 1 && enabled.contains("true")) {
...
}
Otherwise, if you just want to get the element but don't know what it is, the iterator you suggested works fine.
String getOnlyElement(Set<String> enabled, String default) {
return (enabled == null || enabled.size() != 1) ? default : enabled.iterator().next();
}
I like having null checks but it depends on what methodToGetValues returns.

Unsure of what the use case is that would drive using a Set<String> for this data but here is an option:
// check size = 1 over two checks, use contains rather than grabbing an iterator
if (set.size() == 1 && set.contains("true")) {
...
}

public Set<String> getValues(final String key){
.....
}
public String getValue(final String key) {
final Set<String> values = getValues(key);
if (values == null || values.size() != 1) {
throw new IllegalStateException("Invalid configuration for give key :" + key);
}
return values.iterator().next();
}
public Boolean getValueAsBoolean(final String key) {
return Boolean.valueOf(getValue(key));
}
You can modify method to have accept argument to return default value when keys are not found. You can add different methods to return specific type object like inte, boolean, this way code looks cleaner

Check if all values in a map are null

I need to check if all values in a map are null, I have this method that I want to replace by a built-in one if possible. Limitations: Java 5 and access to Apache Commons libraries.
/* Checks if all values are null */
public static boolean isEmpty(Map<Dboid,?> aMap){
boolean isEmpty = true;
Iterator<?> it = aMap.entrySet().iterator();
while(it.hasNext() && isEmpty){
Object value = it.next();
if(value != null) {
isEmpty = false;
}
}
return isEmpty;
}

I know the question is for Java 5, but for those who will come here from google search as I did:
For Java >= 8 you can do:
boolean allValuesAreNull = yourMap.values()
.stream()
.allMatch(Objects::isNull);
with one nuance: it will be true for empty map.

Another solution without using any third party libraries.
Collections.frequency(aMap.values(), null) == aMap.size()

As such there is no direct method for this, but you can use Apache Commons CollectionUtils.countMatches() method, and pass a NullPredicate instance to it. Of course, you would do pass the values in the map using Map#values() method:
public static <K, V> boolean hasAllNullValues(Map<K, V> map) {
int size = map.size();
return CollectionUtils.countMatches(map.values(), NullPredicate.INSTANCE) == size;
}
or even better, use CollectionUtils.exists() method, to check there is at least one element that satisfies the NotNullPredicate passed as second argument:
public static <K, V> boolean hasAllNullValues(Map<K, V> map) {
return !CollectionUtils.exists(map.values(), NotNullPredicate.INSTANCE);
}

There is no built-in method to do this. In particular, there's nothing that provides a means of "finding an element that isn't equal to something".
However, if a map that contains only null values is defined by your business rules to be "empty", that seems to imply that null values mean "not present", in which case you may wish to construct the code such that null values are never added in the first place. Then you can just use the built in isEmpty().

how about
return CollectionUtils.find(aMap.values(),NotNullPredicate.INSTANCE).isEmpty();

There is no API that will give you that, however you could optimize that method a little bit.
No need to check the isEmpty variable on every iteration.
That is a minor optimization.
/* Checks if all values are null */
public static <K,V> boolean isMapEmpty(Map<K,V> aMap){
for (V v: aMap.values()) {
if (v != null) { return false; }
}
return true;
}

TreeMap returning null for value that should exist for some object keys

I have an issue with a TreeMap that we have defined a custom key object for. The issue is that after putting a few objects into the map, and trying to retrieve with the same key used to put on the map, I get a null. I believe this is caused by the fact that we have 2 data points on the key. One value is always populated and one value is not always populated. So it seems like the issue lies with the use of compareTo and equals. Unfortunately the business requirement for how our keys determine equality needs to be implemented this way.
I think this is best illustrated with code.
public class Key implements Comparable<Key> {
private String sometimesPopulated;
private String alwaysPopulated;
public int compareTo(Key aKey){
if(this.equals(aKey)){
return 0;
}
if(StringUtils.isNotBlank(sometimesPopulated) && StringUtils.isNotBlank(aKey.getSometimesPopulated())){
return sometimesPopulated.compareTo(aKey.getSometimesPopulated());
}
if(StringUtils.isNotBlank(alwaysPopulated) && StringUtils.isNotBlank(aKey.getAlwaysPopulated())){
return alwaysPopulated.compareTo(aKey.getAlwaysPopulated());
}
return 1;
}
public boolean equals(Object aObject){
if (this == aObject) {
return true;
}
final Key aKey = (Key) aObject;
if(StringUtils.isNotBlank(sometimesPopulated) && StringUtils.isNotBlank(aKey.getSometimesPopulated())){
return sometimesPopulated.equals(aKey.getSometimesPopulated());
}
if(StringUtils.isNotBlank(alwaysPopulated) && StringUtils.isNotBlank(aKey.getAlwaysPopulated())){
return alwaysPopulated.equals(aKey.getAlwaysPopulated());
}
return false;
}
So the issue occurs when trying to get a value off the map after putting some items on it.
Map<Key, String> map = new TreeMap<Key, String>();
Key aKey = new Key(null, "Hello");
map.put(aKey, "world");
//Put some more things on the map...
//they may have a value for sometimesPopulated or not
String value = map.get(aKey); // this = null
So why is the value null after just putting it in? I think the algorithm used by the TreeMap is sorting the map in an inconsistent manner because of the way I'm using compareTo and equals. I am open to suggestions on how to improve this code. Thanks

Your comparator violates the transitivity requirement.
Consider three objects:
Object A: sometimesPopulated="X" and alwaysPopulated="3".
Object B: sometimesPopulated="Y" and alwaysPopulated="1".
Object C: sometimesPopulated is blank and alwaysPopulated="2".
Using your comparator, A<B and B<C. Transitivity requires that A<C. However, using your comparator, A>C.
Since the comparator doesn't fulfil its contract, TreeMap is unable to do its job correctly.

I think the problem is that you are returning 1 from your compareTo if either of the sometimesPopulated values is blank or either of the alwaysPopulated values is blank. Remember that compareTo can be thought of returning the value of a subtraction operation and your's is not transitive. (a - b) can == (b - a) even when a != b.
I would return -1 if the aKey sometimesPopulated is not blank and the local sometimesPopulated is blank. If they are the same then I would do the same with alwaysPopulated.
I think your logic should be something like:
public int compareTo(Key aKey){
if(this.equals(aKey)){
return 0;
}
if (StringUtils.isBlank(sometimesPopulated)) {
if (StringUtils.isNotBlank(aKey.getSometimesPopulated())) {
return -1;
}
} else if (StringUtils.isBlank(aKey.getSometimesPopulated())) {
return 1;
} else {
int result = sometimesPopulated.compareTo(aKey.getSometimesPopulated());
if (result != 0) {
return result;
}
}
// same logic with alwaysPopulated
return 0;
}

I believe the problem is that you are treating two keys with both blank fields as greater than each other which could confuse the structure.
class Main {
public static void main(String... args) {
Map<Key, String> map = new TreeMap<Key, String>();
Key aKey = new Key(null, "Hello");
map.put(aKey, "world");
//Put some more things on the map...
//they may have a value for sometimesPopulated or not
String value = map.get(aKey); // this = "world"
System.out.println(value);
}
}
class Key implements Comparable<Key> {
private final String sometimesPopulated;
private final String alwaysPopulated;
Key(String alwaysPopulated, String sometimesPopulated) {
this.alwaysPopulated = defaultIfBlank(alwaysPopulated, "");
this.sometimesPopulated = defaultIfBlank(sometimesPopulated, "");
}
static String defaultIfBlank(String s, String defaultString) {
return s == null || s.trim().isEmpty() ? defaultString : s;
}
#Override
public int compareTo(Key o) {
int cmp = sometimesPopulated.compareTo(o.sometimesPopulated);
if (cmp == 0)
cmp = alwaysPopulated.compareTo(o.alwaysPopulated);
return cmp;
}
}

I think your equals, hashCode and compareTo methods should only use the field that is always populated. It's the only way to ensure the same object will always be found in the map regardless of if its optional field is set or not.
Second option, you could write an utility method that tries to find the value in the map, and if no value is found, tries again with the same key but with (or without) the optional field set.

Strange Java HashMap behavior - can't find matching object

I've been encountering some strange behavior when trying to find a key inside a java.util.HashMap, and I guess I'm missing something. The code segment is basically:
HashMap<Key, Value> data = ...
Key k1 = ...
Value v = data.get(k1);
boolean bool1 = data.containsKey(k1);
for (Key k2 : data.keySet()) {
boolean bool2 = k1.equals(k2);
boolean bool3 = k2.equals(k1);
boolean bool4 = k1.hashCode() == k2.hashCode();
break;
}
That strange for loop is there because for a specific execution I happen to know that data contains only one item at this point and it is k1, and indeed bool2, bool3 and bool4 will be evaluated to true in that execution. bool1, however, will be evaluated to false, and v will be null.
Now, this is part of a bigger program - I could not reproduce the error on a smaller sample - but still it seems to me that no matter what the rest of the program does, this behavior should never happen.
EDIT: I have manually verified that the hash code does not change between the time the object was inserted to the map and the time it was queried. I'll keep checking this venue, but is there any other option?

This behavior could happen if the hash code of the key were changed after it was inserted in to the map.
Here's an example with the behavior you described:
public class Key
{
int hashCode = 0;
#Override
public int hashCode() {
return hashCode;
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Key other = (Key) obj;
return hashCode == other.hashCode;
}
public static void main(String[] args) throws Exception {
HashMap<Key, Integer> data = new HashMap<Key, Integer>();
Key k1 = new Key();
data.put(k1, 1);
k1.hashCode = 1;
boolean bool1 = data.containsKey(k1);
for (Key k2 : data.keySet()) {
boolean bool2 = k1.equals(k2);
boolean bool3 = k2.equals(k1);
boolean bool4 = k1.hashCode() == k2.hashCode();
System.out.println("bool1: " + bool1);
System.out.println("bool2: " + bool2);
System.out.println("bool3: " + bool3);
System.out.println("bool4: " + bool4);
break;
}
}
}

From the API description of the Map interface:
Note: great care must be exercised if
mutable objects are used as map keys.
The behavior of a map is not specified
if the value of an object is changed
in a manner that affects equals
comparisons while the object is a key
in the map. A special case of this
prohibition is that it is not
permissible for a map to contain
itself as a key. While it is
permissible for a map to contain
itself as a value, extreme caution is
advised: the equals and hashCode
methods are no longer well defined on
such a map.
Also, there are very specific requirements on the behavior of equals() and hashCode() for types used as Map keys. Failure to follow the rules here will result in all sorts of undefined behavior.

If you're certain the hash code does not change between the time the key is inserted and the time you do the contains check, then there is something seriously wrong somewhere. Are you sure you're using a java.util.HashMap and not a subclass of some sort? Do you know what implementation of the JVM you are using?
Here's the source code for java.util.HashMap.getEntry(Object key) from Sun's 1.6.0_20 JVM:
final Entry<K,V> getEntry(Object key) {
int hash = (key == null) ? 0 : hash(key.hashCode());
for (Entry<K,V> e = table[indexFor(hash, table.length)];
e != null;
e = e.next) {
Object k;
if (e.hash == hash &&
((k = e.key) == key || (key != null && key.equals(k))))
return e;
}
return null;
As you can see, it retrieves the hashCode, goes to the corresponding slot in the table, then does an equals check on each element in that slot. If this is the code you're running and the hash code of the key has not changed, then it must be doing an equals check which must be failing.
The next step would be for you to give us some more code or context - the hashCode and equals methods of your Key class at a minimum.
Alternatively, I would recommend hooking up to a debugger if you can. Watch what bucket your key is hashed to, and step through the containsKey check to see where it's failing.

Is this application multi-threaded? If so, another thread could change the data between the data.containsKey(k1) call and the data.keySet() call.

If equals() returns true for two objects, then hashCode() should return the same value. If equals() returns false, then hashCode() should return different values.
For Reference:
http://www.ibm.com/developerworks/java/library/j-jtp05273.html

Perhaps the Key class looks like
Key
{
boolean equals = false ;
public boolean equals ( Object oth )
{
try
{
return ( equals ) ;
}
finally
{
equals = true ;
}
}
}