Develop Reference

Comparing two large lists in java

I have to Array lists with 1000 objects in each of them. I need to remove all elements in Array list 1 which are there in Array list 2. Currently I am running 2 loops which is resulting in 1000 x 1000 operations in worst case.
List<DataClass> dbRows = object1.get("dbData");
List<DataClass> modifiedData = object1.get("dbData");
List<DataClass> dbRowsForLog = object2.get("dbData");
for (DataClass newDbRows : dbRows) {
boolean found=false;
for (DataClass oldDbRows : dbRowsForLog) {
if (newDbRows.equals(oldDbRows)) {
found=true;
modifiedData.remove(oldDbRows);
break;
}
}
}
public class DataClass{
private int categoryPosition;
private int subCategoryPosition;
private Timestamp lastUpdateTime;
private String lastModifiedUser;
// + so many other variables
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
DataClass dataClassRow = (DataClass) o;
return categoryPosition == dataClassRow.categoryPosition
&& subCategoryPosition == dataClassRow.subCategoryPosition && (lastUpdateTime.compareTo(dataClassRow.lastUpdateTime)==0?true:false)
&& stringComparator(lastModifiedUser,dataClassRow.lastModifiedUser);
}
public String toString(){
return "DataClass[categoryPosition="+categoryPosition+",subCategoryPosition="+subCategoryPosition
+",lastUpdateTime="+lastUpdateTime+",lastModifiedUser="+lastModifiedUser+"]";
}
public static boolean stringComparator(String str1, String str2){
return (str1 == null ? str2 == null : str1.equals(str2));
}
public int hashCode() {
int hash = 7;
hash = 31 * hash + (int) categoryPosition;
hash = 31 * hash + (int) subCategoryPosition
hash = 31 * hash + (lastModifiedUser == null ? 0 : lastModifiedUser.hashCode());
return hash;
}
}
The best work around i could think of is create 2 sets of strings by calling tostring() method of DataClass and compare string. It will result in 1000 (for making set1) + 1000 (for making set 2) + 1000 (searching in set ) = 3000 operations. I am stuck in Java 7. Is there any better way to do this? Thanks.

Let Java's builtin collections classes handle most of the optimization for you by taking advantage of a HashSet. The complexity of its contains method is O(1). I would highly recommend looking up how it achieves this because it's very interesting.
List<DataClass> a = object1.get("dbData");
HashSet<DataClass> b = new HashSet<>(object2.get("dbData"));
a.removeAll(b);
return a;
And it's all done for you.
EDIT: caveat
In order for this to work, DataClass needs to implement Object::hashCode. Otherwise, you can't use any of the hash-based collection algorithms.
EDIT 2: implementing hashCode
An object's hash code does not need to change every time an instance variable changes. The hash code only needs to reflect the instance variables that determine equality.
For example, imagine each object had a unique field private final UUID id. In this case, you could determine if two objects were the same by simply testing the id value. Fields like lastUpdateTime and lastModifiedUser would provide information about the object, but two instances with the same id would refer to the same object, even if the lastUpdateTime and lastModifiedUser of each were different.
The point is that if you really want to want to optimize this, include as few fields as possible in the hash computation. From your example, it seems like categoryPosition and subCategoryPosition might be enough.
Whatever fields you choose to include, the simplest way to compute a hash code from them is to use Objects::hash rather than running the numbers yourself.

It is a Set A-B operation(only retain elements in Set A that are not in Set B = A-B)
If using Set is fine then we can do like below. We can use ArrayList as well in place of Set but in AL case for each element to remove/retain check it needs to go through an entire other list scan.
Set<DataClass> a = new HashSet<>(object1.get("dbData"));
Set<DataClass> b = new HashSet<>(object2.get("dbData"));
a.removeAll(b);
If ordering is needed, use TreeSet.
Try to return a set from object1.get("dbData") and object2.get("dbData") that skips one more intermediate collection creation.

Binary search not detecting duplicates?

I have an array of items, cards, all with String names, so
Card c1= new card("TheCard")
Card c2= new card("TheOtherCard")
And then I am using a quicksort to sort the list and then trying a binary search to see if cards already exist before adding more
So,
if(cards.contains(c3)==true)
//do nothing
else
cards.add(c3)
And my cards.contains method is
Comparator<Card> c = new Comparator<Card>() {
#Override
public int compare(Card u1, Card u2) {
return u1.getName().compareTo(u2.getName());
}
};
int index;
index = Collections.binarySearch(cards, it, c);
if (index == -1) {
return false;
} else {
return true;
}
But the problem is that it's searching the cards array, finding cards that aren't in the list and saying they are and saying cards that are in the list aren't
I am trying to add 10,000 cards, 8,000 of them being unique, but the contains method is returning 2,000 unique cards and when I check the list, they're not even unique https://i.imgur.com/N9kQtms.png
I've tried running the code un-sorted and that just returns about 4,000 results with the same problem of repeating cards, when I brute force and just use the base .contains, that works but it is super slow
(Also sorry if I messed up something in my post, it is my first time posting here)

The javadoc states the following:
Searches the specified list for the specified object using the binary search algorithm. The list must be sorted into ascending order according to the specified comparator (as by the sort(List, Comparator) method), prior to making this call. If it is not sorted, the results are undefined. If the list contains multiple elements equal to the specified object, there is no guarantee which one will be found.
It also states that it returns:
the index of the search key, if it is contained in the list; otherwise, (-(insertion point) - 1). The insertion point is defined as the point at which the key would be inserted into the list: the index of the first element greater than the key, or list.size() if all elements in the list are less than the specified key. Note that this guarantees that the return value will be >= 0 if and only if the key is found.
Your list should be therefore sorted beforehand or it won't return anything that make sense. Then you, it does return either the index or the insertion point of the element. Beware of this technicality. You should check after execution that the element at the index is in fact the correct one and not only the index at which you would insert your element it.
There you could have this test to see if it is your card:
// Test if the card at the index found has got the same name than the card you are actually looking for.
return !index == cards.length && cards[index].getName().equals(it.getName()));
You could also override equals to have something that is closer to:
return !index == cards.length && cards[index].equals(it);
In both case, we ensure that we won't have an ArrayOutOfBoundException if the insertion point is at the end of the list.

The binarySearch gives a non-negative index when it finds an item.
It gives the complement of the insert position: ~index == -index-1 when it is not found.
Search d in a b d e gives 2.
Search d in a b e g gives ~2 == -3, the insert position being 2.
So the check is:
int index = Collections.binarySearch(cards, it, c);
return index >= 0;
Furthermore Card should have a correct equality:
public class Card implements Comparable<Card> {
...
#Override
public int compareTo(Card other) {
return name.compareTo(other.name);
}
#Override
public boolean equals(Object obj) {
if (!(obj instanceOf Card)) {
return false;
}
Card other = (Card) obj;
return name.equals(other.name);
}
#Override
public int hashCode() {
return name.hashCode();
}
}
In this case instead of a Comparator you can implement Comparable<Card> as the name is the read identification of a card. Comparator is more for sorting persons on last name + first name, or first name + last name, or on city.
The hashCode allows using HashMap<Card, ...>.

Why does .contains method on ArrayList of custom instances work?

I've been developing a small application for work, and I've come across something I can't figure out.
In the following code, I have an ArrayList of a Custom Class called 'Product' that contains data of type 'String'. I use the .contains method on this ArrayList to ensure it doesn't contain a certain String.
My IDE gives me the warning of 'Suspicious call to java.util.Collections.contains: Given object cannot contain instances of String (expected Product)'.
I completely understand the above message, because I'm comparing two different Types, so how can it ever evaluate correctly? I'm thinking it must be because the 'Product' class contains the data I want to compare, it is defaulting to using the toString method on the Product class (I override this in the Class) and comparing it with the String I want to compare it against.
It seems like JVM black magic to me.
private void createOrderListing(List<String[]> orderList)
{
//For each line of the order list file
for(String[] s : orderList)
{
if(s.length >= 28) //OrderLine should be of this length
{
if (!s[0].equalsIgnoreCase("ProductCode") && !s[0].isEmpty()) //Makes sure we're not including headers
{
//How does this bit work?
if(!productListing.contains(s[0]))
{
OrderLine order = new OrderLine();
//References product code of Product against Order Line, if match, then pack sizes and other basic fields ammended as appropriate
boolean productFound = false;
for (Product p : productListing)
{
if (s[0].contentEquals(p.getProductCode()))
{
order.initialAmendOrderLine(p.getProductCode(), p.getProductName(), p.getPackSize(), p.getProductType());
productFound = true;
}
}
if(productFound)
{
order.setOrderValues(s);
orderListing.add(order);
}
}
//System.out.println("\nOrder Product is: " + order.getProductName()+ "\nOrder Pack Size is: " + order.getInternalPackSize());
}
}
}
}
UPDATE
The reason this works as pointed out in the comments is that the block is always true (the .contains method is always false, the ! inverses this, hence true). Sorry for the confusion and pointing out my carelessness.

Here is an implementation of contains method in ArrayList that I have in OpenJDK:
public boolean contains(Object o) {
return indexOf(o) >= 0;
}
public int indexOf(Object o) {
if (o == null) {
for (int i = 0; i < size; i++)
if (elementData[i]==null)
return i;
} else {
for (int i = 0; i < size; i++)
if (o.equals(elementData[i]))
return i;
}
return -1;
}
Basically, there is nothing complex in it. It iterates through the all elements of your ArrayList and checks whether your given object is equal to the current one. If the condition is true then element exists in the list.
So let's imagine that you are passing String "SomeValue" to this method. Elements of ArrayList are iterated and following action is executed: "SomeValue".equals(elementData[i]) where elementData[i] is a product.
Since equals method of String class cannot compare String with a Product it returns false and as a result, you get false from contains method.
To fix this situation you can iterate over ArrayList manually and compare some Product's field with your string. E.g. you can implement following contains method:
public boolean contains(List<Product> products, String yourStringValue) {
for (Product p : products) {
if(p.getProductCode().equals(yourStringValue)){
return true;
}
}
return false;
}

productListing is a list of Product objects. Yet you are asking the list if it contains a specific String object -- which shouldn't ever happen.
What you should do is check if your Product#getProductCode is equal to your specific String. This can be acheived by using streams:
if(!productListing.contains(s[0])) // replace this
// with this
if (!productListing.stream().filter(o -> o.getProductCode().equals(s[0])).findFirst().isPresent())
What does this code do? It checks all your Product elements to find one whose myStringData attribute is equal to the String you're comparing.

since contains relays on equals implementation, when you do
if(!productListing.contains(s[0]))
you are asking the list OF ARRAYS OF STRINGS if its contains a String.
that will return always false because the type are different, so is not that is working at all, is that your condition will always return false

Correct way to implement Map<MyObject,ArrayList<MyObject>>

I was asked this in interview. using Google Guava or MultiMap is not an option.
I have a class
public class Alpha
{
String company;
int local;
String title;
}
I have many instances of this class (in order of millions). I need to process them and at the end find the unique ones and their duplicates.
e.g.
instance --> instance1, instance5, instance7 (instance1 has instance5 and instance7 as duplicates)
instance2 --> instance2 (no duplicates for instance 2)
My code works fine
declare datastructure
HashMap<Alpha,ArrayList<Alpha>> hashmap = new HashMap<Alpha,ArrayList<Alpha>>();
Add instances
for (Alpha x : arr)
{
ArrayList<Alpha> list = hashmap.get(x); ///<<<<---- doubt about this. comment#1
if (list == null)
{
list = new ArrayList<Alpha>();
hashmap.put(x, list);
}
list.add(x);
}
Print instances and their duplicates.
for (Alpha x : hashmap.keySet())
{
ArrayList<Alpha> list = hashmap.get(x); //<<< doubt about this. comment#2
System.out.println(x + "<---->");
for(Alpha y : list)
{
System.out.print(y);
}
System.out.println();
}
Question: My code works, but why? when I do hashmap.get(x); (comment#1 in code). it is possible that two different instances might have same hashcode. In that case, I will add 2 different objects to the same List.
When I retrieve, I should get a List which has 2 different instances. (comment#2) and when I iterate over the list, I should see at least one instance which is not duplicate of the key but still exists in the list. I don't. Why?. I tried returning constant value from my hashCode function, it works fine.
If you want to see my implementation of equals and hashCode,let me know.
Bonus question: Any way to optimize it?
Edit:
#Override
public boolean equals(Object obj) {
if (obj==null || obj.getClass()!=this.getClass())
return false;
if (obj==this)
return true;
Alpha guest = (Alpha)obj;
return guest.getLocal()==this.getLocal()
&& guest.getCompany() == this.getCompany()
&& guest.getTitle() == this.getTitle();
}
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + (title==null?0:title.hashCode());
result = prime * result + local;
result = prime * result + (company==null?0:company.hashCode());
return result;
}

it is possible that two different instances might have same hashcode
Yes, but hashCode method is used to identify the index to store the element. Two or more keys could have the same hashCode but that's why they are also evaluated using equals.
From Map#containsKey javadoc:
Returns true if this map contains a mapping for the specified key. More formally, returns true if and only if this map contains a mapping for a key k such that (key==null ? k==null : key.equals(k)). (There can be at most one such mapping.)
Some enhancements to your current code:
Code oriented to interfaces. Use Map and instantiate it by HashMap. Similar to List and ArrayList.
Compare Strings and Objects in general using equals method. == compares references, equals compares the data stored in the Object depending the implementation of this method. So, change the code in Alpha#equals:
public boolean equals(Object obj) {
if (obj==null || obj.getClass()!=this.getClass())
return false;
if (obj==this)
return true;
Alpha guest = (Alpha)obj;
return guest.getLocal().equals(this.getLocal())
&& guest.getCompany().equals(this.getCompany())
&& guest.getTitle().equals(this.getTitle());
}
When navigating through all the elements of a map in pairs, use Map#entrySet instead, you can save the time used by Map#get (since it is supposed to be O(1) you won't save that much but it is better):
for (Map.Entry<Alpha, List<Alpha>> entry : hashmap.keySet()) {
List<Alpha> list = entry.getValuee();
System.out.println(entry.getKey() + "<---->");
for(Alpha y : list) {
System.out.print(y);
}
System.out.println();
}

Use equals along with hashCode to solve the collision state.
Steps:
First compare on the basis of title in hashCode()
If the title is same then look into equals() based on company name to resolve the collision state.
Sample code
class Alpha {
String company;
int local;
String title;
public Alpha(String company, int local, String title) {
this.company = company;
this.local = local;
this.title = title;
}
#Override
public int hashCode() {
return title.hashCode();
}
#Override
public boolean equals(Object obj) {
if (obj instanceof Alpha) {
return this.company.equals(((Alpha) obj).company);
}
return false;
}
}
...
Map<Alpha, ArrayList<Alpha>> hashmap = new HashMap<Alpha, ArrayList<Alpha>>();
hashmap.put(new Alpha("a", 1, "t1"), new ArrayList<Alpha>());
hashmap.put(new Alpha("b", 2, "t1"), new ArrayList<Alpha>());
hashmap.put(new Alpha("a", 3, "t1"), new ArrayList<Alpha>());
System.out.println("Size : "+hashmap.size());
Output
Size : 2

Recursive isMember method with only two arguments!

I need to create a recursive Boolean method named isMemeber. The method should accept two arguments ONLY: an array and a value. The method should return true if the value is found in the array, or false if the value is not found in the array.
I think that the base case will be if the passed array is empty, but I need help with the recursive case:
public static boolean isMember(int[] array, int value)
{
if(array.length==0){
return false;
}else{
return isMember(???);
}
}
Here is how it looks with position variable:
public static boolean isMember(int[] array, int value, int position)
{
if (position > -1)
{
if (array[position] == value)
{
return true;
}
else
{
return isMember(array, value, position - 1);
}
}
return false;
}

If you need to use recursion you can copy the array on each recursion. This is inefficent, but using recursion is inefficient compared with using a loop. e.g. Arrays.indexOf()
public static boolean isMember(int[] array, int value) {
if(array.length == 0) return false;
if(array[0] == value) return true;
int[] array2 = new int[array.length-1];
System.arraycopy(array,1,array2,0,array2.length);
return isMember(array2, value);
}

There is a slight issue with your problem. If you are going to use recursion then each array element needs to have a subsey of elements otherwise whay do you passed to the recursive method? If this is not the casr and the case is as you stated then solving this problem with recursion isnot appropriate. Also you are missing the value comparison.

See the MSDN Array class. This looks like it is c#. Maybe try the Array.Find<T> method.
Update:
For Java, I'd recommend looking at Arrays (Java 2 Platform):
binarySearch
public static int binarySearch(int[]
a,
int key)
Searches the specified array of ints for the specified value using the binary search algorithm. The array must be sorted (as by the sort method above) prior to making this call. If
it is not sorted, the results are
undefined. If the array contains
multiple elements with the specified
value, there is no guarantee which one
will be found.
Parameters:
a - the array to be searched.
key - the value to be searched for.
Returns:
index of the search key, if it is contained in the list; otherwise,> (-(insertion point) - 1).
The insertion point is defined as the point at which the key would be inserted into the list: the index of the first element greater than the key, or list.size(), if all elements
in the list are less than the specified key. Note that this guarantees that the return value will be >= 0 if and only if the key is found. See Also: sort(int[])

If this is homework and they want it recursive, then maybe you should:
1 look for the middle value of the array and check if it matches. If it matches, return true
2 apply the function to the first half of the array. If it returns true, return true
3 apply the function to the second half of the aray. If it returns true, return true
4 return false
No code since it is homework.
EDIT: Is the array ordered?

I was just doing the question, and checking answers for alternative ways. Maybe this might be useful when you have to match names to String arrays.
public class Recursion {
public static void main(String[] args) {
String[] array = {"Tom", "Mary"};
if(isMember(array,"John"))
System.out.print("Found!");
else
System.out.println("Not Found!");
}
public static boolean isMember(String[] array, String name)
{
int i = array.length;
if(array.length == 0)
return false;
if(array[i - 1].equals(name))
return true;
else
{
String[] array2 = new String[array.length - 1];
for(int b = 0; b< array.length -1; b++)
{
array2[b] = array[b];
}
return isMember(array2, name);
}
}
}