Develop Reference

How to remove duplicate from list of object in java using stream API

I have searched for a solution for how to remove duplicates from a list
with Stream API
found only this question
How to remove duplicates from list of objects by id
I have a list of Person i need to filter by the person name, tried with the below snippet but it doesn't filter by the name
private static Map<Person, Integer> getUniqueByBiggerId(Collection<Person> persons) {
return persons.stream().collect(Collectors.toMap(
persons ->
persons,
Person::getId,
(id1, id2) -> {
if (id2 > id1)
return id2;
else
return id1;
}
));
}
public static void main(String args[]){
//.........
List<Person> Persons
= Arrays.asList(Person1, Person2,Person2,Person1,Person2, Person1);
getUniqueByBiggerId(Persons);
}

You were very close to the solution:
public class Test {
public static void main(String[] args) throws Exception {
List<Person> persons = Arrays.asList(new Person("a", 2), new Person("b", 1), new Person("a", 3));
persons = getUniqueByBiggerId(persons);
System.out.println(persons);
}
private static List<Person> getUniqueByBiggerId(Collection<Person> persons) {
return new ArrayList<>(
persons.stream().collect(Collectors.toMap(Person::getName, Function.identity(), (p1, p2) -> {
if (p1.getId() > p2.getId())
return p1;
else
return p2;
})).values());
}
}
record Person(String name, int id) {
public String getName() {
return name;
}
public int getId() {
return id;
}
}
Output:
[Person[name=a, id=3], Person[name=b, id=1]]

private static Map<String, Person> getUniqueByBiggerId(Collection<Person> person) {
return harmonizedDimensions.stream().collect(Collectors.toMap(()
person ->
person.getName(),
Function.identity(),
(id1, id2) -> {
if (id2.getId() > id1.getId())
return id2;
else
return id1;
}
)).values();
}

Assuming you have a Person class which has fields id and name and the field id is the field you want to filter by you just need to override equals and hashCode methods:
public class Person {
private final int id;
private final String name;
public Person(int id, String name) {
this.id = id;
this.name = name;
}
public int getId() {
return id;
}
public String getName() {
return name;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Person person = (Person) o;
return id == person.id;
}
#Override
public int hashCode() {
return Objects.hash(id);
}
Then you can just create a List:
Person personOne = new Person(1, "Name1");
Person personTwo = new Person(2, "Name2");
Person personThree = new Person(1, "Name3");
List<Person> people = Arrays.asList(personOne, personTwo, personThree);
And to eliminate duplicates just write:
List<Person> collect = people.stream().distinct().collect(Collectors.toList());
System.out.println(collect.size());
That will produce the output 2.

Eliminate duplicate from List containing objects using collection in Java

I have created a student class with details studentId, StudentName and studentSemester. Then added a constructor to create new students. in the main method i have given the below.
Student s1 = new Student(101, "Ron", 1);
Student s2 = new Student(102, "Angela", 9);
Student s3 = new Student(103, "Hazel", 5);
Student s4 = new Student(104, "David", 3);
Student s5 = new Student(104, "Alan", 4);
Student s6 = new Student(103, "Hazel", 5);
Student s7 = new Student(101, "Ron", 1);
List<Student> reEvaluationDetails = new ArrayList<Student>();
reEvaluationDetails.add(s1);
reEvaluationDetails.add(s2);
reEvaluationDetails.add(s3);
reEvaluationDetails.add(s4);
reEvaluationDetails.add(s5);
reEvaluationDetails.add(s6);
reEvaluationDetails.add(s7);
from the list, i have to eliminate duplicates by using another collection. I have tried adding the elements to a LinkedHashset but it doesnt remove the duplicates. Could you please suggest any solutions.

Implement equals and hashCode methods in Strudent and the use Set,
class Student {
public String name;
public int id;
public int semester;
public Student(int id, String name, int semester) {
this.id = id;
this.name = name;
this.semester = semester;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Student student = (Student) o;
return id == student.id &&
semester == student.semester &&
Objects.equals(name, student.name);
}
#Override
public int hashCode() {
return Objects.hash(name, id, semester);
}
}
Then use Set,
LinkedHashSet<Student> students = new LinkedHashSet<>(reEvaluationDetails);

You should consider using distinct doc
reEvaluationDetails .stream().distinct().collect(Collectors.toList()).
The equals() method of you student class should be overriden to detect equality between Student object

converting the ArrayList to a HashSet effectively removes duplicates, if you need to preserve insertion order you can also use this
Set<String> reEvaluationDetails = new LinkedHashSet<>(list);
reEvaluationDetails.add(s1);
reEvaluationDetails.add(s2);
reEvaluationDetails.add(s3);
reEvaluationDetails.add(s4);
reEvaluationDetails.add(s5);
reEvaluationDetails.add(s6);
reEvaluationDetails.add(s7);

Use a Set.
A Set is a Collection that, unlike List, does not allow duplicates.
Set<Student> reEvaluationDetails = new HashSet<Student>();
reEvaluationDetails.add(s1);
reEvaluationDetails.add(s2);
reEvaluationDetails.add(s3);
reEvaluationDetails.add(s4);
reEvaluationDetails.add(s5);
reEvaluationDetails.add(s6);
reEvaluationDetails.add(s7);
And override the equals and hashcode:
class Student {
int id;
String name;
int whatever;
Student(int id, String name, int whatever) {
this.id = id;
this.name = name;
this.whatever = whatever;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Student student = (Student) o;
return id == student.id &&
whatever == student.whatever &&
Objects.equals(name, student.name);
}
#Override
public int hashCode() {
return Objects.hash(id, name, whatever);
}
}
When you check what's in the Set you'll see that there are no duplicates.
It's the most efficient way of doing it.
You can check it running here.

Remove Duplicate Elements from ArrayList

We have a ArrayList. It contains duplicate employee objects with only difference in their age, but name and id will be same. So while removing the duplicates, we have to keep the employee with maximum age and remove all other duplicates. This is one of the questions asked by an interviewer in one of the interviews.
I tried solving this. It is giving me correct result, but I am not sure with my approach as I am changing the state of the object in equals method. Is there any other approach to solve this problem?
Code Snippet below: -
package practice;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
class Employee {
private int id;
private String name;
private int age;
Employee(int id, String name, int age) {
this.id = id;
this.name = name;
this.age = age;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
#Override
public int hashCode() {
return (31*(name.hashCode()) + 31);
}
#Override
public boolean equals(Object obj) {
if ((obj instanceof Employee)) {
if (((Employee)obj).getId() == this.id && (((Employee)obj).getName().equalsIgnoreCase(this.name))) {
if(this.age > ((Employee)obj).getAge()) {
((Employee)obj).setAge(this.age);
}
return true;
} else
return false;
} else
return false;
}
}
public class ListDuplicateRemoval {
public static List<Employee> removeDuplicates(List<Employee> employees) {
Set<Employee> set = new HashSet<>();
for (int i = 0; i < employees.size(); i++) {
set.add(employees.get(i));
}
/*for (int i = 0; i < set.size(); i++) {
System.out.println(set.iterator().next().getAge());
}*/
employees.removeAll(employees);
employees.addAll(set);
return employees;
}
public static void main(String[] args) {
Employee e1 = new Employee(1, "Mike", 20);
Employee e2 = new Employee(1, "Mike", 21);
List <Employee> list = new ArrayList<>();
list.add(e1);
list.add(e2);
removeDuplicates(list);
System.out.println(list.size());
System.out.println(list.get(0).getAge());
}
}

No this solution is really awful. equals should never, ever, modify the state of the objects it's comparing.
Create a class containing the information that identifies employees uniquely, and which properly override equal() and hashCode(). Then use a Map containing these identication info as key, and the employee with the largest age as value. Then get the values and make it a list:
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.function.Function;
import java.util.stream.Collectors;
class Employee {
private int id;
private String name;
private int age;
Employee(int id, String name, int age) {
this.id = id;
this.name = name;
this.age = age;
}
int getId() {
return id;
}
String getName() {
return name;
}
int getAge() {
return age;
}
#Override
public String toString() {
return "Employee{" +
"id=" + id +
", name='" + name + '\'' +
", age=" + age +
'}';
}
}
class DuplicateRemoval {
public static void main(String[] args) {
List<Employee> employeeList = Arrays.asList(
new Employee(1, "Joe", 23),
new Employee(2, "Joe", 23),
new Employee(1, "Joe", 21),
new Employee(1, "Jane", 22),
new Employee(1, "Jane", 20)
);
Map<EmployeeKey, Employee> map = employeeList.stream().collect(
Collectors.toMap(e -> new EmployeeKey(e.getId(), e.getName()),
Function.identity(),
(e1, e2) -> e1.getAge() > e2.getAge() ? e1 : e2)
);
List<Employee> result = new ArrayList<>(map.values());
System.out.println("result = " + result);
}
private static class EmployeeKey {
private int id;
private String name;
EmployeeKey(int id, String name) {
this.id = id;
this.name = name;
}
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (o == null || getClass() != o.getClass()) {
return false;
}
EmployeeKey that = (EmployeeKey) o;
return id == that.id &&
Objects.equals(name, that.name);
}
#Override
public int hashCode() {
return Objects.hash(id, name);
}
}
}

Implement a Comparator<Employee>
Have the compare method take into account age.
Have the equals method ignore age.
use equals to identify duplicates.
use compare to determine which duplicate to keep.

How to add object with unique field to Set

How can I fill a Set by objects with unique field ?
For example I have a class Person which has an unique field called name thus if I add to Set an object with duplicate name it should not be added.
public class Test {
public static void main(String[] args) {
// TODO Auto-generated method stub
Set<Person> objList = new HashSet<Person>();
objList.add(new Person("John", "New York", "Accountant"));
objList.add(new Person("Bill", "London", "Manager"));
objList.add(new Person("John", "New York", "Driver"));// this object should not be added
for(Person o : objList){
System.out.println(o.name);//here should printed only John and Bill
}
}
}
class Person {
String name;//must be unique
String city;
String position;
public Person(String c_name, String c_city, String c_position){
this.name = c_name;
this.city = c_city;
this.position = c_position;
}
}

In other words, you mean the person's name defines it's identity. You can produce such a behavior by overriding the equals(Object) and hashCode methods to include only it:
public class Person {
// members, constructors, etc.
#Override
public boolean equals(Object other) {
if (!(other instanceof Person)) {
return false;
}
return name.equals(((Person)other).name);
}
#Override
public int hashCode() {
return name.hashCode();
}
}

You need to overrride the equals() and hashcode() methods of Object class in Person class to achive this.
Since you have not done that for HashSet all object of Person class is different and hashSet will allow you to add all Person.
Below is the code snippet for Person class for unique name only.
class Person{
String name;//must be unique
String city;
String position;
public Person(String c_name, String c_city, String c_position){
this.name = c_name;
this.city = c_city;
this.position = c_position;
}
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result + ((name == null) ? 0 : name.hashCode());
return result;
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (getClass() != obj.getClass())
return false;
Person other = (Person) obj;
if (name == null) {
if (other.name != null)
return false;
} else if (!name.equals(other.name))
return false;
return true;
}
}

You can use UnifiedSetWithHashingStrategy from Eclipse Collections to define your own custom hashing strategy for your Set. The following code will work for your example.
HashingStrategy<Person> uniqueNameStrategy = new HashingStrategy<Person>() {
#Override
public int computeHashCode(Person person) {
return person.name.hashCode();
}
#Override
public boolean equals(Person person1, Person person2) {
return person1.name.equals(person2.name);
}
};
UnifiedSetWithHashingStrategy<Person> people =
UnifiedSetWithHashingStrategy.newSet(uniqueNameStrategy);
people.add(new Person("John", "New York", "Accountant"));
people.add(new Person("Bill", "London", "Manager"));
people.add(new Person("John", "New York", "Driver"));
people.each(person -> System.out.println(person.name + "/" + person.position));
// Bill/Manager
// John/Accountant
Note: I am a committer for Eclipse Collections .

Which collection for sorted unique dataset?

I want to have a Collection with the following requirements:
1.) before inserting an element it checks if there is already a element with the same name, if there is, the element is not inserted
2.) after inserting, the collection is automatically sorted by the salary of the persons in the collection
I insert persons with name and salary as attributes.
Which collection does fit these requirements?

What about a TreeSet ?
You may need to use the constructor with a Comparator.

You can use TreeSet and implement Comparator to sort the needed proprty. See
this sample.
To discard the duplicate entry, you need to return 0 value;
public int compare(Empl e1, Empl e2) {
if(e1.getName().equals(e2.getName())) {
return 0;
}else if(e1.getSalary() > e2.getSalary()){
return 1;
}else if(e1.getSalary() == e2.getSalary() && (e1.getName().compareTo(e2.getName()) > 1)) {
return 1; // if the salary is equal, sort it based on name.
}
else {
return -1;
}
}

Louis is wrong!
A SortedSet would be the perfect.
You only have to implement the Comparable interface and compare the salary of the people.

it will be fine for the SortedSet, but don't forget to implements the interface COMPARABLE.

You can use a TreeSet over a HashSet, but Person has to implement Comparable interface (int compareTo(Person o) method) and override equals and hashCode methods to check the identity/unique based only on name field.
First create Person and implement/overrride methods :
public class Person implements Comparable<Person> {
public Person(String name, double salary) {
this.name = name;
this.salary = salary;
}
public Person() {
}
private String name;
private Double salary;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public double getSalary() {
return salary;
}
public void setSalary(double salary) {
this.salary = salary;
}
#Override
public String toString() {
return "Person{" + "name=" + name + ", salary=" + salary + '}';
}
public int compareTo(Person o) {
return salary.compareTo(o.getSalary());
}
#Override
public int hashCode() {
int hash = 7;
hash = 53 * hash + (this.name != null ? this.name.hashCode() : 0);
return hash;
}
#Override
public boolean equals(Object obj) {
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
final Person other = (Person) obj;
if ((this.name == null) ? (other.name != null) : !this.name.equals(other.name)) {
return false;
}
return true;
}
}
then create a HashSet and fill it with Person elements. The set will contain only elements with unique name attribute, that's done by using only name field in equals and hashCode methods.
To keep collection sorted - create a TreeSet using parameterized constructor - new TreeSet(Collection coll) , you have collection already populated with unique elements, so the sorting will take place during TreeSet initialization.
Here is the rest of snippet
public class TestPerson {
public static void main(String [] args){
Person p1 = new Person("first", 1000);
Person p2 = new Person("second", 2000);
Person p3 = new Person("third", 3000);
Person p4 = new Person("first", 4000);
Person p5 = new Person("second", 5000);
TreeSet<Person> personSet = new TreeSet<Person>();
personSet.add(p1);
personSet.add(p2);
personSet.add(p3);
personSet.add(p4);
personSet.add(p5);
for (Person person : personSet){
System.out.println(" === person element sorted : "+person);
}
HashSet<Person> personHashSet = new HashSet<Person>();
personHashSet.add(p1);
personHashSet.add(p2);
personHashSet.add(p3);
personHashSet.add(p4);
personHashSet.add(p5);
TreeSet<Person> treePersonSet = new TreeSet<Person>(personHashSet);
for (Person person : treePersonSet){
System.out.println(" === person element: "+person);
}
}
}