Develop Reference

Java: Encountering ConcurrentModificationException. There's something wrong with my iterators and/or HashMaps, and I don't know what it is

I am creating a program that takes two .txt files and prints out the words that appear in both texts and the number of times each shared word appears in each text. I declared two file objects that have valid paths. However, when I try to create two Scanner objects that use the two .txt files, I get FileNotFoundException compiler errors for both lines of code that are declaring the new Scanner objects.
FYI, I use scannerObject.hasNext() in a while loop that adds each word from scannerObject.Next() as a new key in a HashMap variable with a value of 1 or, if the word is already a key in the HashMap, increasing the value (number of occurrences) by 1.
I have tried running the following with both file paths and the simple program below runs without error and outputs "It worked! Hehehe":
import java.io.*;
import java.util.*;
public class readingFilesPractice {
public static void main(String[] args) {
try{
File x = new File("C:\\Users\\aravd.000\\Desktop\\Book1.txt");
Scanner sc = new Scanner(x);
while(sc.hasNext()){
System.out.println(sc.next());
}
sc.close();
System.out.println("It worked! Hehehe");
}
catch (Exception e){
System.out.println("Error!");
}
}
}
By the way, the .txt files has areas where there are multiple spaces in succession and stuff like "1.".
The code below runs into two FileNotFoundExceptions (without the try and catch blocks) and in Visual Studios, new Scanner(book1) and new Scanner(book2) have a red squiggly line that states "Unhandled exception type FileNotFoundExceptionJava(16777384)" when I hover over it with my mouse. My complete code for reference is below.
import java.io.*;
import java.util.*;
public class program1 {
public static void main(String[] args) {
try {
File book1 = new File("C:\\Users\\aravd.000\\Desktop\\Book1.txt");
File book2 = new File("C:\\Users\\aravd.000\\Desktop\\Book2.txt");
// Counting the number of occurences of each word in book1
Scanner readBook1 = new Scanner(book1);
HashMap<String, Integer> wordsInBook1 = new HashMap<String, Integer>();
while (readBook1.hasNext()) {
String word = readBook1.next();
if (wordsInBook1.containsKey(word)) {
int occurences = wordsInBook1.get(word) + 1;
wordsInBook1.put(word, occurences);
} else {
wordsInBook1.put(word, 1);
}
}
readBook1.close();
// Counting the number of occurences of each word in book2
Scanner readBook2 = new Scanner(book2);
HashMap<String, Integer> wordsInBook2 = new HashMap<String, Integer>();
while (readBook2.hasNext()) {
String word = readBook2.next();
if (wordsInBook2.containsKey(word)) {
int occurences = wordsInBook2.get(word) + 1;
wordsInBook2.put(word, occurences);
} else {
wordsInBook2.put(word, 1);
}
}
readBook2.close();
// Creating two iterators for each HashMap
Iterator wordsInB1Iter = wordsInBook1.entrySet().iterator();
Iterator wordsInB2Iter = wordsInBook2.entrySet().iterator();
// Running the wordsInB1Iter iterator to find and delete unique keys in
// wordsInBook1
while (wordsInB1Iter.hasNext()) {
Map.Entry pair = (Map.Entry) wordsInB1Iter.next();
if (!wordsInBook2.containsKey(pair.getKey())) {
wordsInBook1.remove(pair.getKey());
}
}
// Running the wordsInB2Iter iterator to find and delete unique keys
while (wordsInB2Iter.hasNext()) {
Map.Entry pair = (Map.Entry) wordsInB2Iter.next();
if (!wordsInBook1.containsKey(pair.getKey())) {
wordsInBook2.remove(pair.getKey());
}
}
System.out.println(wordsInBook1);
System.out.println(wordsInBook2);
} catch (Exception e) {
e.printStackTrace();
}
}
}
If the other parts of the code are broken, I wouldn't know because I haven't debugged that yet. If you find an error elsewhere, let me know if you want. Thank you for your effort and please let me know if there's anything that needs further clarification!
UPDATE: When I changed my catch block to Exception e and used the e.printStackTrace, my code outputted the following:
java.util.ConcurrentModificationException
at java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1493)
at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1526)
at java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1524)
at prorgam1.main(program1.java:50)
Link to error descriptions within the "PROBLEMS" tab in VisualStudios
The picture above may provide more details about the issues with my iterators and HashMaps.

The same answer than #Pedro Borges but
Please use generics! Your code is full of cast while it should not.
Use Iterator.remove() to remove current value instead of using the source collection. This is the reason your are getting a ConcurrentModificationException.
If you don't need the Map.Entry, you may use keySet() instead.
You are using Java > 8. If this is Java 11, you may also use var.
Your code:
Iterator<Map.Entry<String, Integer>>> wordsInB1Iter = wordsInBook1.entrySet().iterator();
Iterator<Map.Entry<String, Integer>>> wordsInB2Iter = wordsInBook2.entrySet().iterator();
// Running the wordsInB1Iter iterator to find and delete unique keys in
// wordsInBook1
while (wordsInB1Iter.hasNext()) {
Map.Entry<String,Integer> pair = wordsInB1Iter.next();
if (!wordsInBook2.containsKey(pair.getKey())) {
wordsInB1Iter.remove();
}
}
// Running the wordsInB2Iter iterator to find and delete unique keys
while (wordsInB2Iter.hasNext()) {
Map.Entry<String,Integer> pair = wordsInB2Iter.next();
if (!wordsInBook1.containsKey(pair.getKey())) {
wordsInB2Iter.remove();
}
}
And while I'm at it, you may also consider refactoring how your read words:
By using a method instead of duplicating the code
By using try with resource (Java 7++)
By using Map.merge (Java 8++)
As in:
void words(File file) {
try (Scanner scanner = new Scanner(file)) {
var result = new HashMap<String,Integer>();
while (scanner.hasNext()) {
var word = scanner.next();
result.merge(word, 1, Integer::sum); // or (a, b) -> a + b
}
return result;
}
}
You may (should?) use a MutableInteger (from common-lang3) to avoid unboxing from Integer to int for performance reasons.

The ConcurrentModificationException comes from the fact you are removing elements from a Set while you're iterating it. That happens because under the hood the iterator is backed by the set, it's not a copy of it.
One way to corner it, although not tremendously elegant is to iterate over a copy of the Set.
If you replace
Iterator wordsInB1Iter = wordsInBook1.entrySet().iterator();
Iterator wordsInB2Iter = wordsInBook2.entrySet().iterator();
with
Iterator wordsInB1Iter = new HashSet<>(wordsInBook1.entrySet()).iterator();
Iterator wordsInB2Iter = new HashSet<>(wordsInBook2.entrySet()).iterator();
you will no longer have concurrent modification.

Displaying word frequencies of 0 in an ArrayList

i'm looking for some assistance. I've made a program that uses two classes - that i've also made. The first class is called CollectionOfWords that reads in text-files and store the words contained in the text-files within a HashMap. The second is called WordFrequencies that calls an object called Collection from the CollectionOfWords class, which in turn reads in another document and to see if the documents contents are in the Collection. This then outputs an ArrayList with the frequencies counted in the document.
Whilst this works and returns the frequencies of the words found in both the collection and document, i'd like it to be able to produce zero values for the words that are in the collection, but not in the document, if that makes sense? For example, test3 returns [1, 1, 1], but i'd like it to return [1, 0, 0, 0, 1, 0, 1] - where the zeroes represent the words in the collection, but are not found in test3.
The test text-files i use can be found here:
https://drive.google.com/open?id=1B1cDpjmZZo01HizxJUSWSVIlHcQke2mU
Cheers
WordFrequencies
public class WordFrequencies {
static HashMap<String, Integer> collection = new HashMap<>();
private static ArrayList<Integer> processDocument(String inFileName) throws IOException {
// Rests collections frequency values to zero
collection.clear();
// Reads in the new document file to an ArrayList
Scanner textFile = new Scanner(new File(inFileName));
ArrayList<String> file = new ArrayList<String>();
while(textFile.hasNext()) {
file.add(textFile.next().trim().toLowerCase());
}
/* Iterates the ArrayList of words -and- updates collection with
frequency of words in the document */
for(String word : file) {
Integer dict = collection.get(word);
if (!collection.containsKey(word)) {
collection.put(word, 1);
} else {
collection.put(word, dict + 1);
}
}
textFile.close();
// Stores the frequency values in an ArrayList
ArrayList<Integer> values = new ArrayList<>(collection.values());
return values;
}
public static void main(String[] args) {
// Stores text files for the dictionary (collection of words)
List<String> textFileList = Arrays.asList("Test.txt", "Test2.txt");
// Declares empty ArrayLists for output of processDocument function
ArrayList<Integer> test3 = new ArrayList<Integer>();
ArrayList<Integer> test4 = new ArrayList<Integer>();
// Creates a new CollectionOfWords object called dictionary
CollectionOfWords dictionary = new CollectionOfWords(collection);
// Reads in the ArrayLists text files and processes it
for (String text : textFileList) {
dictionary.scanFile(text);
}
try {
test3 = processDocument("test3.txt");
test4 = processDocument("test4.txt");
} catch(IOException e){
e.printStackTrace();
}
System.out.println(test3);
System.out.println(test4);
}
}
CollectionOfWords
public class CollectionOfWords {
// Declare set in a higher scope (making it a property within the object)
private HashMap<String, Integer> collection = new HashMap<String, Integer>();
// Assigns the value of the parameter to the field of the same name
public CollectionOfWords(HashMap<String, Integer> collection) {
this.collection = collection;
}
// Gets input text file, removes white spaces and adds to dictionary object
public void scanFile(String textFileName) {
try {
Scanner textFile = new Scanner(new File(textFileName));
while (textFile.hasNext()) {
collection.put(textFile.next().trim(), 0);
}
textFile.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
}
public void printDict(HashMap<String, Integer> dictionary) {
System.out.println(dictionary.keySet());
}
}

I didn't go through the trouble of figuring out your entire code, so sorry if this answer is stupid.
As a solution to your problem, you could initialize the map with every word in the dictionary mapping to zero. Right now, you use the clear method on the hashmap, this does not set everything to zero, but removes all the mappings.
The following code should work, use it instead of collection.clear()
for (Map.Entry<String, Integer> entry : collection.entrySet()) {
entry.setValue(0);
}

Reduce for loops in the code below

I am trying to get friends recommendations for a user based on their direct friends and rank them on their frequency(i.e number of times the recommended friend appeared on user's direct friend lists). Below is the working code to solve the problem.
public List<String> getFriendsRecommendations(String user)
{
Recommendations rd = new Recommendations();
List<String> result= new ArrayList<String>();
List<String> drfriend = rd.getdirectfriends(user); //return list of direct friend for the user.
List<ArrayList<String>> allfriends = new ArrayList<ArrayList<String>>();
Map<String, Integer> mapfriend = new HashMap<String, Integer>();
List<String> userfriend = rd.getfriends(user); //returns the list of all the friends for a given user.
int counter =0;
for(String s: drfriend)
{
allfriends.add(new ArrayList<String>(rd.getfriends(s)));
rd.intersection(userfriend, allfriends.get(counter), mapfriend);
counter++;
}
result.addAll(mapfriend.keySet());
//Sorting based on the value of hashmap. friend with highest value will be recommended first
Collections.sort(result, new Comparator<String>(){
public int compare(String s1, String s2)
{
if(mapfriend.get(s1) > mapfriend.get(s2))
return -1;
else if(mapfriend.get(s1) < mapfriend.get(s2))
return 1;
else if(mapfriend.get(s1) == mapfriend.get(s2))
{
return s1.compareTo(s2);
}
return 0;
}
});
return result;
}
public void intersection(List<String> lt1, ArrayList<String> lt2, Map<String, Integer> ranked)
{
lt2.removeAll(lt1); // ignoring the friends that user is already connected to
for(String st: lt2)
{
boolean val = ranked.containsKey(st);
if(val)
{
int getval = ranked.get(st);
ranked.put(st, getval+1); //friend name as a key and the value would be the count.
}
else
{
ranked.put(st, 1);
}
}
}
I would like to know if there is more efficient way to solve the above problem instead of using 2 for loops?

Quick tip for your Comparator: Get the two values you are interested in comparing at the start and store them in variables, that way you only do a maximimum of 2 get calls instead of 6 in your current worst case scenario (each get call will hash the String, so less is better).
As for simplifying the for loops, could you just get a list of friends of friends and count the occurrences of each friend in that list? Then afterwards, remove any friends you are already friends with.

List of string with occurrences count and sort

I'm developing a Java Application that reads a lot of strings data likes this:
1 cat (first read)
2 dog
3 fish
4 dog
5 fish
6 dog
7 dog
8 cat
9 horse
...(last read)
I need a way to keep all couple [string, occurrences] in order from last read to first read.
string occurrences
horse 1 (first print)
cat 2
dog 4
fish 2 (last print)
Actually i use two list:
1) List<string> input; where i add all data
In my example:
input.add("cat");
input.add("dog");
input.add("fish");
...
2)List<string> possibilities; where I insert the strings once in this way:
if(possibilities.contains("cat")){
possibilities.remove("cat");
}
possibilities.add("cat");
In this way I've got a sorted list where all possibilities.
I use it like that:
int occurrence;
for(String possible:possibilities){
occurrence = Collections.frequency(input, possible);
System.out.println(possible + " " + occurrence);
}
That trick works good but it's too slow(i've got millions of input)... any help?
(English isn’t my first language, so please excuse any mistakes.)

Use a Map<String, Integer>, as #radoslaw pointed, to keep the insertion sorting use LinkedHashMap and not a TreeMap as described here:
LinkedHashMap keeps the keys in the order they were inserted, while a TreeMap is kept sorted via a Comparator or the natural Comparable ordering of the elements.
Imagine you have all the strings in some array, call it listOfAllStrings, iterate over this array and use the string as key in your map, if it does not exists, put in the map, if it exists, sum 1 to actual result...
Map<String, Integer> results = new LinkedHashMap<String, Integer>();
for (String s : listOfAllStrings) {
if (results.get(s) != null) {
results.put(s, results.get(s) + 1);
} else {
results.put(s, 1);
}
}

Make use of a TreeMap, which will keep ordering on the keys as specified by the compare of your MyStringComparator class handling MyString class which wraps String adding insertion indexes, like this:
// this better be immutable
class MyString {
private MyString() {}
public static MyString valueOf(String s, Long l) { ... }
private String string;
private Long index;
public hashcode(){ return string.hashcode(); }
public boolean equals() { // return rely on string.equals() }
}
class MyStringComparator implements Comparator<MyString> {
public int compare(MyString s1, MyString s2) {
return -s1.getIndex().compareTo(s2.gtIndex());
}
}
Pass the comparator while constructing the map:
Map<MyString,Integer> map = new TreeMap<>(new MyStringComparator());
Then, while parsing your input, do
Long counter = 0;
while (...) {
MyString item = MyString.valueOf(readString, counter++);
if (map.contains(item)) {
map.put(map.get(item)+1);
} else {
map.put(item,1);
}
}
There will be a lot of instantiation because of the immutable class, and the comparator will not be consistent with equals, but it should work.
Disclaimer: this is untested code just to show what I'd do, I'll come back and recheck it when I get my hands on a compiler.

Here is the complete solution for your problem,
import java.util.ArrayList;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class DataDto implements Comparable<DataDto>{
public int count = 0;
public String string;
public long lastSeenTime;
public DataDto(String string) {
this.string = string;
this.lastSeenTime = System.currentTimeMillis();
}
public boolean equals(Object object) {
if(object != null && object instanceof DataDto) {
DataDto temp = (DataDto) object;
if(temp.string != null && temp.string.equals(this.string)) {
return true;
}
}
return false;
}
public int hashcode() {
return string.hashCode();
}
public int compareTo(DataDto o) {
if(o != null) {
return o.lastSeenTime < this.lastSeenTime ? -1 : 1;
}
return 0;
}
public String toString() {
return this.string + " : " + this.count;
}
public static final void main(String[] args) {
String[] listOfAllStrings = {"horse", "cat", "dog", "fish", "cat", "fish", "dog", "cat", "horse", "fish"};
Map<String, DataDto> results = new HashMap<String, DataDto>();
for (String s : listOfAllStrings) {
DataDto dataDto = results.get(s);
if(dataDto != null) {
dataDto.count = dataDto.count + 1;
dataDto.lastSeenTime = System.nanoTime();
} else {
dataDto = new DataDto(s);
results.put(s, dataDto);
}
}
List<DataDto> finalResults = new ArrayList<DataDto>(results.values());
System.out.println(finalResults);
Collections.sort(finalResults);
System.out.println(finalResults);
}
}
Ans
[horse : 1, cat : 2, fish : 2, dog : 1]
[fish : 2, horse : 1, cat : 2, dog : 1]
I think this solution will be suitable for your requirement.

If you know that your data is not going to exceed your memory capacity when you read it all into memory, then the solution is simple - using a LinkedList or a and a LinkedHashMap.
For example, if you use a Linked list:
LinkedList<String> input = new LinkedList();
You then proceed to use input.add() as you did originally. But when the input list is full, you basically use Jordi Castilla's solution - but put the entries in the linked list in reverse order. To do that, you do:
Iterator<String> iter = list.descendingIterator();
LinkedHashMap<String,Integer> map = new LinkedHashMap<>();
while (iter.hasNext()) {
String s = iter.next();
if ( map.containsKey(s)) {
map.put( s, map.get(s) + 1);
} else {
map.put(s, 1);
}
}
Now, the only real difference between his solution and mine is that I'm using list.descendingIterator() which is a method in LinkedList that gives you the entries in backwards order, from "horse" to "cat".
The LinkedHashMap will keep the proper order - whatever was entered first will be printed first, and because we entered things in reverse order, then whatever was read last will be printed first. So if you print your map the result will be:
{horse=1, cat=2, dog=4, fish=2}
If you have a very long file, and you can't load the entire list of strings into memory, you had better keep just the map of frequencies. In this case, in order to keep the order of entry, we'll use an object such as this:
private static class Entry implements Comparable<Entry> {
private static long nextOrder = Long.MIN_VALUE;
private String str;
private int frequency = 1;
private long order = nextOrder++;
public Entry(String str) {
this.str = str;
}
public String getString() {
return str;
}
public int getFrequency() {
return frequency;
}
public void updateEntry() {
frequency++;
order = nextOrder++;
}
#Override
public int compareTo(Entry e) {
if ( order > e.order )
return -1;
if ( order < e.order )
return 1;
return 0;
}
#Override
public String toString() {
return String.format( "%s: %d", str, frequency );
}
}
The trick here is that every time you update the entry (add one to the frequency), it also updates the order. But the compareTo() method orders Entry objects from high order (updated/inserted later) to low order (updated/inserted earlier).
Now you can use a simple HashMap<String,Entry> to store the information as you read it (I'm assuming you are reading from some sort of scanner):
Map<String,Entry> m = new HashMap<>();
while ( scanner.hasNextLine() ) {
String str = scanner.nextLine();
Entry entry = m.get(str);
if ( entry == null ) {
entry = new Entry(str);
m.put(str, entry);
} else {
entry.updateEntry();
}
}
Scanner.close();
Now you can sort the values of the entries:
List<Entry> orderedList = new ArrayList<Entry>(m.values());
m = null;
Collections.sort(orderedList);
Running System.out.println(orderedList) will give you:
[horse: 1, cat: 2, dog: 4, fish: 2]
In principle, you could use a TreeMap whose keys contained the "order" stuff, rather than a plain HashMap like this followed by sorting, but I prefer not having either mutable keys in a map, nor changing the keys constantly. Here we are only changing the values as we fill the map, and each key is inserted into the map only once.

What you could do:
Reverse the order of the list using
Collections.reverse(input). This runs in linear time - O(n);
Create a Set from the input list. A Set garantees uniqueness.
To preserve insertion order, you'll need a LinkedHashSet;
Iterate over this set, just as you did above.
Code:
/* I don't know what logic you use to create the input list,
* so I'm using your input example. */
List<String> input = Arrays.asList("cat", "dog", "fish", "dog",
"fish", "dog", "dog", "cat", "horse");
/* by the way, this changes the input list!
* Copy it in case you need to preserve the original input. */
Collections.reverse(input);
Set<String> possibilities = new LinkedHashSet<String>(strings);
for (String s : possibilities) {
System.out.println(s + " " + Collections.frequency(strings, s));
}
Output:
horse 1
cat 2
dog 4
fish 2

Adding values of repeated elements in ArrayList

I have an Arraylist of Records.
package com.demo.myproject;
public class Records
{
String countryName;
long numberOfDays;
public String getCountryName() {
return countryName;
}
public void setCountryName(String countryName) {
this.countryName = countryName;
}
public long getNumberOfDays() {
return numberOfDays;
}
public void setNumberOfDays(long numberOfDays) {
this.numberOfDays = numberOfDays;
}
Records(long days,String cName)
{
numberOfDays=days;
countryName=cName;
}
}
My Arraylist<Records> is containing the values
Singapore 12
Canada 3
United Sates 12
Singapore 21
I need to modify it such that my output is
Canada 3
Singapore 33
United States 12
Please help me with solution,approach.

You could store your Records in a Map, where the key would be the country.
When you receive a new Record, check if the country already is in the map, if it is, add the number of days, if not create it.
Map<String, Record> map = new HashMap<String, Record> ();
addRecord(map, someRecord);
private void addRecord(Map<String, Record> map, Record record) {
Record inMap = map.get(record.getCountryName());
if (inMap == null) {
inMap = record;
} else {
inMap.setNumberOfDays(inMap.getNumberOfDays() + record.getNumberOfDays());
}
map.put(record.getCountryName(), inMap);
}
Notes:
I have assumed that it is fine to modify the records - if not just create a new one using the sum of the days.
you can still get the collection of records by calling map.values(); and iterate over them
ArrayList is not very well suited for your use case. If you really need to stick to ArrayList, for evey new record, you would need to loop over the list, check if one of the records in the list has the same country as the new record, update that record if you find it, or add a new record if not.

public class RecordsMain {
static ArrayList<Records> al = new ArrayList<Records>();
static boolean flag = false;
public static void main(String[] args) {
Records rec1 = new Records(12,"Singapore");
Records rec2 = new Records(3,"Canada");
Records rec3 = new Records(12,"United States");
Records rec4 = new Records(21,"Singapore");
addToList(rec1);
addToList(rec2);
addToList(rec3);
addToList(rec4);
for (int i = 0; i < al.size(); i++) {
System.out.println(al.get(i).getCountryName() + " :: " + al.get(i).getNumberOfDays());
}
}
public static void addToList(Records records) {
for (int i = 0; i < al.size(); i++) {
if(al.get(i).getCountryName().equals(records.getCountryName())) {
al.get(i).setNumberOfDays(al.get(i).getNumberOfDays()+records.getNumberOfDays());
flag=true;
}
}
if (flag == false)
al.add(records);
}
}
Note:
The function addToList adds records and while adding itself checks whether the CountryNames are duplicate, if they are it adds the No of days and does not marks any new entry to the ArrayList.
I was not sure if you were looking for sorting of the List too, thus did not try that.

I suppose you create these records on your own. If you don't need any specific order of the elements you should use the HashMap and as assylias said - create country elements only when they doesn't exist. When you need to keep the order of elements (or sort them later by name etc) you can still use the ArrayList and "indexOf()" method to easily find them.

I dont know what exactly you want to do there but if you want to sort it with specific criteria then You could use comparable or comparator interfaces to sort your records using your criteria in ArrayList And use collections.sort() method to sort it.