Develop Reference

Comparators and HashMaps for Markov Chain

I am trying to make a markov chain in Java/Processing, that will read a book then be able to cut it up in probabilistic ways. Programming is a hobby…
I had the idea that the way to do it was to use a HashMap, and store a Word Object within it. I could easily do this with a String, but within each unique Word it needs to have another HashMap that will store more yet more Word Objects for the Words that follow it, and so on until we have made a model with a sufficient level of complexity.
The problems are that I can’t seem to be able to check whether or not a Word Object is already within the Map by its String name.
Through looking around on SO I can see that it is likely that I will need a Comparator — but all the examples that I have seen use compare or compareTo, when I think that I need something that is more like equals? I don’t need anything at all to do with Sorting, the order will be worked out in the second part of the program.
The code below is pretty horrible — I have been hacking away at this problem for ages but I can’t find an explanation that is sufficiently dumbed down enough for me to understand it.
In Pseudo:
read book
If the Word is not in the Map, put it in there
If the Word is in the Map, iterate the key
Check the Words that follow this Word, and check in the same way if they are within the first Word’s Map, adding as necessary… repeat…
When this is complete
Using the Integer values as probabilities, pick a word
from that Word’s Map, find a Word that is probable to follow it
repeat until desired length is achieved
Code so far:
///markovs
import java.util.HashSet;
import java.util.Comparator;
HashMap<Word, Integer> book;
void setup()
{
book = new HashMap<Word, Integer>();
String[] rows = loadStrings("crash.txt");
for (int i = 0; i < rows.length; i++)
{
if (trim(rows[i]).length() == 0)
{
continue;
}
String[] pieces = split(rows[i], " ");
for (int j = 0; j<pieces.length; j++)
{
Word temp = new Word(pieces[j]);
//c++;
if (book.compare(temp)) {
println("this worked for once");
//iterate here
} else {
book.put(temp, 1);
println("didn’t work");
//book.add(temp);
book.put(temp, 1);
}
}
}
println(book.size());
//println(c);
//println(book);
}
class WordComparator implements Comparator<Word> {
#Override
public int compare(Word w1, Word w2) {
String w1name = w1.name;
String w2name = w2.name;
if (w1name.equals(w2name)) {
return 1;
} else {
return 0;
}
}
}
class Word
{
String name;
int value=1;
int depth;
HashMap<String, Integer> list;
Word(String name_)
{
this.name = name_;
}
int compareTo(Word w) {
if (w.name.equals(this.name)) {
return 0;
} else {
return -1;
}
}
Word(Word w)
{
this.depth = w.depth+1;
}
void nextWord(String word)
{
}
void count() {
value++;
}
void makeHash()
{
list = new HashMap<String, Integer>();
}
}

To use an Object as a key in a HashMap, you need to override two methods: equals() and hashCode(). I'm not exactly sure what you're going for, but a simple example that just uses the name variable would look like this:
public boolean equals(Object other){
if(other instanceof Word){
return this.name.equals(((Word)other).name);
}
return false;
}
public int hashCode(){
return name.hashCode();
}
However, if you're just using the name variable anyway, you might be looking for a multimap, which is just a Map that contains a Map that contains...
HashMap<String, HashMap<String, Integer>> bookMap;
Furthermore, while HashMap does not use the compareTo function, the way you've implemented it seems off. First of all, you need to implement Comparable on your class:
class Word implements Comparable<Word>{
And secondly, the compareTo function should return one of 3 values: negative, zero, or positive. Right now you're only returning zero or negative, which doesn't make any sense.
I think you might be better off taking a step back and describing what you're actually trying to do, as your code contains a lot of confusing logic right now.

As for comparing, you can override Object's inherited equals method, something like:
# Override
boolean equals(Object o) {
return o instanceof Word
? o.name.equals(name) : false;
}

Be aware of using your own types as keys for the HashMap, in this case Word. That only works out well if you provide a sensible implementation of .hashCode() and .equals() on Word.
Here it looks like you could just use String instead. String already has the required method implementations. If you really do want to use Word, you could use those methods from String. e.g.
class Word {
String letters;
public int hashCode() {
return letters.hashCode();
}
public boolean equals(Object o) {
if (o == null || o.getClass() != getClass()) return false;
return letters.equals(((Word) o).letters);
}
}
You don't need a compare or compareTo, just these two.

Does java have some convention to indicate that object is in empty state?

Of course, empty definition can differ. I'm used to PHP's empty though, which calls empty everything that evaluates to false. I'd like to call these things empty in my Java application:
null
String of zero length
0 Integer, Float or Double
false
Any array of zero length
Empty ArrayList or HashMap
Java has, for example, toString convention. Every object is granted to give you some string representation. In my Settings class I operate with HashMap<String, Object>. My empty method looks now like this:
public boolean empty(String name) {
Object val = settings.get(name);
if(val!=null) {
return false;
}
return true;
}
I'd like to extend it in a conventional manner, rather than if(val instanceof XXX) chain.

No, there is no standard convention for this in Java. Also, in Java there is no such thing as "evaluate to false" (except for booleans and Booleans, of course).
You will have to write a method (or rather, a series of overloaded methods for each type you need it for) which implements your notion of "empty". For example:
public static boolean isEmpty(String s) {
return (s == null) || (s.isEmpty());
}
public static boolean isEmpty(int i) {
return i == 0;
}
...

You could use overloading to describe all the "empty" objects:
public static boolean empty(Object o) {
return o == null;
}
public static boolean empty(Object[] array) {
return array == null || array.length == 0;
}
public static boolean empty(int[] array) { //do the same for other primitives
return array == null || array.length == 0;
}
public static boolean empty(String s) {
return s == null || s.isEmpty();
}
public static boolean empty(Number n) {
return n == null || n.doubleValue() == 0;
}
public static boolean empty(Collection<?> c) {
return c == null || c.isEmpty();
}
public static boolean empty(Map<?, ?> m) {
return m == null || m.isEmpty();
}
Examples:
public static void main(String[] args) {
Object o = null;
System.out.println(empty(o));
System.out.println(empty(""));
System.out.println(empty("as"));
System.out.println(empty(new int[0]));
System.out.println(empty(new int[] { 1, 2}));
System.out.println(empty(Collections.emptyList()));
System.out.println(empty(Arrays.asList("s")));
System.out.println(empty(0));
System.out.println(empty(1));
}

AFAIK there is no such convention. It's fairly common to see project specific utility classes with methods such as:
public static boolean isEmpty(String s) {
return s == null || s.isEmpty();
}
However I personally think its use is a bit of a code smell in Java. There's a lot of badly written Java around, but well written Java shouldn't need null checks everywhere, and you should know enough about the type of an object to apply type-specific definitions of "empty".
The exception would be if you were doing reflection-oriented code that worked with Object variables who's type you don't know at compile time. That code should be so isolated that it's not appropriate to have a util method to support it.
Python's duck-typing means the rules are sort of different.

How about creating an interface EmptinessComparable or something similar, and having all your classes implement that? So you can just expect that, and not have to ask instanceof every time.

Java does not, but Groovy does. Groovy runs on the Java VM alongside Java code and provides many shortcuts and convenient conventions such as this. A good approach is write foundation and crital project components in Java and use Groovy for less critical higher level components.

If you want to use the one approach, I would overload a utility method:
public class MyUtils {
public static boolean isEmpty(String s) {
return s == null || s.isEmpty();
}
public static boolean isEmpty(Boolean b) {
return b == null || !b;
}
// add other versions of the method for other types
}
Then your code always looks like:
if (MyUtils.isEmpty(something))
If the type you're checking isn't supported, you'll get a compiler error, and you can implement another version as you like.

There are ways to establish the notion of emptiness but it's not standardized across all Java classes. For example, the Map (implementation) provides the Map#containsKey() method to check if a key exists or not. The List and String (implementations) provide the isEmpty() method but the List or String reference itself could be null and hence you cannot avoid a null check there.
You could however come up with a utility class of your own that takes an Object and using instanceof adapts the empty checks accordingly.
public final class DataUtils {
public static boolean isEmpty(Object data) {
if (data == null) {
return false;
}
if (data instanceof String) {
return ((String) data).isEmpty();
}
if (data instanceof Collection) {
return ((Collection) data).isEmpty();
}
}
}

The Guava Libraries already contains Defaults class that do just that.
Calling defaultValue will return the default value for any primitive type (as specified by the JLS), and null for any other type.
You can use it like shown below:
import com.google.common.base.Defaults;
Defaults.defaultValue(Integer.TYPE); //will return 0
Below is example code on how to use it:
import com.google.common.base.Defaults;
public class CheckingFieldsDefault
{
public static class MyClass {
private int x;
private int y = 2;
}
public static void main() {
MyClass my = new MyClass();
System.out.println("x is defualt: " + (my.x == Defaults.defaultValue(box(my.x).TYPE)));
System.out.println("y is defualt: " + (my.y == Defaults.defaultValue(box(my.y).TYPE)));
}
private static <T extends Object> T box(T t) {
return t;
}
}

Get specific objects from ArrayList when objects were added anonymously?

I have created a short example of my problem. I'm creating a list of objects anonymously and adding them to an ArrayList. Once items are in the ArrayList I later come back and add more information to each object within the list. Is there a way to extract a specific object from the list if you do not know its index?
I know only the Object's 'name' but you cannot do a list.get(ObjectName) or anything. What is the recommended way to handle this? I'd rather not have to iterate through the entire list every time I want to retrieve one specific object.
public class TestCode{
public static void main (String args []) {
Cave cave = new Cave();
// Loop adds several Parties to the cave's party list
cave.parties.add(new Party("FirstParty")); // all anonymously added
cave.parties.add(new Party("SecondParty"));
cave.parties.add(new Party("ThirdParty"));
// How do I go about setting the 'index' value of SecondParty for example?
}
}
class Cave {
ArrayList<Party> parties = new ArrayList<Party>();
}
class Party extends CaveElement{
int index;
public Party(String n){
name = n;
}
// getter and setter methods
public String toString () {
return name;
}
}
class CaveElement {
String name = "";
int index = 0;
public String toString () {
return name + "" + index;
}
}

Given the use of List, there's no way to "lookup" a value without iterating through it...
For example...
Cave cave = new Cave();
// Loop adds several Parties to the cave's party list
cave.parties.add(new Party("FirstParty")); // all anonymously added
cave.parties.add(new Party("SecondParty"));
cave.parties.add(new Party("ThirdParty"));
for (Party p : cave.parties) {
if (p.name.equals("SecondParty") {
p.index = ...;
break;
}
}
Now, this will take time. If the element you are looking for is at the end of the list, you will have to iterate to the end of the list before you find a match.
It might be better to use a Map of some kind...
So, if we update Cave to look like...
class Cave {
Map<String, Party> parties = new HashMap<String, Party>(25);
}
We could do something like...
Cave cave = new Cave();
// Loop adds several Parties to the cave's party list
cave.parties.put("FirstParty", new Party("FirstParty")); // all anonymously added
cave.parties.put("SecondParty", new Party("SecondParty"));
cave.parties.put("ThirdParty", new Party("ThirdParty"));
if (cave.parties.containsKey("SecondParty")) {
cave.parties.get("SecondParty").index = ...
}
Instead...
Ultimately, this will all depend on what it is you want to achieve...

List.indexOf() will give you what you want, provided you know precisely what you're after, and provided that the equals() method for Party is well-defined.
Party searchCandidate = new Party("FirstParty");
int index = cave.parties.indexOf(searchCandidate);
This is where it gets interesting - subclasses shouldn't be examining the private properties of their parents, so we'll define equals() in the superclass.
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof CaveElement)) {
return false;
}
CaveElement that = (CaveElement) o;
if (index != that.index) {
return false;
}
if (name != null ? !name.equals(that.name) : that.name != null) {
return false;
}
return true;
}
It's also wise to override hashCode if you override equals - the general contract for hashCode mandates that, if x.equals(y), then x.hashCode() == y.hashCode().
#Override
public int hashCode() {
int result = name != null ? name.hashCode() : 0;
result = 31 * result + index;
return result;
}

If you want to lookup objects based on their String name, this is a textbook case for a Map, say a HashMap. You could use a LinkedHashMap and convert it to a List or Array later (Chris has covered this nicely in the comments below).
LinkedHashMap because it lets you access the elements in the order you insert them if you want to do so. Otherwise HashMap or TreeMap will do.
You could get this to work with List as the others are suggesting, but that feels Hacky to me.. and this will be cleaner both in short and long run.
If you MUST use a list for the object, you could still store a Map of the object name to the index in the array. This is a bit uglier, but you get almost the same performance as a plain Map.

You could use list.indexOf(Object) bug in all honesty what you're describing sounds like you'd be better off using a Map.
Try this:
Map<String, Object> mapOfObjects = new HashMap<String, Object>();
mapOfObjects.put("objectName", object);
Then later when you want to retrieve the object, use
mapOfObjects.get("objectName");
Assuming you do know the object's name as you stated, this will be both cleaner and will have faster performance besides, particularly if the map contains large numbers of objects.
If you need the objects in the Map to stay in order, you can use
Map<String, Object> mapOfObjects = new LinkedHashMap<String, Object>();
instead

As per your question requirement , I would like to suggest that Map will solve your problem very efficient and without any hassle.
In Map you can give the name as key and your original object as value.
Map<String,Cave> myMap=new HashMap<String,Cave>();

I would suggest overriding the equals(Object) of your Party class. It might look something like this:
public boolean equals(Object o){
if(o == null)
return false;
if(o instanceof String)
return name.equalsIgnoreCase((String)o);
else if(o instanceof Party)
return equals(((Party)o).name);
return false;
}
After you do that, you could use the indexOf(Object) method to retrieve the index of the party specified by its name, as shown below:
int index = cave.parties.indexOf("SecondParty");
Would return the index of the Party with the name SecondParty.
Note: This only works because you are overriding the equals(Object) method.

You could simply create a method to get the object by it's name.
public Party getPartyByName(String name) {
for(Party party : parties) {
if(name.equalsIgnoreCase(party.name)) {
return party;
}
}
return null;
}

contains giving faulty results

I have a class 'CoAutoria' that's suposed to hold 2 instances of an 'Author' class (which only has a name, for now) and the number of articles those authors have in common.
In order to figure out the top 10 of co-authors (regarding number of articles) I created a TreeSet of 'CoAutoria', to hold the total of articles, for each pair.
I need to cycle through a Map of years, gather the different authors and their respective Set of co-Authors. Then, for each pair, create an instance of 'CoAutoria' and: add it to the treeset (if it doesn't already exists); or simply sum its number of articles to the one existing on the set.
I already created the compareTo method, to insert it on the treeset, and created the equals method so that the order of the authors doesn't matter.
Here's the main code:`
public class CoAutoria implements Comparable<CoAutoria>
{
private Autor autor1;
private Autor autor2;
private int artigosComum;
(...)
}
#Override
public int compareTo(CoAutoria a2)
{
String thisAutor1 = autor1.getNome();
String thisAutor2 = autor2.getNome();
String caAutor1 = a2.getAutor1().getNome();
String caAutor2 = a2.getAutor2().getNome();
if((autor1.equals(a2.getAutor1()) && autor2.equals(a2.getAutor2())) || (autor1.equals(a2.getAutor2()) && autor2.equals(a2.getAutor1())))
{
return 0;
}
else
{
return 1;
}
}
#Override
public boolean equals(Object o)
{
if(this == o)
{
return true;
}
if( o == null || o.getClass() != this.getClass())
return false;
CoAutoria ca = (CoAutoria) o;
String thisAutor1 = autor1.getNome();
String thisAutor2 = autor2.getNome();
String caAutor1 = ca.getAutor1().getNome();
String caAutor2 = ca.getAutor2().getNome();
if((thisAutor1.equals(caAutor1) && thisAutor2.equals(caAutor2)) || (thisAutor1.equals(caAutor2) && thisAutor2.equals(caAutor1)))
{
return true;
}
else
{
return false;
}
}
The main problem is: When I check if the set already has a certain instance of 'CoAutoria', (I'm using the contains() method of TreeSet), it gives me faulty results...sometimes it checks correctly that the Pair A-B already exists in that set (on the form of B-A), but sometimes it doesn't... For what I've read, the contains uses the equals method, so that's not suposed to happen..right?
[EDIT:]
Since the first post I started to think that maybe the problem resided on the compareTo..So I changed it to
public int compareTo(CoAutoria a2)
{
String thisAutor1 = autor1.getNome();
String thisAutor2 = autor2.getNome();
String caAutor1 = a2.getAutor1().getNome();
String caAutor2 = a2.getAutor2().getNome();
if(this.equals(a2))
{
System.out.println("return 0");
return 0;
}
else
{
int aux = thisAutor1.compareTo(caAutor1);
if(aux != 0)
{
return aux;
}
else
{
return thisAutor2.compareTo(caAutor2);
}
}
}
But it still gives my bad results..I thought I'd figured it now: if it's the same 'CoAutoria', I return 0, if not I go through the names, and order it by their compareTo values..but something's missing

Your contains method is breaking, because your compareTo method is always returning 0 or positive, no negatives. This means your compareTo is inconsistent. A correct implementation should return 0 if the authors are the same, or positive and negative values when the authors are different.
Example (assuming author1 is different than author2):
int i = author1.compareTo(author2); // i should be positive or negative
int j = author2.compareTo(author1); // j should be the opposite of i
Yours will return 1 for both of the above cases, which will make ordered Collections not work as no element is ever smaller. As another example imagine if you had a Binary Tree(an ordered collection) that had the elements [1-10]. If you were searching for the element 5, your binary tree when comparing 5 against any element would always say that it was equal or greater.
How exactly you should change it is up to you. But an idea would be to sort the authors by name, then iterate over both collections and compare the authors together lexicographically.
EDIT: Even after your edit to your methods they are still not consistent. Try the following, they aren't the most efficient but should work unless you really want to optimize for speed. Notice they first sort to make sure author1 and author2 are in order before they are compared with the other CoAutor which is also sorted. I don't do any null checking and assume both are valid authors.
#Override
public boolean equals(Object o){
if (o == null || !(o instanceof CoAutoria)) return false;
if (o == this) return true;
return this.compareTo((CoAutoria)o) == 0;
}
#Override
public int compareTo(CoAutoria o) {
List<String> authors1 = Arrays.asList(autor1.getNome(), autor2.getNome());
List<String> authors2 = Arrays.asList(o.autor1.getNome(), o.autor2.getNome());
Collections.sort(authors1);
Collections.sort(authors2);
for (int i=0;i<authors1.size();i++){
int compare = authors1.get(i).compareTo(authors2.get(i));
if (compare != 0)
return compare;
}
return 0;
}

TreeMap returning null for value that should exist for some object keys

I have an issue with a TreeMap that we have defined a custom key object for. The issue is that after putting a few objects into the map, and trying to retrieve with the same key used to put on the map, I get a null. I believe this is caused by the fact that we have 2 data points on the key. One value is always populated and one value is not always populated. So it seems like the issue lies with the use of compareTo and equals. Unfortunately the business requirement for how our keys determine equality needs to be implemented this way.
I think this is best illustrated with code.
public class Key implements Comparable<Key> {
private String sometimesPopulated;
private String alwaysPopulated;
public int compareTo(Key aKey){
if(this.equals(aKey)){
return 0;
}
if(StringUtils.isNotBlank(sometimesPopulated) && StringUtils.isNotBlank(aKey.getSometimesPopulated())){
return sometimesPopulated.compareTo(aKey.getSometimesPopulated());
}
if(StringUtils.isNotBlank(alwaysPopulated) && StringUtils.isNotBlank(aKey.getAlwaysPopulated())){
return alwaysPopulated.compareTo(aKey.getAlwaysPopulated());
}
return 1;
}
public boolean equals(Object aObject){
if (this == aObject) {
return true;
}
final Key aKey = (Key) aObject;
if(StringUtils.isNotBlank(sometimesPopulated) && StringUtils.isNotBlank(aKey.getSometimesPopulated())){
return sometimesPopulated.equals(aKey.getSometimesPopulated());
}
if(StringUtils.isNotBlank(alwaysPopulated) && StringUtils.isNotBlank(aKey.getAlwaysPopulated())){
return alwaysPopulated.equals(aKey.getAlwaysPopulated());
}
return false;
}
So the issue occurs when trying to get a value off the map after putting some items on it.
Map<Key, String> map = new TreeMap<Key, String>();
Key aKey = new Key(null, "Hello");
map.put(aKey, "world");
//Put some more things on the map...
//they may have a value for sometimesPopulated or not
String value = map.get(aKey); // this = null
So why is the value null after just putting it in? I think the algorithm used by the TreeMap is sorting the map in an inconsistent manner because of the way I'm using compareTo and equals. I am open to suggestions on how to improve this code. Thanks

Your comparator violates the transitivity requirement.
Consider three objects:
Object A: sometimesPopulated="X" and alwaysPopulated="3".
Object B: sometimesPopulated="Y" and alwaysPopulated="1".
Object C: sometimesPopulated is blank and alwaysPopulated="2".
Using your comparator, A<B and B<C. Transitivity requires that A<C. However, using your comparator, A>C.
Since the comparator doesn't fulfil its contract, TreeMap is unable to do its job correctly.

I think the problem is that you are returning 1 from your compareTo if either of the sometimesPopulated values is blank or either of the alwaysPopulated values is blank. Remember that compareTo can be thought of returning the value of a subtraction operation and your's is not transitive. (a - b) can == (b - a) even when a != b.
I would return -1 if the aKey sometimesPopulated is not blank and the local sometimesPopulated is blank. If they are the same then I would do the same with alwaysPopulated.
I think your logic should be something like:
public int compareTo(Key aKey){
if(this.equals(aKey)){
return 0;
}
if (StringUtils.isBlank(sometimesPopulated)) {
if (StringUtils.isNotBlank(aKey.getSometimesPopulated())) {
return -1;
}
} else if (StringUtils.isBlank(aKey.getSometimesPopulated())) {
return 1;
} else {
int result = sometimesPopulated.compareTo(aKey.getSometimesPopulated());
if (result != 0) {
return result;
}
}
// same logic with alwaysPopulated
return 0;
}

I believe the problem is that you are treating two keys with both blank fields as greater than each other which could confuse the structure.
class Main {
public static void main(String... args) {
Map<Key, String> map = new TreeMap<Key, String>();
Key aKey = new Key(null, "Hello");
map.put(aKey, "world");
//Put some more things on the map...
//they may have a value for sometimesPopulated or not
String value = map.get(aKey); // this = "world"
System.out.println(value);
}
}
class Key implements Comparable<Key> {
private final String sometimesPopulated;
private final String alwaysPopulated;
Key(String alwaysPopulated, String sometimesPopulated) {
this.alwaysPopulated = defaultIfBlank(alwaysPopulated, "");
this.sometimesPopulated = defaultIfBlank(sometimesPopulated, "");
}
static String defaultIfBlank(String s, String defaultString) {
return s == null || s.trim().isEmpty() ? defaultString : s;
}
#Override
public int compareTo(Key o) {
int cmp = sometimesPopulated.compareTo(o.sometimesPopulated);
if (cmp == 0)
cmp = alwaysPopulated.compareTo(o.alwaysPopulated);
return cmp;
}
}

I think your equals, hashCode and compareTo methods should only use the field that is always populated. It's the only way to ensure the same object will always be found in the map regardless of if its optional field is set or not.
Second option, you could write an utility method that tries to find the value in the map, and if no value is found, tries again with the same key but with (or without) the optional field set.