Related
I'm creating a chess engine as a practice in Java, I know it's not recommended due to speed issues but I'm doing it just for practice.
After implementing minimax with alpha-beta pruning, I thought of implementing a time-limit to find the score of a given move.
Here is the code
private int minimax(MoveNode node, MoveNodeType nodeType, int alpha, int beta, Side side, int depth) throws Exception {
// isInterestingLine(prevscores, node, side);
if (depth <= 0) {
count++;
return node.evaluateBoard(side);
}
// Generate Child nodes if we haven't.
if (node.childNodes == null || node.childNodes.size() == 0) {
node.createSingleChild();
}
if (nodeType == MoveNodeType.MAX) {
int bestValue = -1000;
for (int i = 0; i < node.childNodes.size(); i++) {
if (node.childNodes.get(i) == null) continue;
int value = minimax(node.childNodes.get(i), MoveNodeType.MIN, alpha, beta, side, depth - 1);
bestValue = Math.max(bestValue, value);
alpha = Math.max(alpha, bestValue);
if (beta <= alpha) {
break;
}
node.createSingleChild();
}
// reCalculateScore();
return bestValue;
} else {
int bestValue = 1000;
for (int i = 0; i < node.childNodes.size(); i++) {
if (node.childNodes.get(i) == null) continue;
int value = minimax(node.childNodes.get(i), MoveNodeType.MAX, alpha, beta, side, depth - 1);
bestValue = Math.min(bestValue, value);
beta = Math.min(beta, bestValue);
if (beta <= alpha) {
break;
}
node.createSingleChild();
}
// reCalculateScore();
return bestValue;
}
}
and the driver code.
void evaluateMove(Move mv, Board brd) throws Exception {
System.out.println("Started Comparing! " + this.tree.getRootNode().getMove().toString());
minmaxThread = new Thread(new Runnable() {
#Override
public void run() {
try {
bestMoveScore = minimax(tree.getRootNode(), MoveNodeType.MIN, -1000, 1000, side, MAX_DEPTH);
} catch (Exception e) {
e.printStackTrace();
}
}
});
minmaxThread.start();
}
This is how I implemented time-limit.
long time = System.currentTimeMillis();
moveEvaluator.evaluateMove(move, board.clone());
while((System.currentTimeMillis() - time) < secToCalculate*1000 && !moveEvaluator.minmaxThread.isAlive()) {
}
System.out.println("Time completed! score = " + moveEvaluator.bestMoveScore + " move = " + move + " depth = " + moveEvaluator.searchDepth) ;
callback.callback(move, moveEvaluator.bestMoveScore);
Now, Here is the problem
You see, it only calculated Bb7, because of the depth-first search time runs out before even calculating another line.
So I want a way to calculate like following in a time-limit based solution.
Here are a few solutions I taught of.
Implementing an isInteresting() function. which takes all the previous scores and checks if the current line is interesting/winning if yes then and only then calculates next child nodes.
e.g.
[0,0,0,0,0,0] can be interpreted as a drawn line.
[-2,-3,-5,-2,-1] can be interpreted as a losing line.
Searching for small depth-first and then elimination all losing lines.
for (int i = min_depth; i <= max_depth; i ++) {
scores = [];
for(Node childnode : NodesToCalculate) {
scores.push(minimax(childnode, type, alpha, beta, side, i));
}
// decide which child node to calculate for next iterations.
}
But, none of the solutions is perfect and efficient, In the first one, we are just making a guess and In second on we are calculating one node more than once.
Is there a better way to do this?
The solution to this problem used by every chess engine is iterative deepening.
Instead of searching to a fixed depth (MAX_DEPTH in your example) you start by searching to a depth of one, then when this search is done you start again with a depth of two and you continue to increase depth like this until you are out of time. When you are out of time you can play the move of the last search that was completed.
It may seem like lots of time will be spent on lower depth iteration that are later replaced by deeper search and that the time sent doing so is completely lost, but in practice it's not true. Since searching to a depth N is so much longer than searching at depth N-1 the time spent on the lower depth search is always much less than the time spent on the last (deeper) search.
If your engine use a transposition table, the data in the transposition table from previous iteration will help the later iterations. The alpha-beta algorithm's performance is really sensitive to the order move are searched. The time saved by alpha-beta over minimax is optimal when the best move is searched first. If you did a search for depth N-1 before the search for depth N, the transposition table will probably contains a good guess of the best move for most positions that can then be searched first.
In practice, in a engine using a transposition table and ordering move at the root based on previous iteration it's faster to use iterative deepening than not using it. I mean for exemple it's faster to do a depth 1 search, then en depth 2 search, then a depth3 search until say a depth 10 search than it is doing a depth 10 search right away. Plus you get the option to stop the search whenever you want and still have a move to play,=.
Here is the revised code, the other classes don't matter I hope. If you need the other classes, tell me and I'll add it. When I run this I get the naming error when I try to retrieve the spaces that are possible to move in.
public static void main(String[] args) {
// TODO Auto-generated method stub
List<Space> openlist = new ArrayList<Space>();
int g = 0;
Bot Dave = new Bot("Dave");
Goal goal = new Goal();
Obstacle first = new Obstacle("First");
int numofobst = 1;
Space right = new Space(Dave.getX()+1, Dave.getY());
Space left = new Space(Dave.getX()-1, Dave.getY());
Space up = new Space(Dave.getX(), Dave.getY()+1);
Space down = new Space(Dave.getX(), Dave.getY()-1);
int openpossible= 0;
//now its creating an array of each space and getting the fs of each one.
/*time to check which spaces are possible for the bot to move. if they are possible add the space to a possible array list.
* then we check to see which f is smaller by addign a min value.
* we then sort it and get the first space.
* we move to that space.
*/
if (Dave.checkob(first, right, numofobst) == right){
openlist.add(right);
}
if (Dave.checkob(first, left, numofobst) == left){
openlist.add(left);
}
if (Dave.checkob(first, up, numofobst) == up){
openlist.add(up);}
if (Dave.checkob(first, down, numofobst) == down){
openlist.add(down);}
for (int i = 0; i < openlist.size(); i++){
System.out.println("Space available is" + openlist.get(i));
}
System.out.println("Space available is" + openlist);
}
You're code is missing a lot of things (mostly everything).
First try to implement a simple Dijkstra. Do the one in O(V^2), then upgrade it to O(ElogV). It's a bit slower than A* but a lot simpler to understand. Once you get it you can upgrade it to A* by changing a few lines of code.
So I'm trying to teach myself how to implement a binary search in Java, as the topic might have given away, but am having some trouble.
See, I tend to be a little stubborn, and I'd rather not just copy some implementation off the internet.
In order to teach myself this, I created a very (VERY) rough little class which looks as follows:
public class bSearch{
/**
* #param args
*/
public static void main(String[] args) {
int one = 1;
int two = 2;
int three = 3;
int four = 4;
int five = 5;
int six = 6;
ArrayList tab = new ArrayList();
tab.add(one);
tab.add(two);
tab.add(three);
tab.add(four);
tab.add(five);
tab.add(six);
System.out.println(bSearch(tab, 53));
}
#SuppressWarnings({ "rawtypes", "unchecked" })
public static int bSearch(ArrayList tab, int key) {
if (tab.size() == 0)
return 0;
if ((int) tab.get(tab.size() / 2) == key)
return key;
ArrayList smallerThanKey = new ArrayList();
ArrayList largerThanKey = new ArrayList();
for (int i = 0; i < (tab.size() + 1) / 2; i++) {
smallerThanKey.add(tab.get(i));
}
System.out.println("Smaller array = " + smallerThanKey);
for (int i = (tab.size() + 1) / 2; i < tab.size(); i++) {
largerThanKey.add(tab.get(i));
}
System.out.println("Larger array = " + largerThanKey);
if (key < (int) tab.get(tab.size() / 2)) {
bSearch(smallerThanKey, key);
} else {
bSearch(largerThanKey, key);
}
return key;
}
}
As you can see, it's pretty far from beautiful, but it's clear enough for a noobie like myself to understand, anyway.
Now, here's the problem; when I feed it a number that is in the ArrayList, it feeds the number back to me (hurray!), but when I feed it a number that's not in the ArrayList, it still feeds me my number back to me (boo!).
I have a feeling my error is very minor, but I just can't see it.
Or am I all wrong, and there is some larger fundamental error?
Your help is deeply appreciated!
UPDATE
Thanks for all the constructive comments and answers! Many helpful pointer in the right direction by several of you. +1 for everyone who bumped me along the right path.
By following the advice you gave, mostly relating to my recursions not ending properly, I added a few return statements, as follows;
if (key < (int) tab.get(tab.size() / 2)) {
return bSearch(smallerThanKey, key);
} else {
return bSearch(largerThanKey, key);
}
Now, what this does is one step closer to what I want to achieve.
I now get 0 if the number is nowhere to be found, and the number itself if it is to be found. Thus progress is being made!
However, it does not work if I have it search for a negative number or zero (not that I know why I should, but just throwing that out there).
Is there a fix for this, or am I barking up the wrong tree in questioning?
EDIT
Just as a quick solution to the exact question you're asking: you need to change the last few lines to the following
if (key < (int) tab.get(tab.size() / 2)) {
return bSearch(smallerThanKey, key);
} else {
return bSearch(largerThanKey, key);
}
}
Having said that, let me point out a few more issues that I see here:
(a) you can use generics. That is use ArrayList<Integer> rather than just ArrayList this will save you from all those casts.
(b) Instead of returning the value that you found you'd be better off returning the index in the ArrayList where the value is located, or -1 if it was not found. Here's why: returning the key provides the caller with very little new information. I mean - the caller already known what key is. If you return the index to the key you let the caller know if the key was found or not, and if it was found where in the list it resides.
(c) You essentially copying the entire list each time you go into bSearch(): you copy roughly half of the list into smallerThanKey and (roughly) half into greaterThanKey. This means that the complexity of this implementation is not O(log n) but instead O(n).
(EDIT #2)
Summarizing points (a), (b), (c) here's how one could write that method:
public static int bSearch(ArrayList<Integer> tab, int key) {
return bSearch(tab, 0, tab.size(), key);
}
public static int bSearch(ArrayList<Integer> tab, int begin, int end, int key) {
int size = end - begin;
if (size <= 0)
return -1;
int midPoint = (begin + end) / 2;
int midValue = tab.get(midPoint);
if (midValue == key)
return midPoint;
if (key < midValue) {
return bSearch(tab, begin, midPoint, key);
} else {
return bSearch(tab, midPoint + 1, end, key);
}
}
As you can see, I added a second method that takes a begin, end parameters. These parameters let the method which part of the list it should look at. This is much cheaper than creating a new list and copying elements to it. Instead, the recursive function just uses the list object and simply calls itself with new begin, end values.
The return value is now the index of the key inside the list (or -1 if not found).
Your recursion is not properly ended. At the end of the method you recursively call the bSearchmethod for the left or right part of the array. At that point you need to return the search result of the recursive calls.
The idea of the binary search is: If your current node is not the key, look at the left if the value of the current node is bigger than the key or look at the right if it is smaller. So after looking there you need to return the search result from there.
if (key < (int) tab.get(tab.size() / 2)) {
return bSearch(smallerThanKey, key);
} else {
return bSearch(largerThanKey, key);
}
As a side remark, have a look at System.arraycopy and it is always a good idea to not suppress warnings.
I think the issue is here:
if (key < (int) tab.get(tab.size() / 2)) {
bSearch(smallerThanKey, key);
} else {
bSearch(largerThanKey, key);
}
return key;
You're just throwing away the result of your recursive call to bSearch and returning key. So it isn't really much of a surprise you get back whatever number you feed into the method.
Remember how binary search is supposed to work -- if the value isn't in the middle, return the result of searching in the left/right half of the array. So you need to do something with those recursive calls....
And with binary search, you really should be more concerned about finding the location of whatever you're looking for, not its value -- you know that already! So what you think was the binary search working right was a bit mistaken -- searching for 1 should have returned 0 -- the index/location of 1.
Also, you shouldn't need to deal with copying arrays and such -- that's an operation that is unnecessary for searches. Just use parameters to indicate where to begin/end searching.
Dijkstra algorithm has a step which mentions "chose the node with shortest path". I realize that this step is unnecessary if we dont throw a node out of the graph/queue. This works great in my knowledge with no known disadvantage. Here is the code. Please instruct me if it fails ? if it does then how ? [EDIT => THIS CODE IS TESTED AND WORKS WELL, BUT THERE IS A CHANCE MY TEST CASES WERE NOT EXHAUSTIVE, THUS POSTING IT ON STACKOVERFLOW]
public Map<Integer, Integer> findShortest(int source) {
final Map<Integer, Integer> vertexMinDistance = new HashMap<Integer, Integer>();
final Queue<Integer> queue = new LinkedList<Integer>();
queue.add(source);
vertexMinDistance.put(source, 0);
while (!queue.isEmpty()) {
source = queue.poll();
List<Edge> adjlist = graph.getAdj(source);
int sourceDistance = vertexMinDistance.get(source);
for (Edge edge : adjlist) {
int adjVertex = edge.getVertex();
if (vertexMinDistance.containsKey(adjVertex)) {
int vertexDistance = vertexMinDistance.get(adjVertex);
if (vertexDistance > (sourceDistance + edge.getDistance())) {
//previous bug
//vertexMinDistance.put(adjVertex, vertexDistance);
vertexMinDistance.put(adjVertex, sourceDistance + edge.getDistance())
}
} else {
queue.add(adjVertex);
vertexMinDistance.put(adjVertex, edge.getDistance());
}
}
}
return vertexMinDistance;
}
Problem 1
I think there is a bug in the code where it says:
int vertexDistance = vertexMinDistance.get(adjVertex);
if (vertexDistance > (sourceDistance + edge.getDistance())) {
vertexMinDistance.put(adjVertex, vertexDistance);
}
because this has no effect (vertexMinDistance for adjVertex is set back to its original value).
Better would be something like:
int vertexDistance = vertexMinDistance.get(adjVertex);
int newDistance = sourceDistance + edge.getDistance();
if (vertexDistance > newDistance ) {
vertexMinDistance.put(adjVertex, newDistance );
}
Problem 2
You also need to add the adjVertex into the queue using something like:
int vertexDistance = vertexMinDistance.get(adjVertex);
int newDistance = sourceDistance + edge.getDistance();
if (vertexDistance > newDistance ) {
vertexMinDistance.put(adjVertex, newDistance );
queue.add(adjVertex);
}
If you don't do this then you will get an incorrect answer for graphs such as:
A->B (1)
A->C (10)
B->C (1)
B->D (10)
C->D (1)
The correct path is A->B->C->D of weight 3, but without the modification then I believe your algorithm will choose a longer path (as it doesn't reexamine C once it has found a shorter path to it).
High level response
With these modifications I think this approach is basically sound, but you should be careful about the computational complexity.
Dijkstra will only need to go round the main loop V times (where V is the number of vertices in the graph), while your algorithm may need many more loops for certain graphs.
You will still get the correct answer, but it may take longer.
Although the worst-case complexity will be much worse than Dijkstra, I would be interested in how well it performs in practice. My guess is that it will work well for sparse almost tree-like graphs, but less well for dense graphs.
A few days ago I had interview in some big company, name is not required :), and interviewer asked me to find solution to the next task:
Predefined:
There is dictionary of words with unspecified size, we just know that all words in dictionary are sorted (for example by alphabet). Also we have just a one method
String getWord(int index) throws IndexOutOfBoundsException
Needs:
Need to develop algorithm to find some input word in dictionary using java. For this we should implement method
public boolean isWordInTheDictionary(String word)
Limitations:
We cannot change the internal structure of dictionary, we have no access to internal structure, we do not know counts of elements in dictionary.
Issues:
I have developed modified-binary search, and will publish my variant(works variant) of algorithm, but are there another variants with logarithmic complexity? My variant has complexity O(logN).
My variant of implementation:
public class Dictionary {
private static final int BIGGEST_TOP_MASK = 0xF00000;
private static final int LESS_TOP_MASK = 0x0F0000;
private static final int FULL_MASK = 0xFFFFFF;
private String[] data;
private static final int STEP = 100; // for real test step should be Integer.MAX_VALUE
private int shiftIndex = -1;
private static final int LESS_MASK = 0x0000FF;
private static final int BIG_MASK = 0x00FF00;
public Dictionary() {
data = getData();
}
String getWord(int index) throws IndexOutOfBoundsException {
return data[index];
}
public String[] getData() {
return new String[]{"a", "aaaa", "asss", "az", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "test", "u", "v", "w", "x", "y", "z"};
}
public boolean isWordInTheDictionary(String word) {
boolean isFound = false;
int constantIndex = STEP; // predefined step
int flag = 0;
int i = 0;
while (true) {
i++;
if (flag == FULL_MASK) {
System.out.println("Word is not found ... Steps " + i);
break;
}
try {
String data = getWord(constantIndex);
if (null != data) {
int compareResult = word.compareTo(data);
if (compareResult > 0) {
if ((flag & LESS_MASK) == LESS_MASK) {
constantIndex = prepareIndex(false, constantIndex);
if (shiftIndex == 1)
flag |= BIGGEST_TOP_MASK;
} else {
constantIndex = constantIndex * 2;
}
flag |= BIG_MASK;
} else if (compareResult < 0) {
if ((flag & BIG_MASK) == BIG_MASK) {
constantIndex = prepareIndex(true, constantIndex);
if (shiftIndex == 1)
flag |= LESS_TOP_MASK;
} else {
constantIndex = constantIndex / 2;
}
flag |= LESS_MASK;
} else {
// YES!!! We found word.
isFound = true;
System.out.println("Steps " + i);
break;
}
}
} catch (IndexOutOfBoundsException e) {
if (flag > 0) {
constantIndex = prepareIndex(true, constantIndex);
flag |= LESS_MASK;
} else constantIndex = constantIndex / 2;
}
}
return isFound;
}
private int prepareIndex(boolean isBiggest, int constantIndex) {
shiftIndex = (int) Math.ceil(getIndex(shiftIndex == -1 ? constantIndex : shiftIndex));
if (isBiggest)
constantIndex = constantIndex - shiftIndex;
else
constantIndex = constantIndex + shiftIndex;
return constantIndex;
}
private double getIndex(double constantIndex) {
if (constantIndex <= 1)
return 1;
return constantIndex / 2;
}
}
It sounds like the part they really want you to think about is how to handle the fact that you don't know the size of the dictionary. I think they assume that you can give them a binary search. So the real question is how do you manipulate the range of the search as it progresses.
Once you have found a value in the dictionary that is greater than your search target (or out of bounds), the rest looks like standard binary search. The hard part is how do you optimally expand the range when the target value is greater than the dictionary value that you've looked up. It looks like you are expanding by a factor of 1.5. This could be really problematic with a huge dictionary and a small fixed initial step like you have (100). Think if there were 50 million words how many times your algorithm would have to expand the range upwards if you're searching for 'zebra'.
Here's an idea: use the ordered nature of the collection to your advantage by assuming the first letter of each word is evenly distributed amongst the letters of the alphabet (this will never be true, but without knowing more about the collection of words it's probably the best you can do). Then weight the amount of your range expansion by how far from the end you would expect the dictionary word to be.
So if you took your initial step of 100 and looked up the dictionary word at that index and it was 'aardvark', you would expand your range a lot more for the next step than if it was 'walrus.' Still O(log n) but probably much better for most collections of words.
Here is an alternative implementation that uses Collections.binarySearch. It fails if one of the words in the list starts with the Character '\uffff' (that is Unicode 0xffff and not a legal not a valid unicode character).
public static class ListProxy extends AbstractList<String> implements RandomAccess
{
#Override public String get( int index )
{
try {
return getWord( index );
} catch( IndexOutOfBoundsException ex ) {
return "\uffff";
}
}
#Override public int size()
{
return Integer.MAX_VALUE;
}
}
public static boolean isWordInTheDictionary( String word )
{
return Collections.binarySearch( new ListProxy(), word ) >= 0;
}
Update: I modified it so that it implements RandomAccess since the binarySearch in Collections would otherwise use a iterator based search on such a large list which would be extremely slow. This should now however be decently fast since the binary search will need only 31 iterations even though the List pretends to be as large as possible.
Here is a slightly modified version that remembers the smallest failed index to converge its proclaimed size to the actual size of the dictionary en passant and thus avoids almost all exceptions in successive lookups. Although you would need to create a new ListProxy instance whenever the size of the dictionary could have changed.
public static class ListProxy extends AbstractList<String> implements RandomAccess
{
private int size = Integer.MAX_VALUE;
#Override public String get( int index )
{
try {
if( index < size )
return getWord( index );
} catch( IndexOutOfBoundsException ex ) {
size = index;
}
return "\uffff";
}
#Override public int size()
{
return size;
}
}
private static ListProxy listProxy = new ListProxy();
public static boolean isWordInTheDictionary( String word )
{
return Collections.binarySearch( listProxy , word ) >= 0;
}
You have the right idea, but I think your implementation is overly complicated. You want to do a binary search, but you don't know what the upper bound is. So instead of starting at the middle, you start at index 1 (assuming dictionary indexes start at 0).
If the word you're looking for is "less than" the current dictionary word, halve the distance between the current index and your "low" value. ("low" starts at 0, of course).
If the word you're looking for is "greater than" the word at the index you just examined, then either halve the distance between the current index and your "high" value ("high" starts at 2) or, if index and "high" are the same, double the index.
If doubling the index gives you an out of range exception, you halve the distance between the current value and the doubled value. So if going from 16 to 32 throws an exception, try 24. And, of course, keep track of the fact that 32 is more than the max.
So a search sequence might look like 1, 2, 4, 8, 16, 12, 14 - found!
It's the same concept as a binary search, but rather than starting with low = 0, high = n-1, you start with low = 0, high = 2, and double the high value when you need to. It's still O(log N), although the constant is going to be a bit larger than with a "normal" binary search.
You can incur a one-time cost of O(n), if you know that the dictionary will not change. You can add all the words in the dictionary to a hashtable, and then any subsequent calls to isWordInDictionary() will be O(1) (in theory).
Use the getWord() API to copy the entire contents of the dictionary into a more sensible data structure (e.g. hash table, trie, perhaps even augmented by a Bloom filter). ;-)
In a different language:
#!/usr/bin/perl
$t=0;
$cur=1;
$under=0;
$EOL=int(rand(1000000))+1;
$TARGET=int(rand(1000000))+1;
if ($TARGET>$EOL)
{
$x=$EOL;
$EOL=$TARGET;
$TARGET=$x;
}
print "Looking for $TARGET with EOL $EOL\n";
sub testWord($)
{
my($a)=#_;
++$t;
return 0 if ($a eq $TARGET);
return -2 if ($a > $EOL);
return 1 if ($a > $TARGET);
return -1;
}
while ($r = testWord($cur))
{
print "Tested $cur, got $r\n";
if ($r == 1) { $over=$cur; }
if ($r == -1) { $under=$cur; }
if ($r == -2) { $over = $cur; }
if ($over)
{
$cur = int(($over-$under)/2)+$under;
$cur++ if ($cur <= $under);
$cur-- if ($cur >= $over);
}
else
{
$cur *= 2;
}
}
print "Found $TARGET at $r in $t tests\n";
The main benefit of this one is it is a bit simpler to understand. I think it may be more efficient if your first guesses are below the target since I don't think you are taking advantage of the space you have already "searched", but that is just with a quick glance at your code. Since it is looking for numbers for simplicity, it doesn't have to deal with not finding the target, but that is an easy extension.
#Sergii Zagriichuk hope the interview went well. Good luck with that.
I think just as #alexcoco said Binary Search is the answer.
Other options I see are only available if you could extend the dictionary. You could make it slightly better. E.g. You could count the words on each letter, and keep their track this way you would effectively had to work only on a subset of words.
Or yea as guys are saying to entirely implement your own dictionary structure.
I know this doesn't answer you question properly. But I cannot see other possibilities.
BTW would be nice to see your algorithm.
EDIT:
Expanding on my comment under answer of bshields...
#Sergii Zagriichuk even better it would be to remember the last index where we had null (no word), I think. Then at each run you could check if it is still true. If not then expand the range to a 'previous index' obtained by reversing the binary search behaviour, so we have null again. This way you would always adjust the size of the range of your search algorithm, thus adapting to the current state of the dictionary as needed. Plus the changes would have to be significant in order to cause your range adjustment so the adjustment wouldn't have any real negative impact on the algorithm. Also dictionaries tend to be static in nature so this should work :)
On one hand yes you are right with binary search implementation. But on the other hand in case dictionary is static and is not changed between lookups - we could suggest different algorithm. Here we have common problem - string sorting/search is different comparing to sorting/searching int array, so getWord(int i).compareTo(string) is O(min(length0, length1)).
Suppose we have request to find words w0, w1, ... wN, during lookup we could build up a tree with indicies (probably some suffix tree will good enough for this task).
During next lookup request we have following set a1, a2, ... aM, so to decrease average time we could first decrease range by searching position in the tree.
The problem with this implementation is concurrency and memory usage, so next step is implementing strategy to make search tree smaller.
PS: main aim was to check ideas and problems you suggest.
Well i think the info that dictionary is sorted can be utilized in a better way.
Say you are looking for a word "Zebra" , whereas the first guess search resulted in "abcg".
So we can use this info in chossing the second guess index . like in my case the resulted word is starting with a , whereas i am looking for something starting with z. So rather than making a static jump , i can make some calculated jump based on the current result and desired result. So in this way suppose if my next jump takes me to the word "yvu" , i now i am very near , so i will make a rather slow small jump than in the prev case.
Here is my solution.. uses O(logn) operations. First part of the code tries to find a estimate of the length and then the second part takes advantage of the fact that the dictionary is sorted and performs a binary search.
boolean isWordInTheDictionary(String word){
if (word == null){
return false;
}
// estimate the length of the dictionary array
long len=2;
String temp= getWord(len);
while(true){
len = len * 2;
try{
temp = getWord(len);
}catch(IndexOutOfBoundsException e){
// found upped bound break from loop
break;
}
}
// Do a modified binary search using the estimated length
long beg = 0 ;
long end = len;
String tempWrd;
while(true){
System.out.println(String.format("beg: %s, end=%s, (beg+end)/2=%s ", beg,end,(beg+end)/2));
if(end - beg <= 1){
return false;
}
long idx = (beg+end)/2;
tempWrd = getWord(idx);
if(tempWrd == null){
end=idx;
continue;
}
if ( word.compareTo(tempWrd) > 0){
beg = idx;
}
else if(word.compareTo(tempWrd) < 0){
end= idx;
}else{
// found the word..
System.out.println(String.format("getword at index: %s, =%s", idx,getWord(idx)));
return true;
}
}
}
Assuming the dictionary is 0-based, I would decompose the search in two parts.
First, given that the index to parameter to getWord() is an integer, and assuming that the index must be a number between 0 and the maximum positive integer, perform a binary search over that range in order to find the maximum valid index (irrespective of the word values). This operation is O(log N), since is a simple binary search.
Once obtained the size of the dictionary, a second ordinary binary search (again of complexity O(log N)) will bring on the desired answer.
Since O(log N)+O(log N) is O(log N), this algorithm complies with your requirement.
I'm in a hiring proccess which asked me this same problem...
My approach was a bit different, and considering the dictionary (webservice) I have, it's about 30% more efficient (for the words I've tested).
Here is the solution:
https://github.com/gustavompo/wordfinder
I'll not post the whole solution here because it's decoupled through classes and methods, but the core algorithm is this:
public WordFindingResult FindWord(string word)
{
var callsCount = 0;
var lowerLimit = new WordFindingLimit(0, null);
var upperLimit = new WordFindingLimit(int.MaxValue, null);
var wordToFind = new Word(word);
var wordIndex = _initialIndex;
while (callsCount <= _maximumCallsCount)
{
if (CouldNotFindWord(lowerLimit, upperLimit))
return new WordFindingResult(callsCount, -1, string.Empty, WordFindingResult.ErrorCodes.NOT_FOUND);
var wordFound = RetrieveWordAt(wordIndex);
callsCount++;
if (wordToFind.Equals(wordFound))
return new WordFindingResult(callsCount, wordIndex, wordFound.OriginalWordString);
else if (IsIndexTooHigh(wordToFind, wordFound))
{
upperLimit = new WordFindingLimit(wordIndex, wordFound);
wordIndex = IndexConsideringTooHighPreviousResult(lowerLimit, wordIndex);
}
else
{
lowerLimit = new WordFindingLimit(wordIndex, wordFound);
wordIndex = IndexConsideringTooLowPreviousResult(lowerLimit, upperLimit, wordToFind);
}
}
return new WordFindingResult(callsCount, -1, string.Empty, WordFindingResult.ErrorCodes.CALLS_LIMIT_EXCEEDED);
}
private int IndexConsideringTooHighPreviousResult(WordFindingLimit maxLowerLimit, int current)
{
return BinarySearch(maxLowerLimit.Index, current);
}
private int IndexConsideringTooLowPreviousResult(WordFindingLimit maxLowerLimit, WordFindingLimit minUpperLimit, Word target)
{
if (AreLowerAndUpperLimitsDefined(maxLowerLimit, minUpperLimit))
return BinarySearch(maxLowerLimit.Index, minUpperLimit.Index);
var scoreByIndexPosition = maxLowerLimit.Index / maxLowerLimit.Word.Score;
var indexOfTargetBasedInScore = (int)(target.Score * scoreByIndexPosition);
return indexOfTargetBasedInScore;
}