Develop Reference

Java Path-finding puzzle game

I need some advise in an approach I may need to take to solve a gaming problem which is a puzzle (NxN), the puzzle consists of positive numbers and stored in a two dimensional array. For simplistic reasons is i´ll list a simple example
2 1 2 2
1 3 2 1
1 0 2 1
3 1 2 0
So the starting point is at (0,0) => 2 and the goal location is to (3,3) => 0
The number in the array location tells you how far to move. (0,0)=> 2 can move to either (0,2) or (2,0) and so on (moves allowed left, right, up, or down)
So you end up with a solution like this for example (0,2)=>(2,2)=>(2,0)=>(3,0)=>(3,3).
so my question is what sort of algorithm i should be looking into and whether any of you have done something similar to this?

You have plenty of solutions here:
A* algorithm
Dijkstra
Depth-first
Breadth-first
The first two will give you an optimal solution if one exists. A* is typically faster than Dijkstra if the heuristic is well chosen. Breadth- first will also give you an optimal implementation. Depth-first may give you non-optimal solutions in this problem.
The main difference between A* and Djisktra is that A* defines a heuristic, namely a function that tries to estimate if a move is better than another one.
The main difference between depth-first and breadth-first is the order in which they explore the space of solutions. Breadth-first will start by looking for all solutions of length 1 then all solutions of length 2, etc, while depth-first will fully explore an entire path until it either cannot go any further or finds a solution.
A* and Dijkstra are typically implemented in imperative style and are probably more sophisticated than the other two, especially A*. Breadth-first is also naturally expressed in imperative style. Depth-first is generally expressed recursively, which can be a problem if your solutions can exceed a length of several thousands moves (depending on the size of your stack, you will generally only be able to make 7-10k recursive calls before you get a StackOverflowError).
To sum up:
A* is generally the most efficient of the algorithms listed below
A* is the most difficult to implement
Dijkstra is a special case of A* with similar performance but potentially less efficient
Breadth-first is straightforward to implement and is resilient to long solutions
Depth-first is straightforward to implement but it is limited by the length of the longest possible path if it is implemented recursively
All these algorithms except depth-first guarantee an optimal solution
Code example:
I found this Scala implementation of A* in one of my repositories. Might help.

I have some grid search algorithms (Best-First, Breadth-First, Depth-First) implemented here in Object Pascal (Delphi), that you could easily adapt to Java if this was a classic grid search:
https://github.com/Zoomicon/GridSearchDemo/tree/master/Object%20Pascal/algorithms
You can try the GridSearchDemo application here to see how those algorithms behave when searching in a grid with start and target point and obstacles in various grid cells (you can set them):
https://github.com/Zoomicon/GridSearchDemo/releases
In general, I prefer the A* algorithm, which is an example of a Best-First algorithm (https://en.wikipedia.org/wiki/Best-first_search)
In your case, this is not a grid really, but a graph, since you seem to have jump links to other cells (or at least this is how you explain the number in your question, although you call it "how far" at first)

I have written a program in java to solve this problem. It uses A*-algorithm with heuristic functions Manhatten and hamming. It depends on the person whether he uses hamming or manhatten distance but Manhatten is better.
Here is my code in java: 8-puzzle
Btw it's not an easy approach and many problems can't be solved.

Fastest algorithm for locating an object in a field

What would be the best algorithm in terms of speed for locating an object in a field?
The field consists of 18 by 18 squares with side length 30.48 cm. The robot is placed in the square (0,0) and its job is to reach the light source while avoiding obstacles along the way. To locate the light source, the robot does a 360 degree turn to find the angle with the highest light reading and then travels towards the source. It can reliably detect a light source from 100 cm.
The way I'm implementing this presently is I'm storing the information about each tile in a 2x2 array. The possible values of the tiles are unexplored (default), blocked (there's an obstacle), empty (there's nothing in there). I'm thinking of using the DFS algorithm where the children are at position (i+3,j) or (i,j+3). However, considering the fact that I will be doing a rotation to locate the angle with the highest light reading at each child, I think there may be an algorithm which may be able to locate the light source faster than DFS. Also, I will only be travelling in the x and y directions since the robot will be using the grid lines on the floor to make corrections to it's x and y positions.
I would appreciate it if a fast and reliable algorithm could be suggested to accomplish this task.

This is a really broad question, and I'm not an expert so my answer is based on "first principles" thinking rather than experience in the field.
(I'm assuming that your robot has generally unobstructed line of sight and movement; i.e. it is an open area with scattered obstacles, not in a maze.)
The problem is interpreting the information that you get back from a 360 degree scan.
If the robot sees the light source, then traversing a route to the light source is either trivial, or a "simple" maze walking task.
The difficulty is when you don't see the source. It might mean that the source is not within the circle of visibility. But it could also mean that the light is behind an obstacle. And unfortunately, a simple sensor like you are describing cannot distinguish these two cases.
If your sensor system allowed you to see the obstacles, you could plot the locations of the "shadow" regions (regions behind obstacles), and use that to keep track of the places that are left to search. So your strategy would be to visit a small number of locations and do a scan at each, then methodically "tidy up" a small number of areas that were in shadow.
But since you cannot easily tell where the shadow areas are, you need an algorithm that (ultimately) searches everywhere. DFS is a general strategy that searches everywhere, but it does it by (in effect) looking in the nooks and crannies first. A better strategy is to a breadth first search, and only visit the nooks and crannies if the wide-scale scans didn't find the light source.
I would appreciate it if a fast and reliable algorithm could be suggested to accomplish this task.
I think you are going to need to develop one yourself. (Isn't this the point of the problem / task / competition?)

Although it may not look like it, this looks a more like a maze following problem than anything. I suppose this is some kind of challenge or contest situation, where there's always a path from start to target, but suppose there's not for a moment. One of the successful results for a robot navigating a beacon fully surrounded by obstacles would be a report with a description of a closed path of obstacles surrounding a signal. If there's not such a closed path, then you can find a hole in somewhere; this is why is looks like maze following.
So the basic algorithm I'd choose is to start with a spiraling-inward tranversal, sweeping out a path narrow enough so that you're sure to see a beacon if one is present. If there are no obstacles (a degenerate case), this finds the target in minimal time. (Hint: each turn reduces the number of cells your sensor can locate per step.)
Take the spiral traversal to be counter-clockwise. What you have then is related to the rule for solving mazes by keeping your right hand on the wall and following the generated path. In this case, you have the complication that, while the start of the maze is on the boundary, the end may not be. It's possible of the right-hand-touching path to fail in such a situation. Detecting this situation requires looking for "cavities" in the region swept out by adjacency to the wall.

The King's Maze

I'm practicing up for a programming competition, and i'm going over some difficult problems that I wasn't able to answer in the past. One of them was the King's Maze. Essentially you are given an NxN array of numbers -50<x<50 that represent "tokens". You have to start at position 1,1(i assume that's 0,0 in array indexes) and finish at N,N. You have to pick up tokens on cells you visit, and you can't step on a cell with no token (represented by a 0). If you get surrounded by 0's you lose. If there is no solution to a maze you output "No solution". Else, you output the highest possible number you can get from adding up the tokens you pick up.
I have no idea how to solve this problem. I figured you could write a maze algorithm to solve it, but that takes time, and in programming contests you are only given two hours to solve multiple problems. I'm guessing there's some sort of pattern i'm missing. Anyone know how I should approach this?
Also, it might help to mention that this problem is meant for high school students.

This type of problem is typically solved using dynamic programming or memoization.
Basically you formulate a recursive solution, and solve it bottom up while remembering and reusing previously computed results.

The simple approach (i.e. simplest to code) is try all the possible paths - try each first step; for each first step try each second step; for each first step/second step combination try each third step; and so on. However depending on how big the maze is this may take too long to run (or it may not).
Your next step is to think about how you can do this faster. The first step is usually to eliminate moves that you know can't lead to a finish, or can't lead to a finish with higher points than the one you already have. Since this is practice for a competition we'll leave you to do this work yourself.

Think "graph" algorithms: The Algorithm Design Manual

Building an EFFICIENT Sudoku Solver

Yes, I know this is nothing new and there are many questions already out there (it even has its own tag), but I'd like to create a Sudoku Solver in Java solely for the purpose of training myself to write code that is more efficient.
Probably the easiest way to do this in a program is have a ton of for loops parse through each column and row, collect the possible values of each cell, then weed out the cells with only one possibility (whether they contain only 1 number, or they're the only cell in their row/column that contains this number) until you have a solved puzzle. Of course, a sheer thought of the action should raise a red flag in every programmer's mind.
What I'm looking for is the methodology to go about solving this sucker in the most efficient way possible (please try not to include too much code - I want to figure that part out, myself).
I want to avoid mathematical algorithms if at all possible - those would be too easy and 100% not my work.
If someone could provide a step-by-step, efficient thought process for solving a Sudoku puzzle (whether by a human or computer), I would be most happy :). I'm looking for something that's vague (so it's a challenge), but informative enough (so I'm not totally lost) to get me started.
Many thanks,
Justian Meyer
EDIT:
Looking at my code, I got to thinking: what would be some of the possibilities for storing these solving states (i.e. the Sudoku grid). 2D Arrays and 3D Arrays come to mind. Which might be best? 2D might be easier to manage from the surface, but 3D Arrays would provide the "box"/"cage" number as well.
EDIT:
Nevermind. I'm gonna go with a 3D array.

It depends on how you define efficient.
You can use a brute force method, which searches through each column and row, collects the possible values of each cell, then weeds out the cells with only one possibility.
If you have cells remaining with more than one possibility, save the puzzle state, pick the cell with the fewest possibilities, pick one of the possibilities, and attempt to solve the puzzle. If the possibility you picked leads to a puzzle contradiction, restore the saved puzzle state, go back to the cell and choose a different possibility. If none of the possibilities in the cell you picked solves the puzzle, pick the next cell with the fewest possibilities. Cycle through the remaining possibilities and cells until you've solved the puzzle.
Attempt to solve the puzzle means searching through each column and row, collecting the possible values of each cell, then weeding out the cells with only one possibility. When all of the cells are weeded out, you've solved the puzzle.
You can use a logical / mathematical method, where your code tries different strategies until the puzzle is solved. Search Google with "sudoku strategies" to see the different strategies. Using logical / mathematical methods, your code can "explain" how the puzzle was solved.

When I made mine, I thought I could solve every board using a set of rules without doing any backtracking. This proved impossible as even puzzles targeting human players potentially require making a few hypothesis.
So I starting with implementing the basic "rules" for solving a puzzle, trying to find the next rule to implement that would allow the resolution of where it stopped last time. In the end, I was forced to add a brute forcing recursive algorithm, but most puzzles are actually solved without using that.
I wrote a blog post about my sudoku solver. Just read through the "The algorithm" section and you'll get a pretty good idea how I went about it.
http://www.byteauthor.com/2010/08/sudoku-solver/

Should anyone need a reference Android implementation, I wrote a solution that uses the algorithm from the post above.
Full open-source code here: https://github.com/bizz84/SudokuSolver
Additionally, this solution loads Sudoku Puzzles in JSON format from a web server and posts back the results.

You should think about reducing the Sudoku Problem to a SATisfiability problem.
This method will avoid you to think too mathematically but more logically about the AI.
The goal step by step is basically :
* Find all the constraints that a Sudoku has. (line, column, box).
* Write these constraints as boolean constraints.
* Put all these constraints in a Boolean Satisfiability Problem.
* Run a SAT solver (or write your own ;) ) on this problem.
* Transform the SAT solution into the solution of the initial Sudoku.
It has been done by Ivor Spence by using SAT4J and you can find the Java Applet of his work here : http://www.cs.qub.ac.uk/~I.Spence/SuDoku/SuDoku.html.
You can also download directly the Java code from SAT4J website, to see how it look like : http://sat4j.org/products.php#sudoku.
And finally, the big advantage of this method is : You can solve N*N Sudokus, and not only the typical 9*9, which is I think, much more challenging for an AI :).

Boosting my GA with Neural Networks and/or Reinforcement Learning

As I have mentioned in previous questions I am writing a maze solving application to help me learn about more theoretical CS subjects, after some trouble I've got a Genetic Algorithm working that can evolve a set of rules (handled by boolean values) in order to find a good solution through a maze.
That being said, the GA alone is okay, but I'd like to beef it up with a Neural Network, even though I have no real working knowledge of Neural Networks (no formal theoretical CS education). After doing a bit of reading on the subject I found that a Neural Network could be used to train a genome in order to improve results. Let's say I have a genome (group of genes), such as
1 0 0 1 0 1 0 1 0 1 1 1 0 0...
How could I use a Neural Network (I'm assuming MLP?) to train and improve my genome?
In addition to this as I know nothing about Neural Networks I've been looking into implementing some form of Reinforcement Learning, using my maze matrix (2 dimensional array), although I'm a bit stuck on what the following algorithm wants from me:
(from http://people.revoledu.com/kardi/tutorial/ReinforcementLearning/Q-Learning-Algorithm.htm)
1. Set parameter , and environment reward matrix R
2. Initialize matrix Q as zero matrix
3. For each episode:
* Select random initial state
* Do while not reach goal state
o Select one among all possible actions for the current state
o Using this possible action, consider to go to the next state
o Get maximum Q value of this next state based on all possible actions
o Compute
o Set the next state as the current state
End Do
End For
The big problem for me is implementing a reward matrix R and what a Q matrix exactly is, and getting the Q value. I use a multi-dimensional array for my maze and enum states for every move. How would this be used in a Q-Learning algorithm?
If someone could help out by explaining what I would need to do to implement the following, preferably in Java although C# would be nice too, possibly with some source code examples it'd be appreciated.

As noted in some comments, your question indeed involves a large set of background knowledge and topics that hardly can be eloquently covered on stackoverflow. However, what we can try here is suggest approaches to get around your problem.
First of all: what does your GA do? I see a set of binary values; what are they? I see them as either:
bad: a sequence of 'turn right' and 'turn left' instructions. Why is this bad? Because you're basically doing a random, brute-force attempt at solving your problem. You're not evolving a genotype: you're refining random guesses.
better: every gene (location in the genome) represents a feature that will be expressed in the phenotype. There should not be a 1-to-1 mapping between genome and phenotype!
Let me give you an example: in our brain there are 10^13ish neurons. But we have only around 10^9 genes (yes, it's not an exact value, bear with me for a second). What does this tell us? That our genotype does not encode every neuron. Our genome encodes the proteins that will then go and make the components of our body.
Hence, evolution works on the genotype directly by selecting features of the phenotype. If I were to have 6 fingers on each hand and if that would made me a better programmer, making me have more kids because I'm more successful in life, well, my genotype would then be selected by evolution because it contains the capability to give me a more fit body (yes, there is a pun there, given the average geekiness-to-reproducibily ratio of most people around here).
Now, think about your GA: what is that you are trying to accomplish? Are you sure that evolving rules would help? In other words -- how would you perform in a maze? What is the most successful thing that can help you: having a different body, or having a memory of the right path to get out? Perhaps you might want to reconsider your genotype and have it encode memorization abilities. Maybe encode in the genotype how much data can be stored, and how fast can your agents access it -- then measure fitness in terms of how fast they get out of the maze.
Another (weaker) approach could be to encode the rules that your agent uses to decide where to go. The take-home message is, encode features that, once expressed, can be selected by fitness.
Now, to the neural network issue. One thing to remember is that NNs are filters. They receive an input. perform operations on it, and return an output. What is this output? Maybe you just need to discriminate a true/false condition; for example, once you feed a maze map to a NN, it can tell you if you can get out from the maze or not. How would you do such a thing? You will need to encode the data properly.
This is the key point about NNs: your input data must be encoded properly. Usually people normalize it, maybe scale it, perhaps you can apply a sigma function to it to avoid values that are too large or too small; those are details that deal with error measures and performance. What you need to understand now is what a NN is, and what you cannot use it for.
To your problem now. You mentioned you want to use NNs as well: what about,
using a neural network to guide the agent, and
using a genetic algorithm to evolve the neural network parameters?
Rephrased like so:
let's suppose you have a robot: your NN is controlling the left and right wheel, and as input it receives the distance of the next wall and how much it has traveled so far (it's just an example)
you start by generating a random genotype
make the genotype into a phenotype: the first gene is the network sensitivity; the second gene encodes the learning ratio; the third gene.. so on and so forth
now that you have a neural network, run the simulation
see how it performs
generate a second random genotype, evolve second NN
see how this second individual performs
get the best individual, then either mutate its genotype or recombinate it with the loser
repeat
there is an excellent reading on the matter here: Inman Harvey Microbial GA.
I hope I did you some insight on such issues. NNs and GA are no silver bullet to solve all problems. In some they can do very much, in others they are just the wrong tool. It's (still!) up to us to get the best one, and to do so we must understand them well.
Have fun in it! It's great to know such things, makes everyday life a bit more entertaining :)

There is probably no 'maze gene' to find,
genetic algorithms are trying to setup a vector of properties and a 'filtering system' to decide by some kind of 'surival of the fittest' algorithm to find out which set of properties would do the best job.
The easiest way to find a way out of a maze is to move always left (or right) along a wall.
The Q-Algorithm seems to have a problem with local maxima this was workaround as I remember by kicking (adding random values to the matrix) if the results didn't improve.
EDIT: As mentioned above a backtracking algorithm suits this task better than GA or NN.
How to combine both algorithm is described here NeuroGen descibes how GA is used for training a NN.

Try using the free open source NerounDotNet C# library for your Neural networks instead of implementing it.
For Reinforcement Learning library, I am currently looking for one, especially for Dot NET framework..