I am trying to write a simple AI for a "Get four" game.
The basic game principles are done, so I can throw in coins of different color, and they stack on each other and fill a 2D Array and so on and so forth.
until now this is what the method looks like:
public int insert(int x, int color) //0 = empty, 1=player1 2=player2"
X is the horizontal coordinate, as the y coordinate is determined by how many stones are in the array already, I think the idea is obvious.
Now the problem is I have to rate specific game situations, so find how many new pairs, triplets and possible 4 in a row I can get in a specific situation to then give each situation a specific value. With these values I can setup a "Game tree" to then decide which move would be best next (later on implementing Alpha-Beta-Pruning).
My current problem is that I can't think of an efficient way to implement a rating of the current game situation in a java method.
Any ideas would be greatly appreciated!
I'm guessing that this is a homework assignment, and that you mean you want to write the evaluation function and don't know what tricks to use?
The game is called "Connect 4" in English, so you can google for
"connect 4 evaluation function".
You can find enough people discussion heuristics.
Please don't copy an actual source code, it's an important exercise :)
The search space for Connect 4 isn't impossibly large. For a simple implementation, albeit one that'll take a while to run (perhaps tens of minutes) do a minimax search until someone wins, or the game ends. Assign +1 or -1 for a win for one player or the other, and 0 for a draw.
bollocks. search space is huge. you need to use a predefined table if you want to do that.
Related
I'm making my own 5 in a row game where the board is bigger than 5x5, the size isn't decided but let's say 10x10. I'm designing this with a minimax algorithm and alfa-beta pruning. I've decided that a winning situation has the utility function of: (empty places + chosen place)*5, so if computer finds a win where there will be 2 empty places left then the value will be (2+1)*5 = 15. Same calculation for lose but times -5 instead of 5. For a draw it'll evaluate 0. Now, the issue that I'm writing this for is this: How do I determine the utility for an unfinished scenario where the depth limit is reached? Simply making it 0 isn't good enough, there must be some kind of "guess" or ranking.
An idea I had was to count all the X's in every row, every column and every possible diagonal and do empty places+chosen place times biggest found sequence of X's. The issue there is that the calculations take time and are a pain to write and then you'd also have to take into account the edges of the sequences found: are they bordered by O's? Should we count X's with empty gaps?
It seems awfully complicated and I wondered therefore if anyone has any good advise for me in regards to the unfinished scenarios. Thank you!
I have no idea what you are talking about in your first paragraph, a win should be scored as a win (minus depth to always find shortest win). If I understand you correctly you are looking to create an evaluation/heuristics function to be able to input a board state and output a static score for that state? Gomoku (5 in a row) is a popular game that has many engines, Google "evaluation function gomoku" and you will find lots of information and paper on the subject.
For a game like chess the evaluation is done e.g. by counting each sides material balance, piece positions, king safety etc. For X-in-a-row the usual approach is to give all possible scenarios a score and then add them up together. The scenarios for Gomoku are usually as follows:
FiveInRow (Win)
OpenFour
ClosedFour
OpenThree
ClosedThree
OpenTwo
ClosedTwo
Open means there are no blocking enemy pieces next to the friendly pieces:
OpenThree: -O--XXX---
ClosedThree: --OXXX---
How to sum them all up, and what score to give each scenario, is up to you and that is where the fun and experimenting begins!
So I am working on a word problem, and it goes something like this:
! section removed
I am not posting the exact question or diagram just in case someone cannot stop themselves from posting a direct answer.
I am trying to write a program in Java for this, but I can't really get a grip on how to do this logically. I know I can start of with an array for each square like this:
int square1 = //four sides with each liquid value
but I do not know how to include the unlimited ones (which are diagonal), or how to continue once I figure it out.
Any help with this programming logic is appreciated.
I'd say go through each pipe and record the maximum going through it, which should be the lower of its capacity or the sum of the maxima going into it. The unlimited ones then just take the total going into them (or, at the very beginning, Integer.MAX_VALUE). Or you could use a bool for tracking whether it's infinite. Or, just for fun, Double.POSITIVE_INFINITY.
Note that if there are any cycles in the pipes, you may need to cycle through and update a couple times, until you get a steady answer.
Edit: After looking a bit at the max flow problem linked in a comment and thinking about it, I'm not so sure this is the right way to go, as I don't think it accounts for splitting current between two outgoing pipes. Perhaps some distant relative of Ohm's law, adapted for water?
Honestly, I only knew of such a game recently and I wonder how one can create a solving algorithm using the recursive search method?
There are 15 holes in total in the triangular board. Making that 14 pegs with a total of 13 moves.
I don't know where to start with this in C++ nor Java. I have studied C++ for an year before. So I'm familiar with the concepts of stacks, linked lists etc.
I just don't know how to start the code. The program firstly asks the user where they want to start (How is this done?)
Then once it solves it , a certain number of pegs more than just one will be left and the program will ask the user for a better solution (like this until the board is left to just one peg.)
I certainly cannot think of how to make the moves possible ( How do I write a code that "SHOWS" that one peg moves over a hole ,into another?)
I'd love some coding assistance here. It would really be appreciated.
Try treating the board a linked list of holes or positions. Each data field in a node would represent a hole. Each node would have a vector of links to other holes, depending on its position relative to the edge of the board.
To move a peg, iterate over the possible links.
This is just one method, there are probably better ones out there.
Take a look at my answer here: Timeout on a php Peg Puzzle solver. The 15-peg puzzle was the first program that I ever wrote (over 10 years ago), after just learning c++.
The posted solution is a re-write that I did several years later.
Since I was the answerer I can tell you that a triplet is a term that I made up for a move. There are three pegs involved in a move (a triplet). Each peg is represented by a bit. In order for a move to be legal two consecutive pegs (bits) need to be set and the other (the target location) needs to be clear.
The entire solution involves a depth first search using a 16-bit integer to represent the board. Simple bit manipulation can be used to update the board state. For example if you xor the current state with a move (triplet) it will make the move against the board. The same operation applied a second time against the board is an undo of the move.
To be successful as a programmer, you need to develop the skill of examining a problem, figuring out how it can be solved, and come up with a reasonable in-program representation that enables you to solve it.
For problems like this, I find it helpful to have the puzzle in front of me, so I can try it out by hand. Failing that, at least having a picture of the thing will help:
http://www.obsidiandesigns.com/pyramidsol.jpg
Here's another way to look at it, and possibly consider representing it in memory like this:
OXXXX
XXXX
XXX
XX
X
Each "X" is a peg. The "O" is a hole. One can change "OXX" to "XOO" by jumping the 3rd X over the middle one, into the hole. Also consider vertical and diagonal jumps.
Linked lists could make a lot of sense. You might actually want "2D" links, instead of the usual "1D" links. That is each "hole" instance could contain pointers to the two or three holes next to it.
Once you have a representation, you need a way to find valid moves. And to make valid moves.
For this "game," you may want to search the whole space of possible moves, so you'll probably want to either have each move produce a new board, or to have a way to "undo" moves.
A valid move is where one peg jumps over another peg, into an empty hole. The jumped peg is removed. You can't jump from "just anywhere" to anywhere, over any peg you wish -- the three holes have to be next to each other and in line.
The above should give you some hints that will help you get started.
I'm practicing up for a programming competition, and i'm going over some difficult problems that I wasn't able to answer in the past. One of them was the King's Maze. Essentially you are given an NxN array of numbers -50<x<50 that represent "tokens". You have to start at position 1,1(i assume that's 0,0 in array indexes) and finish at N,N. You have to pick up tokens on cells you visit, and you can't step on a cell with no token (represented by a 0). If you get surrounded by 0's you lose. If there is no solution to a maze you output "No solution". Else, you output the highest possible number you can get from adding up the tokens you pick up.
I have no idea how to solve this problem. I figured you could write a maze algorithm to solve it, but that takes time, and in programming contests you are only given two hours to solve multiple problems. I'm guessing there's some sort of pattern i'm missing. Anyone know how I should approach this?
Also, it might help to mention that this problem is meant for high school students.
This type of problem is typically solved using dynamic programming or memoization.
Basically you formulate a recursive solution, and solve it bottom up while remembering and reusing previously computed results.
The simple approach (i.e. simplest to code) is try all the possible paths - try each first step; for each first step try each second step; for each first step/second step combination try each third step; and so on. However depending on how big the maze is this may take too long to run (or it may not).
Your next step is to think about how you can do this faster. The first step is usually to eliminate moves that you know can't lead to a finish, or can't lead to a finish with higher points than the one you already have. Since this is practice for a competition we'll leave you to do this work yourself.
Think "graph" algorithms: The Algorithm Design Manual
As I have mentioned in previous questions I am writing a maze solving application to help me learn about more theoretical CS subjects, after some trouble I've got a Genetic Algorithm working that can evolve a set of rules (handled by boolean values) in order to find a good solution through a maze.
That being said, the GA alone is okay, but I'd like to beef it up with a Neural Network, even though I have no real working knowledge of Neural Networks (no formal theoretical CS education). After doing a bit of reading on the subject I found that a Neural Network could be used to train a genome in order to improve results. Let's say I have a genome (group of genes), such as
1 0 0 1 0 1 0 1 0 1 1 1 0 0...
How could I use a Neural Network (I'm assuming MLP?) to train and improve my genome?
In addition to this as I know nothing about Neural Networks I've been looking into implementing some form of Reinforcement Learning, using my maze matrix (2 dimensional array), although I'm a bit stuck on what the following algorithm wants from me:
(from http://people.revoledu.com/kardi/tutorial/ReinforcementLearning/Q-Learning-Algorithm.htm)
1. Set parameter , and environment reward matrix R
2. Initialize matrix Q as zero matrix
3. For each episode:
* Select random initial state
* Do while not reach goal state
o Select one among all possible actions for the current state
o Using this possible action, consider to go to the next state
o Get maximum Q value of this next state based on all possible actions
o Compute
o Set the next state as the current state
End Do
End For
The big problem for me is implementing a reward matrix R and what a Q matrix exactly is, and getting the Q value. I use a multi-dimensional array for my maze and enum states for every move. How would this be used in a Q-Learning algorithm?
If someone could help out by explaining what I would need to do to implement the following, preferably in Java although C# would be nice too, possibly with some source code examples it'd be appreciated.
As noted in some comments, your question indeed involves a large set of background knowledge and topics that hardly can be eloquently covered on stackoverflow. However, what we can try here is suggest approaches to get around your problem.
First of all: what does your GA do? I see a set of binary values; what are they? I see them as either:
bad: a sequence of 'turn right' and 'turn left' instructions. Why is this bad? Because you're basically doing a random, brute-force attempt at solving your problem. You're not evolving a genotype: you're refining random guesses.
better: every gene (location in the genome) represents a feature that will be expressed in the phenotype. There should not be a 1-to-1 mapping between genome and phenotype!
Let me give you an example: in our brain there are 10^13ish neurons. But we have only around 10^9 genes (yes, it's not an exact value, bear with me for a second). What does this tell us? That our genotype does not encode every neuron. Our genome encodes the proteins that will then go and make the components of our body.
Hence, evolution works on the genotype directly by selecting features of the phenotype. If I were to have 6 fingers on each hand and if that would made me a better programmer, making me have more kids because I'm more successful in life, well, my genotype would then be selected by evolution because it contains the capability to give me a more fit body (yes, there is a pun there, given the average geekiness-to-reproducibily ratio of most people around here).
Now, think about your GA: what is that you are trying to accomplish? Are you sure that evolving rules would help? In other words -- how would you perform in a maze? What is the most successful thing that can help you: having a different body, or having a memory of the right path to get out? Perhaps you might want to reconsider your genotype and have it encode memorization abilities. Maybe encode in the genotype how much data can be stored, and how fast can your agents access it -- then measure fitness in terms of how fast they get out of the maze.
Another (weaker) approach could be to encode the rules that your agent uses to decide where to go. The take-home message is, encode features that, once expressed, can be selected by fitness.
Now, to the neural network issue. One thing to remember is that NNs are filters. They receive an input. perform operations on it, and return an output. What is this output? Maybe you just need to discriminate a true/false condition; for example, once you feed a maze map to a NN, it can tell you if you can get out from the maze or not. How would you do such a thing? You will need to encode the data properly.
This is the key point about NNs: your input data must be encoded properly. Usually people normalize it, maybe scale it, perhaps you can apply a sigma function to it to avoid values that are too large or too small; those are details that deal with error measures and performance. What you need to understand now is what a NN is, and what you cannot use it for.
To your problem now. You mentioned you want to use NNs as well: what about,
using a neural network to guide the agent, and
using a genetic algorithm to evolve the neural network parameters?
Rephrased like so:
let's suppose you have a robot: your NN is controlling the left and right wheel, and as input it receives the distance of the next wall and how much it has traveled so far (it's just an example)
you start by generating a random genotype
make the genotype into a phenotype: the first gene is the network sensitivity; the second gene encodes the learning ratio; the third gene.. so on and so forth
now that you have a neural network, run the simulation
see how it performs
generate a second random genotype, evolve second NN
see how this second individual performs
get the best individual, then either mutate its genotype or recombinate it with the loser
repeat
there is an excellent reading on the matter here: Inman Harvey Microbial GA.
I hope I did you some insight on such issues. NNs and GA are no silver bullet to solve all problems. In some they can do very much, in others they are just the wrong tool. It's (still!) up to us to get the best one, and to do so we must understand them well.
Have fun in it! It's great to know such things, makes everyday life a bit more entertaining :)
There is probably no 'maze gene' to find,
genetic algorithms are trying to setup a vector of properties and a 'filtering system' to decide by some kind of 'surival of the fittest' algorithm to find out which set of properties would do the best job.
The easiest way to find a way out of a maze is to move always left (or right) along a wall.
The Q-Algorithm seems to have a problem with local maxima this was workaround as I remember by kicking (adding random values to the matrix) if the results didn't improve.
EDIT: As mentioned above a backtracking algorithm suits this task better than GA or NN.
How to combine both algorithm is described here NeuroGen descibes how GA is used for training a NN.
Try using the free open source NerounDotNet C# library for your Neural networks instead of implementing it.
For Reinforcement Learning library, I am currently looking for one, especially for Dot NET framework..