So one of our goals is to update the existing games so that you can switch back and forth between a human and computer player during the game instead of just at startup. Most of the games will be fairly easy to adapt to this system. We generally write the computer players in C++ while the rest of the game logic is in torquescript. This separation has meant that we generally pass everything the AI needs to make a decision each time the AI has to play.

Yacht didn’t have an AI at all, so to adapt it to the new system required me to write a computer player for it.

Yacht is a moderately complex game for a computer to play. The decision of which dice to keep depends not just on what you have rolled, but also on which categories are left to score. To play well, the AI needs to take into account the relative value and difficulty of each category and the chances of getting the bonus.

Fortunately, Yacht has been solved(pdf). Unfortunately, it is still too computationally expensive to use this method while playing a game. I implemented a simpler system. It is fairly simple to find the “best” way to play each round if you ignore future rounds. This is similar to solving for one “widget” as described in the paper linked above. Then, instead of trying to solve for all possible futures, I simply use a set of rules to describe the value of each category. I can also tweak the constants used when applying these rules to make different two computer players play slightly differently.

The computer player plays a pretty good game and is fast enough to not be noticeable. If I wanted the AI to be more sophisticated, I would add more logic for determining the probability of getting the bonus based on the current scores. Then I would try to improve the value that I assign to each category by having AIs play against each other with different settings to see what works best.

Here is what the new GUI looks like:

It is not possible for the computer to look through all the possible games to decide what to do since there are a staggeringly huge number of games. I decided to make the AI score each possible option after each throw. The tricky part is scoring the possibilities. The first step was to build a table of all the possible combinations of the five dice and how likely they are to come up. For example: there is only one way to roll all ones, but getting four ones and one two can happen five ways. For a given roll, the choices that you have depend on the rejects that you have taken so far. So, for each set of three rejects, I looked at each dice combination and built all the choices that you would have. For example: if you are rejecting 1,2 and 3 and you roll 1,3,4,4,5 you can: reject 1, take 7,9; reject 1, take 8,8; reject 3, take 5,9; or reject 3 take 6,8. So this roll give you one way to score 7, three ways to score 8, two ways to score 9, etc.. I then add the number of times you could take each number times that number’s score to the weighting for that number. In the previous example there were three ways to score 8 * 60 ways to roll a 1,3,4,4,5 * 40 points for eight = 19,200. After going through all the possible rolls I had a weighting for each number, repeating for each set of three rejects, I had a strategy of which dice to take based on your rejects.

The next step was to write the actual game AI based on this strategy. The AI played by scoring each possible choice and taking the best scoring choice. The choice’s score is the amount of points you will gain/lose for scoring those dice, plus a factor for getting the set of three rejects that the strategy picked, plus a factor for getting the three rejects picked quickly, plus a factor for keeping the rejects taken a similar number of times, plus a factor for picking scoring pairs that have a high weighting. I picked the relative importance of each of the five factors by hand.

The next step was to make some improvements to the game logic. I wrote some code to handle the fact that scoring two of the same number in one round is not the same as scoring each number separately. I also wrote code to handle the symmetry of the rejects: 1,2,6 is the same as 1,5,6; so if your strategy is to go for 1,2,6 and you have the option to reject a 5, that is just as good. And finally, if you are forced to reject a number outside your strategy, I update the weightings of each scoring number. Once all the play logic was written, the AI was scoring about 0 points for the best set of rejects and -100 to -50 for the other sets of rejects.

The next step was to write code to run the game many many times with different values for the five scoring factors to find the best set. To do this, I create a population of 20 different values and play 50K games. The top scoring set of values is then used to make 20 new strategies. Repeating this 100 times results in a good set of scoring factors. This step improved the AI to an average score of 65 for the best reject set and 10 for the other reject sets. The average human averages -50 and a good player can average +100.

William and I have completed the Land Bridges program. After playing a similar game, we were inspired to make the game with an AI. We imagined that the AI would be fairly easy to write and could play so much better than humans.

William wrote the interface to the game and created all the art. I wrote a back-end model of the game and the AI. The front-end of the game is written in Torque, a C++ game engine.

The AI was much harder than I expected it to be. The number of possible moves for the AI to consider was much larger than I initially thought, and it became impossible for the computer to “see” more than a couple moves into the future without trimming down the move tree.