Approximately 10 years ago I was approached by a company from Israel that was working on some artificial intelligence projects. The owner loved to play poker, so he started on a new venture to try to see if he could develop a program based upon artificial intelligence to help him to become a better poker player.
This project involved the development and programming of the software, physics, mathematical analysis and a variety of gaming and casino marketing experts, including myself.
All of this was before the current boom and popularity of both online poker and casino gaming. Einstein was ahead of his time as well as the father of modern physics. Einstein was a genius; Einstein was a curious type of individual who liked to question “Why?” My poker enthusiast friend was the same type of creative, curious individual but whose passion was playing poker and gambling.
He was convinced that with the help of computers, proper programing and software development involving complex algorithms, etc., that he could learn the secrets and teach himself to be a great poker player and a great gambler.
I am still not sure whether he wanted to learn these skills purely for enjoyment or to make lots of money as a semi-professional, highly skilled player. He never told me.
Call Me AI
Artificial intelligence, or AI, can replicate or mimic many of the human brain functions. Computer programmers have often played with new software applications to try to match or improve upon human brain functions. This became very evident with the game of Chess. Deep Blue was the name given to a chess-playing computer developed by IBM.
On May 11, 1997, the machine, with human intervention between games, won the second six-game match against world champion Garry Kasparov by two wins to one with three draws. Kasparov accused IBM of cheating, and demanded a rematch. IBM refused and retired Deep Blue. Kasparov had beaten a previous version of Deep Blue in 1996. IBM used one of its faster computers in 1997, but that was 16 years ago.
Computer chips, RAM and software have been improving dramatically and geometrically every year since 1997. It is amazing how powerful some of the new computers for gaming are today compared to just a few years ago.
It is no wonder that other curious and creative individuals around the world have also tried to use AI to help them to become better poker players or better casino game players. I know of one group in particular from Russia that is actively involved in the development of some really creative AI applications for use with poker.
Poker is an incredibly popular game—or as some view it, a sport—and therefore provides a great new opportunity for all types of AI applications which can aid in helping millions of people to become better skilled players. There are now several different software applications available online for playing both social and gambling games of poker.
However, the task is complicated, because the programmers encounter many different and complicated situations such as game theory, random number generation, enormous game tree for more than three agents or players, substantial complexity with finding equilibrium strategies and finally, possible “deception” from other agents.
AI estimates the pot odds based upon its hole card and other information available to all players. AI simultaneously simulates all possible hole cards and actions of other players as well as possible board cards for the subsequent betting rounds. The most complex task is to determine the probability for the current hand to follow one or another path of the game tree.
AI is used to solve this problem by using opponent modeling. Each model contains a set of parameters, which are created for each opponent based on his actions in the previous games. Some of these parameters are quite common and widely used in statistical poker software while others are proprietary to each AI software developer.
Currently, applied AI studies have achieved considerable success developing innovative solutions for both private and public sectors. Successful commercial applications include automated trading systems, expert systems for medical diagnosis, customer services, toys and entertainment, whether it is for gaming or for non-gaming purposes.
AI and poker make an interesting combination, but it is also a relatively new testing ground for complicated AI research. While making computer agents that are able to play poker, scientists encounter many interesting and complicated problems, which need to be carefully addressed and tested. One Russian software company has developed a number of new artificial intelligences to play poker at a professional level.
The poker games include both no-limit Texas hold ‘em (two-10 players) and limit Texas hold ‘em (two-10 players). This particular Russian company now promotes that it has developed professional-level AI poker players, or “bots” (robots), and that this unique know-how can now be used in an advanced training system. Currently, the company’s lab is working on a new poker training service which will enable people to improve their poker skills with visually demonstrated deviation from optimal play, along with suggested strategy adjustments in real time.
They report that this is not only a stand-alone product but a product that could also be seamlessly integrated into any gambling portal or online poker room. With further detailed research, I also found an American software company offering a series of professionally produced tutorials that are also partially based upon AI to help train members to become better poker players. They have produced more than 200 educational videos ranging in skill level from beginner to intermediate to advanced.
The best poker bots in the world include those from the University of Alberta Computer Poker Research Group, which is nearly 20 years old. Professor Michael Bowling, who has led the group since 2005, says the breakthrough came in 2003, when researchers decided to change their approach, shifting away from the methodology used to build chess bots.
In 2006, the inaugural Annual Computer Poker Competition created more interest in poker-playing computers and established a friendly rivalry between the University of Alberta and Professor Tuomas Sandholm’s poker research group at Carnegie Mellon University in Pittsburgh. Basically, Sandholm recently said, poker bots “can rival good poker players, but not the best—yet.” His last word “yet” is what concerns many people.
In summary, AI estimates the probability of a player’s actions using expert rules and opponent models. In case the expert rules fail to provide the necessary estimation, AI would employ neural networks trained on historical data. The poker AI architecture is constantly being updated with various techniques from the artificial intelligence and game theory domains, which may include regret minimization, gradient search, equilibrium approximation, recursive search methods and others.
If all of this sounds pretty complicated, it is. So why even mention or talk about AI? The main reason is to educate people. AI is a fact of life today. It exists. AI is no longer science fiction or something that you only see in the movies. AI is out there, and it is helping to educate and train people to be better poker players and better gamblers.
It is not perfect and it is not foolproof, but I sure would like to know if I was risking my money playing against a computer poker bot or against another human being. And for that matter, I would also like to know if that human being has already improved or fine-tuned his poker or casino gaming skills by utilizing AI. Many professional poker players use bots to play multiple games at one time, allowing them to get the biggest advantage in the shortest time frame.
Therefore, a proactive approach or strategy seems to make the most sense today, so my advice to players with interest in improving or fine-tuning their poker skills is that they spend a little time first to educate themselves either via poker playing bots or educational poker training videos that may also involve the principles of AI. This is also an issue that should concern online poker room operators, and how they might regulate the use of poker bots.
What are poker bots? Poker bots are basically computer programs based upon AI that one can actually play against. The logic used by some poker bots utilizes different combinations of neural networks, regret minimization and gradient search equilibrium approximation, decision trees, recursive search methods and other expert algorithms from professional poker players in different types of poker games.
Neural networks learning for opponent modeling is based upon a database with millions of hands of history from real poker players. The poker bot analyzes accumulated statistical data very quickly, which allows AI to adjust its style of play against opponents similar to the model we presented earlier of IBM’s Deep Blue computer playing chess against the reigning world champion.
It is also interesting to note that most poker bots play fairly due to the fact that AI does not know your hole cards or what cards will be dealt for decision-making and opponent modeling. Poker bots are normally tested against real poker players, and demonstrate excellent results over a large number of hands. Therefore, “player beware,” as the saying goes.
We also have to wonder about the strength of poker bots. A major assumption in determining “bot” strength is that all hands are played straight up. That is, all starting hands, flops, turns and rivers are distributed randomly for bots as well as for players. Consequently, some experts still feel that poker is a very difficult game to model using AI technology.
They admit that there are some pretty good poker bots out there, but they still feel that a truly world-class poker-playing bot has yet to be developed and still may be some time off. With the growth of online poker over the past few years, other people feel that considerable money could be made by anyone who actually manages to achieve this.
It is also important to note that AI is not bad. AI is a tool with many different uses, ranging from training people to learn to play poker or play casino games at different levels to uses in many other industries as well. Improving one’s skill level can be very beneficial to the player.
AI can also be used in several other applications by operators or by casinos, such as detecting cheating players, analysis of trends, analysis of poker room ecology, etc. So, once again, be prepared and do your homework.