Artificial Intelligence
In yet another major milestone in the development of artificial intelligence, a poker playing algorithm has bested a large number of human pros. 'DeepStack' consistently beat professional players at the extremely popular and challenging game with the help of a synthetic 'gut feeling' learned through playing thousands of rounds of the game.
An artificial intelligence poker bot developed by researchers in Canada and the Czech Republic has defeated several professional players in one-on-one games of no-limit Texas hold’em poker. The system's creators say their program beat its human opponents by using an approximation approach that they compare to “gut feeling.”
Heads-up no-limit Texas hold’em is a version of poker played between two people who can bet as many of the chips as they possess. This game has been shown to be too difficult for machines to play expertly. In each hand there are a staggering 10160 possible paths of play.
DeepStack, the poker-playing software that has already bested some professional players, was developed by a team led by Michael Bowling, a professor of computer science at the University of Alberta, and included researchers from Charles University and Czech Technical University in the Czech Republic.
"In a study involving dozens of participants and 44,000 hands of poker, DeepStack becomes the first computer program to beat professional poker players in heads-up no-limit Texas hold’em," write the researchers. "Furthermore, we show this approach dramatically reduces worst-case exploitability
compared to the abstraction paradigm that has been favored for over a decade."
Poker contains levels of uncertainty, that are not as involved in other games. For instance, the most well known part of the game is bluffing. Poker players cannot see their opponents’ hands, meaning that, in contrast to checkers, chess, or Go, not all of the information contained within the game is available to them.
"For the first time the gap between the largest perfect and imperfect information games to have been mastered is mostly closed."
DeepStack plays like a human might. With so many possibilities to consider, the system relies on a form of programmatic intuition. According to the researchers “[DeepStack] does not compute and store a complete strategy prior to play....It avoids reasoning about the entire remainder of the game by substituting the computation beyond a certain depth with a fast approximate estimate. This estimate can be thought of as DeepStack’s intuition: a gut feeling of the value of holding any possible private cards in any possible poker situation.”Related articles
DeepStack learned to play poker by playing hands against itself. After each game, it reviews and refines the strategy used, leading to a more optimized approach. With the complexity of no-limit poker, this approach normally involves practicing with a more limited version of the game. The DeepStack team coped with this complexity by applying a fast approximation technique that they refined by feeding previous poker situations into a deep-learning algorithm.Measuring the performance of a poker player by looking at the amount won, relative to the amount bet at his or her table, over many games is how the researchers compared DeepStack to human players. The analysis showed DeepStack had a win rate roughly nine times better than what would be considered good for a professional player. Many argue that the system still has not competed against the top poker pros.
Soon at a tournament at a Pittsburgh casino, several world-class poker players will play the same version of poker against a program developed at Carnegie Mellon University. Tuomas Sandholm, a professor of computer science at CMU who is leading the effort, says the human players involved are considerably stronger than those in the competition with the Alberta researchers. At the contest, over 100,000 hands of Texas Hold 'em
will be played over 20 days. The tournament could confirm that AI has indeed mastered a game that has long seemed far too complex and subtle for computers.According to Bowling and his team, the implications of DeepStack go beyond just being a significant milestone for artificial intelligence. "DeepStack is a paradigmatic shift in approximating solutions to large, sequential imperfect information games," they conclude. "
Abstraction and offline computation of complete strategies has been the dominant approach for almost 20 years. DeepStack allows computation to be focused on specific
situations that arise when making decisions and the use of automatically trained value functions. These are two of the core principles that have powered successes in perfect information games, albeit conceptually simpler to implement in those settings. As a result, for the first time the gap between the largest perfect and imperfect information games to have been mastered is mostly closed.
Does DeepStack foreshadow the oncoming development of artificial general intelligence? The algorithm has implications for seeing powerful AI applied more in settings that do not fit the perfect information assumption say the researchers. "The old paradigm for handling imperfect information has shown promise in applications like defending strategic resources, and robust decision making as needed for medical treatment recommendations." These are just a few of the potential impacts of this technology.
0 comments:
Post a Comment