Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userReinforcement Learning Produces Dominant Strategies for the Iterated Prisoners Dilemma
57
0
0.0
(
0
)
Added by
Vincent Knight Dr
Publication date
2017
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
We present tournament results and several powerful strategies for the Iterated Prisoners Dilemma created using reinforcement learning techniques (evolutionary and particle swarm algorithms). These strategies are trained to perform well against a corpus of over 170 distinct opponents, including many well-known and classic strategies. All the trained strategies win standard tournaments against the total collection of other opponents. The trained strategies and one particular human made designed strategy are the top performers in noisy tournaments also.
rate research
Read More
The Prisoners Dilemma game has a long history stretching across the social, biological, and physical sciences. In 2012, Press and Dyson developed a method for analyzing the mapping of the 8-dimensional strategy profile onto the 2-dimensional payoff space in an infinitely iterated Prisoners Dilemma game, based on Markov chain analysis and memory-one strategies. We generalize this approach and introduce the concept of strategy parameter to show that linear relations among player payoffs are a ubiquitous feature of the infinitely iterated Prisoners Dilemma game. Our extended analysis is applied to various strategy profiles including tit-for-tat, win-stay-lose-shift, and other randomized strategy sets. Strategy profiles are identified that map onto the vertices, edges, and interior of the Prisoners Dilemma quadrilateral in the 2-dimensional payoff (score) space. A DaMD strategy is defined based solely on Defection after Mutual Defection and leads to linear relations between player scores using strategy parameter analysis. The DaMD strategy is shown to result in an equal (reciprocal) or larger (extortive) score for its user compare to the other player, independent of the strategy of the other player. The extortive scores occur when the probabilities for the DaMD player to cooperate after conflicting plays (cooperate-defect or defect-cooperate) sum to less than 1. The equal reciprocal scores occur when the probabilities for the DaMD player to cooperate after conflicting plays (cooperate-defect or defect-cooperate) sum to 1. When one player selects the extortive DaMD, the opposing player can force the equal punishment payoffs for both players in the infinitely iterated Prisoners dilemma by also choosing the DaMD strategy. Possible pathways to mutual cooperation based on DaMD are discussed.
The Axelrod library is an open source Python package that allows for reproducible game theoretic research into the Iterated Prisoners Dilemma. This area of research began in the 1980s but suffers from a lack of documentation and test code. The goal of the library is to provide such a resource, with facilities for the design of new strategies and interactions between them, as well as conducting tournaments and ecological simulations for populations of strategies. With a growing collection of 139 strategies, the library is a also a platform for an original tournament that, in itself, is of interest to the game theoretic community. This paper describes the Iterated Prisoners Dilemma, the Axelrod library and its development, and insights gained from some novel research.
Since the introduction of zero-determinant strategies, extortionate strategies have received considerable interest. While an interesting class of strategies, the definitions of extortionate strategies are algebraically rigid, apply only to memory-one strategies, and require complete knowledge of a strategy (memory-one cooperation probabilities). We describe a method to detect extortionate behaviour from the history of play of a strategy. When applied to a corpus of 204 strategies this method detects extortionate behaviour in well-known extortionate strategies as well others that do not fit the algebraic definition. The highest performing strategies in this corpus are able to exhibit selectively extortionate behavior, cooperating with strong strategies while exploiting weaker strategies, which no memory-one strategy can do. These strategies emerged from an evolutionary selection process and their existence contradicts widely-repeated folklore in the evolutionary game theory literature: complex strategies can be extraordinarily effective, zero-determinant strategies can be outperformed by non-zero determinant strategies, and longer memory strategies are able to outperform short memory strategies. Moreover, while resistance to extortion is critical for the evolution of cooperation, the extortion of weak opponents need not prevent cooperation between stronger opponents, and this adaptability may be crucial to maintaining cooperation in the long run.
We present insights and empirical results from an extensive numerical study of the evolutionary dynamics of the iterated prisoners dilemma. Fixation probabilities for Moran processes are obtained for all pairs of 164 different strategies including classics such as TitForTat, zero determinant strategies, and many more sophisticated strategies. Players with long memories and sophisticated behaviours outperform many strategies that perform well in a two player setting. Moreover we introduce several strategies trained with evolutionary algorithms to excel at the Moran process. These strategies are excellent invaders and resistors of invasion and in some cases naturally evolve handshaking mechanisms to resist invasion. The best invaders were those trained to maximize total payoff while the best resistors invoke handshake mechanisms. This suggests that while maximizing individual payoff can lead to the evolution of cooperation through invasion, the relatively weak invasion resistance of payoff maximizing strategies are not as evolutionarily stable as strategies employing handshake mechanisms.
The Iterated Prisoners Dilemma with Choice and Refusal (IPD/CR) is an extension of the Iterated Prisoners Dilemma with evolution that allows players to choose and to refuse their game partners. From individual behaviors, behavioral population structures emerge. In this report, we examine one particular IPD/CR environment and document the social network methods used to identify population behaviors found within this complex adaptive system. In contrast to the standard homogeneous population of nice cooperators, we have also found metastable populations of mixed strategies within this environment. In particular, the social networks of interesting populations and their evolution are examined.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
