使用大规模实验和机器学习推动人类决策行为的理论发现

[chengjun]

使用大规模实验和机器学习推动人类决策行为的理论发现

来自: 苏格拉底大王(I am serious with Socrates.) 组长 2021-07-05 12:17:16

Peterson, J. C., Bourgin, D. D., Agrawal, M., Reichman, D. & Griffiths, T. L. Using large-scale experiments and machine learning to discover new theories of human decision-making. Science 372, 1209–1214 (2021).

https://science.sciencemag.org/content/372/6547/1209

Imagine a choice between two gambles: getting $100 with a probability of 20% or getting $50 with a probability of 80%.

In 1979, Kahneman and Tversky published prospect theory (1), a mathematically specified descriptive theory of how people make risky choices such as these. They explained numerous documented violations of expected utility theory, the dominant theory at the time, by using nonlinear psychophysical functions for perceiving underlying probabilities and evaluating resulting payoffs.

Prospect theory revolutionized the study of choice behavior, showing that researchers could build formal models of decision-making based on realistic psychological principles (2). But in the ensuing decades, as dozens of competing theories have been proposed (3), there has been theoretical fragmentation, redundancy, and stagnation. There is little consensus on the best decision theory or model. On page 1209 of this issue, Peterson et al. (4) demonstrate the power of a more recent approach:

Instead of relying on the intuitions and (potentially limited) intellect of human researchers, the task of theory generation can be outsourced to powerful machine-learning algorithms.

Fig. 1. Applying large-scale experimentation and theory-driven machine learning to risky choice.

(A) Experiment interface in which participants made choices between pairs of gambles (“choice problems”) and were paid at the end of the experiment based on their choice in a single randomly selected gamble. (B) Each pair of gambles can be described by a vector of payoffs and probabilities. Reducing the resulting space to two dimensions (2D) allows us to visualize coverage by different experiments. Each point is a different choice problem, and colors show reconstructions of the problems used in influential experiments (green), the previous largest dataset (red), and our 9831 problems, which provide much broader coverage of the problem space. This 2D embedding results from applying t-distributed stochastic neighbor embedding (t-SNE) to the hiddenlayer representation of our best-performing neural network model. (C) We define a hierarchy of theoretical assumptions expressed as partitions over function space that can be searched. More complex classes of functions contain simpler classes as special cases, allowing us to systematically search the space of theories and identify the impact of constraints that correspond to psychologically meaningful theoretical commitments. All model classes are described in the main text. (D) Differentiable decision theories use the formal structure of classic theories to constrain the architecture of the neural network. For example, our EU model uses a neural network to define the utility function but combines those utilities in a classic form, resulting in a fully differentiable model that can be optimized by gradient descent.

Fig. 2. Comparing classic theories proposed by human researchers with differentiable decision theories discovered through machine learning.

(A) EU. In the left panel, EU with a learned optimal utility function outperforms classic models (gray lines) given enough data. Performance is assessed in terms of prediction error (MSE) on ~1000 unseen choice problems as a function of the amount of training data used for model fitting (~9000 choice problems, average of 50 runs). The right panel shows that the utility function identified through this optimization method reproduces many of the characteristics suggested by human theorists. (B) PT. In the left panel, PT with learned utility and probability weighting functions outperforms classic models (gray lines) given enough data. CPT, the modern variant of PT, performs better with small amounts of data, on the scale of previous experiments, but slightly worse with more data. The right panel shows the optimized utility and probability weighting functions for PT.

Fig. 3. Complex decision theories exhibit better predictive performance than simpler ones.

(A) Performance of differentiable decision theories. As model flexibility increases, performance increases, along with data requirements. (B) Performance comparison between differentiable decision theories and 21 other well-known theories across the risky choice literature, none of which outperforms PT (see the supplementary materials for more details)

Fig. 4. The MOT model assigns a fixed learned decision probability for gambles that are dominated and determines the value of gambles in all other choice problems using mixtures of two utility and two probabilityweighting functions.

(A) MOT obtains slightly better performance than the best learned decision theory, even when the mixture networks and thus context effects are limited to information about outcomes only. (B) Components of the model are interpretable by design and resemble the classic forms for EU and PT: Utility function 1 (UF 1) is symmetric, whereas UF 2 shows loss aversion. Probability weighting function 1 (PWF 1) is linear, whereas PWF 2 shows overweighting of lower probabilities. 2D t-SNE embeddings of the choice problems based on the hidden unit representations of the best-performing context-dependent neural network exhibit distinct clusters that correspond to problems where M

ACKNOWLEDGMENTS We thank F. Callaway, N. Daw, P. Ortoleva, R. Dubey, and reviewers for providing comments on the manuscript. Preliminary analyses of our dataset were presented at the International Conference on Machine Learning. The analyses presented here are entirely new and go substantially beyond that early work. Funding: This work was supported by the Future of Life Institute, the Open Philanthropy Foundation, the NOMIS Foundation, DARPA (cooperative agreement D17AC00004), and the National Science Foundation (grant no. 1718550). Author contributions: Conceptualization: J.C.P., T.L.G., D.D.B., D.R.; Data curation: D.D.B.; Formal analysis: J.C.P., D.D.B., M.A., T.L.G.; Funding acquisition: T.L.G.; Investigation: J.C.P., D.D.B., M.A.; Methodology: J.C.P., T.L.G.; Project administration: D.R., T.L.G.; Software: J.C.P., D.D.B.; Supervision: T.L.G., D.R.; Visualization: J.C.P., M.A.; Writing – original draft: J.C.P., M.A.; Writing – review and editing: J.C.P., T.L.G., D.D.B., M.A., D.R. Competing interests: The authors declare no competing interests.

Data and materials availability:

All data are available to the public without registration at https://github.com/jcpeterson/choices13k