Value-based decision making is a cognitive process in which an animal selects a specific behavior from a set of alternatives. The selection is based on the anticipated reward associated with each behavioral alternative (i.e., subjective values). Subjective values are established, in part, by reinforcement learning (RL). Substantial progress is being made in identifying neural systems, microcircuits and cellular mechanisms of decision making. However, dynamical and structural complexity make it difficult to achieve a comprehensive understanding of mechanisms that underlie value-based decision making. Computational models can help address this issue. Computational models provide a quantitative framework for simultaneously studying multiple levels of organization, testing the validity of assumptions, and assessing the roles of component processes. Moreover, modeling studies can help identify general principles that apply to a variety of animal species and to diverse behavioral circumstances, and that can be adapted to artificial systems. We are using modeling studies to investigate the ways in which an identified neural circuit in Aplysia selects between two alternative feeding behaviors: ingestion vs. rejection [4]. The neurosimulator SNNAP [2] was used to develop a neurobiologically plausible model network with cells B4, B8, B31, B34, B35, B51, B52, and B64 [3]. These cells are elements in a central pattern generator (CPG) that mediates feeding. The model simulated features of fictive feeding. Currently, the model is being extended by: 1) including an autapse in B31; 2) adding CPG cells B20, B30, B65, B65 and CBIs 2-4; and 3) incorporating identified correlates of operant conditioning. Simulations indicated: 1) the autapse and positive feedback among B31, B34, B35, B63 and B65 mediated the decision to initiate fictive feeding; 2) incorporating the known neuronal correlates of operant conditioning [1,5] (i) reduced the threshold for eliciting fictive feeding, (ii) biased the neural activity toward fictive ingestion; the bias toward ingestion resulted from changes in B51, whereas the reduced threshold resulted from changes in cells B63 and B65 and the electrical coupling among cells B30, B63, and B65. Finally, the results suggested that as yet unidentified modifications are necessary to produce more complete ingestion-like neural activity. Our computational studies suggested that value-based decision making involve multiple sites of plasticity. These sites mediate the initial commitment to respond and assignment of subjective values. The interplay among these sites biased the fictive feeding behavior toward a single, highly valued response, which previously had been associated with positive reinforcement. Finally, these studies are providing an opportunity to simultaneously investigate decision making at theoretical, algorithmic, and implementation levels and are providing insights into cognitive processes.
Computational study of neuronal mechanisms underlying value-based decision making
CATALDO, ENRICO;
2012-01-01
Abstract
Value-based decision making is a cognitive process in which an animal selects a specific behavior from a set of alternatives. The selection is based on the anticipated reward associated with each behavioral alternative (i.e., subjective values). Subjective values are established, in part, by reinforcement learning (RL). Substantial progress is being made in identifying neural systems, microcircuits and cellular mechanisms of decision making. However, dynamical and structural complexity make it difficult to achieve a comprehensive understanding of mechanisms that underlie value-based decision making. Computational models can help address this issue. Computational models provide a quantitative framework for simultaneously studying multiple levels of organization, testing the validity of assumptions, and assessing the roles of component processes. Moreover, modeling studies can help identify general principles that apply to a variety of animal species and to diverse behavioral circumstances, and that can be adapted to artificial systems. We are using modeling studies to investigate the ways in which an identified neural circuit in Aplysia selects between two alternative feeding behaviors: ingestion vs. rejection [4]. The neurosimulator SNNAP [2] was used to develop a neurobiologically plausible model network with cells B4, B8, B31, B34, B35, B51, B52, and B64 [3]. These cells are elements in a central pattern generator (CPG) that mediates feeding. The model simulated features of fictive feeding. Currently, the model is being extended by: 1) including an autapse in B31; 2) adding CPG cells B20, B30, B65, B65 and CBIs 2-4; and 3) incorporating identified correlates of operant conditioning. Simulations indicated: 1) the autapse and positive feedback among B31, B34, B35, B63 and B65 mediated the decision to initiate fictive feeding; 2) incorporating the known neuronal correlates of operant conditioning [1,5] (i) reduced the threshold for eliciting fictive feeding, (ii) biased the neural activity toward fictive ingestion; the bias toward ingestion resulted from changes in B51, whereas the reduced threshold resulted from changes in cells B63 and B65 and the electrical coupling among cells B30, B63, and B65. Finally, the results suggested that as yet unidentified modifications are necessary to produce more complete ingestion-like neural activity. Our computational studies suggested that value-based decision making involve multiple sites of plasticity. These sites mediate the initial commitment to respond and assignment of subjective values. The interplay among these sites biased the fictive feeding behavior toward a single, highly valued response, which previously had been associated with positive reinforcement. Finally, these studies are providing an opportunity to simultaneously investigate decision making at theoretical, algorithmic, and implementation levels and are providing insights into cognitive processes.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
