What is value—accumulated reward or evidence?

dc.contributor.authorFriston, Karl J.en
dc.contributor.authorAdams, Ricken
dc.contributor.authorMontague, P. Readen
dc.date.accessioned2017-09-08T14:18:49Zen
dc.date.available2017-09-08T14:18:49Zen
dc.date.issued2012-11-02en
dc.description.abstractWhy are you reading this abstract? In some sense, your answer will cast the exercise as valuable-but what is value? In what follows, we suggest that value is evidence or, more exactly, log Bayesian evidence. This implies that a sufficient explanation for valuable behavior is the accumulation of evidence for internal models of our world. This contrasts with normative models of optimal control and reinforcement learning, which assume the existence of a value function that explains behavior, where (somewhat tautologically) behavior maximizes value. In this paper, we consider an alternative formulation-active inference-that replaces policies in normative models with prior beliefs about the (future) states agents should occupy. This enables optimal behavior to be cast purely in terms of inference: where agents sample their sensorium to maximize the evidence for their generative model of hidden states in the world, and minimize their uncertainty about those states. Crucially, this formulation resolves the tautology inherent in normative models and allows one to consider how prior beliefs are themselves optimized in a hierarchical setting. We illustrate these points by showing that any optimal policy can be specified with prior beliefs in the context of Bayesian inference. We then show how these prior beliefs are themselves prescribed by an imperative to minimize uncertainty. This formulation explains the saccadic eye movements required to read this text and defines the value of the visual sensations you are soliciting.en
dc.description.sponsorshipWellcome Trusten
dc.format.extent25 pagesen
dc.format.mimetype25 pagesen
dc.identifier.doihttps://doi.org/10.3389/fnbot.2012.00011en
dc.identifier.urihttp://hdl.handle.net/10919/78835en
dc.identifier.volume6en
dc.language.isoen_USen
dc.publisherFrontiersen
dc.rightsCreative Commons Attribution 4.0 Internationalen
dc.rights.urihttp://creativecommons.org/licenses/by/4.0/en
dc.subjectfree energyen
dc.subjectactive inferenceen
dc.subjectvalueen
dc.subjectevidenceen
dc.subjectsurpriseen
dc.subjectself-organizationen
dc.subjectselectionen
dc.subjectBayesianen
dc.titleWhat is value—accumulated reward or evidence?en
dc.title.serialFrontiers in Neurobioticsen
dc.typeArticle - Refereeden
dc.type.dcmitypeTexten

Files

Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
MontagueWhatisValue2012.pdf
Size:
1.53 MB
Format:
Adobe Portable Document Format
Description: