Two steps to risk sensitivity

11/12/2021
by   Chris Gagne, et al.
0

Distributional reinforcement learning (RL) – in which agents learn about all the possible long-term consequences of their actions, and not just the expected value – is of great recent interest. One of the most important affordances of a distributional view is facilitating a modern, measured, approach to risk when outcomes are not completely certain. By contrast, psychological and neuroscientific investigations into decision making under risk have utilized a variety of more venerable theoretical models such as prospect theory that lack axiomatically desirable properties such as coherence. Here, we consider a particularly relevant risk measure for modeling human and animal planning, called conditional value-at-risk (CVaR), which quantifies worst-case outcomes (e.g., vehicle accidents or predation). We first adopt a conventional distributional approach to CVaR in a sequential setting and reanalyze the choices of human decision-makers in the well-known two-step task, revealing substantial risk aversion that had been lurking under stickiness and perseveration. We then consider a further critical property of risk sensitivity, namely time consistency, showing alternatives to this form of CVaR that enjoy this desirable characteristic. We use simulations to examine settings in which the various forms differ in ways that have implications for human and animal planning and behavior.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/30/2022

One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning

Offline reinforcement learning (RL) is suitable for safety-critical doma...
research
12/30/2022

Risk-Sensitive Policy with Distributional Reinforcement Learning

Classical reinforcement learning (RL) techniques are generally concerned...
research
06/11/2021

Automatic Risk Adaptation in Distributional Reinforcement Learning

The use of Reinforcement Learning (RL) agents in practical applications ...
research
01/14/2023

Risk-Averse Reinforcement Learning via Dynamic Time-Consistent Risk Measures

Traditional reinforcement learning (RL) aims to maximize the expected to...
research
02/19/2020

A censored mixture model for modeling risk taking

Risk behavior can have substantial consequences for health, well-being, ...
research
01/13/2023

Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learning

In safety-critical decision-making scenarios being able to identify wors...
research
03/23/2022

Towards Scalable Risk Analysis for Stochastic Systems Using Extreme Value Theory

We aim to analyze the behaviour of a finite-time stochastic system, whos...

Please sign up or login with your details

Forgot password? Click here to reset