AI Chat AI Image Generator AI Video Text to Speech

Exponential Family Model-Based Reinforcement Learning via Score Matching

12/28/2021

∙

by Gene Li, et al.

∙

∙

We propose an optimistic model-based algorithm, dubbed SMRL, for finite-horizon episodic reinforcement learning (RL) when the transition model is specified by exponential family distributions with d parameters and the reward is bounded and known. SMRL uses score matching, an unnormalized density estimation technique that enables efficient estimation of the model parameter by ridge regression. Under standard regularity assumptions, SMRL achieves Õ(d√(H^3T)) online regret, where H is the length of each episode and T is the total number of interactions (ignoring polynomial dependence on structural scale parameters).

Gene Li
3 publications
Junbo Li
2 publications
Nathan Srebro
96 publications
Zhaoran Wang
121 publications
Zhuoran Yang
110 publications

page 1

page 2

page 3

page 4

research

∙ 06/01/2020

Model-Based Reinforcement Learning with Value-Targeted Regression

This paper studies model-based reinforcement learning (RL) for regret mi...

11 Alex Ayoub, et al. ∙

research

∙ 07/13/2021

Model Selection with Near Optimal Rates for Reinforcement Learning with General Model Classes

We address the problem of model selection for the finite horizon episodi...

0 Avishek Ghosh, et al. ∙

research

∙ 12/27/2022

Model-Based Reinforcement Learning with Multinomial Logistic Function Approximation

We study model-based reinforcement learning (RL) for episodic Markov dec...

0 TaeHyun Hwang, et al. ∙

research

∙ 05/15/2023

Horizon-free Reinforcement Learning in Adversarial Linear Mixture MDPs

Recent studies have shown that episodic reinforcement learning (RL) is n...

4 Kaixuan Ji, et al. ∙

research

∙ 07/06/2023

Provably Efficient Iterated CVaR Reinforcement Learning with Function Approximation

Risk-sensitive reinforcement learning (RL) aims to optimize policies tha...

0 Yu Chen, et al. ∙

research

∙ 01/23/2013

Relative Loss Bounds for On-line Density Estimation with the Exponential Family of Distributions

We consider on-line density estimation with a parameterized density from...

0 Katy S. Azoury, et al. ∙

research

∙ 06/02/2022

Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning

In this work, we propose a novel Kernelized Stein Discrepancy-based Post...

8 Souradip Chakraborty, et al. ∙