Deep Hierarchical Reinforcement Learning Based Recommendations via Multi-goals Abstraction

by   Dongyang Zhao, et al.

The recommender system is an important form of intelligent application, which assists users to alleviate from information redundancy. Among the metrics used to evaluate a recommender system, the metric of conversion has become more and more important. The majority of existing recommender systems perform poorly on the metric of conversion due to its extremely sparse feedback signal. To tackle this challenge, we propose a deep hierarchical reinforcement learning based recommendation framework, which consists of two components, i.e., high-level agent and low-level agent. The high-level agent catches long-term sparse conversion signals, and automatically sets abstract goals for low-level agent, while the low-level agent follows the abstract goals and interacts with real-time environment. To solve the inherent problem in hierarchical reinforcement learning, we propose a novel deep hierarchical reinforcement learning algorithm via multi-goals abstraction (HRL-MG). Our proposed algorithm contains three characteristics: 1) the high-level agent generates multiple goals to guide the low-level agent in different stages, which reduces the difficulty of approaching high-level goals; 2) different goals share the same state encoder parameters, which increases the update frequency of the high-level agent and thus accelerates the convergence of our proposed algorithm; 3) an appreciate benefit assignment function is designed to allocate rewards in each goal so as to coordinate different goals in a consistent direction. We evaluate our proposed algorithm based on a real-world e-commerce dataset and validate its effectiveness.


page 1

page 2

page 3

page 4


DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) has made notable progress in c...

Feudal Reinforcement Learning by Reading Manuals

Reading to act is a prevalent but challenging task which requires the ab...

Reinforcement Learning for LTLf/LDLf Goals

MDPs extended with LTLf/LDLf non-Markovian rewards have recently attract...

Learning World Graphs to Accelerate Hierarchical Reinforcement Learning

In many real-world scenarios, an autonomous agent often encounters vario...

Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation

This paper investigates the multi-agent navigation problem, which requir...

SHIRO: Soft Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) algorithms have been demonstra...

Learning High-level Representations from Demonstrations

Hierarchical learning (HL) is key to solving complex sequential decision...

Please sign up or login with your details

Forgot password? Click here to reset