Planner-Reasoner Inside a Multi-task Reasoning Agent

02/01/2022
by   Daoming Lyu, et al.
0

We consider the problem of multi-task reasoning (MTR), where an agent can solve multiple tasks via (first-order) logic reasoning. This capability is essential for human-like intelligence due to its strong generalizability and simplicity for handling multiple tasks. However, a major challenge in developing effective MTR is the intrinsic conflict between reasoning capability and efficiency. An MTR-capable agent must master a large set of "skills" to tackle diverse tasks, but executing a particular task at the inference stage requires only a small subset of immediately relevant skills. How can we maintain broad reasoning capability and also efficient specific-task performance? To address this problem, we propose a Planner-Reasoner framework capable of state-of-the-art MTR capability and high efficiency. The Reasoner models shareable (first-order) logic deduction rules, from which the Planner selects a subset to compose into efficient reasoning paths. The entire model is trained in an end-to-end manner using deep reinforcement learning, and experimental studies over a variety of domains validate its effectiveness.

READ FULL TEXT
research
09/12/2018

Multi-task Deep Reinforcement Learning with PopArt

The reinforcement learning community has made great strides in designing...
research
05/11/2022

Developing cooperative policies for multi-stage reinforcement learning tasks

Many hierarchical reinforcement learning algorithms utilise a series of ...
research
10/20/2022

Disentangling Reasoning Capabilities from Language Models with Compositional Reasoning Transformers

This paper presents ReasonFormer, a unified reasoning framework for mirr...
research
05/29/2023

Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning

Diffusion models have demonstrated highly-expressive generative capabili...
research
11/30/2018

BlockPuzzle - A Challenge in Physical Reasoning and Generalization for Robot Learning

In this work we propose a novel task framework under which a variety of ...
research
09/28/2021

DeepPSL: End-to-end perception and reasoning with applications to zero shot learning

We introduce DeepPSL a variant of Probabilistic Soft Logic (PSL) to prod...
research
10/18/2021

In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specifications

We address the problem of building agents whose goal is to satisfy out-o...

Please sign up or login with your details

Forgot password? Click here to reset