Learning to Generate All Feasible Actions

by   Mirco Theile, et al.

Several machine learning (ML) applications are characterized by searching for an optimal solution to a complex task. The search space for this optimal solution is often very large, so large in fact that this optimal solution is often not computable. Part of the problem is that many candidate solutions found via ML are actually infeasible and have to be discarded. Restricting the search space to only the feasible solution candidates simplifies finding an optimal solution for the tasks. Further, the set of feasible solutions could be re-used in multiple problems characterized by different tasks. In particular, we observe that complex tasks can be decomposed into subtasks and corresponding skills. We propose to learn a reusable and transferable skill by training an actor to generate all feasible actions. The trained actor can then propose feasible actions, among which an optimal one can be chosen according to a specific task. The actor is trained by interpreting the feasibility of each action as a target distribution. The training procedure minimizes a divergence of the actor's output distribution to this target. We derive the general optimization target for arbitrary f-divergences using a combination of kernel density estimates, resampling, and importance sampling. We further utilize an auxiliary critic to reduce the interactions with the environment. A preliminary comparison to related strategies shows that our approach learns to visit all the modes in the feasible action space, demonstrating the framework's potential for learning skills that can be used in various downstream tasks.


page 7

page 8

page 9

page 13

page 15

page 16


Threshold-aware Learning to Generate Feasible Solutions for Mixed Integer Programs

Finding a high-quality feasible solution to a combinatorial optimization...

Offloading and Resource Allocation with General Task Graph in Mobile Edge Computing: A Deep Reinforcement Learning Approach

In this paper, we consider a mobile-edge computing system, where an acce...

A New Constructive Heuristic driven by Machine Learning for the Traveling Salesman Problem

Recent systems applying Machine Learning (ML) to solve the Traveling Sal...

Answering Complex Logical Queries on Knowledge Graphs via Query Computation Tree Optimization

Answering complex logical queries on incomplete knowledge graphs is a ch...

Energy Minimization in UAV-Aided Networks: Actor-Critic Learning for Constrained Scheduling Optimization

In unmanned aerial vehicle (UAV) applications, the UAV's limited energy ...

Density-based Feasibility Learning with Normalizing Flows for Introspective Robotic Assembly

Machine Learning (ML) models in Robotic Assembly Sequence Planning (RASP...

POWERPLAY: Training an Increasingly General Problem Solver by Continually Searching for the Simplest Still Unsolvable Problem

Most of computer science focuses on automatically solving given computat...

Please sign up or login with your details

Forgot password? Click here to reset