Translating Natural Language to Planning Goals with Large-Language Models

by   Yaqi Xie, et al.

Recent large language models (LLMs) have demonstrated remarkable performance on a variety of natural language processing (NLP) tasks, leading to intense excitement about their applicability across various domains. Unfortunately, recent work has also shown that LLMs are unable to perform accurate reasoning nor solve planning problems, which may limit their usefulness for robotics-related tasks. In this work, our central question is whether LLMs are able to translate goals specified in natural language to a structured planning language. If so, LLM can act as a natural interface between the planner and human users; the translated goal can be handed to domain-independent AI planners that are very effective at planning. Our empirical results on GPT 3.5 variants show that LLMs are much better suited towards translation rather than planning. We find that LLMs are able to leverage commonsense knowledge and reasoning to furnish missing details from under-specified goals (as is often the case in natural language). However, our experiments also reveal that LLMs can fail to generate goals in tasks that involve numerical or physical (e.g., spatial) reasoning, and that LLMs are sensitive to the prompts used. As such, these models are promising for translation to structured planning languages, but care should be taken in their use.


page 1

page 2

page 3

page 4


Language Models of Code are Few-Shot Commonsense Learners

We address the general task of structured commonsense reasoning: given a...

Distilling Script Knowledge from Large Language Models for Constrained Language Planning

In everyday life, humans often plan their actions by following step-by-s...

Integrating AI Planning with Natural Language Processing: A Combination of Explicit and Tacit Knowledge

Automated planning focuses on strategies, building domain models and syn...

Language Models Meet World Models: Embodied Experiences Enhance Language Models

While large language models (LMs) have shown remarkable capabilities acr...

Chain-of-Symbol Prompting Elicits Planning in Large Langauge Models

In this paper, we take the initiative to investigate the performance of ...

High Fidelity Vector Space Models of Structured Data

Machine learning systems regularly deal with structured data in real-wor...

Please sign up or login with your details

Forgot password? Click here to reset