Formal Verification of Robustness and Resilience of Learning-Enabled State Estimation Systems for Robotics

by   Wei Huang, et al.

This paper presents a formal verification guided approach for a principled design and implementation of robust and resilient learning-enabled systems. We focus on learning-enabled state estimation systems (LE-SESs), which have been widely used in robotics applications to determine the current state (e.g., location, speed, direction, etc.) of a complex system. The LE-SESs are networked systems composed of a set of connected components including Bayes filters for localisation, and neural networks for processing sensory input. We study LE-SESs from the perspective of formal verification, which determines the satisfiability of a system model against the specified properties. Over LE-SESs, we investigate two key properties - robustness and resilience - and provide their formal definitions. To enable formal verification, we reduce the LE-SESs to a novel class of labelled transition systems, named PO2-LTS in the paper, and formally express the properties as constrained optimisation objectives. We prove that the robustness verification is NP-complete. Based on PO2-LTS and the optimisation objectives, practical verification algorithms are developed to check the satisfiability of the properties on the LE-SESs. As a major case study, we interrogate a real-world dynamic tracking system which uses a single Kalman Filter (KF) - a special case of Bayes filter - to localise and track a ground vehicle. Its perception system, based on convolutional neural networks, processes a high-resolution Wide Area Motion Imagery (WAMI) data stream. Experimental results show that our algorithms can not only verify the properties of the WAMI tracking system but also provide representative examples, the latter of which inspired us to take an enhanced LE-SESs design where runtime monitors or joint-KFs are required. Experimental results confirm the improvement of the robustness of the enhanced design.


page 1

page 12

page 13

page 14

page 15


Reliability Validation of Learning Enabled Vehicle Tracking

This paper studies the reliability of a real-world learning-enabled syst...

Neuro-Symbolic Verification of Deep Neural Networks

Formal verification has emerged as a powerful approach to ensure the saf...

Decentralized Runtime Verification for LTL Properties Using Global

Runtime verification is the process of verifying critical behavioral pro...

Decentralized Runtime Verification for LTL Properties Using Global Clock

Runtime verification is the process of verifying critical behavioral pro...

Transition-Oriented Programming: Developing Verifiable Systems

It is extremely challenging to develop verifiable systems that are regul...

Formal Verification of CNN-based Perception Systems

We address the problem of verifying neural-based perception systems impl...

The Design and Regulation of Exchanges: A Formal Approach

We use formal methods to specify, design, and monitor continuous double ...

Please sign up or login with your details

Forgot password? Click here to reset