Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems

by   Michael Yuhas, et al.

When machine learning (ML) models are supplied with data outside their training distribution, they are more likely to make inaccurate predictions; in a cyber-physical system (CPS), this could lead to catastrophic system failure. To mitigate this risk, an out-of-distribution (OOD) detector can run in parallel with an ML model and flag inputs that could lead to undesirable outcomes. Although OOD detectors have been well studied in terms of accuracy, there has been less focus on deployment to resource constrained CPSs. In this study, a design methodology is proposed to tune deep OOD detectors to meet the accuracy and response time requirements of embedded applications. The methodology uses genetic algorithms to optimize the detector's preprocessing pipeline and selects a quantization method that balances robustness and response time. It also identifies several candidate task graphs under the Robot Operating System (ROS) for deployment of the selected design. The methodology is demonstrated on two variational autoencoder based OOD detectors from the literature on two embedded platforms. Insights into the trade-offs that occur during the design process are provided, and it is shown that this design methodology can lead to a drastic reduction in response time in relation to an unoptimized OOD detector while maintaining comparable accuracy.


Embedded out-of-distribution detection on an autonomous robot platform

Machine learning (ML) is actively finding its way into modern cyber-phys...

Co-Design of Out-of-Distribution Detectors for Autonomous Emergency Braking Systems

Learning enabled components (LECs), while critical for decision making i...

Demo Abstract: Real-Time Out-of-Distribution Detection on a Mobile Robot

In a cyber-physical system such as an autonomous vehicle (AV), machine l...

ConAML: Constrained Adversarial Machine Learning for Cyber-Physical Systems

Recent research demonstrated that the superficially well-trained machine...

Improving Variational Autoencoder based Out-of-Distribution Detection for Embedded Real-time Applications

Uncertainties in machine learning are a significant roadblock for its ap...

WiP Abstract : Robust Out-of-distribution Motion Detection and Localization in Autonomous CPS

Highly complex deep learning models are increasingly integrated into mod...

Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systems

Deep Neural Networks are actively being used in the design of autonomous...

Please sign up or login with your details

Forgot password? Click here to reset