A Safety Framework for Critical Systems Utilising Deep Neural Networks

by   Xingyu Zhao, et al.
University of Liverpool
Heriot-Watt University

Increasingly sophisticated mathematical modelling processes from Machine Learning are being used to analyse complex data. However, the performance and explainability of these models within practical critical systems requires a rigorous and continuous verification of their safe utilisation. Working towards addressing this challenge, this paper presents a principled novel safety argument framework for critical systems that utilise deep neural networks. The approach allows various forms of predictions, e.g., future reliability of passing some demands, or confidence on a required reliability level. It is supported by a Bayesian analysis using operational data and the recent verification and validation techniques for deep learning. The prediction is conservative – it starts with partial prior knowledge obtained from lifecycle activities and then determines the worst-case prediction. Open challenges are also identified.


page 1

page 2

page 3

page 4


The theory of homogeneity of nonlinear structural systems – A general basis for structural safety assessment

The paper develops a novel and general methodology to characterize the n...

Assessing Safety-Critical Systems from Operational Testing: A Study on Autonomous Vehicles

Context: Demonstrating high reliability and safety for safety-critical s...

Performance Analysis of Out-of-Distribution Detection on Various Trained Neural Networks

Several areas have been improved with Deep Learning during the past year...

Using Quantifier Elimination to Enhance the Safety Assurance of Deep Neural Networks

Advances in the field of Machine Learning and Deep Neural Networks (DNNs...

Towards Probability-based Safety Verification of Systems with Components from Machine Learning

Machine learning (ML) has recently created many new success stories. Hen...

Verification of deep probabilistic models

Probabilistic models are a critical part of the modern deep learning too...

Please sign up or login with your details

Forgot password? Click here to reset