Layerwise Knowledge Extraction from Deep Convolutional Networks

by   Simon Odense, et al.

Knowledge extraction is used to convert neural networks into symbolic descriptions with the objective of producing more comprehensible learning models. The central challenge is to find an explanation which is more comprehensible than the original model while still representing that model faithfully. The distributed nature of deep networks has led many to believe that the hidden features of a neural network cannot be explained by logical descriptions simple enough to be comprehensible. In this paper, we propose a novel layerwise knowledge extraction method using M-of-N rules which seeks to obtain the best trade-off between the complexity and accuracy of rules describing the hidden features of a deep network. We show empirically that this approach produces rules close to an optimal complexity-error tradeoff. We apply this method to a variety of deep networks and find that in the internal layers we often cannot find rules with a satisfactory complexity and accuracy, suggesting that rule extraction as a general purpose method for explaining the internal logic of a neural network may be impossible. However, we also find that the softmax layer in Convolutional Neural Networks and Autoencoders using either tanh or relu activation functions is highly explainable by rule extraction, with compact rules consisting of as little as 3 units out of 128 often reaching over 99 useful component for explaining parts (or modules) of a deep neural network.


Rule Extraction from Binary Neural Networks with Convolutional Rules for Model Validation

Most deep neural networks are considered to be black boxes, meaning thei...

ExpDNN: Explainable Deep Neural Network

In recent years, deep neural networks have been applied to obtain high p...

Functional Rule Extraction Method for Artificial Neural Networks

The idea I propose in this paper is a method that is based on comprehens...

Eclectic Extraction of Propositional Rules from Neural Networks

Artificial Neural Network is among the most popular algorithm for superv...

Descriptive complexity for neural networks via Boolean networks

We investigate the descriptive complexity of a class of neural networks ...

Knowledge Enhanced Neural Networks for relational domains

In the recent past, there has been a growing interest in Neural-Symbolic...

Diagnostic Rule Extraction Using Neural Networks

The neural networks have trained on incomplete sets that a doctor could ...

Please sign up or login with your details

Forgot password? Click here to reset