A Set Membership Approach to Discovering Feature Relevance and Explaining Neural Classifier Decisions

by   Stavros P. Adam, et al.

Neural classifiers are non linear systems providing decisions on the classes of patterns, for a given problem they have learned. The output computed by a classifier for each pattern constitutes an approximation of the output of some unknown function, mapping pattern data to their respective classes. The lack of knowledge of such a function along with the complexity of neural classifiers, especially when these are deep learning architectures, do not permit to obtain information on how specific predictions have been made. Hence, these powerful learning systems are considered as black boxes and in critical applications their use tends to be considered inappropriate. Gaining insight on such a black box operation constitutes a one way approach in interpreting operation of neural classifiers and assessing the validity of their decisions. In this paper we tackle this problem introducing a novel methodology for discovering which features are considered relevant by a trained neural classifier and how they affect the classifier's output, thus obtaining an explanation on its decision. Although, feature relevance has received much attention in the machine learning literature here we reconsider it in terms of nonlinear parameter estimation targeted by a set membership approach which is based on interval analysis. Hence, the proposed methodology builds on sound mathematical approaches and the results obtained constitute a reliable estimation of the classifier's decision premises.


page 8

page 11

page 12

page 14

page 15

page 16

page 17

page 19


A Logic of "Black Box" Classifier Systems

Binary classifiers are traditionally studied by propositional logic (PL)...

On Deciding Feature Membership in Explanations of SDD Related Classifiers

When reasoning about explanations of Machine Learning (ML) classifiers, ...

Best-Effort Adversarial Approximation of Black-Box Malware Classifiers

An adversary who aims to steal a black-box model repeatedly queries the ...

Discovering the Rationale of Decisions: Experiments on Aligning Learning and Reasoning

In AI and law, systems that are designed for decision support should be ...

A Model Explanation System: Latest Updates and Extensions

We propose a general model explanation system (MES) for "explaining" the...

SCRIB: Set-classifier with Class-specific Risk Bounds for Blackbox Models

Despite deep learning (DL) success in classification problems, DL classi...

Please sign up or login with your details

Forgot password? Click here to reset