A New Computationally Simple Approach for Implementing Neural Networks with Output Hard Constraints

by   Andrei V. Konstantinov, et al.

A new computationally simple method of imposing hard convex constraints on the neural network output values is proposed. The key idea behind the method is to map a vector of hidden parameters of the network to a point that is guaranteed to be inside the feasible set defined by a set of constraints. The mapping is implemented by the additional neural network layer with constraints for output. The proposed method is simply extended to the case when constraints are imposed not only on the output vectors, but also on joint constraints depending on inputs. The projection approach to imposing constraints on outputs can simply be implemented in the framework of the proposed method. It is shown how to incorporate different types of constraints into the proposed method, including linear and quadratic constraints, equality constraints, and dynamic constraints, constraints in the form of boundaries. An important feature of the method is its computational simplicity. Complexities of the forward pass of the proposed neural network layer by linear and quadratic constraints are O(n*m) and O(n^2*m), respectively, where n is the number of variables, m is the number of constraints. Numerical experiments illustrate the method by solving optimization and classification problems. The code implementing the method is publicly available.


RAYEN: Imposition of Hard Convex Constraints on Neural Networks

This paper presents RAYEN, a framework to impose hard convex constraints...

No one-hidden-layer neural network can represent multivariable functions

In a function approximation with a neural network, an input dataset is m...

Pulling back error to the hidden-node parameter technology: Single-hidden-layer feedforward network without output weight

According to conventional neural network theories, the feature of single...

Extremal learning: extremizing the output of a neural network in regression problems

Neural networks allow us to model complex relationships between variable...

Interpretable Ensembles of Hyper-Rectangles as Base Models

A new extremely simple ensemble-based model with the uniformly generated...

Imposing Hard Constraints on Deep Networks: Promises and Limitations

Imposing constraints on the output of a Deep Neural Net is one way to im...

Approximate Uncertain Program

Chance constrained program where one seeks to minimize an objective over...

Please sign up or login with your details

Forgot password? Click here to reset