A Solver-Free Framework for Scalable Learning in Neural ILP Architectures

by   Yatin Nandwani, et al.

There is a recent focus on designing architectures that have an Integer Linear Programming (ILP) layer within a neural model (referred to as Neural ILP in this paper). Neural ILP architectures are suitable for pure reasoning tasks that require data-driven constraint learning or for tasks requiring both perception (neural) and reasoning (ILP). A recent SOTA approach for end-to-end training of Neural ILP explicitly defines gradients through the ILP black box (Paulus et al. 2021) - this trains extremely slowly, owing to a call to the underlying ILP solver for every training data point in a minibatch. In response, we present an alternative training strategy that is solver-free, i.e., does not call the ILP solver at all at training time. Neural ILP has a set of trainable hyperplanes (for cost and constraints in ILP), together representing a polyhedron. Our key idea is that the training loss should impose that the final polyhedron separates the positives (all constraints satisfied) from the negatives (at least one violated constraint or a suboptimal cost value), via a soft-margin formulation. While positive example(s) are provided as part of the training data, we devise novel techniques for generating negative samples. Our solution is flexible enough to handle equality as well as inequality constraints. Experiments on several problems, both perceptual as well as symbolic, which require learning the constraints of an ILP, show that our approach has superior performance and scales much better compared to purely neural baselines and other state-of-the-art models that require solver-based training. In particular, we are able to obtain excellent performance in 9 x 9 symbolic and visual sudoku, to which the other Neural ILP solver is not able to scale.


page 1

page 2

page 3

page 4


End-to-End Neuro-Symbolic Architecture for Image-to-Image Reasoning Tasks

Neural models and symbolic algorithms have recently been combined for ta...

Neural-Symbolic Solver for Math Word Problems with Auxiliary Tasks

Previous math word problem solvers following the encoder-decoder paradig...

Relational reasoning and generalization using non-symbolic neural networks

Humans have a remarkable capacity to reason about abstract relational st...

A Solver + Gradient Descent Training Algorithm for Deep Neural Networks

We present a novel hybrid algorithm for training Deep Neural Networks th...

Combining Symbolic and Function Evaluation Expressions In Neural Programs

Neural programming involves training neural networks to learn programs f...

Object-based attention for spatio-temporal reasoning: Outperforming neuro-symbolic models with flexible distributed architectures

Neural networks have achieved success in a wide array of perceptual task...

Neuro-Symbolic Constraint Programming for Structured Prediction

We propose Nester, a method for injecting neural networks into constrain...

Please sign up or login with your details

Forgot password? Click here to reset