Synthetic Petri Dish: A Novel Surrogate Model for Rapid Architecture Search

by   Aditya Rawal, et al.

Neural Architecture Search (NAS) explores a large space of architectural motifs – a compute-intensive process that often involves ground-truth evaluation of each motif by instantiating it within a large network, and training and evaluating the network with thousands of domain-specific data samples. Inspired by how biological motifs such as cells are sometimes extracted from their natural environment and studied in an artificial Petri dish setting, this paper proposes the Synthetic Petri Dish model for evaluating architectural motifs. In the Synthetic Petri Dish, architectural motifs are instantiated in very small networks and evaluated using very few learned synthetic data samples (to effectively approximate performance in the full problem). The relative performance of motifs in the Synthetic Petri Dish can substitute for their ground-truth performance, thus accelerating the most expensive step of NAS. Unlike other neural network-based prediction models that parse the structure of the motif to estimate its performance, the Synthetic Petri Dish predicts motif performance by training the actual motif in an artificial setting, thus deriving predictions from its true intrinsic properties. Experiments in this paper demonstrate that the Synthetic Petri Dish can therefore predict the performance of new motifs with significantly higher accuracy, especially when insufficient ground truth data is available. Our hope is that this work can inspire a new research direction in studying the performance of extracted components of models in an alternative controlled setting.


page 1

page 2

page 3

page 4


Neural Architecture Search with Random Labels

In this paper, we investigate a new variant of neural architecture searc...

PEng4NN: An Accurate Performance Estimation Engine for Efficient Automated Neural Network Architecture Search

Neural network (NN) models are increasingly used in scientific simulatio...

Self-Learning for Received Signal Strength Map Reconstruction with Neural Architecture Search

In this paper, we present a Neural Network (NN) model based on Neural Ar...

Data-Free Neural Architecture Search via Recursive Label Calibration

This paper aims to explore the feasibility of neural architecture search...

Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data

This paper investigates the intriguing question of whether we can create...

Latency-Controlled Neural Architecture Search for Streaming Speech Recognition

Recently, neural architecture search (NAS) has attracted much attention ...

Inference of cell dynamics on perturbation data using adjoint sensitivity

Data-driven dynamic models of cell biology can be used to predict cell r...

Please sign up or login with your details

Forgot password? Click here to reset