Redefining Neural Architecture Search of Heterogeneous Multi-Network Models by Characterizing Variation Operators and Model Components

by   Unai Garciarena, et al.

With neural architecture search methods gaining ground on manually designed deep neural networks -even more rapidly as model sophistication escalates-, the research trend shifts towards arranging different and often increasingly complex neural architecture search spaces. In this conjuncture, delineating algorithms which can efficiently explore these search spaces can result in a significant improvement over currently used methods, which, in general, randomly select the structural variation operator, hoping for a performance gain. In this paper, we investigate the effect of different variation operators in a complex domain, that of multi-network heterogeneous neural models. These models have an extensive and complex search space of structures as they require multiple sub-networks within the general model in order to answer to different output types. From that investigation, we extract a set of general guidelines, whose application is not limited to that particular type of model, and are useful to determine the direction in which an architecture optimization method could find the largest improvement. To deduce the set of guidelines, we characterize both the variation operators, according to their effect on the complexity and performance of the model; and the models, relying on diverse metrics which estimate the quality of the different parts composing it.


page 1

page 5

page 12


Task-Aware Neural Architecture Search

The design of handcrafted neural networks requires a lot of time and res...

LISSNAS: Locality-based Iterative Search Space Shrinkage for Neural Architecture Search

Search spaces hallmark the advancement of Neural Architecture Search (NA...

ShuffleNASNets: Efficient CNN models through modified Efficient Neural Architecture Search

Neural network architectures found by sophistic search algorithms achiev...

Contrastive Embeddings for Neural Architectures

The performance of algorithms for neural architecture search strongly de...

Analyzing and Mitigating Interference in Neural Architecture Search

Weight sharing has become the de facto approach to reduce the training c...

Towards modular and programmable architecture search

Neural architecture search methods are able to find high performance dee...

Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems

Neural architecture search is a promising area of research dedicated to ...

Please sign up or login with your details

Forgot password? Click here to reset