Reinforced Evolutionary Neural Architecture Search
share
Neural architecture search (NAS) is an important task in network design, but it remains challenging due to high computational consumption in most methods and low stability in evolution algorithm (EA) based NAS. In this paper, we propose the Reinforced Evolutionary Neural Architecture Search (RENAS), an evolutionary method with reinforced mutation for NAS to address these two issues. Specifically, we integrate reinforced mutation into an EA based NAS method by adopting a mutation controller to learn the effects of slight modifications and make mutation actions. For this reason, the proposed method is more like the process of model design by human experts than typical RL-based NAS methods that construct networks sequentially. Furthermore, as the models are trained by fine-tuning rather than from scratch in model evaluation, the cell-wise search process becomes much more efficient and only takes less than 1.5 days using 4 GPUs (Titan xp). Experimental results demonstrate the effectiveness and efficiency of our method. Moreover, the architecture searched on CIFAR-10 sets a new state-of-the-art on ImageNet in the mobile setting (top-1/5 accuracy = 75.7
READ FULL TEXT