Text

MIGO-NAS : Towards Fast and Generalizable Neural Architecture Search

Neural architecture search (NAS) has achieved unprecedented performance in various computer vision tasks. However, most existing NAS methods are defected in search efficiency and model generalizability. In this paper, we propose a novel NAS framework, termed MIGO-NAS, with the aim to guarantee the efficiency and generalizability in arbitrary search spaces. On the one hand, we formulate the search space as a multivariate probabilistic distribution, which is then optimized by a novel multivariate information-geometric optimization (MIGO). By approximating the distribution with a sampling, training, and testing pipeline, MIGO guarantees the memory efficiency, training efficiency, and search flexibility. Besides, MIGO is the first time to decrease the estimation error of natural gradient in multivariate distribution. On the other hand, for a set of specific constraints, the neural architectures are generated by a novel dynamic programming network generation (DPNG), which significantly reduces the training cost under various hardware environments. Experiments validate the advantages of our approach over existing methods by establishing a superior accuracy and efficiency i.e., 2.39 test error on CIFAR-10 benchmark and 21.7 on ImageNet benchmark, with only 1.5 GPU hours and 96 GPU hours for searching, respectively. Besides, the searched architectures can be well generalize to computer vision tasks including object detection and semantic segmentation, i.e., 25×25× FLOPs compression, with 6.4 mAP gain over Pascal VOC dataset, and 29.9×29.9× FLOPs compression, with only 1.41 percent performance drop over Cityscapes dataset. The code is publicly available.

Ketersediaan

Informasi Detail

Judul Seri

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE; Vol.43 No.9 September 2021

No. Panggil

-

Penerbit

: .,

Deskripsi Fisik

p. 2936 - 2952

Bahasa

English

ISBN/ISSN

-

Klasifikasi

-

Tipe Isi

-

Tipe Media

-

Tipe Pembawa

-

Edisi

-

Subjek

DYNAMIC PROGRAMMING
NEURAL ARCHITECTURE SEARCH
MULTIVARIATE INFORMATION-GEOMETRIC OPTIMIZATION

Info Detail Spesifik

DOI: 10.1109/TPAMI.2021.3065138

Pernyataan Tanggungjawab

Xiawu Zheng, Rongrong Ji, Yuhang Chen, Qiang Wang, Baochang Zhang, Jie Chen, Qixiang Ye, Feiyue Huang, Yonghong Tian

Versi lain/terkait

Lampiran Berkas

Komentar