Text

Investigating Bi-Level Optimization for Learning and Vision From a Unified Perspective: A Survey and Beyond

Bi-Level Optimization (BLO) is originated from the area of economic game theory and then introduced into the optimization community. BLO is able to handle problems with a hierarchical structure, involving two levels of optimization tasks, where one task is nested inside the other. In machine learning and computer vision fields, despite the different motivations and mechanisms, a lot of complex problems, such as hyper-parameter optimization, multi-task and meta learning, neural architecture search, adversarial learning and deep reinforcement learning, actually all contain a series of closely related subproblms. In this paper, we first uniformly express these complex learning and vision problems from the perspective of BLO. Then we construct a best-response-based single-level reformulation and establish a unified algorithmic framework to understand and formulate mainstream gradient-based BLO methodologies, covering aspects ranging from fundamental automatic differentiation schemes to various accelerations, simplifications, extensions and their convergence and complexity properties. Last but not least, we discuss the potentials of our unified BLO framework for designing new algorithms and point out some promising directions for future research. A list of important papers discussed in this survey, corresponding codes, and additional resources on BLOs are publicly available at: https://github.com/vis-opt-group/BLO .

Ketersediaan

Informasi Detail

Judul Seri

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE; Vol.44 No.12 Part 3 December 2022

No. Panggil

-

Penerbit

: .,

Deskripsi Fisik

p. 10045-10067

Bahasa

English

ISBN/ISSN

-

Klasifikasi

NONE

Tipe Isi

-

Tipe Media

-

Tipe Pembawa

-

Edisi

-

Subjek

BI-LEVEL OPTIMIZATION
VALUE-FUNCTION-BASED REFORMULATION
BEST-RESPONSE MAPPING
EXPLICIT AND IMPLICIT GRADIENTS

Info Detail Spesifik

10.1109/TPAMI.2021.3132674

Pernyataan Tanggungjawab

Risheng Liu, Jiaxin Gao, Jin Zhang, Deyu Meng, Zhouchen Lin

Versi lain/terkait

Lampiran Berkas

Komentar