Text

High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks

High-dimensional statistical learning (HDSL) has wide applications in data analysis, operations research, and decision making. Despite the availability of multiple theoretical frameworks, most existing HDSL schemes stipulate the following two conditions: (a) the sparsity and (b) restricted strong convexity (RSC). This paper generalizes both conditions via the use of the folded concave penalty (FCP). More specifically, we consider an M-estimation problem where (i) (conventional) sparsity is relaxed into the approximate sparsity and (ii) RSC is completely absent. We show that the FCP-based regularization leads to poly-logarithmic sample complexity; the training data size is only required to be poly-logarithmic in the problem dimensionality. This finding can facilitate the analysis of two important classes of models that are currently less understood: high-dimensional nonsmooth learning and (deep) neural networks (NNs). For both problems, we show that poly-logarithmic sample complexity can be maintained. In particular, our results indicate that the generalizability of NNs under overparameterization can be theoretically ensured with the aid of regularization.

Ketersediaan

Informasi Detail

Judul Seri

OPERATIONS RESEARCH; Vol.70 No.6 November-December 2022

No. Panggil

-

Penerbit

: .,

Deskripsi Fisik

p. 3176-3197

Bahasa

English

ISBN/ISSN

-

Klasifikasi

NONE

Tipe Isi

-

Tipe Media

-

Tipe Pembawa

-

Edisi

-

Subjek

NEURAL NETWORK
SPARSITY
SUPPORT VECTOR MACHINE
RESTRICTED STRONG CONVEXITY
FOLDED CONCAVE PENALTY
HIGH-DIMENSIONAL LEARNING
NONSMOOTH LEARNING

Info Detail Spesifik

https://doi.org/10.1287/opre.2021.2217

Pernyataan Tanggungjawab

Hongcheng Liu, Yinyu Ye, Hung Yi Lee

Versi lain/terkait

Lampiran Berkas

Komentar