Text

Universal Weighting Metric Learning for Cross-Modal Retrieval

Cross-modal retrieval has recently attracted growing attention, which aims to match instances captured from different modalities. The performance of cross-modal retrieval methods heavily relies on the capability of metric learning to mine and weight the informative pairs. While various metric learning methods have been developed for unimodal retrieval tasks, the cross-modal retrieval tasks, however, have not been explored to its fullest extent. In this paper, we develop a universal weighting metric learning framework for cross-modal retrieval, which can effectively sample informative pairs and assign proper weight values to them based on their similarity scores so that different pairs favor different penalty strength. Based on this framework, we introduce two types of polynomial loss for cross-modal retrieval, self-similarity polynomial loss and relative-similarity polynomial loss. The former provides a polynomial function to associate the weight values with self-similarity scores, and the latter defines a polynomial function to associate the weight values with relative-similarity scores. Both self and relative-similarity polynomial loss can be freely applied to off-the-shelf methods and further improve their retrieval performance. Extensive experiments on two image-text retrieval datasets, three video-text retrieval datasets and one fine-grained image retrieval dataset demonstrate that our proposed method can achieve a noticeable boost in retrieval performance.

Ketersediaan

Informasi Detail

Judul Seri

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE; Vol.44 No.10 Part 1 October 2022

No. Panggil

-

Penerbit

: .,

Deskripsi Fisik

p. 6534-6545

Bahasa

English

ISBN/ISSN

-

Klasifikasi

NONE

Tipe Isi

-

Tipe Media

-

Tipe Pembawa

-

Edisi

-

Subjek

CROSS-MODAL RETRIEVAL
UNIVERSAL WEIGHTING METRIC LEARNING
SELF-SIMILARITY POLYNOMIAL LOSS
RELATIVE-SIMILARITY POLYNOMIAL LOSS

Info Detail Spesifik

DOI: 10.1109/TPAMI.2021.3088863

Pernyataan Tanggungjawab

Jiwei Wei, Yang Yang, Xing Xu, Xiaofeng Zhu, Heng Tao Shen

Versi lain/terkait

Lampiran Berkas

Komentar