The Community for Technology Leaders
2017 IEEE International Conference on Multimedia and Expo (ICME) (2017)
Hong Kong, Hong Kong
July 10, 2017 to July 14, 2017
ISSN: 1945-788X
ISBN: 978-1-5090-6068-9
pp: 1153-1158
Li He , Qualcomm Technologies, Inc., San Diego, CA
Xing Xu , University of Electronic Science and Technology of China, China
Huimin Lu , Kyushu Institute of Technology, Japan
Yang Yang , University of Electronic Science and Technology of China, China
Fumin Shen , University of Electronic Science and Technology of China, China
Heng Tao Shen , University of Electronic Science and Technology of China, China
ABSTRACT
The core of existing cross-modal retrieval approaches is to close the gap between different modalities either by finding a maximally correlated subspace or by jointly learning a set of dictionaries. However, the statistical characteristics of the transformed features were never considered. Inspired by recent advances in adversarial learning and domain adaptation, we propose a novel Unsupervised Cross-modal retrieval method based on Adversarial Learning, namely UCAL. In addition to maximizing the correlations between modalities, we add an additional regularization by introducing adversarial learning. In particular, we introduce a modality classifier to predict the modality of a transformed feature. This can be viewed as a regularization on the statistical aspect of the feature transforms, which ensures that the transformed features are also statistically indistinguishable. Experiments on popular multimodal datasets show that UCAL achieves competitive performance compared to state of the art supervised cross-modal retrieval methods.
INDEX TERMS
Correlation, Transforms, Machine learning, Neural networks, Visualization, Dictionaries, Gallium nitride
CITATION

L. He, X. Xu, H. Lu, Y. Yang, F. Shen and H. T. Shen, "Unsupervised cross-modal retrieval through adversarial learning," 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong, Hong Kong, 2017, pp. 1153-1158.
doi:10.1109/ICME.2017.8019549
84 ms
(Ver 3.3 (11022016))