For better user experience and business effectiveness, Click-Through Rate
(CTR) prediction has been one of the most important tasks in E-commerce.
Although extensive CTR prediction models have been proposed, learning good
representation of items from multimodal features is still less investigated,
considering an item in E-commerce usually contains multiple heterogeneous
modalities. Previous works either concatenate the multiple modality features,
that is equivalent to giving a fixed importance weight to each modality; or
learn dynamic weights of different modalities for different items through
technique like attention mechanism. However, a problem is that there usually
exists common redundant information across multiple modalities. The dynamic
weights of different modalities computed by using the redundant information may
not correctly reflect the different importance of each modality. To address
this, we explore the complementarity and redundancy of modalities by
considering modality-specific and modality-invariant features differently. We
propose a novel Multimodal Adversarial Representation Network (MARN) for the
CTR prediction task. A multimodal attention network first calculates the
weights of multiple modalities for each item according to its modality-specific
features. Then a multimodal adversarial network learns modality-invariant
representations where a double-discriminators strategy is introduced. Finally,
we achieve the multimodal item representations by combining both
modality-specific and modality-invariant representations. We conduct extensive
experiments on both public and industrial datasets, and the proposed method
consistently achieves remarkable improvements to the state-of-the-art methods.
Moreover, the approach has been deployed in an operational E-commerce system
and online A/B testing further demonstrates the effectiveness.

提出了一种新的基于多模态对抗表示网络 (MARN) 的点击率预测模型，它通过计算多模态特征中的模态加权值和使用模态不变特征来学习表征，以解决在电子商务中从多模态特征学习良好的物品表示的问题，并在公共和工业数据集上实现了显着的改进。

对抗式多模态表征学习用于点击率预测

Adversarial Multimodal Representation Learning for Click-Through Rate  Prediction

Human face-to-face communication is a complex multimodal signal. We use words
(language modality), gestures (vision modality) and changes in tone (acoustic
modality) to convey our intentions. Humans easily process and understand
face-to-face communication, however, comprehending this form of communication
remains a significant challenge for Artificial Intelligence (AI). AI must
understand each modality and the interactions between them that shape human
communication. In this paper, we present a novel neural architecture for
understanding human communication called the Multi-attention Recurrent Network
(MARN). The main strength of our model comes from discovering interactions
between modalities through time using a neural component called the
Multi-attention Block (MAB) and storing them in the hybrid memory of a
recurrent component called the Long-short Term Hybrid Memory (LSTHM). We
perform extensive comparisons on six publicly available datasets for multimodal
sentiment analysis, speaker trait recognition and emotion recognition. MARN
shows state-of-the-art performance on all the datasets.

本文介绍了一种名为多注意力循环网络（MARN）的人类交流理解神经架构，通过使用多注意力块（MAB）的神经组件，MARN 致力于发现和存储影响人际交往的互动，并在多模态情感分析、说话者特质识别和情感识别方面显示出最先进的性能。