In recent years, live streaming platforms have gained immense popularity as
they allow users to broadcast their videos and interact in real-time with hosts
and peers. Due to the dynamic changes of live content, accurate recommendation
models are crucial for enhancing user experience. However, most previous works
treat the live as a whole item and explore the Click-through-Rate (CTR)
prediction framework on item-level, neglecting that the dynamic changes that
occur even within the same live room. In this paper, we proposed a ContentCTR
model that leverages multimodal transformer for frame-level CTR prediction.
First, we present an end-to-end framework that can make full use of multimodal
information, including visual frames, audio, and comments, to identify the most
attractive live frames. Second, to prevent the model from collapsing into a
mediocre solution, a novel pairwise loss function with first-order difference
constraints is proposed to utilize the contrastive information existing in the
highlight and non-highlight frames. Additionally, we design a temporal
text-video alignment module based on Dynamic Time Warping to eliminate noise
caused by the ambiguity and non-sequential alignment of visual and textual
information. We conduct extensive experiments on both real-world scenarios and
public datasets, and our ContentCTR model outperforms traditional
recommendation models in capturing real-time content changes. Moreover, we
deploy the proposed method on our company platform, and the results of online
A/B testing further validate its practical significance.

本文提出了一种基于多模态 Transformer 的 ContentCTR 模型，采用帧级别的 CTR 预测来充分利用视觉帧、音频和评论等多模态信息，借助具有一阶差分约束的新型成对损失函数来利用亮点和非亮点间的对比信息，以及基于动态时间规整的时间文本 - 视频对齐模块以捕捉实时内容变化，实验表明该方法在真实场景和公共数据集上进行推荐时均优于传统方法，应用于公司平台后的在线 A/B 测试也进一步验证了其实际意义。