Deception detection is gaining increasing interest due to ethical and
security concerns. This paper explores the application of convolutional neural
networks for the purpose of multimodal deception detection. We use a dataset
built by interviewing 104 subjects about two topics, with one truthful and one
falsified response from each subject about each topic. In particular, we make
three main contributions. First, we extract linguistic and physiological
features from this data to train and construct the neural network models.
Second, we propose a fused convolutional neural network model using both
modalities in order to achieve an improved overall performance. Third, we
compare our new approach with earlier methods designed for multimodal deception
detection. We find that our system outperforms regular classification methods;
our results indicate the feasibility of using neural networks for deception
detection even in the presence of limited amounts of data.

利用卷积神经网络对多模态欺骗检测进行研究，从数据中提取语言和生理特征以训练神经网络模型，提出了融合两种模态的卷积神经网络模型，与早期方法相比，我们的系统表现更优，初步证明在数据有限的情况下使用神经网络进行欺骗检测是可行的。