A demonstration of a real-time and continuous turn-taking prediction system
is presented. The system is based on a voice activity projection (VAP) model,
which directly maps dialogue stereo audio to future voice activities. The VAP
model includes contrastive predictive coding (CPC) and self-attention
transformers, followed by a cross-attention transformer. We examine the effect
of the input context audio length and demonstrate that the proposed system can
operate in real-time with CPU settings, with minimal performance degradation.

提出了一个基于语音活动投影（VAP）模型的实时连续交替预测系统演示。该系统基于对话立体声音频直接预测未来语音活动。VAP 模型包括对比预测编码（CPC）和自注意力变换器，之后是交叉注意力变换器。我们考察了输入上下文音频长度的影响，并演示了该系统可以在 CPU 设置下以实时方式运行，并具有最小的性能降低。

实时连续的话语交替预测：使用声音活动投影

Real-time and Continuous Turn-taking Prediction Using Voice Activity  Projection

In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal
policy -- the optimal policy when facing an unknown task that is sampled from
some known task distribution. Previous approaches tackled this problem by
inferring a belief over task parameters, using variational inference methods.
Motivated by recent successes of contrastive learning approaches in RL, such as
contrastive predictive coding (CPC), we investigate whether contrastive methods
can be used for learning Bayes-optimal behavior. We begin by proving that
representations learned by CPC are indeed sufficient for Bayes optimality.
Based on this observation, we propose a simple meta RL algorithm that uses CPC
in lieu of variational belief inference. Our method, ContraBAR, achieves
comparable performance to state-of-the-art in domains with state-based
observation and circumvents the computational toll of future observation
reconstruction, enabling learning in domains with image-based observations. It
can also be combined with image augmentations for domain randomization and used
seamlessly in both online and offline meta RL settings.

本文提出了一种基于对比学习方法的元 RL 算法 ContraBAR，可以在状态观测及基于图像观测的领域中有效地学习 Bayes 最优行为，并可以与图像增强相结合，用于领域随机化，并可以无缝地应用于在线和离线元 RL 设置。