We propose an end-to-end learned video compression scheme for low-latency
scenarios. Previous methods are limited in using the previous one frame as
reference. Our method introduces the usage of the previous multiple frames as
references. In our scheme, the motion vector (MV) field is calculated between
the current frame and the previous one. With multiple reference frames and
associated multiple MV fields, our designed network can generate more accurate
prediction of the current frame, yielding less residual. Multiple reference
frames also help generate MV prediction, which reduces the coding cost of MV
field. We use two deep auto-encoders to compress the residual and the MV,
respectively. To compensate for the compression error of the auto-encoders, we
further design a MV refinement network and a residual refinement network,
taking use of the multiple reference frames as well. All the modules in our
scheme are jointly optimized through a single rate-distortion loss function. We
use a step-by-step training strategy to optimize the entire scheme.
Experimental results show that the proposed method outperforms the existing
learned video compression methods for low-latency mode. Our method also
performs better than H.265 in both PSNR and MS-SSIM. Our code and models are
publicly available.

该研究提出了一种端到端的学习视频压缩方案，使用多个参考帧来提高数据压缩效率，其中包括运动向量 (MV) 场的计算、残差和 MV 的深度自编码压缩，以及一个利用多个参考帧的 MV 细化网络和残差细化网络。

M-LVC: 学习视频压缩的多帧预测

M-LVC: Multiple Frames Prediction for Learned Video Compression

Beamforming has been extensively investigated for multi-channel audio
processing tasks. Recently, learning-based beamforming methods, sometimes
called \textit{neural beamformers}, have achieved significant improvements in
both signal quality (e.g. signal-to-noise ratio (SNR)) and speech recognition
(e.g. word error rate (WER)). Such systems are generally non-causal and require
a large context for robust estimation of inter-channel features, which is
impractical in applications requiring low-latency responses. In this paper, we
propose filter-and-sum network (FaSNet), a time-domain, filter-based
beamforming approach suitable for low-latency scenarios. FaSNet has a two-stage
system design that first learns frame-level time-domain adaptive beamforming
filters for a selected reference channel, and then calculate the filters for
all remaining channels. The filtered outputs at all channels are summed to
generate the final output. Experiments show that despite its small model size,
FaSNet is able to outperform several traditional oracle beamformers with
respect to scale-invariant signal-to-noise ratio (SI-SNR) in reverberant speech
enhancement and separation tasks. Moreover, when trained with a
frequency-domain objective function on the CHiME-3 dataset, FaSNet achieves
14.3\% relative word error rate reduction (RWERR) compared with the baseline
model. These results show the efficacy of FaSNet particularly in reverberant
and noisy signal conditions.

提出了一种适用于低延迟场景的时域、基于过滤的波束成形方法 ——FaSNet，该方法可在含噪声和混响语音增强中表现出色，同时还可在 CHiME-3 数据集上通过训练基于频域目标函数的 FaSNet 模型实现与基准模型相比的相对字错误率降低 14.3%。