In this work we introduce a time- and memory-efficient method for structured
prediction that couples neuron decisions across both space at time. We show
that we are able to perform exact and efficient inference on a densely
connected spatio-temporal graph by capitalizing on recent advances on deep
Gaussian Conditional Random Fields (GCRFs). Our method, called VideoGCRF is (a)
efficient, (b) has a unique global minimum, and (c) can be trained end-to-end
alongside contemporary deep networks for video understanding. We experiment
with multiple connectivity patterns in the temporal domain, and present
empirical improvements over strong baselines on the tasks of both semantic and
instance segmentation of videos.

本研究提出了一种名为 VideoGCRF 的方法，采用深度高斯条件随机场对密集连接的时空图进行精确和高效的推断，能够与现代深度网络一起进行端到端的视频理解训练，并在语义分割和实例分割的任务中呈现出实证上的优越性。

用于高效视频分割的深度时空随机场

Deep Spatio-Temporal Random Fields for Efficient Video Segmentation

In this work we propose a structured prediction technique that combines the
virtues of Gaussian Conditional Random Fields (G-CRF) with Deep Learning: (a)
our structured prediction task has a unique global optimum that is obtained
exactly from the solution of a linear system (b) the gradients of our model
parameters are analytically computed using closed form expressions, in contrast
to the memory-demanding contemporary deep structured prediction approaches that
rely on back-propagation-through-time, (c) our pairwise terms do not have to be
simple hand-crafted expressions, as in the line of works building on the
DenseCRF, but can rather be `discovered' from data through deep architectures,
and (d) out system can trained in an end-to-end manner. Building on standard
tools from numerical analysis we develop very efficient algorithms for
inference and learning, as well as a customized technique adapted to the
semantic segmentation task. This efficiency allows us to explore more
sophisticated architectures for structured prediction in deep learning: we
introduce multi-resolution architectures to couple information across scales in
a joint optimization framework, yielding systematic improvements. We
demonstrate the utility of our approach on the challenging VOC PASCAL 2012
image segmentation benchmark, showing substantial improvements over strong
baselines. We make all of our code and experiments available at
{this https URL}

本文提出了一种结构化预测技术，其将高斯条件随机场与深度学习相结合，能够在保证唯一全局最优解的同时，使用精简的闭式表达式计算模型参数的梯度。通过在语义分割任务中引入多分辨率结构化优化框架，该系统在 PASCAL 2012 图像分割基准上显示出明显的改进。