Recently, the community has made tremendous progress in developing effective
methods for point cloud video understanding that learn from massive amounts of
labeled data. However, annotating point cloud videos is usually notoriously
expensive. Moreover, training via one or only a few traditional tasks (e.g.,
classification) may be insufficient to learn subtle details of the
spatio-temporal structure existing in point cloud videos. In this paper, we
propose a Masked Spatio-Temporal Structure Prediction (MaST-Pre) method to
capture the structure of point cloud videos without human annotations. MaST-Pre
is based on spatio-temporal point-tube masking and consists of two
self-supervised learning tasks. First, by reconstructing masked point tubes,
our method is able to capture the appearance information of point cloud videos.
Second, to learn motion, we propose a temporal cardinality difference
prediction task that estimates the change in the number of points within a
point tube. In this way, MaST-Pre is forced to model the spatial and temporal
structure in point cloud videos. Extensive experiments on MSRAction-3D,
NTU-RGBD, NvGesture, and SHREC'17 demonstrate the effectiveness of the proposed
method.

本文提出了一种无需人工注释的点云视频结构预测方法（MaST-Pre），通过空间 - 时间点管掩蔽和自监督学习任务，捕捉了点云视频的外观信息和运动变化，有效地模拟了点云视频的空间和时间结构。大量实验证明了该方法的有效性。

点云视频的遮蔽空间 - 时间结构预测自监督学习

Masked Spatio-Temporal Structure Prediction for Self-supervised Learning  on Point Cloud Videos

In the field of behavior-related brain computation, it is necessary to
meaningfully align raw neural population activities against the drastic shift
between them. However, the alignment is non-trivial since most neural
population activities are in a multivariate time-series manner. An instrumental
framework within neuroscience research posits that trial-based neural
population activities rely on low-dimensional latent dynamics. Focusing on such
latent dynamics greatly facilitates the alignment procedure. Despite the
considerable progress we have reached, existing methods usually ignore the
intrinsic spatio-temporal structures within latent dynamics. Thus, those
solutions lead to poor quality in dynamics structures and overall performance
after alignment. To tackle this problem, we propose a method leveraging the
expressiveness of diffusion model to relieve such issues. Specifically, the
latent dynamics structures of the source domain are first extracted by the
diffusion model. Then, such structures are well-recovered through a maximum
likelihood alignment procedure on the target domain. We first demonstrate the
effectiveness of our proposed method on a synthetic dataset. Then, when applied
to neural recordings from primate motor cortex, under both cross-day and
inter-subject settings, our method consistently manifests its capability of
preserving the spatio-temporal structure of latent dynamics and outperforms
existing approaches in alignment quality.

该论文提出了一种基于扩散模型的方法，用于在跨域和跨主体领域中提高神经人口活动的对齐质量，维护潜在动态的时空结构，并实现超过现有方法的对齐质量。

潜在动力学中的时空结构提取与恢复：扩散模型对齐

Extraction and Recovery of Spatio-Temporal Structure in Latent Dynamics  Alignment with Diffusion Model

Contrastive learning methods have significantly narrowed the gap between
supervised and unsupervised learning on computer vision tasks. In this paper,
we explore their application to geo-located datasets, e.g. remote sensing,
where unlabeled data is often abundant but labeled data is scarce. We first
show that due to their different characteristics, a non-trivial gap persists
between contrastive and supervised learning on standard benchmarks. To close
the gap, we propose novel training methods that exploit the spatio-temporal
structure of remote sensing data. We leverage spatially aligned images over
time to construct temporal positive pairs in contrastive learning and
geo-location to design pre-text tasks. Our experiments show that our proposed
method closes the gap between contrastive and supervised learning on image
classification, object detection and semantic segmentation for remote sensing.
Moreover, we demonstrate that the proposed method can also be applied to
geo-tagged ImageNet images, improving downstream performance on various tasks.
Project Webpage can be found at this link geography-aware-ssl.github.io.

本研究旨在将对比学习方法应用于地理定位的数据集上，通过利用遥感数据的时空结构设计预处理任务和构建时空正对来消除对比学习和监督学习之间的差距，并在遥感图像分类、目标检测和语义分割等任务上展示该方法的有效性，同时还表明该方法可以应用于地理标记的 ImageNet 图像并提高相应任务的性能。

地理感知的自监督学习

Geography-Aware Self-Supervised Learning

Typical human actions last several seconds and exhibit characteristic
spatio-temporal structure. Recent methods attempt to capture this structure and
learn action representations with convolutional neural networks. Such
representations, however, are typically learned at the level of a few video
frames failing to model actions at their full temporal extent. In this work we
learn video representations using neural networks with long-term temporal
convolutions (LTC). We demonstrate that LTC-CNN models with increased temporal
extents improve the accuracy of action recognition. We also study the impact of
different low-level representations, such as raw values of video pixels and
optical flow vector fields and demonstrate the importance of high-quality
optical flow estimation for learning accurate action models. We report
state-of-the-art results on two challenging benchmarks for human action
recognition UCF101 (92.7%) and HMDB51 (67.2%).

通过使用长时序卷积神经网络和高质量的光流估计，该研究获得了 UCF101 和 HMDB51 数据集上最先进的人类动作识别结果，具备良好的行为表示和准确性。