Video synthesis has recently made remarkable strides benefiting from the
rapid development of diffusion models. However, it still encounters challenges
in terms of semantic accuracy, clarity and spatio-temporal continuity. They
primarily arise from the scarcity of well-aligned text-video data and the
complex inherent structure of videos, making it difficult for the model to
simultaneously ensure semantic and qualitative excellence. In this report, we
propose a cascaded I2VGen-XL approach that enhances model performance by
decoupling these two factors and ensures the alignment of the input data by
utilizing static images as a form of crucial guidance. I2VGen-XL consists of
two stages: i) the base stage guarantees coherent semantics and preserves
content from input images by using two hierarchical encoders, and ii) the
refinement stage enhances the video's details by incorporating an additional
brief text and improves the resolution to 1280$\times$720. To improve the
diversity, we collect around 35 million single-shot text-video pairs and 6
billion text-image pairs to optimize the model. By this means, I2VGen-XL can
simultaneously enhance the semantic accuracy, continuity of details and clarity
of generated videos. Through extensive experiments, we have investigated the
underlying principles of I2VGen-XL and compared it with current top methods,
which can demonstrate its effectiveness on diverse data. The source code and
models will be publicly available at https://i2vgen-xl.github.io.

提出了 I2VGen-XL 的级联方法，通过将两个因素解耦并利用静态图像作为关键指导，能够同时提高模型的语义准确性、细节的连续性和视频的清晰度。

I2VGen-XL: 通过级联扩散模型实现高质量图像到视频的合成

I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion  Models

This work proposes an end-to-end multi-camera 3D multi-object tracking (MOT)
framework. It emphasizes spatio-temporal continuity and integrates both past
and future reasoning for tracked objects. Thus, we name it "Past-and-Future
reasoning for Tracking" (PF-Track). Specifically, our method adapts the
"tracking by attention" framework and represents tracked instances coherently
over time with object queries. To explicitly use historical cues, our "Past
Reasoning" module learns to refine the tracks and enhance the object features
by cross-attending to queries from previous frames and other objects. The
"Future Reasoning" module digests historical information and predicts robust
future trajectories. In the case of long-term occlusions, our method maintains
the object positions and enables re-association by integrating motion
predictions. On the nuScenes dataset, our method improves AMOTA by a large
margin and remarkably reduces ID-Switches by 90% compared to prior approaches,
which is an order of magnitude less. The code and models are made available at
this https URL.

本文介绍了一种名为 PF-Track 的多相机 3D 多目标跟踪框架，它重点强调时空连续性，过去和未来的推理，采用 “注意跟踪” 框架，并使用对象查询一致地表示跟踪实例，以明确使用历史线索，显式地引用前面帧和其他对象的查询，从而学习优化轨迹并增强物体特征。