Stories and emotions in movies emerge through the effect of well-thought-out
directing decisions, in particular camera placement and movement over time.
Crafting compelling camera trajectories remains a complex iterative process,
even for skilful artists. To tackle this, in this paper, we propose a dataset
called the Exceptional Trajectories (E.T.) with camera trajectories along with
character information and textual captions encompassing descriptions of both
camera and character. To our knowledge, this is the first dataset of its kind.
To show the potential applications of the E.T. dataset, we propose a
diffusion-based approach, named DIRECTOR, which generates complex camera
trajectories from textual captions that describe the relation and
synchronisation between the camera and characters. To ensure robust and
accurate evaluations, we train on the E.T. dataset CLaTr, a Contrastive
Language-Trajectory embedding for evaluation metrics. We posit that our
proposed dataset and method significantly advance the democratization of
cinematography, making it more accessible to common users.

通过精心考虑的导演决策，特别是相机位置和随时间变化的运动，电影中的故事和情感呈现。为了解决相机轨迹建立的复杂过程，我们提出了一个数据集 Exceptional Trajectories（E.T.），其中包括相机轨迹、字符信息和文字标题，涵盖了相机和字符的描述。我们提出了一种基于扩散的方法 DIRECTOR，该方法根据描述相机与字符之间关系和同步的文本标题生成复杂的相机轨迹，以展示 E.T. 数据集的潜在应用。在 E.T. 数据集上进行训练的 CLaTr 对比学习方法用于评估指标的稳健性和准确性。我们认为我们提出的数据集和方法显著推动了电影摄影术的民主化进程，使其对普通用户更易接触。

E.T. 异常轨迹：具有角色意识的文本到相机轨迹生成

E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation  with character awareness

Video summarization remains a huge challenge in computer vision due to the
size of the input videos to be summarized. We propose an efficient,
language-only video summarizer that achieves competitive accuracy with high
data efficiency. Using only textual captions obtained via a zero-shot approach,
we train a language transformer model and forego image representations. This
method allows us to perform filtration amongst the representative text vectors
and condense the sequence. With our approach, we gain explainability with
natural language that comes easily for human interpretation and textual
summaries of the videos. An ablation study that focuses on modality and data
compression shows that leveraging text modality only effectively reduces input
data processing while retaining comparable results.

通过仅使用通过零样本方法获取的文本标题，我们提出了一种高效的纯文本视频摘要方法，能够以高数据效率实现有竞争力的准确性。我们通过训练语言转换模型并摒弃图像表示来进行视频摘要，从而允许我们在代表性文本向量之间进行筛选并压缩序列。我们的方法能够使自然语言解释能力易于人类理解，以及对视频的文本摘要。调查了模态性和数据压缩的消融研究表明，仅利用文本模态有效地减少了输入的数据处理量，同时保持了可比较的结果。