We propose Subject-Conditional Relation Detection SCoRD, where conditioned on
an input subject, the goal is to predict all its relations to other objects in
a scene along with their locations. Based on the Open Images dataset, we
propose a challenging OIv6-SCoRD benchmark such that the training and testing
splits have a distribution shift in terms of the occurrence statistics of
$\langle$subject, relation, object$\rangle$ triplets. To solve this problem, we
propose an auto-regressive model that given a subject, it predicts its
relations, objects, and object locations by casting this output as a sequence
of tokens. First, we show that previous scene-graph prediction methods fail to
produce as exhaustive an enumeration of relation-object pairs when conditioned
on a subject on this benchmark. Particularly, we obtain a recall@3 of 83.8% for
our relation-object predictions compared to the 49.75% obtained by a recent
scene graph detector. Then, we show improved generalization on both
relation-object and object-box predictions by leveraging during training
relation-object pairs obtained automatically from textual captions and for
which no object-box annotations are available. Particularly, for
$\langle$subject, relation, object$\rangle$ triplets for which no object
locations are available during training, we are able to obtain a recall@3 of
42.59% for relation-object pairs and 32.27% for their box locations.

基于 Open Images 数据集，我们提出了一种挑战性的 OIv6-SCoRD 基准，通过条件化输入主体来预测其与场景中其他对象的所有关系及其位置。我们提出了一种自回归模型，通过将输出解释为一个标记序列，预测主体的关系、对象和对象位置，提高了关系 - 对象和对象 - 框预测的泛化能力。

SCoRD：基于文本增强数据的主体条件关系检测

SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data

We present a significant breakthrough in 3D shape generation by scaling it to
unprecedented dimensions. Through the adaptation of the Auto-Regressive model
and the utilization of large language models, we have developed a remarkable
model with an astounding 3.6 billion trainable parameters, establishing it as
the largest 3D shape generation model to date, named Argus-3D. Our approach
addresses the limitations of existing methods by enhancing the quality and
diversity of generated 3D shapes. To tackle the challenges of high-resolution
3D shape generation, our model incorporates tri-plane features as latent
representations, effectively reducing computational complexity. Additionally,
we introduce a discrete codebook for efficient quantization of these
representations. Leveraging the power of transformers, we enable multi-modal
conditional generation, facilitating the production of diverse and visually
impressive 3D shapes. To train our expansive model, we leverage an ensemble of
publicly-available 3D datasets, consisting of a comprehensive collection of
approximately 900,000 objects from renowned repositories such as ModelNet40,
ShapeNet, Pix3D, 3D-Future, and Objaverse. This diverse dataset empowers our
model to learn from a wide range of object variations, bolstering its ability
to generate high-quality and diverse 3D shapes. Extensive experimentation
demonstrate the remarkable efficacy of our approach in significantly improving
the visual quality of generated 3D shapes. By pushing the boundaries of 3D
generation, introducing novel methods for latent representation learning, and
harnessing the power of transformers for multi-modal conditional generation,
our contributions pave the way for substantial advancements in the field. Our
work unlocks new possibilities for applications in gaming, virtual reality,
product design, and other domains that demand high-quality and diverse 3D
objects.

我们开发了一个名为 Argus-3D 的模型，通过采用自回归模型和大型语言模型的适应性，以及使用 36 亿可训练参数，在 3D 形状生成方面取得了重大突破，通过增强生成的 3D 形状的质量和多样性，解决了现有方法的局限性。

大规模三维形状生成的极限挑战

Pushing the Limits of 3D Shape Generation at Scale

Recent advances in deep learning have expanded possibilities to generate
music, but generating a customizable full piece of music with consistent
long-term structure remains a challenge. This paper introduces MusicFrameworks,
a hierarchical music structure representation and a multi-step generative
process to create a full-length melody guided by long-term repetitive
structure, chord, melodic contour, and rhythm constraints. We first organize
the full melody with section and phrase-level structure. To generate melody in
each phrase, we generate rhythm and basic melody using two separate
transformer-based networks, and then generate the melody conditioned on the
basic melody, rhythm and chords in an auto-regressive manner. By factoring
music generation into sub-problems, our approach allows simpler models and
requires less data. To customize or add variety, one can alter chords, basic
melody, and rhythm structure in the music frameworks, letting our networks
generate the melody accordingly. Additionally, we introduce new features to
encode musical positional information, rhythm patterns, and melodic contours
based on musical domain knowledge. A listening test reveals that melodies
generated by our method are rated as good as or better than human-composed
music in the POP909 dataset about half the time.

本文介绍了 MusicFrameworks，这是一种基于深度学习的分层音乐结构表示方法；并提出了一种多步骤的生成过程，根据长期重复结构、和弦、旋律轮廓和节奏约束生成完整的旋律。研究结果表明，其中一半的旋律比起 POP909 数据集中由人类作曲家创作的音乐更好或同样好。