Generative modeling of complex behaviors from labeled datasets has been a
longstanding problem in decision making. Unlike language or image generation,
decision making requires modeling actions - continuous-valued vectors that are
multimodal in their distribution, potentially drawn from uncurated sources,
where generation errors can compound in sequential prediction. A recent class
of models called Behavior Transformers (BeT) addresses this by discretizing
actions using k-means clustering to capture different modes. However, k-means
struggles to scale for high-dimensional action spaces or long sequences, and
lacks gradient information, and thus BeT suffers in modeling long-range
actions. In this work, we present Vector-Quantized Behavior Transformer
(VQ-BeT), a versatile model for behavior generation that handles multimodal
action prediction, conditional generation, and partial observations. VQ-BeT
augments BeT by tokenizing continuous actions with a hierarchical vector
quantization module. Across seven environments including simulated
manipulation, autonomous driving, and robotics, VQ-BeT improves on
state-of-the-art models such as BeT and Diffusion Policies. Importantly, we
demonstrate VQ-BeT's improved ability to capture behavior modes while
accelerating inference speed 5x over Diffusion Policies. Videos and code can be
found this https URL

通过将连续行为离散化为层次化矢量量化模块，VQ-BeT 是一种能够处理多模态行为预测、条件生成和部分观测的行为生成通用模型，其在七个环境中取得了比 BeT 和 Diffusion Policies 等现有模型更好的效果，并且加速了推断速度 5 倍以上。