We address the task of generating temporally consistent and physically
plausible images of actions and object state transformations. Given an input
image and a text prompt describing the targeted transformation, our generated
images preserve the environment and transform objects in the initial image. Our
contributions are threefold. First, we leverage a large body of instructional
videos and automatically mine a dataset of triplets of consecutive frames
corresponding to initial object states, actions, and resulting object
transformations. Second, equipped with this data, we develop and train a
conditioned diffusion model dubbed GenHowTo. Third, we evaluate GenHowTo on a
variety of objects and actions and show superior performance compared to
existing methods. In particular, we introduce a quantitative evaluation where
GenHowTo achieves 88% and 74% on seen and unseen interaction categories,
respectively, outperforming prior work by a large margin.

生成具有时间一致性和物理可行性的行为和物体状态转换的图像，利用视频数据集和条件扩散模型 GenHowTo，在各种对象和动作上展示出优秀的性能。