We introduce PhotoBot, a framework for automated photo acquisition based on
an interplay between high-level human language guidance and a robot
photographer. We propose to communicate photography suggestions to the user via
a reference picture that is retrieved from a curated gallery. We exploit a
visual language model (VLM) and an object detector to characterize reference
pictures via textual descriptions and use a large language model (LLM) to
retrieve relevant reference pictures based on a user's language query through
text-based reasoning. To correspond the reference picture and the observed
scene, we exploit pre-trained features from a vision transformer capable of
capturing semantic similarity across significantly varying images. Using these
features, we compute pose adjustments for an RGB-D camera by solving a
Perspective-n-Point (PnP) problem. We demonstrate our approach on a real-world
manipulator equipped with a wrist camera. Our user studies show that photos
taken by PhotoBot are often more aesthetically pleasing than those taken by
users themselves, as measured by human feedback.

PhotoBot 框架结合高级人类语言引导和机器摄影师，实现了基于自动话术采集照片的功能。利用视觉语言模型（VLM）和目标检测器，通过文本描述表征参考图片，然后通过大型语言模型（LLM）根据用户的语言查询检索相关的参考图片。利用一个能够跨不同图片捕捉语义相似性的视觉转换器的训练特征，来对应参考图片和观察场景。该方法通过求解透视 n 点（PnP）问题计算了 RGB-D 相机的姿势调整。在配备腕部相机的真实世界操作器上演示了我们的方法。我们的用户研究表明，通过 PhotoBot 拍摄的照片在美学上往往比用户自己拍摄的更令人满意，这可通过人类反馈来衡量。