We introduce PhotoBot, a framework for automated photo acquisition based on
an interplay between high-level human language guidance and a robot
photographer. We propose to communicate photography suggestions to the user via
a reference picture that is retrieved from a curated gallery. We exploit a
visual language model (VLM) and an object detector to characterize reference
pictures via textual descriptions and use a large language model (LLM) to
retrieve relevant reference pictures based on a user's language query through
text-based reasoning. To correspond the reference picture and the observed
scene, we exploit pre-trained features from a vision transformer capable of
capturing semantic similarity across significantly varying images. Using these
features, we compute pose adjustments for an RGB-D camera by solving a
Perspective-n-Point (PnP) problem. We demonstrate our approach on a real-world
manipulator equipped with a wrist camera. Our user studies show that photos
taken by PhotoBot are often more aesthetically pleasing than those taken by
users themselves, as measured by human feedback.

PhotoBot 框架结合高级人类语言引导和机器摄影师，实现了基于自动话术采集照片的功能。利用视觉语言模型（VLM）和目标检测器，通过文本描述表征参考图片，然后通过大型语言模型（LLM）根据用户的语言查询检索相关的参考图片。利用一个能够跨不同图片捕捉语义相似性的视觉转换器的训练特征，来对应参考图片和观察场景。该方法通过求解透视 n 点（PnP）问题计算了 RGB-D 相机的姿势调整。在配备腕部相机的真实世界操作器上演示了我们的方法。我们的用户研究表明，通过 PhotoBot 拍摄的照片在美学上往往比用户自己拍摄的更令人满意，这可通过人类反馈来衡量。

PhotoBot：基于自然语言引导的交互式摄影

PhotoBot: Reference-Guided Interactive Photography via Natural Language

Many photography websites such as Flickr, 500px, Unsplash, and Adobe Behance
are used by amateur and professional photography enthusiasts. Unlike
content-based image search, such users of photography websites are not just
looking for photos with certain content, but more generally for photos with a
certain photographic "aesthetic". In this context, we explore personalized
photo recommendation and propose two aesthetic feature extraction methods based
on (i) color space and (ii) deep style transfer embeddings. Using a dataset
from 500px, we evaluate how these features can be best leveraged by
collaborative filtering methods and show that (ii) provides a significant boost
in photo recommendation performance.

本文研究摄影网站的个性化照片推荐，并提出两种基于颜色空间和深度风格迁移嵌入的美学特征提取方法，使用 500px 数据集进行评估，结果表明第二种方法在照片推荐性能方面有显着提高。