Machine learning has enabled the development of powerful systems capable of editing images from natural language instructions. However, in many common scenarios it is difficult for users to specify precise image transformations with text alone. For example, in an image with several dogs, it is difficult to select a particular dog and move it to a precise location. Doing this with text alone would require a complex prompt that disambiguates the target dog and describes the destination. However, direct manipulation is well suited to visual tasks like selecting objects and specifying locations. We introduce Point and Instruct, a system for seamlessly combining familiar direct manipulation and textual instructions to enable precise image manipulation. With our system, a user can visually mark objects and locations, and reference them in textual instructions. This allows users to benefit from both the visual descriptiveness of natural language and the spatial precision of direct manipulation.

将直接操作和文本指令相结合，可实现精确图像操作。用户可以通过视觉标记对象和位置，然后在文本指令中引用它们，从而在自然语言的视觉描述性和直接操作的空间精度之间实现有益的结合。

点和指导：通过统一直接操作和文本指令实现精确图像编辑