The rapid development of generative AI (GenAI) models in computer vision
necessitates effective evaluation methods to ensure their quality and fairness.
Existing tools primarily focus on dataset quality assurance and model
explainability, leaving a significant gap in GenAI output evaluation during
model development. Current practices often depend on developers' subjective
visual assessments, which may lack scalability and generalizability. This paper
bridges this gap by conducting a formative study with GenAI model developers in
an industrial setting. Our findings led to the development of GenLens, a visual
analytic interface designed for the systematic evaluation of GenAI model
outputs during the early stages of model development. GenLens offers a
quantifiable approach for overviewing and annotating failure cases, customizing
issue tags and classifications, and aggregating annotations from multiple users
to enhance collaboration. A user study with model developers reveals that
GenLens effectively enhances their workflow, evidenced by high satisfaction
rates and a strong intent to integrate it into their practices. This research
underscores the importance of robust early-stage evaluation tools in GenAI
development, contributing to the advancement of fair and high-quality GenAI
models.

通过和产业界内的生成 AI 模型开发人员进行形态研究，本文提出了 GenLens，一个可视化分析接口，用于在模型开发的早期阶段系统评估生成 AI 模型输出的质量和公平性。通过开发人员的用户研究发现，GenLens 有效地提升了他们的工作流程，得到了高度的满意度和将其整合到实践中的积极意图。这项研究强调了在生成 AI 开发中早期评估工具的重要性，为公平和高质量的生成 AI 模型的进展做出了贡献。