Multimodal scene search of conversations is essential for unlocking valuable
insights into social dynamics and enhancing our communication. While experts in
conversational analysis have their own knowledge and skills to find key scenes,
a lack of comprehensive, user-friendly tools that streamline the processing of
diverse multimodal queries impedes efficiency and objectivity. To solve it, we
developed Providence, a visual-programming-based tool based on design
considerations derived from a formative study with experts. It enables experts
to combine various machine learning algorithms to capture human behavioral cues
without writing code. Our study showed its preferable usability and
satisfactory output with less cognitive load imposed in accomplishing scene
search tasks of conversations, verifying the importance of its customizability
and transparency. Furthermore, through the in-the-wild trial, we confirmed the
objectivity and reusability of the tool transform experts' workflow, suggesting
the advantage of expert-AI teaming in a highly human-contextual domain.

通过基于专家研究的设计考虑，我们开发了 Providence 工具，它是一个基于可视化编程的工具，致力于解决多模态场景搜索中存在的效率和客观性问题。该工具使专家能够结合多种机器学习算法捕捉人类行为提示，而无需编写代码。通过研究和实践验证，我们证实了该工具的可用性、输出质量和可定制性，并且证明了专家和人工智能团队在高度人类环境的领域中的优势。