Organizations including companies, nonprofits, governments, and academic institutions are increasingly developing, deploying, and utilizing artificial intelligence (AI) tools. Responsible AI (RAI) governance approaches at organizations have emerged as important mechanisms to address potential AI risks and harms. In this work, we interviewed 17 technical contributors across organization types (Academic, Government, Industry, Nonprofit) and sectors (Finance, Health, Tech, Other) about their experiences with internal RAI governance. Our findings illuminated the variety of organizational definitions of RAI and accompanying internal governance approaches. We summarized the first detailed findings on algorithm review boards (ARBs) and similar review committees in practice, including their membership, scope, and measures of success. We confirmed known robust model governance in finance sectors and revealed extensive algorithm and AI governance with ARB-like review boards in health sectors. Our findings contradict the idea that Institutional Review Boards alone are sufficient for algorithm governance and posit that ARBs are among the more impactful internal RAI governance approaches. Our results suggest that integration with existing internal regulatory approaches and leadership buy-in are among the most important attributes for success and that financial tensions are the greatest challenge to effective organizational RAI. We make a variety of suggestions for how organizational partners can learn from these findings when building their own internal RAI frameworks. We outline future directions for developing and measuring effectiveness of ARBs and other internal RAI governance approaches.

人工智能工具在组织机构中的开发、部署和利用日益增多，负责任的人工智能(RAI)治理方法成为解决潜在AI风险和危害的重要机制。本研究采访了来自学术机构、政府、行业和非营利组织等不同类型和领域的17名技术参与者，研究他们在内部RAI治理方面的经验。研究结果揭示了组织机构对RAI的定义及其内部治理方法的多样性。研究还发现金融行业具备健全的模型治理，并在医疗领域中出现了类似ARB审查委员会的广泛算法和人工智能治理。研究表明ARB是更有影响力的内部RAI治理方法。研究结果对建立组织机构自己的内部RAI框架提供了一些建议，并为ARB和其他内部RAI治理方法的发展和效果评估提出了未来方向。

对组织负责的人工智能治理算法审查委员会的研究