Context: An increasing demand is observed in various domains to employ
Machine Learning (ML) for solving complex problems. ML models are implemented
as software components and deployed in Machine Learning Software Systems
(MLSSs). Problem: There is a strong need for ensuring the serving quality of
MLSSs. False or poor decisions of such systems can lead to malfunction of other
systems, significant financial losses, or even threats to human life. The
quality assurance of MLSSs is considered a challenging task and currently is a
hot research topic. Objective: This paper aims to investigate the
characteristics of real quality issues in MLSSs from the viewpoint of
practitioners. This empirical study aims to identify a catalog of quality
issues in MLSSs. Method: We conduct a set of interviews with
practitioners/experts, to gather insights about their experience and practices
when dealing with quality issues. We validate the identified quality issues via
a survey with ML practitioners. Results: Based on the content of 37 interviews,
we identified 18 recurring quality issues and 24 strategies to mitigate them.
For each identified issue, we describe the causes and consequences according to
the practitioners' experience. Conclusion: We believe the catalog of issues
developed in this study will allow the community to develop efficient quality
assurance tools for ML models and MLSSs. A replication package of our study is
available on our public GitHub repository.

本文旨在从实践者的角度研究机器学习软件系统中的质量问题，通过一系列采访和调研，我们创建了一个包含 18 个质量问题和 24 个解决策略的目录，这将有助于开发高效的质量保证工具。