The rapid progress of Large Models (LMs) has recently revolutionized various
fields of deep learning with remarkable grades, ranging from Natural Language
Processing (NLP) to Computer Vision (CV). However, LMs are increasingly
challenged and criticized by academia and industry due to their powerful
performance but untrustworthy behavior, which urgently needs to be alleviated
in reliable methods. Despite the abundance of literature on trustworthy LMs in
language, a systematic survey specifically delving into the trustworthiness of
LMs in vision remains absent. In order to mitigate this gap, we summarize four
relevant concerns that obstruct the trustworthy usage in vision of LMs in this
survey, including 1) human misuse, 2) vulnerability, 3) inherent issue and 4)
interpretability. By highlighting corresponding challenge, countermeasures, and
discussion in each topic, we hope this survey will facilitate readers'
understanding of the field, promote alignment of LMs with human expectations
and enable trustworthy LMs to serve as welfare rather than disaster for human
society.

大型模型的快速进展在深度学习的各个领域中带来了显著的成绩，但其强大性能而不可信赖的行为成为学术界和工业界面临的挑战。本综述总结了在视觉领域中妨碍大型模型信赖使用的四个相关问题，包括人类误用、易受攻击、内在问题和解释性，通过突出每个主题中的相应挑战、对策和讨论，希望读者能更好地理解该领域，促进大型模型与人类期望的一致性，使可信赖的大型模型成为人类社会的福祉而非灾难。