Artificial intelligence (AI) is being ubiquitously adopted to automate
processes in science and industry. However, due to its often intricate and
opaque nature, AI has been shown to possess inherent vulnerabilities which can
be maliciously exploited with adversarial AI, potentially putting AI users and
developers at both cyber and physical risk. In addition, there is insufficient
comprehension of the real-world effects of adversarial AI and an inadequacy of
AI security examinations; therefore, the growing threat landscape is unknown
for many AI solutions. To mitigate this issue, we propose one of the first red
team frameworks for evaluating the AI security of maritime autonomous systems.
The framework provides operators with a proactive (secure by design) and
reactive (post-deployment evaluation) response to securing AI technology today
and in the future. This framework is a multi-part checklist, which can be
tailored to different systems and requirements. We demonstrate this framework
to be highly effective for a red team to use to uncover numerous
vulnerabilities within a real-world maritime autonomous systems AI, ranging
from poisoning to adversarial patch attacks. The lessons learned from
systematic AI red teaming can help prevent MAS-related catastrophic events in a
world with increasing uptake and reliance on mission-critical AI.

通过使用一种多部分检查表的框架，我们展示了如何高效地评估海上自主系统的人工智能安全，并揭示其中的多个漏洞，从中毒到对抗性补丁攻击。通过系统化的人工智能红队行动，我们可以防止与提高对使命关键人工智能的接受和依赖性相应的严重事件。

保障海上自治系统中人工智能的红队框架

A Red Teaming Framework for Securing AI in Maritime Autonomous Systems

Artificial intelligence (AI) and machine learning (ML) have become
increasingly vital in the development of novel defense and intelligence
capabilities across all domains of warfare. An adversarial AI (A2I) and
adversarial ML (AML) attack seeks to deceive and manipulate AI/ML models. It is
imperative that AI/ML models can defend against these attacks. A2I/AML defenses
will help provide the necessary assurance of these advanced capabilities that
use AI/ML models. The A2I Working Group (A2IWG) seeks to advance the research
and development of assured AI/ML capabilities via new A2I/AML defenses by
fostering a collaborative environment across the U.S. Department of Defense and
U.S. Intelligence Community. The A2IWG aims to identify specific challenges
that it can help solve or address more directly, with initial focus on three
topics: AI Trusted Robustness, AI System Security, and AI/ML Architecture
Vulnerabilities.

人工智能 (AI) 和机器学习 (ML) 在开发新型防御和情报能力方面已变得越来越重要。A2I 攻击旨在欺骗和操纵 AI/ML 模型，因此 AI/ML 模型必须能够抵御这些攻击。A2IWG 旨在通过在美国国防部和情报机构之间建立协作环境，通过新的 A2I/AML 防御手段，推动确保 AI/ML 能力的研究和开发，并针对具体挑战，重点关注 AI 可信鲁棒性，AI 系统安全性和 AI/ML 架构漏洞。