A critical factor in utilizing agents with Artificial Intelligence (AI) is
their robustness to novelty. AI agents include models that are either
engineered or trained. Engineered models include knowledge of those aspects of
the environment that are known and considered important by the engineers.
Learned models form embeddings of aspects of the environment based on
connections made through the training data. In operation, however, a rich
environment is likely to present challenges not seen in training sets or
accounted for in engineered models. Worse still, adversarial environments are
subject to change by opponents. A program at the Defense Advanced Research
Project Agency (DARPA) seeks to develop the science necessary to develop and
evaluate agents that are robust to novelty. This capability will be required,
before AI has the role envisioned within mission critical environments. As part
of the Science of AI and Learning for Open-world Novelty (SAIL-ON), we are
mapping possible military domain novelty types to a domain-independent ontology
developed as part of a theory of novelty. Characterizing the possible space of
novelty mathematically and ontologically will allow us to experiment with agent
designs that are coming from the DARPA SAIL-ON program in relevant military
environments. Utilizing the same techniques as being used in laboratory
experiments, we will be able to measure agent ability to detect, characterize,
and accommodate novelty.

研究 AI 代理人的鲁棒性来应对新奇环境，DARPA 的 SAIL-ON 计划致力于开发必要的科学来评估代理人的鲁棒性，同时利用实验室实验中使用的技术来衡量代理人检测、表征和适应新奇性的能力。