We introduce EvalAI, an open source platform for evaluating and comparing machine learning (ML) and artificial intelligence algorithms (AI) at scale. EvalAI is built to provide a scalable solution to the research community to fulfill the critical need of evaluating machine learning models and agents acting in an environment against annotations or with a human-in-the-loop. This will help researchers, students, and data scientists to create, collaborate, and participate in AI challenges organized around the globe. By simplifying and standardizing the process of benchmarking these models, EvalAI seeks to lower the barrier to entry for participating in the global scientific effort to push the frontiers of machine learning and artificial intelligence, thereby increasing the rate of measurable progress in this domain.

EvalAI是一个开源工具，用于评估和比较规模化的机器学习和人工智能算法，通过标注或与人交互的方式对机器学习模型进行评估，从而帮助全球范围内的研究人员、学生和数据科学家创建、协作和参与人工智能挑战，降低机器学习和人工智能的门槛，从而提高领域内可量化的进展速度。

EvalAI: 为AI代理构建更好的评估系统