Lifelong learning offers a promising paradigm of building a generalist agent
that learns and adapts over its lifespan. Unlike traditional lifelong learning
problems in image and text domains, which primarily involve the transfer of
declarative knowledge of entities and concepts, lifelong learning in
decision-making (LLDM) also necessitates the transfer of procedural knowledge,
such as actions and behaviors. To advance research in LLDM, we introduce
LIBERO, a novel benchmark of lifelong learning for robot manipulation.
Specifically, LIBERO highlights five key research topics in LLDM: 1) how to
efficiently transfer declarative knowledge, procedural knowledge, or the
mixture of both; 2) how to design effective policy architectures and 3)
effective algorithms for LLDM; 4) the robustness of a lifelong learner with
respect to task ordering; and 5) the effect of model pretraining for LLDM. We
develop an extendible procedural generation pipeline that can in principle
generate infinitely many tasks. For benchmarking purpose, we create four task
suites (130 tasks in total) that we use to investigate the above-mentioned
research topics. To support sample-efficient learning, we provide high-quality
human-teleoperated demonstration data for all tasks. Our extensive experiments
present several insightful or even unexpected discoveries: sequential
finetuning outperforms existing lifelong learning methods in forward transfer,
no single visual encoder architecture excels at all types of knowledge
transfer, and naive supervised pretraining can hinder agents' performance in
the subsequent LLDM. Check the website at this https URL for
the code and the datasets.

LIBERO 是一个新的机器人操作的终身学习基准，提出了五个核心研究主题: （1）如何有效地传递申明性知识、程序性知识或两者混合；（2）如何设计有效的政策架构和决策算法；（3）终身学习与任务排序的关系；（4）模型预训练对终身学习的影响；（5）如何在终身学习中进行 Sample efficient learning

LIBERO：针对终身机器人学习的知识转移基准测试

LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning

ClueWeb22, the newest iteration of the ClueWeb line of datasets, provides 10
billion web pages affiliated with rich information. Its design was influenced
by the need for a high quality, large scale web corpus to support a range of
academic and industry research, for example, in information systems,
retrieval-augmented AI systems, and model pretraining. Compared with earlier
ClueWeb corpora, the ClueWeb22 corpus is larger, more varied, of
higher-quality, and aligned with the document distributions in commercial web
search. Besides raw HTML, ClueWeb22 includes rich information about the web
pages provided by industry-standard document understanding systems, including
the visual representation of pages rendered by a web browser, parsed HTML
structure information from a neural network parser, and pre-processed cleaned
document text to lower the barrier to entry. Many of these signals have been
widely used in industry but are available to the research community for the
first time at this scale.

ClueWeb22 是一个高质量的、大规模的 Web 语料库，旨在支持信息系统、检索 - 增强 AI 系统和模型预训练等领域的学术和工业研究。

ClueWeb22: 100 亿个带视觉和语义信息的网络文档

ClueWeb22: 10 Billion Web Documents with Visual and Semantic Information

Recent progress in natural language processing has been driven by advances in
both model architecture and model pretraining. Transformer architectures have
facilitated building higher-capacity models and pretraining has made it
possible to effectively utilize this capacity for a wide variety of tasks.
\textit{Transformers} is an open-source library with the goal of opening up
these advances to the wider machine learning community. The library consists of
carefully engineered state-of-the art Transformer architectures under a unified
API. Backing this library is a curated collection of pretrained models made by
and available for the community. \textit{Transformers} is designed to be
extensible by researchers, simple for practitioners, and fast and robust in
industrial deployments. The library is available at
https://github.com/huggingface/transformers.

自然语言处理的进展主要归功于模型架构和预训练的进步，Transformer 架构为构建更高性能的模型提供了便利，预训练使得这些模型能够在各种任务中被有效利用。Transformers 是一个旨在向更广泛的机器学习社区推广这些进展的开源库，库中包含一系列经过精心设计的 Transformer 架构和社区制作和提供的预训练模型，并且旨在为研究人员提供可扩展性、为从业者提供简便性、为产业部署提供快速和稳健性。