Deep Neural Networks (DNNs) have been a large driver and enabler for AI
breakthroughs in recent years. These models have been getting larger in their
attempt to become more accurate and tackle new upcoming use-cases, including
AR/VR and intelligent assistants. However, the training process of such large
models is a costly and time-consuming process, which typically yields a single
model to fit all targets. To mitigate this, various techniques have been
proposed in the literature, including pruning, sparsification or quantization
of the model weights and updates. While able to achieve high compression rates,
they often incur computational overheads or accuracy penalties. Alternatively,
factorization methods have been leveraged to incorporate low-rank compression
in the training process. Similarly, such techniques (e.g.,~SVD) frequently rely
on the computationally expensive decomposition of layers and are potentially
sub-optimal for non-linear models, such as DNNs. In this work, we take a
further step in designing efficient low-rank models and propose Maestro, a
framework for trainable low-rank layers. Instead of regularly applying a priori
decompositions such as SVD, the low-rank structure is built into the training
process through a generalized variant of Ordered Dropout. This method imposes
an importance ordering via sampling on the decomposed DNN structure. Our
theoretical analysis demonstrates that our method recovers the SVD
decomposition of linear mapping on uniformly distributed data and PCA for
linear autoencoders. We further apply our technique on DNNs and empirically
illustrate that Maestro enables the extraction of lower footprint models that
preserve model performance while allowing for graceful accuracy-latency
tradeoff for the deployment to devices of different capabilities.

设计高效的低秩模型，通过一种广义的 Ordered Dropout 方法将低秩结构内嵌到训练过程中，实现对深度神经网络进行可训练的低秩层插入，从而提取出足够小的模型，保持模型性能并允许在不同性能设备上进行准确性和延迟的权衡。

Maestro: 通过可训练分解揭示低秩结构

Maestro: Uncovering Low-Rank Structures via Trainable Decomposition

Although the prevention of AI vulnerabilities is critical to preserve the
safety and privacy of users and businesses, educational tools for robust AI are
still underdeveloped worldwide. We present the design, implementation, and
assessment of Maestro. Maestro is an effective open-source game-based platform
that contributes to the advancement of robust AI education. Maestro provides
goal-based scenarios where college students are exposed to challenging
life-inspired assignments in a competitive programming environment. We assessed
Maestro's influence on students' engagement, motivation, and learning success
in robust AI. This work also provides insights into the design features of
online learning tools that promote active learning opportunities in the robust
AI domain. We analyzed the reflection responses (measured with Likert scales)
of 147 undergraduate students using Maestro in two quarterly college courses in
AI. According to the results, students who felt the acquisition of new skills
in robust AI tended to appreciate highly Maestro and scored highly on material
consolidation, curiosity, and mastery in robust AI. Moreover, the leaderboard,
our key gamification element in Maestro, has effectively contributed to
students' engagement and learning. Results also indicate that Maestro can be
effectively adapted to any course length and depth without losing its
educational quality.

本研究设计、实施、并评估了 Maestro，一个有效的开源、以游戏为基础的平台，为促进强大的 AI 教育做出了贡献。Maestro 提供了基于目标的场景，让大学生在竞争性编程环境中面对挑战性的仿生作业。根据研究结果，学生感觉在强大的 AI 应用中获得了新技能，往往对 Maestro 评价很高，并在强大的 AI 应用的素材巩固，好奇心和掌握方面得分很高。

Maestro：教授人工智能韧性的游戏化平台

Maestro: A Gamified Platform for Teaching AI Robustness

Open-ended learning methods that automatically generate a curriculum of
increasingly challenging tasks serve as a promising avenue toward generally
capable reinforcement learning agents. Existing methods adapt curricula
independently over either environment parameters (in single-agent settings) or
co-player policies (in multi-agent settings). However, the strengths and
weaknesses of co-players can manifest themselves differently depending on
environmental features. It is thus crucial to consider the dependency between
the environment and co-player when shaping a curriculum in multi-agent domains.
In this work, we use this insight and extend Unsupervised Environment Design
(UED) to multi-agent environments. We then introduce Multi-Agent Environment
Design Strategist for Open-Ended Learning (MAESTRO), the first multi-agent UED
approach for two-player zero-sum settings. MAESTRO efficiently produces
adversarial, joint curricula over both environments and co-players and attains
minimax-regret guarantees at Nash equilibrium. Our experiments show that
MAESTRO outperforms a number of strong baselines on competitive two-player
games, spanning discrete and continuous control settings.

本文提出了 Multi-Agent Environment Design Strategist for Open-Ended Learning (MAESTRO) 算法，该算法在多智能体环境中使用 Unsupervised Environment Design (UED) 生成敌对性的联合任务集，MAESTRO 在竞争性的两人游戏中，跨离散和连续控制环境优于其他基线方法，并在 Nash 均衡点上获得了最小化后悔的保证。

MAESTRO: 多智能体强化学习的开放环境设计

MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement  Learning

We present Maestro, a self-supervised training method to unify
representations learnt from speech and text modalities. Self-supervised
learning from speech signals aims to learn the latent structure inherent in the
signal, while self-supervised learning from text attempts to capture lexical
information. Learning aligned representations from unpaired speech and text
sequences is a challenging task. Previous work either implicitly enforced the
representations learnt from these two modalities to be aligned in the latent
space through multitasking and parameter sharing or explicitly through
conversion of modalities via speech synthesis. While the former suffers from
interference between the two modalities, the latter introduces additional
complexity. In this paper, we propose Maestro, a novel algorithm to learn
unified representations from both these modalities simultaneously that can
transfer to diverse downstream tasks such as Automated Speech Recognition (ASR)
and Speech Translation (ST). Maestro learns unified representations through
sequence alignment, duration prediction and matching embeddings in the learned
space through an aligned masked-language model loss. We establish a new
state-of-the-art (SOTA) on VoxPopuli multilingual ASR with a 8% relative
reduction in Word Error Rate (WER), multidomain SpeechStew ASR (3.7% relative)
and 21 languages to English multilingual ST on CoVoST 2 with an improvement of
2.8 BLEU averaged over 21 languages.

Maestro 是一个自我监督的训练方法，以统一从语音和文本模态中学到的表示。 该算法通过序列对齐，持续预测和匹配来学习统一的表示，以用于自动语音识别（ASR）和语音翻译（ST）等下游实验。