This paper introduces ASTRA, a novel method for improving Automatic Speech
Recognition (ASR) through text injection.Unlike prevailing techniques, ASTRA
eliminates the need for sampling to match sequence lengths between speech and
text modalities. Instead, it leverages the inherent alignments learned within
CTC/RNNT models. This approach offers the following two advantages, namely,
avoiding potential misalignment between speech and text features that could
arise from upsampling and eliminating the need for models to accurately predict
duration of sub-word tokens. This novel formulation of modality (length)
matching as a weighted RNNT objective matches the performance of the
state-of-the-art duration-based methods on the FLEURS benchmark, while opening
up other avenues of research in speech processing.

该论文介绍了 ASTRA，一种通过文本注入来改进自动语音识别（ASR）的新方法，它消除了在语音和文本模态之间匹配序列长度时采样的需要，并利用 CTC/RNNT 模型内部学习到的固有对齐性。这种方法有两个优点，即避免由上采样导致的语音和文本特征之间潜在的对齐问题，以及消除模型准确预测子词标记持续时间的需求。该新颖的模态（长度）匹配公式将加权的 RNNT 目标与基于持续时间方法在 FLEURS 基准测试上的最新技术性能相匹配，同时开辟了语音处理领域的其他研究方向。

ASTRA：无需采样的 ASR 语音和文本表示对齐

ASTRA: Aligning Speech and Text Representations for Asr without Sampling

Recent advances in multimodal single-cell technologies have enabled
simultaneous acquisitions of multiple omics data from the same cell, providing
deeper insights into cellular states and dynamics. However, it is challenging
to learn the joint representations from the multimodal data, model the
relationship between modalities, and, more importantly, incorporate the vast
amount of single-modality datasets into the downstream analyses. To address
these challenges and correspondingly facilitate multimodal single-cell data
analyses, three key tasks have been introduced: $\textit{modality prediction}$,
$\textit{modality matching}$ and $\textit{joint embedding}$. In this work, we
present a general Graph Neural Network framework $\textit{scMoGNN}$ to tackle
these three tasks and show that $\textit{scMoGNN}$ demonstrates superior
results in all three tasks compared with the state-of-the-art and conventional
approaches. Our method is an official winner in the overall ranking of
$\textit{Modality prediction}$ from NeurIPS 2021 Competition, and all
implementations of our methods have been integrated into DANCE
package~https://github.com/OmicsML/dance.

本文介绍了一种名为 scMoGNN 的图神经网络框架，用于多模态单细胞技术数据的模态预测，模态匹配和联合嵌入，取得了 SOTA 的成果，并在 NeurIPS 2021 竞赛中获胜，所有实现都已整合到 DANCE 软件包中。