Large-scale language-agnostic sentence embedding models such as LaBSE (Feng
et al., 2022) obtain state-of-the-art performance for parallel sentence
alignment. However, these large-scale models can suffer from inference speed
and computation overhead. This study systematically explores learning
language-agnostic sentence embeddings with lightweight models. We demonstrate
that a thin-deep encoder can construct robust low-dimensional sentence
embeddings for 109 languages. With our proposed distillation methods, we
achieve further improvements by incorporating knowledge from a teacher model.
Empirical results on Tatoeba, United Nations, and BUCC show the effectiveness
of our lightweight models. We release our lightweight language-agnostic
sentence embedding models LEALLA on TensorFlow Hub.

本研究使用轻量级模型系统地探索学习通用句子嵌入的方法，证明了一种深薄编码器可以构建 109 种语言的强韧低维句子嵌入，并通过引入知识蒸馏方法进一步提高模型性能。实验证明，如此轻量级模型在 Tatoeba，联合国和 BUCC 上展现了出色的效果，并在 TensorFlow Hub 上发布了名为 LEALLA 的语言无关句子嵌入模型。

LEALLA: 通过知识蒸馏学习轻量级、语言无关的句子嵌入

LEALLA: Learning Lightweight Language-agnostic Sentence Embeddings with Knowledge Distillation

The problem of reversing the compilation process, decompilation, is an
important tool in reverse engineering of computer software. Recently,
researchers have proposed using techniques from neural machine translation to
automate the process in decompilation. Although such techniques hold the
promise of targeting a wider range of source and assembly languages, to date
they have primarily targeted C code. In this paper we argue that existing
neural decompilers have achieved higher accuracy at the cost of requiring
language-specific domain knowledge such as tokenizers and parsers to build an
abstract syntax tree (AST) for the source language, which increases the
overhead of supporting new languages. We explore a different tradeoff that, to
the extent possible, treats the assembly and source languages as plain text,
and show that this allows us to build a decompiler that is easily retargetable
to new languages. We evaluate our prototype decompiler, Beyond The C (BTC), on
Go, Fortran, OCaml, and C, and examine the impact of parameters such as
tokenization and training data selection on the quality of decompilation,
finding that it achieves comparable decompilation results to prior work in
neural decompilation with significantly less domain knowledge. We will release
our training data, trained decompilation models, and code to help encourage
future research into language-agnostic decompilation.

使用神经机器翻译技术进行反编译，可以使得反编译的支持范围更广，但过度的语言特定领域知识会增加支持新语言的负担。因此，研究者提出了一种使用神经机器翻译将程序汇编语言和源码语言看作纯文本的方案，并成功地应用于多种语言的反编译中。在参数调整和评估中，该方案相比其他基于神经机器翻译的反编译技术，在较少领域知识的情况下实现了相当的反编译精度。

通过神经机器翻译实现可重定向反汇编

Beyond the C: Retargetable Decompilation using Neural Machine Translation

We present LAReQA, a challenging new benchmark for language-agnostic answer
retrieval from a multilingual candidate pool. Unlike previous cross-lingual
tasks, LAReQA tests for "strong" cross-lingual alignment, requiring
semantically related cross-language pairs to be closer in representation space
than unrelated same-language pairs. Building on multilingual BERT (mBERT), we
study different strategies for achieving strong alignment. We find that
augmenting training data via machine translation is effective, and improves
significantly over using mBERT out-of-the-box. Interestingly, the embedding
baseline that performs the best on LAReQA falls short of competing baselines on
zero-shot variants of our task that only target "weak" alignment. This finding
underscores our claim that languageagnostic retrieval is a substantively new
kind of cross-lingual evaluation.

介绍了 LAReQA - 一项新的挑战性基准测试，可从多语言候选池中进行语言无关的答案检索。在基于多语言 BERT 的基础上，研究了实现强交叉语言对齐的不同策略，发现通过机器翻译增加训练数据是有效的，同时，该基线在 LAReQA 上表现最好，但在仅瞄准 “弱” 对齐的任务中却不及竞争基线，这突显了语言无关检索是一种实质性新型跨语言评估。