This paper addresses the problem of sentiment analysis for Jopara, a
code-switching language between Guarani and Spanish. We first collect a corpus
of Guarani-dominant tweets and discuss on the difficulties of finding quality
data for even relatively easy-to-annotate tasks, such as sentiment analysis.
Then, we train a set of neural models, including pre-trained language models,
and explore whether they perform better than traditional machine learning ones
in this low-resource setup. Transformer architectures obtain the best results,
despite not considering Guarani during pre-training, but traditional machine
learning models perform close due to the low-resource nature of the problem.

该论文研究了跨越瓜拉尼语和西班牙语的 Jopara 语言的情感分析问题，并训练了一组神经模型，探讨了它们在低资源环境下的表现。Transformer 神经网络模型表现最佳，但传统机器学习模型在这个问题的低资源情况下也表现接近最佳水平。

Jopara 情感分析的后勤难题和发现

On the logistical difficulties and findings of Jopara Sentiment Analysis

Recent years have witnessed significant improvement in ASR systems to
recognize spoken utterances. However, it is still a challenging task for noisy
and out-of-domain data, where substitution and deletion errors are prevalent in
the transcribed text. These errors significantly degrade the performance of
downstream tasks. In this work, we propose a BERT-style language model,
referred to as PhonemeBERT, that learns a joint language model with phoneme
sequence and ASR transcript to learn phonetic-aware representations that are
robust to ASR errors. We show that PhonemeBERT can be used on downstream tasks
using phoneme sequences as additional features, and also in low-resource setup
where we only have ASR-transcripts for the downstream tasks with no phoneme
information available. We evaluate our approach extensively by generating noisy
data for three benchmark datasets - Stanford Sentiment Treebank, TREC and ATIS
for sentiment, question and intent classification tasks respectively. The
results of the proposed approach beats the state-of-the-art baselines
comprehensively on each dataset.

本文提出 PhonemeBERT 模型，利用音素信息和 ASR 转录文本的联合语言模型学习语音感知表示，从而提高 ASR 系统在噪声和领域外数据中的准确性，同时为下游任务提供额外的特征，并在低资源环境下进行了广泛评估，取得了在情感分析、问题分类和意图分类等任务上显著优于目前最好方法的效果。

Phoneme-BERT: 音素序列和 ASR 转录的联合语言建模

Phoneme-BERT: Joint Language Modelling of Phoneme Sequence and ASR  Transcript

Neural encoder-decoder models of machine translation have achieved impressive
results, while learning linguistic knowledge of both the source and target
languages in an implicit end-to-end manner. We propose a framework in which our
model begins learning syntax and translation interleaved, gradually putting
more focus on translation. Using this approach, we achieve considerable
improvements in terms of BLEU score on relatively large parallel corpus (WMT14
English to German) and a low-resource (WIT German to English) setup.

提出了一种神经编码器 - 解码器机器翻译模型框架，该模型开始交替学习语法和翻译，逐渐将重点放在翻译上，实现了在相对较大的平行语料库（WMT14 英语到德语）和低资源（WIT 德语到英语）设置中的 BLEU 得分显着提高。