Automatic diagnosis has attracted increasing attention but remains
challenging due to multi-step reasoning. Recent works usually address it by
reinforcement learning methods. However, these methods show low efficiency and
require taskspecific reward functions. Considering the conversation between
doctor and patient allows doctors to probe for symptoms and make diagnoses, the
diagnosis process can be naturally seen as the generation of a sequence
including symptoms and diagnoses. Inspired by this, we reformulate automatic
diagnosis as a symptoms Sequence Generation (SG) task and propose a simple but
effective automatic Diagnosis model based on Transformer (Diaformer). We
firstly design the symptom attention framework to learn the generation of
symptom inquiry and the disease diagnosis. To alleviate the discrepancy between
sequential generation and disorder of implicit symptoms, we further design
three orderless training mechanisms. Experiments on three public datasets show
that our model outperforms baselines on disease diagnosis by 1%, 6% and 11.5%
with the highest training efficiency. Detailed analysis on symptom inquiry
prediction demonstrates that the potential of applying symptoms sequence
generation for automatic diagnosis.

通过考虑医生和病人之间的对话，我们将自动诊断重新定义为症状序列生成任务，并基于变压器（Diaformer）提出了一个简单而有效的自动诊断模型，采用症状注意机制和三种无序的训练机制来提高自动诊断的准确性。在三个公共数据集上的实验结果表明，Diaformer 模型在疾病诊断方面优于基线模型，准确率分别提高了 1%、6%、11.5%。

Diaformer: 症状序列生成自动诊断

Diaformer: Automatic Diagnosis via Symptoms Sequence Generation

Conducting text retrieval in a dense learned representation space has many
intriguing advantages over sparse retrieval. Yet the effectiveness of dense
retrieval (DR) often requires combination with sparse retrieval. In this paper,
we identify that the main bottleneck is in the training mechanisms, where the
negative instances used in training are not representative of the irrelevant
documents in testing. This paper presents Approximate nearest neighbor Negative
Contrastive Estimation (ANCE), a training mechanism that constructs negatives
from an Approximate Nearest Neighbor (ANN) index of the corpus, which is
parallelly updated with the learning process to select more realistic negative
training instances. This fundamentally resolves the discrepancy between the
data distribution used in the training and testing of DR. In our experiments,
ANCE boosts the BERT-Siamese DR model to outperform all competitive dense and
sparse retrieval baselines. It nearly matches the accuracy of
sparse-retrieval-and-BERT-reranking using dot-product in the ANCE-learned
representation space and provides almost 100x speed-up.

本文提出了一种新型的训练机制 --- ANCE (Approximate nearest neighbor Negative Contrastive Estimation), 它使用了一个近似最近邻索引来构建负面示例，以更好地解决稠密检索中负面实例训练不足的问题，并且通过实验证明了其在 BERT 模型上的有效性能达到了最佳效果。