Generative retrieval is a promising new paradigm in text retrieval that
generates identifier strings of relevant passages as the retrieval target. This
paradigm leverages powerful generative language models, distinct from
traditional sparse or dense retrieval methods. In this work, we identify a
viable direction to further enhance generative retrieval via distillation and
propose a feasible framework, named DGR. DGR utilizes sophisticated ranking
models, such as the cross-encoder, in a teacher role to supply a passage rank
list, which captures the varying relevance degrees of passages instead of
binary hard labels; subsequently, DGR employs a specially designed distilled
RankNet loss to optimize the generative retrieval model, considering the
passage rank order provided by the teacher model as labels. This framework only
requires an additional distillation step to enhance current generative
retrieval systems and does not add any burden to the inference stage. We
conduct experiments on four public datasets, and the results indicate that DGR
achieves state-of-the-art performance among the generative retrieval methods.
Additionally, DGR demonstrates exceptional robustness and generalizability with
various teacher models and distillation losses.

通过蒸馏技术，提出了一种名为 DGR 的可行框架，利用排名模型作为教师角色，通过专门设计的蒸馏 RankNet loss 来优化生成式检索模型，从而提升了当前生成式检索系统的性能。

蒸馏增强的生成式检索

Distillation Enhanced Generative Retrieval

Quantum hardware suffers from high error rates and noise, which makes
directly running applications on them ineffective. Quantum Error Correction
(QEC) is a critical technique towards fault tolerance which encodes the quantum
information distributively in multiple data qubits and uses syndrome qubits to
check parity. Minimum-Weight-Perfect-Matching (MWPM) is a popular QEC decoder
that takes the syndromes as input and finds the matchings between syndromes
that infer the errors. However, there are two paramount challenges for MWPM
decoders. First, as noise in real quantum systems can drift over time, there is
a potential misalignment with the decoding graph's initial weights, leading to
a severe performance degradation in the logical error rates. Second, while the
MWPM decoder addresses independent errors, it falls short when encountering
correlated errors typical on real hardware, such as those in the 2Q
depolarizing channel.
We propose DGR, an efficient decoding graph edge re-weighting strategy with
no quantum overhead. It leverages the insight that the statistics of matchings
across decoding iterations offer rich information about errors on real quantum
hardware. By counting the occurrences of edges and edge pairs in decoded
matchings, we can statistically estimate the up-to-date probabilities of each
edge and the correlations between them. The reweighting process includes two
vital steps: alignment re-weighting and correlation re-weighting. The former
updates the MWPM weights based on statistics to align with actual noise, and
the latter adjusts the weight considering edge correlations.
Extensive evaluations on surface code and honeycomb code under various
settings show that DGR reduces the logical error rate by 3.6x on average-case
noise mismatch with exceeding 5000x improvement under worst-case mismatch.

提出了一种无量子开销的解码图形边缘重新加权策略 DGR，通过统计解码匹配过程中的边缘出现次数和边缘之间的相关性，对 MWPM 权重进行更新，从而在真实的量子硬件上降低逻辑错误率。