Representation-based Siamese networks have risen to popularity in lightweight
text matching due to their low deployment and inference costs. While word-level
attention mechanisms have been implemented within Siamese networks to improve
performance, we propose Feature Attention (FA), a novel downstream block
designed to enrich the modeling of dependencies among embedding features.
Employing "squeeze-and-excitation" techniques, the FA block dynamically adjusts
the emphasis on individual features, enabling the network to concentrate more
on features that significantly contribute to the final classification. Building
upon FA, we introduce a dynamic "selection" mechanism called Selective Feature
Attention (SFA), which leverages a stacked BiGRU Inception structure. The SFA
block facilitates multi-scale semantic extraction by traversing different
stacked BiGRU layers, encouraging the network to selectively concentrate on
semantic information and embedding features across varying levels of
abstraction. Both the FA and SFA blocks offer a seamless integration capability
with various Siamese networks, showcasing a plug-and-play characteristic.
Experimental evaluations conducted across diverse text matching baselines and
benchmarks underscore the indispensability of modeling feature attention and
the superiority of the "selection" mechanism.

基于表征的孪生网络在轻量级文本匹配中因其低部署和推理成本而变得流行。我们提出了 Feature Attention（FA），一种新颖的下游模块，旨在丰富嵌入特征之间的依赖建模。该模块动态调整对个体特征的强调，使网络更集中于对最终分类有重要贡献的特征。在 FA 的基础上，我们引入了一种名为 Selective Feature Attention（SFA）的动态 “选择” 机制，利用了堆叠 BiGRU Inception 结构。该模块通过遍历不同的堆叠 BiGRU 层，促进网络集中关注跨不同抽象级别的语义信息和嵌入特征，从而实现多尺度语义提取。FA 和 SFA 模块都能无缝集成到各种孪生网络中，并展示了即插即用的特性。对多样的文本匹配基线和基准进行的实验评估强调了建模特征注意力的必要性以及 “选择” 机制的优越性。