Summarization systems make numerous "decisions" about summary properties
during inference, e.g. degree of copying, specificity and length of outputs,
etc. However, these are implicitly encoded within model parameters and specific
styles cannot be enforced. To address this, we introduce HydraSum, a new
summarization architecture that extends the single decoder framework of current
models to a mixture-of-experts version with multiple decoders. We show that
HydraSum's multiple decoders automatically learn contrasting summary styles
when trained under the standard training objective without any extra
supervision. Through experiments on three summarization datasets (CNN, Newsroom
and XSum), we show that HydraSum provides a simple mechanism to obtain
stylistically-diverse summaries by sampling from either individual decoders or
their mixtures, outperforming baseline models. Finally, we demonstrate that a
small modification to the gating strategy during training can enforce an even
stricter style partitioning, e.g. high- vs low-abstractiveness or high- vs
low-specificity, allowing users to sample from a larger area in the generation
space and vary summary styles along multiple dimensions.

HydraSum 是一种新的总结架构，通过多个解码器自动学习不同样式的摘要并提供了一种简单机制以获得样式多样的摘要，同时使用训练期间的小改动可以实现对生成空间多个维度的变化。