In the past few years, in the context of fully-supervised semantic
segmentation, several losses -- such as cross-entropy and dice -- have emerged
as de facto standards to supervise neural networks. The Dice loss is an
interesting case, as it comes from the relaxation of the popular Dice
coefficient; one of the main evaluation metric in medical imaging applications.
In this paper, we first study theoretically the gradient of the dice loss,
showing that concretely it is a weighted negative of the ground truth, with a
very small dynamic range. This enables us, in the second part of this paper, to
mimic the supervision of the dice loss, through a simple element-wise
multiplication of the network output with a negative of the ground truth. This
rather surprising result sheds light on the practical supervision performed by
the dice loss during gradient descent. This can help the practitioner to
understand and interpret results while guiding researchers when designing new
losses.

研究神经网络的监督语义分割的标准之一 --Dice 损失函数的梯度，并表明它具有非常小的动态范围，进而通过简单的元素乘法模拟它的监督方法，这一结论有助于理解和解释结果同时指导新损失函数的设计。

论骰子损失函数的梯度及其模拟方法

On the dice loss gradient and the ways to mimic it

Albeit the Dice loss is one of the dominant loss functions in medical image
segmentation, most research omits a closer look at its derivative, i.e. the
real motor of the optimization when using gradient descent. In this paper, we
highlight the peculiar action of the Dice loss in the presence of missing or
empty labels. First, we formulate a theoretical basis that gives a general
description of the Dice loss and its derivative. It turns out that the choice
of the reduction dimensions $\Phi$ and the smoothing term $\epsilon$ is
non-trivial and greatly influences its behavior. We find and propose heuristic
combinations of $\Phi$ and $\epsilon$ that work in a segmentation setting with
either missing or empty labels. Second, we empirically validate these findings
in a binary and multiclass segmentation setting using two publicly available
datasets. We confirm that the choice of $\Phi$ and $\epsilon$ is indeed
pivotal. With $\Phi$ chosen such that the reductions happen over a single batch
(and class) element and with a negligible $\epsilon$, the Dice loss deals with
missing labels naturally and performs similarly compared to recent adaptations
specific for missing labels. With $\Phi$ chosen such that the reductions happen
over multiple batch elements or with a heuristic value for $\epsilon$, the Dice
loss handles empty labels correctly. We believe that this work highlights some
essential perspectives and hope that it encourages researchers to better
describe their exact implementation of the Dice loss in future work.

本文通过理论分析与实验验证的方法，提出了一些关于 Dice Loss 函数及其导数的最佳实践方法，以实现对于医疗图像中的缺失标签和空标签的有效分割，在此过程中重点考虑了降维维度和平滑项的取值对于 Dice Loss 函数表现的影响。

缺失或空标签情况下的 Dice Loss: 引入 $Φ$ 和 $ε$

The Dice loss in the context of missing or empty labels: Introducing $Φ$ and $ε$

Many NLP tasks such as tagging and machine reading comprehension are faced
with the severe data imbalance issue: negative examples significantly outnumber
positive examples, and the huge number of background examples (or easy-negative
examples) overwhelms the training. The most commonly used cross entropy (CE)
criteria is actually an accuracy-oriented objective, and thus creates a
discrepancy between training and test: at training time, each training instance
contributes equally to the objective function, while at test time F1 score
concerns more about positive examples. In this paper, we propose to use dice
loss in replacement of the standard cross-entropy objective for data-imbalanced
NLP tasks. Dice loss is based on the Sorensen-Dice coefficient or Tversky
index, which attaches similar importance to false positives and false
negatives, and is more immune to the data-imbalance issue. To further alleviate
the dominating influence from easy-negative examples in training, we propose to
associate training examples with dynamically adjusted weights to deemphasize
easy-negative examples.Theoretical analysis shows that this strategy narrows
down the gap between the F1 score in evaluation and the dice loss in training.
With the proposed training objective, we observe significant performance boost
on a wide range of data imbalanced NLP tasks. Notably, we are able to achieve
SOTA results on CTB5, CTB6 and UD1.4 for the part of speech tagging task; SOTA
results on CoNLL03, OntoNotes5.0, MSRA and OntoNotes4.0 for the named entity
recognition task; along with competitive results on the tasks of machine
reading comprehension and paraphrase identification.

本研究提出了一种新的 dice loss 网络训练目标替代 cross-entropy 目标，以解决 NLP 任务中数据失衡的问题，并且在多项任务中均取得了显著的性能提升。