Recently, influence functions present an apparatus for achieving
explainability for deep neural models by quantifying the perturbation of
individual train instances that might impact a test prediction. Our objectives
in this paper are twofold. First we incorporate influence functions as a
feedback into the model to improve its performance. Second, in a dataset
extension exercise, using influence functions to automatically identify data
points that have been initially `silver' annotated by some existing method and
need to be cross-checked (and corrected) by annotators to improve the model
performance. To meet these objectives, in this paper, we introduce InfFeed,
which uses influence functions to compute the influential instances for a
target instance. Toward the first objective, we adjust the label of the target
instance based on its influencer(s) label. In doing this, InfFeed outperforms
the state-of-the-art baselines (including LLMs) by a maximum macro F1-score
margin of almost 4% for hate speech classification, 3.5% for stance
classification, and 3% for irony and 2% for sarcasm detection. Toward the
second objective we show that manually re-annotating only those silver
annotated data points in the extension set that have a negative influence can
immensely improve the model performance bringing it very close to the scenario
where all the data points in the extension set have gold labels. This allows
for huge reduction of the number of data points that need to be manually
annotated since out of the silver annotated extension dataset, the influence
function scheme picks up ~1/1000 points that need manual correction.

使用影响函数改进深度神经模型的性能并自动识别需要手动校正的数据点来提高模型性能，达到减少手动标注数据点数量的效果。

InfFeed: 以影响函数作为反馈来提高主观任务的性能

InfFeed: Influence Functions as a Feedback to Improve the Performance of  Subjective Tasks

Current movie captioning architectures are not capable of mentioning
characters with their proper name, replacing them with a generic "someone" tag.
The lack of movie description datasets with characters' visual annotations
surely plays a relevant role in this shortage. Recently, we proposed to extend
the M-VAD dataset by introducing such information. In this paper, we present an
improved version of the dataset, namely M-VAD Names, and its semi-automatic
annotation procedure. The resulting dataset contains 63k visual tracks and 34k
textual mentions, all associated with character identities. To showcase the
features of the dataset and quantify the complexity of the naming task, we
investigate multimodal architectures to replace the "someone" tags with proper
character names in existing video captions. The evaluation is further extended
by testing this application on videos outside of the M-VAD Names dataset.

本文介绍了 M-VAD Names 数据集以及其语义自动注释程序，并探讨了使用多模式架构将 ' 某人 ' 标签替换为适当的角色名称的应用，以解决电影字幕生成中常见的角色无名问题。