Digital technologies have positively transformed society, but they have also
led to undesirable consequences not anticipated at the time of design or
development. We posit that insights into past undesirable consequences can help
researchers and practitioners gain awareness and anticipate potential adverse
effects. To test this assumption, we introduce BLIP, a system that extracts
real-world undesirable consequences of technology from online articles,
summarizes and categorizes them, and presents them in an interactive, web-based
interface. In two user studies with 15 researchers in various computer science
disciplines, we found that BLIP substantially increased the number and
diversity of undesirable consequences they could list in comparison to relying
on prior knowledge or searching online. Moreover, BLIP helped them identify
undesirable consequences relevant to their ongoing projects, made them aware of
undesirable consequences they "had never considered," and inspired them to
reflect on their own experiences with technology.

数字技术正面地改变了社会，但也导致了在设计或开发时未曾预料到的不良后果。我们假设对过去的不良后果的了解可以帮助研究者和从业人员提高意识并预见潜在的不良影响。为了测试这个假设，我们引入了 BLIP 系统，该系统从在线文章中提取技术的现实世界中的不良后果，对它们进行摘要和分类，并在一个交互式的基于网络的界面中展示。在两个用户研究中，我们和 15 位来自各个计算机科学学科的研究人员发现，与依靠以往知识或在线搜索相比，BLIP 大大增加了他们能够列举的不良后果的数量和多样性。此外，BLIP 帮助他们识别与他们正在进行的项目相关的不良后果，使他们意识到他们 “从未考虑过” 的不良后果，并激发他们反思自己对技术的经验。

BLIP：促进数字技术不良后果的探索

BLIP: Facilitating the Exploration of Undesirable Consequences of  Digital Technologies

Neural captioners are typically trained to mimic human-generated references
without optimizing for any specific communication goal, leading to problems
such as the generation of vague captions. In this paper, we show that
fine-tuning an out-of-the-box neural captioner with a self-supervised
discriminative communication objective helps to recover a plain, visually
descriptive language that is more informative about image contents. Given a
target image, the system must learn to produce a description that enables an
out-of-the-box text-conditioned image retriever to identify such image among a
set of candidates. We experiment with the popular ClipCap captioner, also
replicating the main results with BLIP. In terms of similarity to ground-truth
human descriptions, the captions emerging from discriminative finetuning lag
slightly behind those generated by the non-finetuned model, when the latter is
trained and tested on the same caption dataset. However, when the model is used
without further tuning to generate captions for out-of-domain datasets, our
discriminatively-finetuned captioner generates descriptions that resemble human
references more than those produced by the same captioner without finetuning.
We further show that, on the Conceptual Captions dataset, discriminatively
finetuned captions are more helpful than either vanilla ClipCap captions or
ground-truth captions for human annotators tasked with an image discrimination
task.

本文提出在自我监督的证明性沟通目标情况下对预先训练的神经字幕系统进行微调，使其生成更详细的图像描述，并在 Conceptual Captions 数据集上进行验证。

具有判别性微调的跨领域图像字幕生成

Cross-Domain Image Captioning with Discriminative Finetuning

Vision-Language Pre-training (VLP) has advanced the performance for many
vision-language tasks. However, most existing pre-trained models only excel in
either understanding-based tasks or generation-based tasks. Furthermore,
performance improvement has been largely achieved by scaling up the dataset
with noisy image-text pairs collected from the web, which is a suboptimal
source of supervision. In this paper, we propose BLIP, a new VLP framework
which transfers flexibly to both vision-language understanding and generation
tasks. BLIP effectively utilizes the noisy web data by bootstrapping the
captions, where a captioner generates synthetic captions and a filter removes
the noisy ones. We achieve state-of-the-art results on a wide range of
vision-language tasks, such as image-text retrieval (+2.7% in average
recall@1), image captioning (+2.8% in CIDEr), and VQA (+1.6% in VQA score).
BLIP also demonstrates strong generalization ability when directly transferred
to video-language tasks in a zero-shot manner. Code, models, and datasets are
released at this https URL

本文提出了 BLIP 作为新的 VLP 框架，通过引入 captioner 生成合成字幕，并使用 filter 删除噪音数据，能灵活地传输视觉语言理解和生成任务，获得了在一系列视觉语言任务中最先进的结果，同时在零样本任务中也表现出极强的泛化能力。