Perception-based image analysis technologies can be used to help visually
impaired people take better quality pictures by providing automated guidance,
thereby empowering them to interact more confidently on social media. The
photographs taken by visually impaired users often suffer from one or both of
two kinds of quality issues: technical quality (distortions), and semantic
quality, such as framing and aesthetic composition. Here we develop tools to
help them minimize occurrences of common technical distortions, such as blur,
poor exposure, and noise. We do not address the complementary problems of
semantic quality, leaving that aspect for future work. The problem of assessing
and providing actionable feedback on the technical quality of pictures captured
by visually impaired users is hard enough, owing to the severe, commingled
distortions that often occur. To advance progress on the problem of analyzing
and measuring the technical quality of visually impaired user-generated content
(VI-UGC), we built a very large and unique subjective image quality and
distortion dataset. This new perceptual resource, which we call the LIVE-Meta
VI-UGC Database, contains $40$K real-world distorted VI-UGC images and $40$K
patches, on which we recorded $2.7$M human perceptual quality judgments and
$2.7$M distortion labels. Using this psychometric resource we also created an
automatic blind picture quality and distortion predictor that learns
local-to-global spatial quality relationships, achieving state-of-the-art
prediction performance on VI-UGC pictures, significantly outperforming existing
picture quality models on this unique class of distorted picture data. We also
created a prototype feedback system that helps to guide users to mitigate
quality issues and take better quality pictures, by creating a multi-task
learning framework.

该研究利用认知图像分析技术开发了自动化工具，旨在改善视力受损用户拍摄照片的技术质量问题，建立了一个大型的歪曲视力受损用户生成内容（VI-UGC）图像数据库，利用人类感知质量判断和失真标签进行预测模型的训练及其反馈系统的原型设计，为视力受损用户提供拍摄更高质量照片的机会。

帮助视障人士拍摄更高质量的照片

Helping Visually Impaired People Take Better Quality Pictures

Automatic image description systems are commonly trained and evaluated using
crowdsourced, human-generated image descriptions. The best-performing system is
then determined using some measure of similarity to the reference data (BLEU,
Meteor, CIDER, etc). Thus, both the quality of the systems as well as the
quality of the evaluation depends on the quality of the descriptions. As
Section 2 will show, the quality of current image description datasets is
insufficient. I argue that there is a need for more detailed guidelines that
take into account the needs of visually impaired users, but also the
feasibility of generating suitable descriptions. With high-quality data,
evaluation of image description systems could use reference descriptions, but
we should also look for alternatives.

通过产生更好的数据和寻找替代方法来改进图像描述系统的评估，因为当前的图像描述数据集质量不足。需要更详细的指导方针来考虑视觉障碍用户的需求和生成适当的描述。

使用人类参考数据评估自动图像描述

On the use of human reference data for evaluating automatic image  descriptions

Studies have shown that a dominant class of questions asked by visually
impaired users on images of their surroundings involves reading text in the
image. But today's VQA models can not read! Our paper takes a first step
towards addressing this problem. First, we introduce a new "TextVQA" dataset to
facilitate progress on this important problem. Existing datasets either have a
small proportion of questions about text (e.g., the VQA dataset) or are too
small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408
images that require reasoning about text to answer. Second, we introduce a
novel model architecture that reads text in the image, reasons about it in the
context of the image and the question, and predicts an answer which might be a
deduction based on the text and the image or composed of the strings found in
the image. Consequently, we call our approach Look, Read, Reason & Answer
(LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on
our TextVQA dataset. We find that the gap between human performance and machine
performance is significantly larger on TextVQA than on VQA 2.0, suggesting that
TextVQA is well-suited to benchmark progress along directions complementary to
VQA 2.0.

我们研究了盲人用户在看不到图片的情况下经常关注的图像内容，即图片中的文本，并介绍了一种名为 LoRRA 的新模型用于解决这个问题，同时提出了一个名为 TextVQA 的数据集来评估和改进模型表现。