This paper explores the task of radiology report generation, which aims at
generating free-text descriptions for a set of radiographs. One significant
challenge of this task is how to correctly maintain the consistency between the
images and the lengthy report. Previous research explored solving this issue
through planning-based methods, which generate reports only based on high-level
plans. However, these plans usually only contain the major observations from
the radiographs (e.g., lung opacity), lacking much necessary information, such
as the observation characteristics and preliminary clinical diagnoses. To
address this problem, the system should also take the image information into
account together with the textual plan and perform stronger reasoning during
the generation process. In this paper, we propose an observation-guided
radiology report generation framework (ORGAN). It first produces an observation
plan and then feeds both the plan and radiographs for report generation, where
an observation graph and a tree reasoning mechanism are adopted to precisely
enrich the plan information by capturing the multi-formats of each observation.
Experimental results demonstrate that our framework outperforms previous
state-of-the-art methods regarding text quality and clinical efficacy

本文提出了一种以观察为导向的放射学报告生成框架（ORGAN），其通过采用观察图和树型推理机制，将图像信息与文本计划结合进行报告生成，其实验结果表明，相比先前最先进的方法，该框架在文本质量和临床疗效方面表现更佳。

基于树形推理的观察引导放射学报告生成

ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning

What does it take to design a machine that learns to answer natural questions
about a video? A Video QA system must simultaneously understand language,
represent visual content over space-time, and iteratively transform these
representations in response to lingual content in the query, and finally
arriving at a sensible answer. While recent advances in lingual and visual
question answering have enabled sophisticated representations and neural
reasoning mechanisms, major challenges in Video QA remain on dynamic grounding
of concepts, relations and actions to support the reasoning process. Inspired
by the dual-process account of human reasoning, we design a dual process neural
architecture, which is composed of a question-guided video processing module
(System 1, fast and reactive) followed by a generic reasoning module (System 2,
slow and deliberative). System 1 is a hierarchical model that encodes visual
patterns about objects, actions and relations in space-time given the textual
cues from the question. The encoded representation is a set of high-level
visual features, which are then passed to System 2. Here multi-step inference
follows to iteratively chain visual elements as instructed by the textual
elements. The system is evaluated on the SVQA (synthetic) and TGIF-QA datasets
(real), demonstrating competitive results, with a large margin in the case of
multi-step reasoning.

本研究提出了一种基于双过程神经架构的视频问答系统，该系统能够理解语言、表示视频内容，并通过多步推理过程得出答案