Video scene parsing in the wild with diverse scenarios is a challenging and
great significance task, especially with the rapid development of automatic
driving technique. The dataset Video Scene Parsing in the Wild(VSPW) contains
well-trimmed long-temporal, dense annotation and high resolution clips. Based
on VSPW, we design a Temporal Bilateral Network with Vision Transformer. We
first design a spatial path with convolutions to generate low level features
which can preserve the spatial information. Meanwhile, a context path with
vision transformer is employed to obtain sufficient context information.
Furthermore, a temporal context module is designed to harness the inter-frames
contextual information. Finally, the proposed method can achieve the mean
intersection over union(mIoU) of 49.85\% for the VSPW2021 Challenge test
dataset.

本研究使用 VSPW 数据集设计了一个基于时空双边网络和视觉转换器的视频场景解析模型，该模型利用卷积和视觉转换器获得空间和上下文信息，并且使用时间上下文模块获取帧间上下文信息，实验证明该模型可以在 VSPW2021 挑战赛中获得 49.85% 的 mIoU。

TBN-ViT：基于视觉 Transformer 的时间双边网络用于视频场景解析

TBN-ViT: Temporal Bilateral Network with Vision Transformer for Video  Scene Parsing

Artificial intelligence (AI) is an important technology that supports daily
social life and economic activities. It contributes greatly to the sustainable
growth of Japan's economy and solves various social problems. In recent years,
AI has attracted attention as a key for growth in developed countries such as
Europe and the United States and developing countries such as China and India.
The attention has been focused mainly on developing new artificial intelligence
information communication technology (ICT) and robot technology (RT). Although
recently developed AI technology certainly excels in extracting certain
patterns, there are many limitations. Most ICT models are overly dependent on
big data, lack a self-idea function, and are complicated. In this paper, rather
than merely developing next-generation artificial intelligence technology, we
aim to develop a new concept of general-purpose intelligence cognition
technology called Beyond AI. Specifically, we plan to develop an intelligent
learning model called Brain Intelligence (BI) that generates new ideas about
events without having experienced them by using artificial life with an imagine
function. We will also conduct demonstrations of the developed BI intelligence
learning model on automatic driving, precision medical care, and industrial
robots.

该研究论文提出了一种名为 “超越人工智能” 的普适性认知技术，旨在解决当前人工智能技术的局限性，该技术使用人工生命带有想象力的能力，新颖地发掘事件的思想，并将其应用于自动驾驶、医疗护理和工业机器人等方面。