We offer philosophical motivations for a method we call Virtual World
Cognitive Science (VW CogSci), in which researchers use virtual embodied agents
that are embedded in virtual worlds to explore questions in the field of
Cognitive Science. We focus on questions about mental and linguistic
representation and the ways that such computational modeling can add rigor to
philosophical thought experiments, as well as the terminology used in the
scientific study of such representations. We find that this method forces
researchers to take a god's-eye view when describing dynamical relationships
between entities in minds and entities in an environment in a way that
eliminates the need for problematic talk of belief and concept types, such as
the belief that cats are silly, and the concept CAT, while preserving belief
and concept tokens in individual cognizers' minds. We conclude with some
further key advantages of VW CogSci for the scientific study of mental and
linguistic representation and for Cognitive Science more broadly.

我们提供一种我们称之为虚拟世界认知科学（VW CogSci）的方法的哲学动机，在这种方法中，研究人员使用嵌入虚拟世界中的虚拟具身代理来探索认知科学领域的问题。我们着重研究有关心理和语言表示以及这种计算建模如何为哲学思想实验增加严谨性的问题，以及在科学研究中所使用的术语。我们发现这种方法迫使研究人员以上帝的视角描述心智与环境中实体之间的动态关系，从而消除了有关信念和概念类型的问题性讨论，如猫是愚蠢的这种信念和 CAT 这个概念，同时保留了个体认知者心智中的信念和概念实例。最后，我们总结了 VW CogSci 在对心智和语言表示以及认知科学的科学研究方面的一些其他关键优势。

为更严谨的心理哲学和科学进行计算思维实验

Computational Thought Experiments for a More Rigorous Philosophy and  Science of the Mind

As part of human core knowledge, the representation of objects is the
building block of mental representation that supports high-level concepts and
symbolic reasoning. While humans develop the ability of perceiving objects
situated in 3D environments without supervision, models that learn the same set
of abilities with similar constraints faced by human infants are lacking.
Towards this end, we developed a novel network architecture that simultaneously
learns to 1) segment objects from discrete images, 2) infer their 3D locations,
and 3) perceive depth, all while using only information directly available to
the brain as training data, namely: sequences of images and self-motion. The
core idea is treating objects as latent causes of visual input which the brain
uses to make efficient predictions of future scenes. This results in object
representations being learned as an essential byproduct of learning to predict.

利用仅依赖于大脑可访问的信息进行训练的图像序列和自我运动，我们开发了一种新颖的网络架构，能够同时学习从离散图像中分割对象、推断其三维位置和感知深度，从而作为预测学习的副产品有效地学习对象的表示。

通过预测学习三维物体为中心的表示

Learning 3D object-centric representation through prediction

Humans possess a unique social cognition capability; nonverbal communication
can convey rich social information among agents. In contrast, such crucial
social characteristics are mostly missing in the existing scene understanding
literature. In this paper, we incorporate different nonverbal communication
cues (e.g., gaze, human poses, and gestures) to represent, model, learn, and
infer agents' mental states from pure visual inputs. Crucially, such a mental
representation takes the agent's belief into account so that it represents what
the true world state is and infers the beliefs in each agent's mental state,
which may differ from the true world states. By aggregating different beliefs
and true world states, our model essentially forms "five minds" during the
interactions between two agents. This "five minds" model differs from prior
works that infer beliefs in an infinite recursion; instead, agents' beliefs are
converged into a "common mind". Based on this representation, we further devise
a hierarchical energy-based model that jointly tracks and predicts all five
minds. From this new perspective, a social event is interpreted by a series of
nonverbal communication and belief dynamics, which transcends the classic
keyframe video summary. In the experiments, we demonstrate that using such a
social account provides a better video summary on videos with rich social
interactions compared with state-of-the-art keyframe video summary methods.

本文通过结合不同的非语言交流提示，设计了一种能够描绘、学习和推断代理人心理状态的方法，并以此为基础开发了一种新型的能够跟踪和预测所有心理状态的能级能量模型，从而提供了一种基于非语言交流和信念动态的社交事件描述和视频总结方法。