Spiking Neural Networks (SNNs) are recognized as the candidate for the
next-generation neural networks due to their bio-plausibility and energy
efficiency. Recently, researchers have demonstrated that SNNs are able to
achieve nearly state-of-the-art performance in image recognition tasks using
surrogate gradient training. However, some essential questions exist pertaining
to SNNs that are little studied: Do SNNs trained with surrogate gradient learn
different representations from traditional Artificial Neural Networks (ANNs)?
Does the time dimension in SNNs provide unique representation power? In this
paper, we aim to answer these questions by conducting a representation
similarity analysis between SNNs and ANNs using Centered Kernel Alignment
(CKA). We start by analyzing the spatial dimension of the networks, including
both the width and the depth. Furthermore, our analysis of residual connections
shows that SNNs learn a periodic pattern, which rectifies the representations
in SNNs to be ANN-like. We additionally investigate the effect of the time
dimension on SNN representation, finding that deeper layers encourage more
dynamics along the time dimension. We also investigate the impact of input data
such as event-stream data and adversarial attacks. Our work uncovers a host of
new findings of representations in SNNs. We hope this work will inspire future
research to fully comprehend the representation power of SNNs. Code is released
at this https URL

本文对比分析了脉冲神经网络和人工神经网络在表示相似性方面的表现，发现通过 CKA 的分析可以发现脉冲神经网络有类似于 ANN 的周期模式，并且深层次的脉冲神经网络对时间维度的动态表示更加有效。

使用替代梯度训练的脉冲神经网络表征的揭示

Uncovering the Representation of Spiking Neural Networks Trained with  Surrogate Gradient

Predicting accurate future trajectories of multiple agents is essential for
autonomous systems, but is challenging due to the complex agent interaction and
the uncertainty in each agent's future behavior. Forecasting multi-agent
trajectories requires modeling two key dimensions: (1) time dimension, where we
model the influence of past agent states over future states; (2) social
dimension, where we model how the state of each agent affects others. Most
prior methods model these two dimensions separately, e.g., first using a
temporal model to summarize features over time for each agent independently and
then modeling the interaction of the summarized features with a social model.
This approach is suboptimal since independent feature encoding over either the
time or social dimension can result in a loss of information. Instead, we would
prefer a method that allows an agent's state at one time to directly affect
another agent's state at a future time. To this end, we propose a new
Transformer, AgentFormer, that jointly models the time and social dimensions.
The model leverages a sequence representation of multi-agent trajectories by
flattening trajectory features across time and agents. Since standard attention
operations disregard the agent identity of each element in the sequence,
AgentFormer uses a novel agent-aware attention mechanism that preserves agent
identities by attending to elements of the same agent differently than elements
of other agents. Based on AgentFormer, we propose a stochastic multi-agent
trajectory prediction model that can attend to features of any agent at any
previous timestep when inferring an agent's future position. The latent intent
of all agents is also jointly modeled, allowing the stochasticity in one
agent's behavior to affect other agents. Our method substantially improves the
state of the art on well-established pedestrian and autonomous driving
datasets.

论文提出了一个新的 Transformer 模型 AgentFormer，该模型可以联合模拟时间和社交两个维度，用于预测多智能体行为，同时考虑了多智能体的影响以及不确定性。该模型具有良好的性能，并在公共数据集上取得了 state-of-the-art 的结果。