This work provides empirical evidence that Mamba, a newly proposed selective
structured state space model, has similar in-context learning (ICL)
capabilities as transformers. We evaluated Mamba on tasks involving simple
function approximation as well as more complex natural language processing
problems. Our results demonstrate that across both categories of tasks, Mamba
matches the performance of transformer models for ICL. Further analysis reveals
that like transformers, Mamba appears to solve ICL problems by incrementally
optimizing its internal representations. Overall, our work suggests that Mamba
can be an efficient alternative to transformers for ICL tasks involving longer
input sequences.

这项研究提供了实证证据，证明了一种新提出的选择性结构化状态空间模型 Mamba 具有类似于 transformers 的上下文学习（ICL）能力。我们在涉及简单函数逼近和更复杂的自然语言处理问题的任务上评估了 Mamba。我们的结果表明，在两类任务中，Mamba 与 transformer 模型在 ICL 方面的性能相当。进一步的分析表明，像 transformer 一样，Mamba 似乎通过逐步优化其内部表示来解决 ICL 问题。总体而言，我们的研究表明，对于涉及更长输入序列的 ICL 任务，Mamba 可以作为 transformers 的高效替代品。