Featurizing microscopy images for use in biological research remains a
significant challenge, especially for large-scale experiments spanning millions
of images. This work explores the scaling properties of weakly supervised
classifiers and self-supervised masked autoencoders (MAEs) when training with
increasingly larger model backbones and microscopy datasets. Our results show
that ViT-based MAEs outperform weakly supervised classifiers on a variety of
tasks, achieving as much as a 11.5% relative improvement when recalling known
biological relationships curated from public databases. Additionally, we
develop a new channel-agnostic MAE architecture (CA-MAE) that allows for
inputting images of different numbers and orders of channels at inference time.
We demonstrate that CA-MAEs effectively generalize by inferring and evaluating
on a microscopy image dataset (JUMP-CP) generated under different experimental
conditions with a different channel structure than our pretraining data
(RPI-93M). Our findings motivate continued research into scaling
self-supervised learning on microscopy data in order to create powerful
foundation models of cellular biology that have the potential to catalyze
advancements in drug discovery and beyond.

在生物研究中，将显微镜图像分析成特征仍然是一个重大挑战。本研究探讨了弱监督分类器和自监督掩码自编码器（MAEs）在使用越来越大的模型和显微镜数据集进行训练时的伸缩性能。我们的研究结果表明，基于 ViT 的 MAEs 在多项任务上优于弱监督分类器，在从公共数据库中获取的已知生物关系的召回中相对改进了 11.5%。此外，我们还开发了一种新的通道无关的 MAE 架构（CA-MAE），可以在推理时输入具有不同通道数和顺序的图像。我们证明了 CA-MAEs 通过在具有不同实验条件和通道结构的显微镜图像数据集（JUMP-CP）上进行推断和评估的方法有效地泛化。我们的发现促使继续在显微镜数据上进行自监督学习的规模扩展研究，以创建能够催化药物发现等领域进展的细胞生物学强大基础模型。

显微镜遮蔽自编码器是可扩展的细胞生物学学习模型

Masked Autoencoders for Microscopy are Scalable Learners of Cellular  Biology

We approached the goal of applying meta-learning to self-supervised masked
autoencoders for spatiotemporal learning in three steps. Broadly, we seek to
understand the impact of applying meta-learning to existing state-of-the-art
representation learning architectures. Thus, we test spatiotemporal learning
through: a meta-learning architecture only, a representation learning
architecture only, and an architecture applying representation learning
alongside a meta learning architecture. We utilize the Memory Augmented Neural
Network (MANN) architecture to apply meta-learning to our framework.
Specifically, we first experiment with applying a pre-trained MAE and
fine-tuning on our small-scale spatiotemporal dataset for video reconstruction
tasks. Next, we experiment with training an MAE encoder and applying a
classification head for action classification tasks. Finally, we experiment
with applying a pre-trained MAE and fine-tune with MANN backbone for action
classification tasks.

应用元学习于自监督遮蔽自编码器以实现时空学习的研究中，我们通过三个步骤进行探索。我们测试了应用元学习于现有先进表示学习架构的影响，并通过三种方法进行时空学习的测试，分别是仅应用元学习架构、仅应用表示学习架构，以及应用表示学习与元学习架构的组合。我们使用 Memory Augmented Neural Network (MANN) 架构来应用元学习于我们的框架。具体地，我们首先实验了在小规模时空数据集上应用预训练的 MAE 进行视频重构任务的微调。接下来，我们尝试训练 MAE 编码器并应用分类头来进行动作分类任务。最后，我们实验了在具备 MANN 骨干网络的情况下，应用预训练的 MAE 进行动作分类任务的微调。