This report outlines our approach in the IEEE SPS VIP Cup 2023: Ophthalmic
Biomarker Detection competition. Our primary objective in this competition was
to identify biomarkers from Optical Coherence Tomography (OCT) images obtained
from a diverse range of patients. Using robust augmentations and 5-fold
cross-validation, we trained two vision transformer-based models: MaxViT and
EVA-02, and ensembled them at inference time. We find MaxViT's use of
convolution layers followed by strided attention to be better suited for the
detection of local features while EVA-02's use of normal attention mechanism
and knowledge distillation is better for detecting global features. Ours was
the best-performing solution in the competition, achieving a patient-wise F1
score of 0.814 in the first phase and 0.8527 in the second and final phase of
VIP Cup 2023, scoring 3.8% higher than the next-best solution.

使用视觉变换器模型 MaxViT 和 EVA-02，该报告概述了我们在 IEEE SPS VIP Cup 2023: Ophthalmic Biomarker Detection 竞赛中的方法，通过从多样化病人获得的光学相干断层扫描图像中识别生物标志物，使我们的解决方案成为竞赛中表现最佳的解决方案。

眼科生物标志物检测方法基于集成视觉变形器 ——IEEE SPS VIP Cup 2023 的获胜解决方案

Ophthalmic Biomarker Detection Using Ensembled Vision Transformers --  Winning Solution to IEEE SPS VIP Cup 2023

We launch EVA-02, a next-generation Transformer-based visual representation
pre-trained to reconstruct strong and robust language-aligned vision features
via masked image modeling. With an updated plain Transformer architecture as
well as extensive pre-training from an open & accessible giant CLIP vision
encoder, EVA-02 demonstrates superior performance compared to prior
state-of-the-art approaches across various representative vision tasks, while
utilizing significantly fewer parameters and compute budgets. Notably, using
exclusively publicly accessible training data, EVA-02 with only 304M parameters
achieves a phenomenal 90.0 fine-tuning top-1 accuracy on ImageNet-1K val set.
Additionally, our EVA-02-CLIP can reach up to 80.4 zero-shot top-1 on
ImageNet-1K, outperforming the previous largest & best open-sourced CLIP with
only ~1/6 parameters and ~1/6 image-text training data. We offer four EVA-02
variants in various model sizes, ranging from 6M to 304M parameters, all with
impressive performance. To facilitate open access and open research, we release
the complete suite of EVA-02 to the community at
this https URL.

通过使用 EVA-02，一款基于 Transformer 的视觉表示预训练算法，研究人员证明其在各种代表性视觉任务中具有卓越的性能，并且使用更少的参数和计算预算。