Hyperspectral image segmentation is crucial for many fields such as
agriculture, remote sensing, biomedical imaging, battlefield sensing and
astronomy. However, the challenge of hyper and multi spectral imaging is its
large data footprint. We propose both a novel camera design and a vision
transformer-based (ViT) algorithm that alleviate both the captured data
footprint and the computational load for hyperspectral segmentation. Our camera
is able to adaptively sample image regions or patches at different resolutions,
instead of capturing the entire hyperspectral cube at one high resolution. Our
segmentation algorithm works in concert with the camera, applying ViT-based
segmentation only to adaptively selected patches. We show results both in
simulation and on a real hardware platform demonstrating both accurate
segmentation results and reduced computational burden.

提出一种通过新颖的相机设计和基于 ViT 的算法来减轻高光谱分割的数据占用和计算负担的方法。相机能够自适应地在不同分辨率下采样图像区域或补丁，而不是以高分辨率捕获整个高光谱立方体。分割算法与相机协同工作，仅将基于 ViT 的分割应用于自适应选择的补丁。通过模拟和实际硬件平台展示了精确的分割结果和减少的计算负担。