Our goal is to capture the pose of neuroscience model organisms, without
using any manual supervision, to be able to study how neural circuits
orchestrate behaviour. Human pose estimation attains remarkable accuracy when
trained on real or simulated datasets consisting of millions of frames.
However, for many applications simulated models are unrealistic and real
training datasets with comprehensive annotations do not exist. We address this
problem with a new sim2real domain transfer method. Our key contribution is the
explicit and independent modeling of appearance, shape and poses in an unpaired
image translation framework. Our model lets us train a pose estimator on the
target domain by transferring readily available body keypoint locations from
the source domain to generated target images. We compare our approach with
existing domain transfer methods and demonstrate improved pose estimation
accuracy on Drosophila melanogaster (fruit fly), Caenorhabditis elegans (worm)
and Danio rerio (zebrafish), without requiring any manual annotation on the
target domain and despite using simplistic off-the-shelf animal characters for
simulation, or simple geometric shapes as models. Our new datasets, code, and
trained models will be published to support future neuroscientific studies.

通过新的 sim2real 域转移方法，我们在未配对的图像转换框架中对外观、形状和姿势进行了显式和独立的建模，并在无需手动注释目标域的情况下，在 Drosophila melanogaster（果蝇）、Caenorhabditis elegans（蛔虫）和 Danio rerio（斑马鱼）上实现了改进的姿势估计准确性，从而捕捉神经回路编排行为的姿势。