Tracking the full skeletal pose of the hands and fingers is a challenging
problem that has a plethora of applications for user interaction. Existing
techniques either require wearable hardware, add restrictions to user pose, or
require significant computation resources. This research explores a new
approach to tracking hands, or any articulated model, by using an augmented
rigid body simulation. This allows us to phrase 3D object tracking as a linear
complementarity problem with a well-defined solution. Based on a depth sensor's
samples, the system generates constraints that limit motion orthogonal to the
rigid body model's surface. These constraints, along with prior motion,
collision/contact constraints, and joint mechanics, are resolved with a
projected Gauss-Seidel solver. Due to camera noise properties and attachment
errors, the numerous surface constraints are impulse capped to avoid
overpowering mechanical constraints. To improve tracking accuracy, multiple
simulations are spawned at each frame and fed a variety of heuristics,
constraints and poses. A 3D error metric selects the best-fit simulation,
helping the system handle challenging hand motions. Such an approach enables
real-time, robust, and accurate 3D skeletal tracking of a user's hand on a
variety of depth cameras, while only utilizing a single x86 CPU core for
processing.

本研究利用增强刚体模拟方法追踪手部骨骼姿态，在利用深度传感器样本生成限制运动方式的约束的基础上，使用投影高斯 — 赛德尔求解器解决先前的运动，碰撞 / 接触限制，以及关节力学问题，实现了实时、鲁棒且准确的 3D 骨骼追踪。