Human pose estimation in trampoline gymnastics: how to improve performance on extreme poses

Trampoline gymnastics involves extreme human poses and uncommon viewpoints, on which state-of-the art pose estimation models tend to under-perform. We demonstrate that this problem can be addressed by fine-tuning a pose estimation model on a combination of real extreme poses and domain-specific synthetic poses (STP). We generate STP from motion capture recordings of trampoline routines. We propose a pipeline to fit noisy motion capture data to a parametric human model, then generate multi-view realistic images with high-fidelity keypoint labels. The fine-tuned ViTPose model tested on real multi-view images exhibits accuracy improvements in 2D which translate to improved 3D triangulation. In 2D, we obtain a performance similar to state-of-the-art models on the MS COCO validation set while evaluating on significantly more challenging data, bridging the performance gap between common and extreme poses. In 3D, we reduce the MPJPE by 46.1 mm with our best model, which represents an improvement of 42.7% compared to the pretrained ViTPose model.