4DFA: Four-Dimensional Full-Anatomy Reconstruction of Individualized Digital Human Models Based on Motion Videos

Abstract

Reconstructing personalized models of the human body is a key step for digital twin modeling of sports motion. There have been many studies reconstructing the body surface models based on video sequences, but these algorithms lack the modeling of internal structures (bones, muscles, blood vessels, etc.). Some researchers map internal organ structures into the 3D point cloud scanning of the body surface, but such mapping does not guarantee realistic tissue deformation and is not developed for video processing. This paper focuses on the video-based reconstruction of a personalized full-anatomy digital model in sports motion. Our method creates a four-dimensional (4D) model of the moving process composed of the 3D models of each time moment. We first use a deep network to regress the body surface of each time point and then estimate the internal structures by registering a previously constructed deformable human anatomy atlas to the body surfaces. To mimic realistic internal structure deformation, we applied articulated linear transform to the bony structures to avoid bone shape distortion and applied smooth nonlinear transform to the soft tissue to mimic the motion deformation. We also developed an intersection removal method to prevent the possible intersection between bones and soft tissues during the deformation. Our method is tested with athletes' sports videos and automatically generates full-anatomy motion models which cannot be achieved by previous methods. Our method also surpasses the existing video-based body surface modeling method by providing additional internal structures. The 4D human motion model constructed by our method is useful in sports modeling, biomechanics simulation, wearable device design, etc.