Recently, human motion estimation is an important task and attracting significant attention in sports and medical applications, as it presents both theoretical and practical interest from bio-mechanical, computer vision and robotics perspectives. We focus on developing algorithms to: 1) estimate gait parameters, and 2) analysis human motion during walking activity, with reduced number of sensor count. In this dissertation, gait parameters are estimated as well as human lower-limb motion is reconstructed using a single waist-mounted Intel Realsense Depth Camera D455, which has an integrated 6-DOF Inertial Measurement Unit (IMU).
An inertial navigation algorithm is primarily proposed with only IMU data to obtain spatio-temporal gait parameters, such as: attitude, position, velocity of a human body; and stance phase duration, stride length, and walking distance. However, low-cost inertial sensors come with noise and bias that lead to unavoidable accumulative errors. Therefore, straight-line walking with a constant speed constraints is imposed in the filtering algorithm to improve attitude estimation. Visual odometry (VO) algorithm provides relative pose from image sequence, which plays as a measurement updating role for the filter. Detected stance foot from color and depth image data are used as landmarks to update foot position. Finally, a smoothing algorithm is proposed as a linear optimization problem to minimize estimation errors. Stance foot position is derived and other gait parameters can be calculated from step information.
A deep learning-based segmentation model with custom dataset is proposed to improve foot detection not only in stance phases but also in swing phases. 3-D dual foot trajectories are then calculated from proposed filter results and relative position of dual foot with respect to the camera. However, foot position in between Toe-Off and Mid-Stance phases are missing due to obscurity. To tackle this problem, a Graph Convolutional Network (GCN) based model is proposed to predict pose from previous poses. Complete foot trajectories are finally reconstructed with only a single waist-mounted RGB-D camera.
Through experiments, the proposed system shows promising results and could be applied for human motion estimation in real application with a considerable accuracy.