This is how it started.

Pay walled, but looks nice. Don’t be poor. This explains how to build motion dictionaries, kinematic chains, and other control structures to catch flying bananas. Learning Progressive Joint Propagation for Human Motion Prediction 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII.

Experimental results show that motion prediction with NAE and NAE-DF is superior to other methods and has a good generalization performance on unseen objects. We test our methods on a robot, performing velocity control in real world and respectively achieve 83.3% and 86.7% success rate on a ploy urethane banana and a gourd.
Hongxiang Yu, Dashun Guo, Huan Yin, Anzhe Chen, Kechun Xu, Yue Wang, Rong Xiong

I don’t have time to read this, but maybe you do. Kalman filters are brilliant. https://ieeexplore.ieee.org/document/9281312 Human Motion Prediction Using Adaptable Recurrent Neural Networks and Inverse Kinematics.