An approach using multiple machine learning algorithms based on sensor modeling for gesture matching and recognition

Abstract

Gesture recognition is crucial for applications such as character modeling and humanoid robots, yet gesture recognition and generation remain underexplored, with most studies relying on camera-based tracking, which is limited by single-finger accuracy and computational requirements. This study proposes a hybrid framework that combines a haptic controller with machine learning (ML) to capture high-resolution finger motion data. The haptic controller enhances accuracy by providing additional evaluation of finger position and pressure metrics. We evaluated three ML algorithms - Random Forest (RF), Xgboost, and Support Vector Regression (SVR) to model 45 Euler angle-based joint rotations from 20 input parameters for each hand. Our results show that XGBoost outperforms RF and SVR in all sample sizes (500-3000 data points), achieving the lowest angle error (4431.4°) and distance error (19.8 cm) at 3000 samples. This study provides an innovative method for gesture recognition and provides valuable empirical experience for the application and development of related fields.