Classification algorithms trained on simple (symmetric) lifting data perform poorly in predicting hand loads during complex (free-dynamic) lifting tasks

Abstract

The performance of machine learning (ML) algorithms is dependent on which dataset it has been trained on. While ML algorithms are increasingly used for lift risk assessment, many algorithms are often trained and tested on controlled simulation datasets, lacking the diversity of the lifting conditions. Consequently, concerns arise regarding their applicability in real-world scenarios characterized by substantial variations in lifting scenarios and postures. Our study investigates the impact of different lifting scenarios on the performance of ML algorithms trained on surface electromyography (sEMG) armband sensor data to classify hand-load levels (2.3 and 6.8 kg). Twelve healthy participants (6 male and 6 female) performed repetitive lifting tasks employing various lifting scenarios, including symmetric (S), asymmetric (A), and free-dynamic (F) techniques. Separate algorithms were developed using diverse training datasets (S, A, S+A, and F), ML classifiers, and sEMG features, and tested using the F dataset, representing unconstrained and naturalistic lifts. The mean accuracy and sensitivity were significantly lower in models trained on constrained (S) datasets compared to those trained on naturalistic lifts (F). The accuracy, precision, and sensitivity of models trained with frequency-domain sEMG features were greater than those trained with the time-domain features. In conclusion, ML algorithms trained on controlled symmetric lifts showed poor performance in predicting loads for dynamic, unconstrained lifts; thus, particular attention is needed when using such algorithms in real-world scenarios.