Multimodal Wearable System for Objective Assessment of Dynamic Rotational Knee Biomechanics Following ACL Injury and Reconstruction: A Clinical Validation Study Using Ensemble Deep Learning

BackgroundThe clinical assessment of knee stability after an Anterior Cruciate Ligament (ACL) injury is routinely conducted via operator-dependent physical examination tests (i.e. pivot shift) and standardized patient-reported outcomes. Unfortunately, both are unable to perceive and quantify the subtle rotational biomechanical deficiencies from an ACL tear. Although specialized laboratory-based motion capture systems may provide objective measurements, they are found in research institutions and thus, are not suitable for clinical use. In contrast, GATOR PRO is a clinic-based multimodal wearable sensor system that uses a machine learning (ML) model (ensemble deep learning) to differentiate and classify its data outputs for assessing in-vivo dynamic rotational knee stability. ObjectiveThe purpose of this study is to validate the deep machine learning model and its performance used in GATOR PRO, which integrates knee-mounted Inertial Measurement Units (IMUs) with ultrasound images to derive high-fidelity in-vivo biomechanical rotational data. Based on this data collected by the GATOR PRO, it is hypothesized that the model can effectively classify knee stability after ACL injury and reconstruction. MethodsThis prospective clinical study at Singapore General Hospital (SGH) (CIRB 2019/2766, PDPA-compliant) aimed to enroll 60 patients (30 ACL-deficient, 30 ACL-reconstructed [&ge;]6 months post-surgery). At the halfway point of the clinical trial, 29 patients (8 ACL-deficient, 21 ACL-reconstructed [&ge;]6 months post-surgery) were recruited through physician referral at SGH outpatient clinics to perform standardized chair-stand tests. An ensemble deep learning model that combines convolutional (EfficientNet) and time-series (InceptionTime) classifiers is used to output binary stability classifications (ACL-deficient/ACL-reconstructed). The models performance was evaluated using 10-fold stratified cross-validation with patient-wise splitting, repeated across 100 random seeds to assess variability. ResultsAt the halfway point of the trial, the ensemble model performance with regard to the Receiver Operating Characteristic area under the curve (ROC-AUC) was 0.8365 (SD: 0.042, p-value < 0.001), and the classification accuracy was 75.9% (SD: 3.2%) when the model was tested on the 29 CIRB-approved patients. For the ACL-reconstructed class, the performance indicators were as follows: precision 71.4%, recall 93.8%, F1-score 81.1%. For the ACL-deficient class, the indicators were: precision 87.5%, recall 53.8%, F1-score 66.7%.Against the clinical pivot shift tests low sensitivity (24-32%), the model delivers an almost 2X better sensitivity (53.8%)[2, 3], with a comparable specificity (93.8% vs. 90-98%) ConclusionThe multimodal machine learning model was able to perform at a level that was relevant to clinical classification (AUC-ROC 0.8365, accuracy 75.9%) in differentiating between ACL-deficient and ACL-reconstructed knees. Moreover, the model demonstrated far superior sensitivity than previously published estimates for manual pivot shift testing (53.8% vs. 24-32%). These findings demonstrate that rotational knee instability can be reliably differentiated in clinical settings with a ML model deployed on GATOR PRO data.