Optimization of the automated Sunnybrook Facial Grading System – Improving the reliability of a deep learning network with facial landmarks

Abstract

Objective: The Sunnybrook Facial Grading System (SFGS) is a well-established grading system to assess the severity and progression of a unilateral facial palsy. The automation of the SFGS makes the SFGS more accessible for researchers, students, clinicians in training, or other untrained co-workers and could be implemented in an eHealth environment. This study investigated the impact on the reliability of the automated SFGS by adding a facial landmark layer in a previously developed convolutional neural network (CNN). Methods: An existing dataset of 116 patients with a unilateral peripheral facial palsy and 9 healthy subjects performing the SFGS poses was used to train a CNN with a newly added facial landmark layer. A separate model was trained for each of the 13 elements of the SFGS and then used to calculate the SFGS subscores and composite score. The intra-class coefficient of the automated grading system was calculated based on three clinicians experienced in the grading of facial palsy. Results: The inter-rater reliability of the CNN with the additional facial landmarks increased in performance for all composite scores compared to the previous model. The intra-class coefficient for the composite SFGS score increased from 0.87 to 0.91, the resting symmetry subscore increased from 0.45 to 0.62, the symmetry of voluntary movement subscore increased from 0.89 to 0.92, and the synkinesis subscore increased from 0.75 to 0.78. Conclusion: The integration of a facial landmark layer into the CNN significantly improved the reliability of the automated SFGS, reaching a performance level comparable to human observers. These results were attained without increasing the dataset underscoring the impact of incorporating facial landmarks into a CNN. These findings indicate that the automated SFGS with facial landmarks is a reliable tool for assessing patients with a unilateral peripheral facial palsy and is applicable in an eHealth environment.