Conference Paper Published

Abstract

This study presents a comprehensive framework to enhance oral-motor assessments by integrating real-time digital analyses of the lips, jaws, and oral-facial movements, which are often overlooked in existing methodologies for motor-speech assessment (MSA) that focus primarily on perceptual and subjective judgments of human speech/oral-facial performance. We propose standardized metrics for evaluating a set of 6 oral motor actions, utilizing a dataset from 194 participants aged 20 to 85 years (133 neurologically healthy and 61 with stroke history), which includes annotated facial muscle movements and assessments of orofacial musculature capabilities. Our algorithmic analysis aims to produce standardized action completion scores across different age groups and genders, improving the evaluation of motor-speech performance by applying objective and quantified measurement to the coordination of orofacial musculature. The precise locations of oral-facial keypoints are detected by pre-trained machine learning models - Mediapipe Face Landmarker and custom-trained YOLO Detect models (for detection of jaw bottom position) with transfer learning, which are available for further studies and applications. By highlighting the significance of visual detection for facial muscles, including lips and jaw, the preliminary findings of this research may provide valuable insights for clinicians and researchers in speech pathology and the development of assistive technology in speech diagnostics.

Keywords

Computer Vision, MediaPipe, Object Detection, Speech Diagnostics, Stroke, YOLO