Analysis of facial expressions recorded from patients during psychiatric interviews

Mental health research increasingly focuses on the relationship between psychiatric symptoms and observable manifestations of the face and body1. In recent studies2,3, psychiatric patients have shown distinct patterns in movement, posture and facial expressions, suggesting these elements could enhance clinical diagnostics. The analysis of the facial expressions is grounded on the Facial Action Coding System (FACS)4. FACS provides a systematic method for categorizing facial expressions based on specific muscle movements, enabling detailed analysis of emotional and communicative behaviors. This method combined with recent advancements in Artificial Intelligence (AI) has shown promising results for the detection of the patient mental state. We analyze video data from patients with various psychiatric symptoms, using open-source Python toolboxes for facial expression and body movement analysis. These toolboxes facilitate face detection, facial landmark detection, emotion detection and motion recognition. Specifically, we aim to explore the connection between these physical expressions and established diagnostic tools, like symptom severity scores, and finally enhance psychiatric diagnostics by integrating AI-driven analysis of video data. By providing a more objective and detailed understanding of psychiatric symptoms, this study could lead to earlier detection and more personalized treatment approaches, ultimately improving patient outcomes. The findings will contribute to the development of innovative diagnostic tools that are both efficient and accurate, addressing a critical need in mental health care.