Accuracy and concordance of DMIND AI application with a renowned depression assessment tool in Thai adults

Abstract

Background: In Thailand, the high prevalence of depression, particularly in rural areas with limited mental health services, poses significant challenges. Advanced technologies like digital phenotyping and artificial intelligence (AI), particularly natural language processing (NLP) and Large Language Models (LLMs), offer promising solutions by analyzing digital data to detect early signs of depression.

Objectives: This study evaluated the performance of the "Detection and Monitoring Intelligence Network of Depression (DMIND)" application, an AI-powered screening tool for detecting depression in Thai adults that analyzes behavioral data from participant responses using machine learning algorithms, such as NLP models and LLMs. The study aimed to determine the agreement of the DMIND AI model and the Thai version of the Hamilton depression rating scale (HDRS-17 Thai) in classifying depression severity.

Methods: This cross-sectional study recruited 388 participants from one tertiary care hospital and two psychiatric hospitals. Initially, participants used the DMIND application, where they were asked to answer a series of questions. Their response in the application was recorded and a pre-trained AI model predicted their depression severity. Subsequently, a trained nurse or psychologist then assessed participants using the HDRS-17 Thai to establish a baseline measure of depression severity. Statistical analysis involved comparing the depression severity classifications from the DMIND AI model with the HDRS-17 Thai. Cohen's kappa coefficient, sensitivity, specificity, and predictive values were used to evaluate the agreement between the two assessments.

Results: Our DMIND application demonstrated moderate agreement with the HDRS-17 Thai, indicating substantial consistency in depression severity classification. The tool showed high sensitivity (87.3%) and moderate specificity (59.5%), with strong negative predictive values for detecting depression.

Conclusion: The AI-powered DMIND application effectively screens for depression by analyzing digital data from participants' responses. Its moderate agreement with a traditional clinical assessment and strong diagnostic performance highlights its potential as a scalable, accessible tool for mental health management in Thailand. Integrating AI tools like the DMIND into the public health infrastructure could significantly enhance the accessibility, accuracy, and responsiveness of mental health services, particularly in underserved regions, potentially revolutionizing the management and treatment of depression across the country.