AI in the clinic: Can algorithms diagnose patients better than traditional tests?

Artificial intelligence (AI) can recognize common mental disorders just as effectively as – and sometimes better than – traditional diagnostic tools. According to a paper published in the journal Scientific Reports, a generative AI assistant was also perceived by patients as highly empathetic and supportive. The study was conducted by researchers from Sweden, Norway, Italy, and Poland, and could significantly improve the field of mental health diagnostics.

In an era of an escalating mental health crisis, characterized by a shortage of specialists and high treatment costs, an AI-based system offers the potential for earlier diagnosis.

The end of routine surveys?

Traditional diagnostics used for decades often rely on simple questionnaires where patients answer closed-ended questions. While effective, these tools can be prone to errors and may sometimes fail to distinguish overlapping symptoms.

The alternative presented by the researchers is the TalkToAlba system, which utilizes advanced large language models (LLMs). TalkToAlba is designed to support mental health professionals through various AI-assisted features, including an AI therapist delivering CBT (Cognitive Behavioral Therapy). How does it work? Instead of filling out a dry form, the patient engages in a dynamic, natural conversation with the AI. The system analyzes their responses in real-time, asking follow-up questions much like a clinician would during an initial intake.

Higher accuracy of AI

The study involved 303 participants recruited through the Prolific platform, suffering from nine common disorders, including depression, generalized anxiety disorder (GAD), OCD (obsessive-compulsive disorder), ADHD, and autism. 55 of them were healthy controls.

Participants completed an AI-conducted clinical interview using the TalkToAlba software platform. They could choose to interact with the AI clinician either by typing and reading, or by speaking and listening. The AI system responded to input within a few seconds, simulating a natural conversational pace.

In the first stage, the AI assistant gathered baseline data to form an initial diagnostic hypothesis, based on the DSM-5 framework. During the second phase, the algorithm conducted a structured follow-up to validate its findings. Finally, the system integrated all collected data to calculate the probability of each of the nine target conditions. This synthesized evaluation formed the AI's diagnostic conclusion, which was later compared to standard rating scale results.

The study confirms that AI-powered interviews are capable of identifying common mental health conditions just as effectively as – and sometimes better than – traditional surveys. While the AI demonstrated significantly higher accuracy in flagging major depression, OCD, autism, and bipolar disorder, it was just as reliable as standard tests for diagnosing anxiety, ADHD, and PTSD, said Marta Lasota, a doctoral student at SWPS University’s Interdisciplinary Doctoral School and co-author of the article.

The AI assistant excelled at distinguishing between disorders with overlapping symptoms – for instance, identifying the difference between pure depression and anxiety, which is often a challenge for standard surveys.

Patients’ experience

72% of participants rated the AI interview as a meaningful experience, and more than half considered the bot to be empathetic and understanding of their problems. In patient descriptions, words such as “understanding”, “helpful”, “interesting”, “informative”, and “caring” appeared most frequently. For many people who fear judgment from another human, a sincere conversation with a non-judgmental algorithm was simply easier.

Digital support, not a replacement

Despite the results, the study's authors emphasize that the technology is not intended to replace doctors. Human oversight remains a key element to verify results and ensure ethical standards and patient data privacy.

Artificial intelligence can become a powerful supportive tool that can assist with the most time-consuming stages of diagnostics, Marta Lasota suggests. Due to its scalability and low cost, this solution could find applications where access to psychiatrists is most limited.

The article "Generative AI-assisted clinical interviewing of mental health" by Sverker Sikström, Danilo Garcia, et al. was published in the journal Scientific Reports.