AI models developed using one million health-related Reddit threads matched human raters, suggesting they could support faster mental health triage.
Researchers spent two years building a database of mental health narratives and training two AI models, PaLM 2 and AutoML, to analyse real-world accounts shared on Reddit.
They linked language used in the posts to root-cause classifications from Healthdirect Australia, before a psychiatrist manually classified 800 threads.
Victoria University associate professor Khandakar Ahmed and Saima Rani carried out the research, with psychiatrist and University of Melbourne honorary associate professor Manjula O’Connor carrying out the manual classification.
The expert-verified data was then fed back to the AI models. The systems assessed new threads that were also independently classified by human raters.
PaLM 2 matched the human raters’ judgement in 80 per cent of cases, while AutoML matched their assessments in 68 per cent.
The study found that, once trained on expert-verified data, the models could meaningfully align with human judgement when identifying the root causes behind people’s mental health difficulties.
Researchers said the approaches have limitations but could potentially support mental health screening and triage.
They also suggested possible uses in patient onboarding, reviewing patient stories and digital mental health services.
Ahmed said: “Mental health is deeply personal, and every person has a different story behind their struggles. Two people can experience similar symptoms for very different reasons. We wanted to explore whether AI could look beyond the symptoms and recognise some of the deeper themes people describe in their own words.”
He added: “This is an exciting health research breakthrough. We know AI is being used in some health settings already, such as to assist practitioners with note-taking in consultations, but as we have shown here, the possibilities are much greater.”
O’Connor said: “With about 40 per cent of the population experiencing a mental illness at some time in their lives, this methodology has the potential to detect population-level changes to help prioritise public health approaches to mental illness in Australia.”
