Soul Machines’ ‘digital people’, Sam and Roman

Researchers at the Auckland Bioengineering institute (ABI) have received more than $4 million in Ministry of Business, Innovation and Employment (MBIE) funding to investigate how ‘digital people’ could be used to monitor people’s health and support the management of their conditions. As part of the project, ABI researchers will work with Kiwi artificial intelligence firm Soul Machines’ ‘digital people’, which the company claims interact with humans in real time and in a lifelike manner.

ABI deputy director Professor Merryn Tawhai said the funding will support an international consortium, led by ABI, which will explore how digital people could be used to support the management of type 2 diabetes and cardiovascular disease (CVD). Although both conditions are intrinsically linked, treated in tandem and managed primarily at home by the patient with prescription medications, many patients struggle to understand their condition and how to manage it, she said. “Often, they don’t comply with treatment strategies recommended by their doctor, especially the lifestyle changes needed to get the greatest improvements in health and quality of life. While we know intensive interventions with clinical and personal coaching are effective, their cost puts them out of the reach of most people.”

Prof Tawhai’s team aims to create a platform of interconnected digital tools, starting with a culturally appropriate digital health navigator, which reads data from a patient’s in-home and body-worn sensors and ‘talks’ to them to help them better understand and manage their condition, she explained. “Rather than the typical episodic and reactive doctor-patient relationship in the real world, this would allow for continuous interaction that encourages and supports the management of chronic disease, reducing barriers in access to care and reducing healthcare inequity.”

Professor Mark Billinghurst, head of ABI’s Empathic Computing Lab, will also use the funding to lead Tōku Hoa, a project exploring how digital people – customised to match ethnicity, age and gender – could provide long-term monitoring of mental wellbeing. The digital person will appear on the user’s phone or computer in response to changes in their mood, or when requested. A conversation will then offer feedback to help manage their depression. This project involves developing an interface for live remote therapy sessions, mediated and supported by the user’s digital person, which will have detailed knowledge of the user’s mental-health history, said Prof Billinghurst. The initial target audience is young Māori men, who are overrepresented in mental-health statistics, he added. “This will help address the long wait time and lack of access to mental-health support services in Aotearoa New Zealand, particularly for rural Māori.”