Could artifical intelligence (AI) provide a useful tool to inform parents about their child’s asthma?

New research has shown that artificial intelligence models like ChatGPT could be useful to help parents better understand their child’s condition. 

Background 

The internet is a popular way to find out about lung conditions and more people are starting to use Large Language Models such as ChatGPT for health information. Large Language Models (LLMs) are a type of computer programme that has been trained to understand human language and answer questions using artificial intelligence (AI).  

Researchers wanted to see if the information provided by LLMs is accurate and if they could be trusted to provide lung health information in the future. 

What did the study look at? 

This study explored how well LLMs could answer questions parents asked about their children’s asthma. It was split into three phases: 

• Phase 1: Parents asked ChatGPT questions to better understand their child’s asthma. 

• Phase 2: Researchers looked at the questions asked and grouped them together into common themes. One ‘representative’ question was chosen for the different themes. The same question was asked to 10 different LLMs.  

• Phase 3: Researchers and parents looked at whether the LLM could answer the questions. They checked whether the answers provided were accurate and easy to understand.  

The same LLMs were tested again one month later to see if their responses could be trusted. 

What did the results show? 

The questions asked related mostly to asthma symptoms and how to manage them, along with risk factors and side effects of treatments.  

The results found that most LLMs were rated as ‘good’ or ‘very good’ at answering parents’ questions accurately and that most gave responses that were ‘appropriate’ or ‘very appropriate’ for parents to understand. 

LLMs that are cheaper to access generally performed less well than more expensive versions in terms of how accurate and simple the answers were.  

The researchers also found that responses from LLMs changed when the experiment was repeated, suggesting they are not always reliable. 

Why is this important? 

As online AI tools become more popular and widely available, it is important to understand if the information they provide is correct and if they could improve outcomes. 

This is the first study to consider how well LLMs can answer parents’ questions about their children’s asthma. It shows that although LLMs can provide clear and accurate information, their answers are not reliable. Further studies are needed to confirm this and find out whether LLMs have the same effectiveness when answering questions about other lung conditions. Parents should always speak to their doctor if they are worried about their child’s health.  

Paid-for versions of LLMs were better at answering questions about asthma than their cheaper or free counterparts, raising concerns that this could worsen health inequalities. Action should be taken to prevent this and ensure all LLMs can provide the correct information for patients in the future. 

Read the full paper