A summary of research published in the European Respiratory Journal.
New research has shown that artificial intelligence models like ChatGPT could be useful to help parents better understand their child’s condition.
The internet is a popular way to find out about lung conditions and more people are starting to use Large Language Models such as ChatGPT for health information. Large Language Models (LLMs) are a type of computer programme that has been trained to understand human language and answer questions using artificial intelligence (AI).
Researchers wanted to see if the information provided by LLMs is accurate and if they could be trusted to provide lung health information in the future.
This study explored how well LLMs could answer questions parents asked about their children’s asthma. It was split into three phases:
The same LLMs were tested again one month later to see if their responses could be trusted.
The questions asked related mostly to asthma symptoms and how to manage them, along with risk factors and side effects of treatments.
The results found that most LLMs were rated as ‘good’ or ‘very good’ at answering parents’ questions accurately and that most gave responses that were ‘appropriate’ or ‘very appropriate’ for parents to understand.
LLMs that are cheaper to access generally performed less well than more expensive versions in terms of how accurate and simple the answers were.
The researchers also found that responses from LLMs changed when the experiment was repeated, suggesting they are not always reliable.
As online AI tools become more popular and widely available, it is important to understand if the information they provide is correct and if they could improve outcomes.
This is the first study to consider how well LLMs can answer parents’ questions about their children’s asthma. It shows that although LLMs can provide clear and accurate information, their answers are not reliable. Further studies are needed to confirm this and find out whether LLMs have the same effectiveness when answering questions about other lung conditions. Parents should always speak to their doctor if they are worried about their child’s health.
Paid-for versions of LLMs were better at answering questions about asthma than their cheaper or free counterparts, raising concerns that this could worsen health inequalities. Action should be taken to prevent this and ensure all LLMs can provide the correct information for patients in the future.
