Artificial Intelligence versus Researcher: Conceptual Model Development

Blatt, L. Sams, C. Bessant, M. Abdollahyan, F. Smeraldi, T. Symonds, in: ISOQOL (30th annual conference), Calgary (Canada), Poster 2077, October 18-21, 2023

Abstract

Background:

Conceptual model (CM) development plays a vital role in understanding complex medical conditions, such as Lennox Gastaut Syndrome (LGS). Traditionally, CMs are developed through conducting literature reviews, a relatively time-consuming process for researchers. With the advent of artificial intelligence (AI), methods have emerged as potential tools for automating CM development, for example, use of chatbots (e.g., ChatGPT), and the use of AI to mine online health forums for patient-reported data (e.g., Mebomine).

Methods:

In this study, we compared the process and outcomes of CM development for LGS between researchers, ChatGPT, and Mebomine. Researchers conducted literature reviews to identify relevant features and impacts to construct the CM. In comparison, ChatGPT was utilized independently to generate a CM based on input prompts and Mebomine conducted a search of online health forums utilizing bespoke AI to extract disease-relevant information.

Results:

Our findings indicated that all 3 methods produced a reasonable summary of the key features and impacts to assist in understanding of the disease presentation. General concepts tended to align across the CMs produced, with similar high-level domains identified (e.g., seizures, physical functioning/motor, and behavioral/emotional functioning) and multiple concepts grouped under each domain. A key limitation of ChatGPT identified was the inconsistent results when inputting the same prompts on different occasions.

Conclusions:

Both ChatGPT and Mebomine produced detailed CMs, as well as offering advantages in terms of automation and efficiency. Mebomine results also draw directly from the patient voice. Where possible, literature reviews obtain data from qualitative studies to similarly ensure the patient voice is promoted throughout the CM development. Importantly, researchers bring unique human insights and contextual understanding.
A collaborative approach that combines the strengths of both human researchers and AI may offer the best path forward, allowing for efficient automation while benefiting from the human expertise needed for CM development. The incorporation of data from online health forum posts can also ensure promotion of the patient voice, particularly in cases where published qualitative articles are limited. Further research into the integration of AI into medical research is needed before confidence can be instilled in the results of solely AI-generated work.

(poster)

Backlinks: Publications