PCR3 Assessing Concept Saturation and Sample Representativeness When Using Social Media Data to Inform Concept Elicitation Studies

C Bessant, Y Bridges, M Chowdhury, A Tahsin, M Abdollahyan, F Smeraldi, B Byrom, Abstract and Poster, Value in Health, vol 25, issue 12, supplement, pp S390, 2022

Abstract

Objectives:

Social media (SM) data, such as that posted on online health boards (OHBs), can provide insights to aid concept elicitation in the development of clinical outcome assessments, and other application areas. While SM data samples are often much larger than those achieved in cognitive interview studies, understanding concept saturation remains important. In addition, OHB users are often younger and include greater proportions of females compared to many patient populations, making it important to consider the representativeness of the sample when drawing conclusions.

Methods:

We use OHB posts to elicit understanding of meaningful aspects of physical activity in CHF patients. We used autocorrelation between concepts identified over time to measure the convergence of findings to quantify concept saturation. Further, we implemented weighting adjustments to adjust for the differences in age distributions between our cohort and the CHF population.

Results:

We used 383 posts from 271 individuals with CHF. Ages were determined for 209/271 (77%) of individuals, and were normally distributed (range: 20 – 96, mean: 54, SD: 15 years). Adjusting findings non-parametrically for the true population age distribution did not change the overall conclusions, but increased the importance of certain concepts including “self-care” and “going out”, while lessening concerns around “productivity” and “sick leave”. Saturation analysis showed asymptotic growth in identified concepts, but correlation between the overall frequency distribution of concepts from post to post began to stabilise after the first 30 posts, and continuing to code beyond the first 90 posts had negligible impact on the overall findings of the study.

Conclusions:

SM data may contribute to concept elicitation knowledge important in outcome measure development. The methods we describe help to ensure that findings are representative of the patient population, and that concept saturation is established. These approaches contribute to enhancing the robustness of findings from social listening studies.

(online version, poster)

Backlinks: Publications