Methodological Challenges in Randomized Controlled Trials of mHealth Interventions: Cross-Sectional Survey Study and Consensus-Based Recommendations

Jesus Lopez-Alcalde, Institute for Complementary and Integrative Medicine, University Hospital Zurich, Zurich, Switzerland.
L Susan Wieland, Institute for Complementary and Integrative Medicine, University Hospital Zurich, Zurich, Switzerland.
Yuqian Yan, Institute for Complementary and Integrative Medicine, University Hospital Zurich, Zurich, Switzerland.
Jürgen Barth, Institute for Complementary and Integrative Medicine, University Hospital Zurich, Zurich, Switzerland.
Mohammad Reza Khami, Research Center for Caries Prevention, Dentistry Research Institute, Tehran University of Medical Sciences, Tehran, Iran.
Siddharudha Shivalli, Department of Medical Statistics, Faculty of Epidemiology and Public Health, London School of Hygiene & Tropical Medicine, London, United Kingdom.
Cynthia Lokker, Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada.
Harleen Kaur Rai, Digital Health and Wellness Research Group, Department of Computer and Information Sciences, University of Strathclyde, Glasgow, United Kingdom.
Paul Macharia, Department of Research and Programmes, Kenyatta National Hospital, Nairobi, Kenya.
Sergi Yun, Bio-Heart Cardiovascular Diseases Research Group, Bellvitge Biomedical Research Institute, L'Hospitalet de Llobregat, Barcelona, Spain.
Elvira Lang, Hypnalgesics, Comfort Talk, Brookline, MA, United States.
Agnes Bwanika Naggirinya, Infectious Diseases Institute, College of Health Sciences, Makerere University, Kampala, Uganda.
Concepción Campos-Asensio, Biblioteca Médica, Hospital Universitario de Móstoles, Móstoles, Spain.
Leila Ahmadian, Fakher Mechatronic Research Center, Kerman University of Medical Sciences, Kerman, Iran.
Claudia M. Witt, Institute for Complementary and Integrative Medicine, University of Zurich, Zurich, Switzerland.

Document Type

Journal Article

Publication Date

12-19-2024

Journal

Journal of medical Internet research

Volume

26

DOI

10.2196/53187

Keywords

adherence; consensus; digital health; eHealth; intervention integrity; mHealth; mobile health; mobile phone; randomized controlled trial; recommendations; survey

Abstract

BACKGROUND: Mobile health (mHealth) refers to using mobile communication devices such as smartphones to support health, health care, and public health. mHealth interventions have their own nature and characteristics that distinguish them from traditional health care interventions, including drug interventions. Thus, randomized controlled trials (RCTs) of mHealth interventions present specific methodological challenges. Identifying and overcoming those challenges is essential to determine whether mHealth interventions improve health outcomes. OBJECTIVE: We aimed to identify specific methodological challenges in RCTs testing mHealth interventions' effects and develop consensus-based recommendations to address selected challenges. METHODS: A 2-phase participatory research project was conducted. First, we sent a web-based survey to authors of mHealth RCTs. Survey respondents rated on a 5-point scale how challenging they found 21 methodological aspects in mHealth RCTs compared to non-mHealth RCTs. Nonsystematic searches until June 2022 informed the selection of the methodological challenges listed in the survey. Second, a subset of survey respondents participated in an online workshop to discuss recommendations to address selected methodological aspects identified in the survey. Finally, consensus-based recommendations were developed based on the workshop discussion and email interaction. RESULTS: We contacted 1535 authors of mHealth intervention RCTs, of whom 80 (5.21%) completed the survey. Most respondents (74/80, 92%) identified at least one methodological aspect as more or much more challenging in mHealth RCTs. The aspects most frequently reported as more or much more challenging were those related to mHealth intervention integrity, that is, the degree to which the study intervention was implemented as intended, in particular managing low adherence to the mHealth intervention (43/77, 56%), defining adherence (39/79, 49%), measuring adherence (33/78, 42%), and determining which mHealth intervention components are used or received by the participant (31/75, 41%). Other challenges were also frequent, such as analyzing passive data (eg, data collected from smartphone sensors; 24/58, 41%) and verifying the participants' identity during recruitment (28/68, 41%). In total, 11 survey respondents participated in the subsequent workshop (n=8, 73% had been involved in at least 2 mHealth RCTs). We developed 17 consensus-based recommendations related to the following four categories: (1) how to measure adherence to the mHealth intervention (7 recommendations), (2) defining adequate adherence (2 recommendations), (3) dealing with low adherence rates (3 recommendations), and (4) addressing mHealth intervention components (5 recommendations). CONCLUSIONS: RCTs of mHealth interventions have specific methodological challenges compared to those of non-mHealth interventions, particularly those related to intervention integrity. Following our recommendations for addressing these challenges can lead to more reliable assessments of the effects of mHealth interventions on health outcomes.

Department

Clinical Research and Leadership

Link to Full Text

Share

COinS