Engagement with a Chatbot-based Intervention for the delivery of mailed at-home COVID-19 Testing: a Descriptive Log Analysis Study from the SCALE-UP II Trial

Background Promoting at-home tests (e.g., for COVID-19) using chatbots may be a novel and scalable way to improve uptake across underserved populations. Objective The objective of this study was to assess the navigational patterns (i.e., sequence of interactions) of underserved populations when using a chatbot designed to provide education on COVID-19 testing and free order access for at-home COVID-19 test kits. Methods The study was a descriptive analysis of the original data of the chatbot intervention of the SCALE-UP II trial, which compared different digital health modalities (i.e., chatbots versus simple text messages) to deliver free at-home COVID-19 test kits to minority populations in Utah. SCALE-UP II (registration numbers NCT05533918; NCT05533359) was a multisite, pragmatic clinical trial with patients randomized in a 2x2x2 factorial design (smartphone study) to receive (1) chatbot or text messaging, (2) the option to request patient navigation, and (3) intervention frequency every 10 or 30 days. All other participants were randomized in a 2x2 factorial design (nonsmartphone study) to receive the option to request patient navigation and intervention frequency every 10 or 30 days. Eligible patients (1) had an appointment at one of the participating community health centers (CHC) in the last 3 years, (2) were 18 years and older, and (3) had a valid cellphone number recorded in the CHC electronic health record (EHR). The trial enrolled 2117 in the smartphone study and 31,439 in the nonsmartphone study. In the smartphone study, the proportion of participants who requested test kits in the Chatbot arm was lower than in SMS text messaging. In the nonsmartphone study, test kits was higher if they were messaged every 10 days. Sources of funding included the National Institute on Minority Health and Health Disparities (NIMHD) of the US National Institutes of Health (NIH) grant number 5U01MD017421 and by awards from the National Cancer Institute of the NIH (P30CA042014) and the Huntsman Cancer Foundation. Results: Of 1,051 patients randomized to the chatbot intervention, 309 (29%) launched the chatbot, 196 (63%) interacted with it, and 186 (60%) started the COVID-19 test kit ordering process. Among those who launched the chatbot, 170 (55%) completed a test kit order. One patient (0.3%) accessed the chatbot educational content. The median age was 51, with 66% female, 54% Latino/a, 55% uninsured, and 86% located in an urban area. Conclusion: Ordering of COVID-19 test kits among underserved patients who interacted with the chatbot was high. Thus, chatbots may represent a viable approach to reach underserved populations as a part of public health response in a pandemic. All patients except one placed orders without reviewing educational content. Chatbot design should identify and minimize the number of steps for patients to achieve a specific goal.