SARA - Substance Abuse Research Assistant
Study Information
SARA: Substance Abuse Research Assistant
In the following, we describe our research questions. However, we first provide a brief background of the research project in order to contextualize the research questions. Background: Substance use is an important public health issue. Substance use is associated with risky behaviors (e.g., drunk driving or violence) and has short-term (e.g., injury) and long-term health consequences (e.g., substance use disorders). Adolescents and emerging adults are important age groups to study because substance use often begins in early adolescence, increases across adolescent development, and peaks during emerging adulthood [1]. Mobile health apps can enable the capture of substance use data in-situ, data that can show what happens before and after substance use among adolescents and emerging adults. However, adherence to manual data collection is low for most mobile health apps, including substance use apps. This work investigates different engagement strategies to increase adherence to substance use data collection in a mobile health setting. Research question: This research study’s goal is to investigate the efficacy of different engagement strategies to increase the frequency of completion of self-reports. Specifically, in this study, participants (ages 14-24) who screen positive for past month binge drinking and/or marijuana use are prompted to complete a survey and two active tasks (tapping speed and spatial memory) [2] between 6PM and midnight every day using a mobile phone app. In addition, participants are prompted to complete a weekly survey on Sundays. The study tests four engagement strategies to determine their efficacy to increase the survey and active task completion. The four engagement strategies and their associated research questions are as follows: 1. 4PM push notification: Every day at 4PM, a push notification may or may not be issued with an inspirational quote from a contemporary celebrity. One of our research questions is whether offering (vs. not offering) an inspirational quote increases the likelihood to fully complete current day’s survey and/or active tasks. 2. 6PM push notification: Every day at 6PM, a reminder notification is issued. The notification may be one of two types: a. A simple reminder to complete a survey and two active tasks b. A reminder to complete a survey and two active tasks, along with an additional persuasive message. The additional persuasive message may be i. “Do you know it only takes a minute to fill out the survey and active tasks?” or ii.“Do you know you can earn money if you complete a 3-day streak?” or iii. “You are close to unlocking the next fish for the aquarium.” One of our research questions is whether offering (vs. not offering) the additional persuasive message increases the likelihood to fully complete current day’s survey and/or active tasks. 3. Reinforcement after survey completion: If the survey is completed, then a meme may be issued as a positive reinforcement. The meme can be a static image or an animated gif, and can be either funny or inspirational. One of our research questions is whether offering (vs. not offering) a meme increases the likelihood to fully complete the survey and/or active tasks the following day. 4. Reinforcement after active task completion: If active tasks are completed, then a life-insight may be issued as a positive reinforcement. Life-insights are graphs of data collected (e.g., stress, loneliness) from the participant over the last seven days. One of the research questions is whether offering (vs. not offering) a life-insight increases the likelihood to fully complete the survey and/or active tasks the following day. For more details on each of the engagement strategies of SARA, please see this paper [3]. References: [1] Center for Behavioral Health Statistics and Quality. Behavioral health trends in the United States: Results from the 2014 National Survey on Drug Use and Health. 2015. [2] http://researchkit.org/docs/docs/ActiveTasks/ActiveTasks.html [3] Mashfiqui Rabbi, Meredith Philyaw-Kotov, Jinseok Lee, Anthony Mansour, Laura Dent, Xiaolei Wang, Rebecca Cunningham, Erin Bonar, Inbal Nahum-Shani, Predrag Klasnja, Maureen Walton, and Susan Murphy. 2017. SARA: a mobile app to engage users in health data collection. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers (UbiComp '17). ACM, New York, NY, USA, 781-789. DOI: https://doi.org/10.1145/3123024.3125611 [The camera-ready version of this paper is attached as supplementary file in the files section (filename: rabbi17sara.pdf, https://osf.io/ejp5a/)]
This study involves four randomized engagement strategies to increase adherence of data collection. Each of the four engagement strategies is associated with a hypothesis. These hypotheses are as follows: 1. 4pm notification: Hypothesis: Offering (vs. not offering) an inspirational quote increases the likelihood to fully complete current day’s survey and/or active tasks. This hypothesis is directional; our goal is to increase survey and/or active task completion. 2. 6pm reminder notification: Hypothesis: Offering a reminder with an additional persuasive message (vs. a simple reminder) increases the likelihood to fully complete current day’s survey and/or active tasks. This hypothesis is directional; our goal is to increase survey and/or active task completion. 3. After self-report completion reinforcement: Hypothesis: Offering positive reinforcement in the form of a meme (vs. not offering meme) following survey completion increases the likelihood to fully complete next day’s survey and/or active tasks. This hypothesis is directional; our goal is to increase survey and/or active task completion. 4. After active task completion reinforcement: Hypothesis: Offering positive reinforcement in the form of life insights (vs. not offering life-insights) following active task completion increases the likelihood of to fully complete next day’s survey and/or active tasks. This hypothesis is directional; our goal is to increase survey and/or active task completion. Our primary focus will be on the two following primary hypotheses: 1. The 4PM push notification with inspirational quote will increase the full completion of survey and/or active task the same day as compared to no inspirational quote 2. Among individuals who complete the survey, offering a post-survey-completion meme will increase the full completion of survey and/or action task the next day as compared to not offering a meme after survey completion
Sampling Plan
Every day, four randomizations occur for each participant in SARA: each randomization corresponds to one of the four engagement strategies stated above. Dr. Mashfiqui Rabbi has access to the randomizations (i.e., the manipulated variables). Mashfiqui is the lead programmer of the SARA app and he regularly checks the randomizations to ensure that all the push notifications, memes and life-insights are delivered properly. This is necessary since updates to the phone operating system or server software may disrupt the randomizations. Therefore, someone needs to monitor these changes and respond with fixes if needed. No other study member has access to the randomization. No other study member has knowledge of which randomized engagement strategy is delivered to any participant. The outcome measures, i.e., whether participants complete the survey and active tasks are visible to study staff during the study. The outcome measures are visible to allow study staff in order to deliver an additional support: participants who do not complete the survey and/or active tasks before 9PM and did not complete the survey and/or active tasks on the prior day are sent an SMS message with a generic reminder message. If the participant continues to not complete any survey/active task for 3 additional days then a second generic SMS text message reminder is sent. If the participant continues to not complete the survey/active task for a further additional 2 days (total of five incomplete daily surveys/active tasks), the participant receives a personalized phone call from a study staff member. If a participant continues to not fill out the survey/active tasks after this personalized phone call, study staff repeat the above process until either: 1) the participant completes a survey/active task, 2) the participant has been contacted by either SMS or phone up to 9 times over 3 weeks, following which the participant is no longer contacted, or 3) the participant asks to be removed from the study. Communications with participants do not involve questions about the randomized engagement strategies. Dr. Rabbi, who has access to the randomizations, has no communication with the study participants. Finally, since Dr. Rabbi has access to both randomizations and outcome measures, Dr. Rabbi will not conduct the primary analysis and corresponding data cleaning. A biostatistician who did not have participant contact nor access to the randomizations will conduct the primary analyses.
1. Population: Adolescents (14-17) and emerging adults (18-24) reporting past month binge drinking and/or marijuana use 2. Recruitment: Study participants will be recruited from the Emergency Department of the University of Michigan Health System. Logs of electronic patient records maintained by the UM Emergency Department will allow recruiters to identify potential patients in the age range who are eligible to be approached about the study and likely meet inclusion/exclusion criteria. 3. Eligibility: Patients will be eligible for screening if they are between the ages of 14 and 24, understand English, are medically stable, and able to provide informed consent/assent (e.g., are not cognitively impaired or intoxicated), and accompanied by a parent/guardian (for patients between the ages of 14 and 17). Individuals will be eligible for the study if they: 1) have an Android or iPhone smartphone on which the app can be downloaded; and 2) screen positive for past-month binge drinking (4 or more drinks on one occasion for females; 5 or more drinks on one occasion for males) OR past-month marijuana use without a medical marijuana card. 4. Study time-line: Recruitment for the study began August 21, 2017, and will continue until February 1, 2018. Each participant will use the SARA application for one month. Follow up interviews will continue until April 1, 2018. 5. Payment: Participants receive $20 for the intake session and $30 for the exit interview. During the study, each participant can earn up to $13 depending on his or her level of survey and/or active task completion. Attached files: - Clinicaltrials.gov registration
We will recruit at most 110 participants. One participant is one unit. All units will be included in the analysis.
The funding for recruitment will end on February 1, 2018 thus restricting the sample size.
We will terminate recruitment on February 1, 2018. Since the study involves using the SARA app for 30 days, the data collection and follow-up interviews will continue for up to two months following the last recruitment date (i.e., April 1, 2018).
Variables
Four variables are manipulated (e.g. randomized) in the study: 1. 4pm notification: Participants will either receive a push notification with an inspirational quote at 4PM or will not receive a notification. 2. 6pm reminder notification: Participants will receive a notification with a reminder message, which can take one of the following two types: a. A simple reminder to complete the survey and active tasks. b. A reminder to complete a survey and two active tasks, along with an additional persuasive message. The additional persuasive message may be i. “Do you know it only takes a minute to fill out the survey and active tasks?” or ii. “Do you know you can earn money if you complete a 3-day streak?” or iii. “You are close to unlocking the next fish for the aquarium.” 3. After-survey completion reinforcement: After a participant completes the survey, s/he will receive a meme or will not receive a meme. 4. After-active-task-completion reinforcement: After a participant completes the active tasks, s/he will receive a life insight or will not receive a life-insight.
No files selected
1. 4pm notification a. Primary outcome: participants fully complete the survey and/or active tasks in the evening of the same day. This outcome is binary. b. Independent variable: push notification with an inspirational message vs no push notification with an inspirational message. This variable is binary. c. Covariates: whether the survey and/or active tasks were fully completed prior-day (binary); whether text or phone calls were made in the last 24 hours, i.e. after 4PM from the previous day to before 4PM the current day (binary); whether the app was opened in the prior 72 hours outside of when survey and/or active task were completed, i.e. after 4PM from the 3-days ago to before 4 PM the current day (binary) 2. 6pm reminder notification a. Primary outcome: participants fully complete the survey and/or active tasks in the evening of the same day. This outcome is binary. b. Independent variable: A reminder message which is one of the following two types i. A simple message to complete survey and active task ii. A reminder to complete a survey and two active tasks, along with an additional persuasive message. The additional persuasive message may be 1. “Do you know it only takes a minute to fill out the survey and active tasks?” or 2. “Do you know you can earn money if you complete a 3-day streak?” or 3. “You are close to unlocking the next fish for the aquarium.” This variable is binary. c. Covariates: whether the survey and/or active tasks were fully completed prior-day (binary); whether text or phone calls were made in the last 24 hours, i.e. after 6PM from the previous day to before 6PM the current day (binary); whether the app was opened in the prior 72 hours outside of when survey and/or active task were completed, i.e. after 6PM from the 3-days ago to before 6PM the current day (binary) 3. After-survey-completion reinforcement: a. Primary outcome: whether participants fully complete the survey and/or active tasks the following day. This outcome is binary b. Independent variable: offering a meme vs. not offering a meme after survey completion. This variable is binary. c. Covariates: whether the survey and/or active tasks were fully completed the prior day (binary); whether text or phone calls were made in the last 30 hours (binary); whether the app was opened in the prior 80 hours outside of when survey and/or active task were completed, i.e. after 6PM from the 3-days ago to before data collection time on the current day (binary) 4. After-active-tasks-completion reinforcement: a. Primary outcome: whether participants fully complete the survey and/or active tasks the following day. This outcome is binary b. Independent variabel: offering a life-insight vs. not offering a life-insight after active tasks are completed. This variable is binary. c. Covariates: whether the survey and/or active tasks were fully completed the prior day (binary); whether text or phone calls were made in the last 30 hours (binary); whether the app was opened in the prior 80 hours outside of when survey and/or active task were completed, i.e. after 6PM from the 3-days ago to before data collection time on the current day (binary)
No files selected
We will code the outcomes in the following way: 0 - Neither survey nor active tasks are fully completed by participant 1 - Participant fully completed both or either the active tasks or the survey
No files selected
Design Plan
This study involves a micro-randomized design (a sequential factorial design) [1, 2]. [1] Klasnja, P., Hekler, E. B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., & Murphy, S. A. (2015). Microrandomized trials: An experimental design for developing just-in-time adaptive interventions. Health Psychology, 34(S), 1220. [2] Liao, Peng, Predrag Klasnja, Ambuj Tewari, and Susan A. Murphy. "Sample size calculations for micro-randomized trials in mHealth." Statistics in medicine (2015). [Both the above references are attached as files in this registration]
1. 4pm notification: Every day at 4PM each participant is randomized; either the number 0 or the number 1 is generated using a uniform random number generator. Thus number 1 is selected with probability ½ and number 0 is selected with probability ½. For Android version of SARA, the randomization decision is made on the phone and Math.random() function from the Android SDK is used. For the iOS version of SARA, the randomization decision is made on the cloud and random.randomInt() function from Python SDK is used. If the number 0 is selected, then no push notification is sent; if the number 1 is selected then a message with an inspirational quote is sent to the participant via a push notification. 2. 6PM reminder notification: Every day at 6PM each participant is randomized; either the number 0 or the number 1 is generated using a uniform random number generator. Thus number 1 is selected with probability ½ and number 0 is selected with probability ½. For Android version of SARA, the randomization decision is made on the phone and Math.random() function from the Android SDK is used. For the iOS version of SARA, the randomization decision is made on the cloud and random.randomInt() function from Python SDK is used. If the number 0 is selected, then a simple reminder message is issued to complete the survey and active tasks. If the number 1 is selected, then the reminder message is augmented with an additional persuasive message. 3. After survey completion reinforcement: After the daily survey is completed, each participant is randomized; either the number 0 or the number 1 is selected using a uniform random number generator. Thus number 1 is selected with probability ½ and number 0 is selected with probability ½. For both iOS and Android version of SARA, the randomization decision is made on the phone and Math.random() function from the Javascript SDK is used to pick the random number. If the number 1 is selected, then positive reinforcement in the form of a meme is offered to the participant; if the number 0 is selected, then no meme is offered. 4. After active tasks completion reinforcement: After the active tasks are completed, each participant is randomized; either the number 0 or the number 1 is selected using a uniform number generator. Thus number 1 is selected with probability ½ and number 0 is selected with probability ½. For both iOS and Android version of SARA, the randomization decision is made on the phone and Math.random() function from the Javascript SDK is used to select the random number. If the number 1 is selected, then positive reinforcement in the form of a life-insight is offered to the participant. If number 0 is selected, then no life-insight is offered.
Analysis Plan
We will use a multiplicative structural nested log linear model for a binary outcome. Our primary analysis will concern the two following hypotheses: 1. The 4PM push notification with inspirational quote will increase the full completion of survey and/or active task the same day as compared to no inspirational quote (p-value less than 0.025) 2. Among individuals who complete the survey, offering a post-survey-completion meme will increase the full completion of survey and/or action task the next day as compared to not offering a meme after survey completion (p-value less than 0.025) Since we are testing two hypotheses we divide the standard p-value of 0.05 by two. Please see attached file (SaraAnalysis.pdf) for details on how the hypothesis will be done.
In SARA, the measured outcome and manipulated variables are binary. We will dummy code the outcome and manipulations as follows: Outcome variable on each person-day: 0 - if participants fully completed neither daily survey nor the active tasks 1- if participants fully completed one or both of the daily survey or active tasks Manipulated (randomized engagement strategy) variables: 4PM push notification with an inspirational message: 0 - if a push notification with inspirational messages at 4PM is not provided 1 - if a push notification with inspirational messages at 4PM is provided 6PM reminder push notification: 0 - if the reminder-push-notification does not contain an additional persuasive message 1 - if the reminder-push-notification contains an additional persuasive message If the survey is completed between 6PM and midnight: 0 – if a meme is not provided after the survey is completed 1 – if a meme is provided after the survey is completed If the activity tasks are completed between 6PM and midnight: 0 – if a life-insight is not provided after the active tasks are completed 1 – if a life-insight is provided after the active tasks are completed Covariates: Prior day survey/active task completion 0 - if participants fully completed neither daily survey nor the active tasks the previous day 1- if participants fully completed one or both of the daily survey or active tasks the previous day Text/phone call For 4PM and 6PM push notifications: 0 – if neither text messages nor phone calls are made during the same day and/or the previous day 1- if text messages and/or phone calls are made during the same day and/or the previous day For meme and life-insights: 0 – if neither text messages nor phone calls are made during the same day and/or the following day 1- if text messages and/or phone calls are made during the same day and/or the following day
We will include a linear term in time both as a control variable as well as an interaction term with the indicator of engagement strategy (indicator =1 if engagement strategy provided, indicator =0 otherwise; time will be coded as 0 for the first day of the study and will be incremented by 1 for each additional day in the study.)
Our primary analyses will concern the two following hypotheses: 1. The 4PM push notification with inspirational quote will increase the full completion of survey and/or active task the same day as compared to no inspirational quote (p-value less than 0.025) 2. Among individuals who complete the survey, offering a post-survey-completion meme will increase the full completion of survey and/or action task the next day as compared to not offering a meme after survey completion (p-value less than 0.025) Since we are testing two hypotheses we divide the standard p-value of 0.05 by two.
We exclude data if one of the following two situations happens: (i) if push notifications are not delivered due to a bug in the code or due to lack of access to the internet and (ii) phone malfunction.
As long as the application is on the participant’s phone, we can record the randomization, whether or not the participant fully completes the survey and/or active tasks on any day and whether or not the participant was sent a SMS or received a phone call. As we will be conducting intention-to-treat analyses, missing data can only occur if the participant deletes the application on the phone. Note that even if a participant deletes the application, the outcome measure is observed (e.g. outcome =0 since the participant did not fully complete either the survey or active tasks). Similarly all covariates are observed regardless of application deletion. Furthermore on missing days due to app deletion, no randomization or primary analysis will be done for memes or life-insights since memes or life-insights only occur when participants complete survey or active tasks. Thus the only variables in the analyses that are subject to missingness are the 4pm and 6pm randomizations; these will not occur after a participant deletes the application from their phone. The analyses will include data from both participants who did not delete and participants who deleted the app within the 30-day study. In this analysis, we will multiply impute the 4pm and 6pm randomizations on study days on which a participant did not have the app installed on their phone. We will conduct 4 sensitivity analyses for the two above specified primary analyses: 1. A primary analysis that includes the data from all participants, either to the end of the 30 day study period or until the participant deletes the app. 2. A primary analysis after excluding all data from all the participants who deleted the app within the 30-day study period. 3. A primary analysis after excluding data from all the participants who did not delete the app. We will only use the participant’s data until the time of app deletion.
We plan to the following exploratory analyses: Effect of life-insight: Among individuals who complete the active tasks, offering a post-survey-completion life-insight will increase the full completion of survey and/or action task the next day as compared to not offering a life-insight after active task completion Effect of the persuasive message in the 6PM notification: The 6PM push notification with the additional persuasive message will increase the full completion of the survey and/or active task the same day as compared to the reminder notification without the persuasive message. Longer sequences of rewards: We will re-run the same primary analysis for post-data collection rewards such as memes with the following added covariate: 0 - if participant is not offered a meme after survey completion the day before previous day 1- if participant is offered a meme after survey completion the day before previous day We will include the rewards from two days prior to investigating whether recency of rewards influence the outcome of survey and/or active tasks completion Quality of the rewards: Participants can rate the memes or life-insight with thumbs up and thumbs down. We will investigate the effect of user rating on the outcome. We will re-run the same primary analysis for post-data collection rewards such as memes with the following added covariate: 0 - if participant is not offered a meme after survey completion or participant gave a thumb down to the meme 1- if participant is offered a meme after survey completion and participant gave a thumbs up to the meme Demographic: We will add gender and weekdays/weekend as covariates and analyze their effect on the outcome.
Scripts
Other
In the following, we provide a brief description of SARA. SARA is a mobile application developed for youth and emerging adults at high risk for substance use. The goal is to capture substance use data for extended periods of time in order to inform substance use and addiction science and treatment. The key novelty of SARA is the multitude of different engagement strategies it provides to increase data collection while only offering minimal monetary incentives and human contact [18]. SARA is comprised of two modules: (1) a data collection module (2) an engagement module. In the following, we first briefly describe the data collection module. Then, we discuss the engagement module, which contains different strategies to support data collection. Data collection module Each study participant in SARA is asked to complete one survey and two active tasks each day between 6PM and 12am. Typically the survey contains 7 questions about their daily emotions (e.g., stress level, mood, loneliness, hopefulness), and daily reflections (e.g., amount of free time, excitement of their day). On Sundays, an extra set of questions is asked in addition to the 7 questions. These extra questions ask about past-week substance use (alcohol, marijuana, nicotine, vaping) and motives (alcohol, marijuana), perceived risk of regular substance use, impulsivity, and behavioral intentions to avoid substance use in the following week. In addition to the surveys, participants complete two active tasks. Active tasks are a way to collect data on reaction time, spatial memory data, gait, problem solving skill etc. and were first introduced in the Apple Research Kit. In the current version of SARA, we re-implemented two active tasks. The first active task measures spatial memory. A random sequence of five seashells light up in a two dimensional grid of nine seashells; participants are then asked to repeat the sequence. In the second active task, participant tap two buttons alternately for 10 seconds; the number of completed taps provides a measure of reaction time. Engagement module The centerpiece of SARA’s engagement strategies is a growing aquarium. Each time the participant completes a daily survey or active tasks, the participant earns 30 points for the aquarium. For the longer survey on Sundays, additional 50 points are awarded. When participants reach a certain number of points, different fish are unlocked and become visible in the aquarium. Each time a fish is awarded, SARA also shows a piece of trivia about that fish to further spark interest and curiosity. For example, when the goldfish is unlocked, SARA provides the message: “Do you know that goldfish can recognize faces?” In addition, the aquarium, four other engagement strategies are used in SARA: 1. 4pm notification: Participants will be either offered a push notification with an inspirational quote or will not be offered a notification. 2. 6pm reminder notification: Participants will be offered a notification with a reminder message. The reminder message can be two types. a. A reminder to complete a survey and two active tasks, b. A reminder to complete a survey and two active tasks, along with an additional persuasive message. The additional persuasive message may be i. “Do you know it only takes a minute to fill out the survey and active tasks?” or ii. “Do you know you can earn money if you complete a 3-day streak?” or iii. “You are close to unlocking the next fish for the aquarium.” 3. After-survey completion reinforcement: After a participant completes the survey, s/he will receive a meme or will not receive a meme 4. After-active-task-completion reinforcement: After a participant completes the active tasks, s/he will receive a life insight or will not receive a life-insight. Finally, participants can earn 1 dollar if they complete both the survey and active tasks three days in a row. Theoretical justification of the engagement strategies: Theoretical justification of the different engagement strategies in SARA is as follows: Post completion rewards for survey and active tasks: SARA provides a variety of rewards; namely memes, life-insights, points and fishes; after daily survey and active tasks are completed. Providing rewards post data collection and shaping target behavior follow the operant conditioning theory. The operant conditioning theory states that if rewards are provided after a desired behavior (i.e., data collection in SARA) then the target behavior may happen more in the future. 4PM push notification with inspirational message: SARA often issues a push notification with an inspirational quote. The notification is issued at 4PM, which is two hours before 6PM when the data collection period begins. Our hope is participants will like the inspirational quote and collect data after 6PM. Giving such rewards in advance follows social exchange theories, which states that offering a gift or a gesture with no strings attached triggers the norm of reciprocity (i.e., an internal motivation to return a favor) in the recipient. Here SARA is an entity, which is giving an inspirational message in a manner that is non-contingent upon survey/task completion. To the extent the participant appreciates this gesture, s/he are expected to reciprocate via survey/task completion. 6PM reminder push notification: SARA issues a reminder notification at 6PM. The reminder message may or may not include an additional persuasive message. Providing the reminder notification is grounded in the trigger construct from BJ Fogg’s theory of persuasion. Fogg suggests a trigger is necessary to engage in target behavior. Fogg also argues that the trigger can be augmented with messages that spark interest or decrease perception of required effort. The persuasive message “Do you know it only takes a minute to fill out the survey and active tasks?” is issued to decrease the perception of required effort to collect data. On the other hand, the persuasive messages “Do you know you can earn money if you complete a 3-day streak?” and “You are close unlocking the next fish for the aquarium” are issued to spark interest and increase participant’s momentary motivation to collect data. [A Microsoft word document with the entire submission is attached as (filename: OSFSubmissionSARA2017.doc)]