Personalization Revisited: A Reflective Approach Helps People Better Personalize Health Services and Motivates Them To Increase Physical Activity Min Kyung Lee1, Junsung Kim2, Jodi Forlizzi1, Sara Kiesler1 1
HCI Institute, Carnegie Mellon University Pittsburgh, PA, USA {mklee, forlizzi, kiesler}@cs.cmu.edu
ABSTRACT
Current approaches to personalization either presuppose people’s needs and automatically tailor services or provide formulaic options for people to customize. We propose a complementary approach to personalization: a reflective strategy that helps people realize what matters to them and enables them to better personalize services themselves. To design this strategy, we first studied the practices of eight personal health service providers. We then tested the strategy’s efficacy by building a Fitbit Plan website that encouraged Fitbit users to customize a plan or accept an automatically tailored plan. For one group of users, the website used the reflective strategy to assist in the plan setup process. A two-week between-subjects field experiment showed that the reflective strategy helped motivate users to carry out their plans, increasing their average daily steps by 2,425 steps. Without the reflective strategy, users either set easy goals or failed to carry out system-created plans, ultimately showing no change in their average daily steps. This work suggests that helping people reflect on and connect with their own goals in using a personalized service could advance the effectiveness of the service. Author Keywords
Reflective personalization; service; behavior change; health; fitness; personal informatics; activity tracker ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION
Personalization has become an integral part of technologybased services, with an ever-growing influence on the decisions that people make and the content and services Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. UbiComp '15, September 07–11, 2015, Osaka, Japan. Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-3574-4/15/09…$15.00 http://dx.doi.org/10.1145/2750858.2807552
2
Ottomatika Inc. Pittsburgh, PA, USA
[email protected] they access. Social media feeds on Facebook, advertisements on Amazon, search results on Google, physical activity goals on the Apple Watch, and even interactions at Disney parks are automatically personalized based on people’s behavior traces, including browsing, purchasing, and communication records physical activity levels, and locations [2, 6, 13, 40, 62]. In addition to this system-driven personalization, many technologies allow people to directly customize service features [5, 47] or the user models on which personalization is based [63], making the personalization process user-driven. Prior research has shown that personalization in general offers many benefits: reduced cognitive and physical overloads, faster task completion, and increased persuasion, satisfaction, and loyalty to services [4, 5, 19, 26, 27, 30, 34, 59]. However, we argue that personalization can be further improved. In system-driven personalization, systems infer what users want and need based on the assumption that their user models accurately represent real users. However, this assumption is often false as the models are based on specific, limited instances and treat users as objects of algorithmic optimization [32, 40]. The limitations of the user models can result in irrelevant or even offensive personalization outcomes [11]. The automatic–and thus often imperceptible–implementation of system-driven personalization exacerbates the problem. The hidden assumptions of user models may unconsciously bias decision-making [55], habituate people to follow directions mindlessly [7, 48], or make them focus only on things they already like [44, 58]. User-driven personalization (also known as customization) offers alternatives for people to choose from and tailor. However, the choices are often formulaic and superficial; previous research reveals many reasons why people may not be able to customize or tailor options to best satisfy their needs [46, 53]. People may feel overwhelmed or even experience decreased satisfaction when presented with too many choices [23]. We propose an alternative approach to personalization, which helps people to better personalize services themselves. Rather than assuming what will work best for people or presenting them with options from which to choose, our approach prompts people to reflect on their
goals and priorities, so that they can focus on what is most important to them when using and modifying services. We envision personalization as a form of empowerment, helping people realize what outcomes matter to them and giving them the agency to prioritize those outcomes. In this paper, we designed a reflective personalization approach informed by interviews with personal service providers. We tested the efficacy of the reflective approach using Fitbit [14], a fitness service that tracks steps. Among many services that could benefit from the reflective approach, we focused on goal setting as it is a ubiquitous feature in technology services with a cascading effect in later service use. We chose a personal health service because health is one of the domains where personalization is increasingly seen as a transformative factor. We built Fitbit Plan, a website that let Fitbit users set step goals and track their progress. Fitbit Plan allowed users to either create their own goals or follow suggested plans based on their performance over the previous two weeks. The reflective strategy was used during the set-up process to encourage some users to think deeply about why they wanted to increase their daily steps. We conducted a 2 (Reflective vs. Non-reflective strategy) x 2 (User-driven vs. System-driven personalization) field experiment for two weeks to test the reflective strategy. The results show that adding the reflective strategy motivated users to increase their daily steps by 2,425 steps on average over two weeks. Without the reflective strategy, users either set easy goals or failed to carry out the plans that the system suggested, ultimately showing no change in their average daily steps. This paper makes two contributions. First, we introduce a new approach to personalization that broadens the design space. Second, we demonstrate the efficacy of this approach in a field study and discuss design implications for both reflective service personalization and future research on personalization and behavior change in HCI. REFLECTIVE APPROACHES IN HCI
There is a long history of HCI research on how more active user participation and reflection play out in such domains as office technology design [10], end-user programming and development [15], computer-supported learning [42], reflective design [3, 51], slow technology [18], online discussion participation [28], sustainability [60], and personal health [21, 38, 49]. Collectively, this work reveals the benefits of empowering users by giving them the freedom to interpret technologies and to appropriate them for their own use. Yet, this line of work has not influenced how personalization has been conceptualized and implemented in the fields of HCI, business, or computer science. Drawing from this body of work, we explore a reflective approach to personalization, in which users are given opportunities to think about what matters to them with the guidance of technology-based services. When given this agency, people may be better able to create personalized services that work best for them.
EXPLORING THE PERSONALIZATION OF HUMAN SERVICE PROVIDERS
STRATEGIES
To create our model for personalization technology, we studied personal service providers. Services were being personalized long before computers were invented. Service roles such as aide-de-camp, court jester, tutor, and butler existed centuries ago for high-status and wealthy people. Today, personal service providers perform a myriad of personalized services at many levels of society. In this paper, we ask how personal service providers have navigated the problems of determining and adapting to people’s preferences. Prior work suggests that practitioners use principled approaches to understand unique problem situations and optimize solutions for each problem [50]. Method
We interviewed eight personal service providers (five women) who offered expert services to improve clients’ physical and mental wellbeing in an Eastern city in the U.S. The providers included two personal trainers, two physical therapists, a chiropractor, a counselor, a massage therapist and a personal tutor. They had worked in their occupations for 15.6 years on average. We recruited them using referrals from clients and snowball sampling. We also directly contacted professionals whose business contact information was publicly available. We aimed to recruit providers with experience, as experts are reported to provide better care than novices [24]. Interview protocol
The hour-long interview began with questions about the professionals’ work contexts and services, their typical relationships with clients, and whether any of their work was tailored to different individuals. Further questions probed what information the professionals needed to do a good job with their clients, how they listened to feedback and observed cues in their clients’ behavior, and how they handled breakdowns in interactions with their clients. Analysis
The interview recordings were transcribed for analysis. We grouped personalization instances into three coding categories, based on how service providers collected information about a client, tailored services to the client, and evaluated their services and used client feedback [13]. We also added codes that described details of how providers personalized their services and the reasons they gave for doing so [56]. We re-grouped lower-level codes into clusters that reflected themes of personalization across the three stages, and used the themes to consider current practices of technology–based personalization. Results
Two key results informed the design of our reflective strategy. Providers act as catalysts for reflective service processes
When clients lacked knowledge and expertise about a service domain, service providers played active roles in personalizing the service. The clients provided information
about their goals and motivations, but it was the provider’s role to devise a program with specific tasks. For example, a physical therapist created a simple storefront mockup for a client who wanted to return to her cashier job in a bakery after her injury. Together, they practiced sales interactions and monitored the client’s progress in managing the demands of the job. After the therapist understood the client’s goals, the client’s initial preferences regarding the service played a comparatively insignificant role in determining the ultimate nature of the service. Providers help clients realize deeper goals and motivations
The success of the services that we studied depended on clients’ presence and active participation. Providers explained that understanding clients’ goals was critical for eliciting and maintaining engagement over time. Sometimes, however, clients did not think about or initially share their deeper motivations with service providers. Here, providers guided clients to think more deeply about their goals and motivations, so that they could personalize their interactions and sustain client motivation over time. For example, a personal trainer said he tried to understand the deeper motivations of his clients, even when they initially appeared to have well-defined goals: (My clients) may say that their goal is to lose ten pounds, or lower their BMI, blood pressure or to be more fit or healthy; these are very generic goals, and are not what truly motivates them. For example… they have known that they may die early if they do not change their lifestyles, but this did not prompt or motivate them to change and come here. There is always something more emotional and motivating for different individuals, even though it may not be related to their biggest problem (threats to their health). So I keep asking ‘why’ to truly understand that motivation. At the end, they say it is to fit into their jeans that they used to wear when they were young… or look better to their husband, or get over the break-up. What is noteworthy here is that the service provider’s role is to repeatedly ask questions to get at hidden motivations. Providers used this information to tailor service offerings and interactions. For example, the personal trainer used this information to personalize his exercise programs and his feedback to clients, so that he could support the goals that clients cared most about. A personal tutor used his understanding of a student’s goal of going to pharmacy school in tailoring examples that he used to explain math and chemistry concepts. These shared experiences made the client-provider relationship more collaborative and caring. DESIGNING REFLECTIVE PERSONALIZATION
Our study of human service providers shows that service providers seek to understand clients’ deeper goals and motivations, and use this knowledge to create personalized plans and maintain motivation through tailored feedback. What is critical in this process is the very first step in which providers guide their clients to reflect on and articulate what motivates them, so that clients can orient themselves
to their underlying needs and goals. This focus contrasts starkly with how computer-based services are currently personalized. Most current technology services, such as online shopping and education websites, product and news recommendations, and health management tools do not help users determine their goals. Instead, they either prescribe services that systems think will work best for users, or allow users to personalize the solution spaces of their services on the assumption that the users’ goals at the time of personalization are fully realized. This insight played a key role in designing the reflective strategy, which prompts users to think more deeply about their goals through a series of questions. How questions facilitate reflective personalization
The reflective strategies and processes that human service providers use are greatly nuanced. These processes include rapport and trust creation through shared goals, and interactive, dialogue-based goal and reflection elicitation. We distilled these processes into open-ended questions, focusing particularly on the “why” question, to translate human conversational strategies into computational ones. The simplicity and feasibility of open-ended questions allow for the flexible application of the strategy in a wide array of tasks and domains. We then turned to the literature on reflection in order to gain insights into the effects of reflection beyond those we observed in our interview study. Why “why” works
Previous research suggests that asking reflective questions can help people articulate their underlying needs and goals and increase their motivation to use a service. The reflective strategy may elicit contemplative [25] or metacognitive [16] thinking, allowing people to think about their needs and wants beyond the first answers that come to mind. Research suggests that asking people their reasons for doing an activity triggers their underlying motivations and leads them to focus on higher-level goals [8, 61]. This reasoning can help people overcome potential decision biases [41] and give people opportunities to personalize service features to better satisfy their underlying needs and goals. Reflective strategies may encourage people to focus on the long-term consequences of their choices and make decisions that are more in line with their ideal selves. In one study, people who were asked to think about why they eat snacks before making a choice were more likely to choose healthy options [17]. In another study, people exerted greater self-control tolerating physical discomfort when they were reminded of the purpose of the activity [17, 61]. This suggests that a reflective strategy can be particularly useful for services in which people might benefit from orienting themselves to long-term, deeper goals. Greater awareness of underlying needs and goals attached to ideal selves can increase people’s motivation for using a service. Research shows that people are motivated to satisfy their own needs [39] and to work on tasks that match their identities [1, 52]. In the absence of the reflective strategy,
people may not consider their individual motivations and goals when personalizing services. How to ask “why” to elicit reflection
behavior. Much HCI research has investigated different ways to use digital technologies to improve how people manage their health [9, 12, 19, 20, 29, 36, 38, 43, 49].
The success of reflective questions depends on whether they elicit deeper reflection. When service providers interact with clients, they often ask follow-up questions. There is also social pressure for clients to cooperate with the providers. Without the dialogue and social pressure, people may be less inclined to comply with computer-based services, and provide only superficial answers to questions. In order to address these challenges, we reviewed previous research in which reflective techniques were used successfully and adopted several of these in our design. These included asking “why” questions twice [17], instructing clients to take more time to think about the question, and asking clients to write longer answers than they usually would do [54].
We used Fitbit [14], a physical activity tracker, as the basis for our personalized service design. Walking is a critical activity in personal health, as walking 10,000 steps or more daily has been shown to offer many health benefits [22]. The Fitbit website offers both user-driven and systemdriven personalization features for goal setting and planning. Fitbit users can set up a single goal for their activity by typing in the number of steps they desire to achieve. If Fitbit users want to set up a plan to increase their daily steps, they can enroll in a “personal trainer” service with a yearly subscription fee. This feature presents a plan in which daily step goals increase from a current baseline. Users can follow the plan to increase physical activity.
Goal-setting and planning in personal health tracking
On the Fitbit Plan website, Fitbit users could set up a twoweek goal and track their progress (Figure 1). The Fitbit Plan was built on Google App Engine using Fitbit Developer API. Highcharts JS was used to visualize users’ progress, and a portion of back-end implementation was adopted from [12]. Once users signed up to the website and gave their consent, Fitbit Plan automatically retrieved users’ daily step data from their Fitbit accounts. Fitbit Plan asked six reflective questions designed to guide people in thinking about their underlying motivations and goals, and to discover easy ways to implement them and remove any barriers to healthier lifestyles. Users could then create personalized two-week plans. Alternatively, Fitbit Plan recommended a personalized plan based on baseline steps and recommended guidelines.
We evaluated the efficacy of the reflective strategy in the task of creating personalized plans for health. We built the Fitbit Plan website, where Fitbit users could set up personalized two-week plans for increasing their daily steps with the reflective strategy. Goal-setting and planning
Among the many service aspects that can be personalized, we focused on goal-setting and planning. Goal-setting and planning are widely-used service features in a number of domains [37]. They are only a small part of the larger service process, generally taking place once at the start of a service, but they are a critical part of the service process. They can have a cascading effect; once people set their goals, they tend not to change the settings over the duration of the usage [48, 58]. In addition, allowing for goal-setting and planning has been shown to improve people’s service adherence and motivation [37]. The domain of personal health tracking
Many digital technologies are currently changing how people manage their health by offering aggregated sensor data and tools to document and manage health-related
Fitbit Plan Website
EVALUATING REFLECTIVE PERSONALIZATION
We conducted a 2x2 between-subjects field experiment. The effect of the reflective strategy was evaluated with two main personalization methods – user-driven and systemdriven personalization. Recruitment and administration of the study took place entirely online through email and on the Fitbit Plan website.
Figure 1. ©Min Kyung Lee. Fitbit Plan website
Hypotheses
Based on our goals of articulating needs and increasing motivation to use a personalized service, we explored the following hypotheses in the study: Hypothesis 1. The reflective strategy will help people articulate their underlying needs, goals and motivations for increasing physical activities. Hypothesis 2. People will be more motivated to carry out the personalized health plan by a technology system that employs the reflective strategy than by one that does not. Participants
We recruited Fitbit users who wanted to increase their daily steps and try a daily-steps planning website. We posted a call for research participation on online Fitbit community boards, and on Facebook, Craigslist, and Twitter using the Fitbit hashtag. We used a pre-survey to limit participation to those who were older than 18 years, resided in US, were not pregnant, were proficient English speakers, selfreported that they wore their Fitbit almost everyday, and intended to increase their daily steps. In total, we recruited 62 Fitbit users (74% female, age ranges from 19.5 to 68 (M=38.7 (SE=1.62)). 37 were Caucasian, 9 were Asian American, 2 were Hispanic, 2 were of mixed origins, 1 was African American, and the rest did not wish to report. On average, participants had used Fitbit for 8 months (SE=1.44 month). Participants who completed the two-week long study were compensated with a $10 Amazon gift card. Experimental conditions
The experiment manipulated whether or not participants answered reflective questions and who personalized the two-week plan, the user or the system (Table 1). User-driven System-driven
Reflective N=15 (11) N=17 (11)
Non-reflective N=14 (11) N=16 (7)
Table 1. Field experiment conditions and participant numbers. The first numbers indicate participants who set up their plans and filled out the Day 0 survey. The numbers in parentheses indicate participants who used Fitbit Plan and filled out the Day 14 survey. Reflective strategy conditions
Six reflective questions were presented right before participants were asked to construct their plans. The first question asked why they wanted to increase their daily steps. The second question asked them to elaborate on their first answers (“OK. Think about what you just described. Why is this important to you?”). We encouraged them to think about this question for at least one minute, but we did not force them using a timer. The next questions asked about easy ways to start moving toward their goal, expected barriers, and their plans for addressing those barriers, in order to help them think about realistic goals. Finally, we asked how many minutes they could spend walking without disrupting their routines. Participants’ answers to these
questions were displayed on the page along with a graph of their goals and progress. Personalization conditions
In the user-driven condition, Fitbit Plan asked users to type in daily step goals for two weeks. As a baseline and a reference, the page showed their average steps during the previous two weeks. We asked them to create a plan that was challenging (more than their baseline), yet realistic (achievable without requiring major effort). To encourage success, we also told them to try to accommodate their schedules. Once they typed in their goals, the goals were presented as a graph on the next page. In the system-driven condition, Fitbit Plan devised a personalized two-week plan by following US Health agency recommendations, as well as guidelines for walking programs used in academic research and suggested by popular health information resource sites such as Mayo Clinic. Fitbit Plan used the participant’s average number of steps for the previous two weeks as a baseline. We told participants that we followed US Health agency recommendations to create a challenging yet realistic plan. The system gradually increased their daily steps, raising their baselines by 2,000 additional steps by the end of the first week and by an additional 1,500 by the end of the second week. This resulted in a 3,500 daily step increase on the last day of the two weeks, with an average daily step increase of 2,000 during that time – approximately equivalent to an additional 20 minutes of walking each day. Once participants set up a two-week plan, they could track their progress in relation to their goals on Fitbit Plan. They were not able to modify the plan during the two-week period. We made this choice intentionally; the two-week plan is a good way for participants to explore their limits. If they wanted to continue to increase their daily steps or maintain their increased steps, they could make changes after two weeks. This also allowed us to control the experimental manipulation. Procedure
People who contacted the study organizer were asked to fill out a pre-survey. Qualified participants were randomly assigned to one of the four conditions. We created four versions of Fitbit Plan, in order to satisfy each condition. The study organizer sent each participant an email with a link to an online consent form and the instructions on how to set up a two-week long plan to increase daily steps on Fitbit Plan. Then participants were asked to fill out the Day 0 survey. After two weeks of using Fitbit Plan, participants were asked to fill out the Day 14 survey, and compensated. No reminders were sent to the participants during the two weeks of the study period. Data sources and measures
We collected behavioral and subjective data from the website’s plan and daily step logs and from the surveys.
Two-week plan and daily step logs
We used daily step logs on Fitbit Plan to understand how well the participants carried out their plans. To calculate changes in daily steps, we compared their daily average over two weeks of study participation with their baseline, which was their daily average from the two weeks before the study. There were no differences in the baseline numbers of daily steps across the conditions (Daily M=9,801 (SD=5,541)). To calculate each participant’s plan completion rate, we averaged the ratio of the step goal in the plan with the actual steps that the participant walked for each day over two weeks. For participants who created their own plans (user-driven), we analyzed the goals that they set for two weeks by calculating how many daily steps, on average, participants planned to increase over two weeks. Answers to reflective questions
We examined participants’ answers to the reflective questions to see whether the questions elicited reflection on their underlying motivations and goals. We then categorized their answers by the types of reasons that participants provided, how they felt their goals contributed to their identities, and whether their reasoning was based on emotion, past experience, or knowledge. We also calculated the length of the answers. Surveys
We conducted an initial Day 0 survey right after participants set up their plan, and an exit Day 14 survey after their two-week usage. We asked the same question sets as repeated measures to detect changes. All questions used 7-point Likert-type scales, unless noted otherwise. Initial survey (Day 0). We asked about participants’ experiences of setting up the plan. Perceived control over the personalization process. Two items asked whether people felt that they had an appropriate level of control over the process (Cronbach’s α=.8): “I felt that I had an appropriate level of control over the process of setting up the plan” and “I felt that I could influence the process and outcome of setting up the plan.” Engagement with the personalization process: Two items measured how engaged people were in creating the plan (Cronbach’s α=.8): “I was highly motivated to pay close attention to this plan,” and “While creating this plan, I could have worked harder on it.” Motivation to carry out the personalized plan: Five items adopted from [57] measured how motivated participants were to follow the plan that they created at the moment of creation (Cronbach’s α=.7): “Doing well on this plan is important to me,” “I will put in good effort throughout following this plan,” “I am not concerned about how successfully I will carry out this plan,” “I will put forward my best effort in carrying out this plan,” and “While following this plan, I will persist to completion of the plan.”
Perception of services being personalized: Two items asked how personalized the participants felt their plans were [27] (Cronbach α=.82): “The plan is personalized for me and my context,” and “This plan accommodates what I want.” Perceived difficulty of and satisfaction with the plan: One question asked how much effort participants would need to put into completing the plan. We also asked how satisfied participants were with the plan. Manipulation checks: We asked how helpful Fitbit Plan had been in guiding participants to think about their goals and constraints, and how evenly participants felt the work of setting up the plan had been divided between themselves and the Fitbit Plan website. Exit survey (Day 14). The questions in the Day 0 survey– motivation to carry out the plan, perceived level of personalization and difficulty, and satisfaction with the plan– were rephrased to measure how participants felt while carrying out their plan rather than at the moment of creation. Additional questions asked how often they had remembered their daily goals, how much they enjoyed following the plan, and whether they wanted to decrease, maintain or increase their daily steps going forward. Results
The study evaluated the effects of adding a reflective strategy to prevalent personalization strategies – user-driven and system-driven personalization. Both of our hypotheses were supported, suggesting that the reflective strategy can help users of personalized services achieve their goals. Manipulation check
Our manipulations were successful. Participants in the reflective conditions felt that Fitbit Plan helped them think about their motivations and goals before setting up their plans (Reflective=4.78 (SE=.26), Non-reflective=3.1 (SE=.27), F(1,60)=20.17, p