Figures
Abstract
Background
Women’s physical activity levels decline during their transition to parenthood. Facebook is widely used by Australian mothers and provides the opportunity to target social networks in order to maintain and increase physical activity.
Method
This mixed method study aimed to pilot and assess the usability of the Mums Step It Up Facebook app, a new team-based physical activity intervention for mothers with young children. A purposive sample of five “Captain” women with young children, were recruited through personal contacts. These women used the app to recruit 3–7 Facebook friends (with children under 5) to join their respective teams (total n = 25). The app encourages women to take 10,000 steps a day measured by a pedometer. Women used the app for 28 days to log steps, interact with team mates and monitor progress. Physical activity was assessed at two time points (baseline and final week) using the Active Australia Survey. Usability testing with the five “Captain” women took place over two one hour face-to-face sessions. A questionnaire seeking feedback on the app was completed at time point two.
Citation: Kernot J, Olds T, Lewis LK, Maher C (2014) Usability Testing and Piloting of the Mums Step It Up Program - A Team-Based Social Networking Physical Activity Intervention for Women with Young Children. PLoS ONE 9(10): e108842. https://doi.org/10.1371/journal.pone.0108842
Editor: Daisuke Nishi, National Center of Neurology and Psychiatry, Japan
Received: June 10, 2014; Accepted: September 4, 2014; Published: October 1, 2014
Copyright: © 2014 Kernot et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. All relevant data are within the paper and its Supporting Information files.
Funding: The software development for the Mums Step it Up Facebook app was funded by a research development grant from the Division of Health Sciences at the University of South Australia (http://www.unisa.edu.au). JK is supported by an Australian Postgraduate Award Scholarship. CM is supported by an Australian Research Council Australian Postdoctoral Fellowship. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Insufficient physical activity is linked to many chronic diseases such as cardiovascular disease, diabetes, cancer and osteoporosis [1]. Furthermore, inactive people miss out on the benefits of regular physical activity such as the positive effects on mood, energy, wellbeing, cholesterol levels, insulin sensitivity and aerobic fitness [2], [3]. Women with young children are particularly at risk of physical inactivity [4], [5].
Online social networks offer considerable potential for the delivery of low cost, mass reach health-based interventions [6]. Facebook has 1.11 billion users world-wide, with over 50 per cent of users visiting the site daily [7]. In Australia, social networking accounts for one in every five minutes that Australians spend online [8], with Facebook attracting 13.2 million Australians each month [9] and nine million Australians each day [10].
Women’s use of Facebook increases during their transition to parenthood [11]. Eighty per cent of Australian mothers use Facebook daily, with mums identifying Facebook as their primary means of keeping in touch with friends and family [11], [12]. Key features of Facebook, such as its 24-hour day availability, flexibility and saliency, suggest that it may be a useful platform for the delivery of health interventions for this population.
A recent systematic review identified four studies which have attempted to use Facebook to alter physical activity behaviours [13]. Three of these studies used a Facebook community group with a discussion board, and produced modest results [14], [15], [16]. A strength of Facebook is the ability to recruit users and deliver an intervention via online social networks. One pilot study reporting a team-based physical activity intervention showed promising results, however the intervention was short in duration (5 days) and the sample size was small (n = 10) [17]. In recent times there has been increasing awareness that health issues and behaviours can “spread” through social networks. Recent work by Leahey et al [18], [19] suggests that teams can mediate the effect of health interventions.
In light of such findings, we created a Facebook app, titled “Mums Step It Up”, which is used to deliver a team based physical activity intervention for mothers with young children. This study aimed to determine the usability of the Mums Step It Up Facebook app. The primary aims were to 1) scrutinise: the ease of navigation, appropriateness of language, instructions, interest and overall appeal of the Mums Step It Up app; and 2) evaluate whether the app could successfully allow teams to self-assemble via existing social networks; The secondary aims were to: 3) examine the effectiveness of the app for changing physical activity behaviour; and 4) evaluate user retention and engagement with the app.
Method
This study meets PLOS ONE guidelines for new methods software and databases.
2.1 Ethics
This study was approved by the University of South Australia Human Research Ethics Committee (protocol number: 00000030420). Participants consented to participate via an online registration form which they completed as part of the Mums Step It Up app. Data were collected between April and June 2013.
2.2 Intervention
The Mums Step It Up Facebook app (http://apps.facebook.com/fbexper) is designed to encourage women to reach the recommended public health physical activity goal of 10,000 steps per day [20], [21], [22]. Women used the app for 28 days with the cumulative goal being 280,000 steps. They participated in teams of four to eight friends and measured their daily step count with a pedometer.
The Mums Step It Up app is based on the Theory of Planned Behaviour [23], [24] and Fun Theory [25]. The Theory of Planned Behaviour proposes that an individual’s decision to go ahead with a particular behaviour is influenced by three factors: attitude, subjective norms and perceived behavioural control [23], [24]. The app attempts to address each of these factors by: customising features to ensure they are relevant and appealing to mothers with young children (attitudes, perceived behavioural control); use of teams for peer encouragement and support (subjective norms); and setting small achievable goals (daily step count) which are recorded and contribute to a long term/overall goal (280,000 steps) (attitude, perceived behavioural control).
Fun theory is not a true theory as it is not based on rigorous research, but rather, a philosophical campaign which advocates that if routine activities are adapted to be fun, people are more likely to be motivated to do them [25]. The Mums Step It Up app has been designed to include a number of fun and interactive features; a comedian has assisted with writing daily tips for increasing physical activity, and awards which participants can unlock based upon step count, login and team achievements. Team mates can also send each other virtual gifts for encouragement. Additional feedback is provided regarding step count achievements via a team tally board, graphs and statistics on hours of life gained, fat burned, carbon emissions and transport costs saved. Participants receive weekly emails detailing their progress and reminding them to log their steps. Figure 1 includes a flow chart (based on the Kellogg Foundation Logic Development Guide [26]) of the program theory detailing the rationale behind the Mums Step it Up app. Figure 2 shows the app dashboard.
2.3 Participants
To be eligible to participate, women had to 1) have a child under five years old, 2) be current Facebook users, 3) be able to read and understand English and 4) live in greater metropolitan Adelaide (Australian city of approximately one million people). Women were excluded if they had a medical condition that prevented them from participating in a walking program or if they were pregnant or planning on becoming pregnant in the next three months.
Women were recruited in two stages. Firstly, a purposive sample of five women (“Captains”) was recruited through personal contacts. The second stage involved a snowballing recruitment method in which “Captains” used the team formation processes built into the Facebook app to recruit three to seven of their eligible Facebook friends each to join their teams (“Team Members”).
2.4 Procedure
“Captain” women attended two one-on-one sessions with the principal investigator (JK) where they were observed using the Facebook app. The sessions took place at their homes or at the University of South Australia, with approximately five weeks between sessions. “Team Members” had no face-to-face contact with the principal investigator; rather they participated via the Facebook app, with email and phone contact as needed. Following registration, all participants received a pedometer (NL-1000) in the mail.
This procedure was chosen to test the app as it is intended to be used (i.e. for recruitment/team formation and as a physical activity intervention). The detailed procedures related to each of the study aims are described below.
Usability issues.
Substantial alpha testing of the Facebook app by the researchers and software developers occurred in the five months prior, with the aim of identifying and rectifying technical issues before the commencement of the study.
Usability was scrutinised in two ways, 1) a feedback questionnaire completed by all participants in the last week of the Mums Step It Up challenge, and 2) by direct observation of “Captains” during the two face-to-face sessions. The feedback questionnaire (File S1) comprised 23 items designed to gather participants’ opinions regarding the app including: ease of use, appeal to the target audience, interest and impact of the key features, and suggestions for improvement.
At each face-to-face session, “Captains” were observed using the app, and asked to undertake all key tasks, such as sending invitations, logging daily steps, monitoring their own and their team’s achievements and interacting with team mates via the discussion board. Participants were asked to “think aloud” (i.e. verbally describe thought processes and decision making). The principal investigator observed participants and completed a checklist which detailed each of the steps, whether they were completed successfully and any questions asked. Observations were also made on body language, such as signs of confusion or frustration, and the time taken to complete each step.
Team forming process.
The Mums Step It Up app includes a team building feature. During the first face-to-face session the “Captains” sent invitations to eligible existing Facebook friends. This was completed via the app’s “invite friends page” (Figure 3), which includes a pop up listing of all their Facebook friends. Once they selected friends to invite, they were able to post the invitation with an optional personalised message on to their friends’ Facebook wall. When their friends clicked on the invitation it directed them to the app, where they could watch a video summarising the study information, and either decline or accept by completing an online registration and consent form.
a) Invite Friends Page. b) When participants click on the “invite friends” button, a pop up appears listing their Facebook friends. They select 3–7 friends that they would like to send an invitation to. c) When they click on the “send request” button, another pop up appears which allows them to send a personalised message to post on their friends’ Facebook wall. d)”Captain” women receive daily emails updating them on their friends’ responses to their invitations.
In the ensuing days “Captains” received daily emails informing them who had accepted the invitation to join their team. “Captains” could go back to the app at any stage to invite more friends or delete an invitation if one of their friends had indicated that they did not want to be involved, or hadn’t responded. Once the “Captain” had recruited at least three friends, they could finalise their team (via the app), or wait for more friends to respond.
Effectiveness for increasing physical activity.
Physical activity was assessed at two time points: 1) baseline, immediately prior to commencing the Mums Step It Up program, and 2) in the final week of the intervention. Physical activity was measured using the Active Australia Survey [27] which invites participants to recall their physical activities over the previous seven days. The survey includes questions relating to the frequency and duration of: walking (for exercise, recreation or transport); vigorous physical activities (such as jogging, cycling, aerobics and competitive sport); and moderate physical activity (such as gentle swimming, tennis and golf). The Active Australia Survey has been shown to have moderate reliability (reliability coefficient for frequency/duration ranged from 0.56 to 0.64 for each of the physical activity domains) [28] and moderate validity when compared with weekly pedometer step counts (rho = 0.43) and accelerometery (rho = 0.52) [28].
2.5 Data analysis
Participant characteristics (age, education, number of children, work status, and marital status) were analysed descriptively using means, standard deviations and frequency counts. Usability issues from the observation sessions and the feedback questionnaire were collated and an inventory was created of issues and suggestions for improvement of the app.
The team forming process was scrutinised by examining the app usage history records to determine the total number of invitations sent, number of invitations accepted, and days taken to form a team.
Usage data were descriptively analysed to determine participant retention and engagement.
To investigate the effectiveness of the app, total weekly activity time was calculated from the Active Australia Survey at baseline and 21 days. Since data were non-parametric, the Wilcoxon Mann-Whitney test was used.
Results and Discussion
The results relating to each of the study aims are presented together with discussion in the following section, in order to assist readers to comprehend the issues and subsequent actions that were taken during the study.
Usability of the app
Direct observation of participants using the app for the first time revealed that they were able to successfully navigate the app and undertake key tasks with no issue. Participant feedback obtained via the feedback questionnaires, indicated that the strengths of the app were its ease of use and navigation; that participants enjoyed being part of a team, and that they liked the ability to monitor their own and their team mates’ progress. The key usability issues were related to access of the app using different devices, operating systems and software. Participants reported difficulty finding the app on Apple devices (iPhone/iPad), as well as seeing all features on a Smart phone screen due to the small screen size. Variability in different users’ security settings and operating software were also found to affect some features of the app (e.g. sending invitations during the team-forming process).
Midway through the usability study, the issue of navigating to the app on mobile devices was addressed by providing a web link and instructions for Smart phone and iDevice users in all emails generated by the app including weekly emails (summarising step achievements and reminding participant to log their steps), and emails received when messages are posted on the message wall or when a gift is received. A “help tab” and additional instructions were also included to assist users to troubleshoot if they have difficulty with features due to security settings or operating software. These changes resolved the difficulties during the remainder of the usability study.
Team forming process
Ten women were invited to be “Captains”, of whom six expressed willingness to participate in the first face-to-face session. Each of these six women sent invitations to their Facebook friends to form respective teams, however only five teams proceeded. The sixth “Captain” was highly physically active, and reported difficulty recruiting team mates who were interested in participating in a pilot study related to physical activity, as her friends were also highly active. Since we had originally only planned to recruit five “Captains” to the study, it was decided not to replace this participant and to include five captains. The participation rate for “Team Members” was 68.4 per cent (percentage of women who accepted invitations; note that this includes invitations sent by the sixth “Captain” who did not successfully form a team). A summary of the team forming process is provided in Table 1.
Teams took a median of 13 days to form. During the pilot process it became apparent that team formation was impeded by three key issues: 1) some invitees downloaded the app but did not realise that they also needed to complete a registration page (within the app) to finalise their registration; 2) the app initially did not allow participants to formally decline an invitation, causing participants who did not want to participate to appear in limbo in the system; 3) “Captains” were reluctant to finalise their teams when there were friends who had not responded, even if they had met minimum numbers to form a team.
These issues were dealt with midway through the pilot period. Two teams (11 participants) were recruited following these changes which enabled further testing. A step-by-step graphic (Figure 4) was added to visually highlight the process involved in online registration. A “decline invitation” function was also added, which, rather than keeping non-participants in a limbo status, cleared them from the system, allowing “Captains” to send invitations to new potential participants. Since completion of the usability study, issue three was dealt with by providing “Captains” with written guidance on appropriate time frames (i.e. the team forming screen of the app, as well as the team-forming daily email update recommends they only wait a maximum of 7 days for their friends to respond before finalising their team).
One unexpected result was that during the study, a team which did not include women who had been specifically recruited for the study, formed on its own. This team was not included in the study as recruitment had already been finalised. It is likely that the “Captain” of this unexpected team saw the Mums Step It Up invitation on a friend’s Facebook page, demonstrating preliminary evidence of the potential viral capabilities of the intervention.
Effectiveness of the Mums Step It Up app
A total of 29 participants (5 “Captains” and 24 “Team Members”) joined the study of which 25 completed (5 “Captains” and 21 “Team Members”) (Table 2). Reasons for withdrawal included: major illness (n = 1), going overseas for an extended period (n = 1) and other commitments (n = 2). Due to missing data these four women were not included in the analysis. However, all participants who provided 21 day follow up data were included in the effectiveness analyses, irrespective of intervention compliance.
Participants’ physical activity levels at baseline and 21 days were compared using Wilcoxon Mann-Whitney test. Participants’ physical activity significantly improved across the intervention period from a mean (SD) of 294 (307) minutes per week at baseline, to 471 (437) minutes in the final week of the program (p = 0.01). The number of physical activity sessions per week and the mean minutes per week increased in each of the activity categories. At both time points, walking was the most popular physical activity.
Participant engagement and retention
Usage data recorded by the Mums Step It Up app is summarised in Table 3. In general, usage rates were high. Of the 25 participants who completed assessments at both time points 18 logged steps for all 28 days; 22 participants logged steps for more than 21 days; and 24 logged steps for more than 14 days. The minimum number of days steps were logged was 13.
The mean number of weekly step entries (ranging from 6.9 (SD 0.3) to 5.5 (SD 2.8)) was high compared to other social networking interventions which have included logging of physical activity [16] (maximum mean entries per week over 12 weeks was 2.1 (SD 3.4)) or dietary behaviour [29] (maximum entries per week over 6 months was 2.9 (SD 1.9)).
A modest decline in log in rates (the number of logins, numbers of days for which steps were logged and number of steps taken) was observed towards the end of the 28 day period, which is consistent with other web-based physical activity interventions [30].
The attrition rate for this study was 14 per cent, showing promise compared to other web-delivered physical activity interventions (average attrition of 27%) [30]. However, it must be acknowledged that the intervention period in this study was relatively short and attrition would likely increase over a longer time frame.
As expected, logging steps was the most used feature. Sending gifts wasn’t well utilized, and feedback by participants at the end of the program suggested that this may have been due to lack of clarity about the gifts and/or the limited range of gifts available. This has since been addressed by increasing the range of gifts available (new gifts are released during the program as participants reach particularly step count milestones) and a short description of the gift (e.g. “sent flowers for encouragement”) appears when users hover their mouse over the gift.
Conclusion
This study aimed to assess the usability and pilot the Mums Step It Up app, a new social networking physical activity intervention for women with young children. The key findings highlighted that women with young children found the Mums Step It Up app engaging (with high usage rates and weekly step entries). The online team-building process which is a unique feature of the app appears to be feasible. In addition, the study provides preliminary evidence that a social networking intervention is effective in bringing about behaviour change.
The technical challenges of delivering a physical activity intervention in the Facebook environment where a wide variety of devices, operating systems and software platforms are in use, was highlighted. Given that these technologies are rapidly evolving, addressing such issues will be an ongoing process, and presents a sizable challenge to programmers working in the online social-network environment who wish to create robust systems.
The snowballing recruitment method and self-organising nature of this intervention were strengths of this study. It is acknowledged that the small sample size, homogenous nature of the sample (largely tertiary educated, active women), lack of control group, and use of a self-reported physical activity measure (which had moderate reliability and validity; similar to other self-report physical activity surveys [31]) limit the evidence regarding the effectiveness of the Mums Step It Up program. A randomised controlled trial is needed to see if behaviour change can be replicated with a higher quality research design, and maintained over a longer period.
Acknowledgments
We thank Portal Australia, a South Australian based company, who was employed to design the Mums Step It Up app.
Author Contributions
Conceived and designed the experiments: JK CM TO LKL. Performed the experiments: JK. Analyzed the data: JK CM. Contributed reagents/materials/analysis tools: TO. Contributed to the writing of the manuscript: JK TO CM LKL.
References
- 1. Warburton DER, Nicol CW, Bredin SSD (2006) Health benefits of physical activity: the evidence. Canadian Medical Association Journal 174: 801.
- 2. Larson-Meyer DE (2002) Effect of postpartum exercise on mothers and their offspring: a review of the literature. Obesity 10: 841–853.
- 3. Blum JW, Beaudoin CM, Caton-Lemos L (2004) Physical activity patterns and maternal well-being in postpartum women. Maternal and Child Health Journal 8: 163–169.
- 4. Brown WJ, Trost SG (2003) Life transitions and changing physical activity patterns in young women. American Journal of Preventive Medicine 25: 140–143.
- 5. Pereira MA, Rifas-Shiman SL, Kleinman KP, Rich-Edwards JW, Peterson KE, et al. (2007) Predictors of change in physical activity during and after pregnancy: Project Viva. American Journal of Preventive Medicine 32: 312–319.
- 6.
Chou WYS, Hunt YM, Beckjord EB, Moser RP, Hesse BW (2009) Social media use in the United States: implications for health communication. Journal of medical Internet research 11.
- 7.
Statistics Brain (2013) Facebook Statistics. Available: http://www.statisticbrain.com/facebook-statistics/. Accessed 20 October 2013..
- 8.
Public Relations Institute of Australia (2012) 13 million Australians spend 18 hours a day online. Available: http://www.pria.com.au/industrynews/13-million-australians-spend-18-hours-a-day-online. Accessed 4 June 2014.
- 9.
Cowling D (2014) Social Media Statistics Australia - June. Available: http://www.socialmedianews.com.au/social-media-statistics-australia-june-2014/. Accessed 20 July 2014.
- 10.
Campaign Brief (2013) Facebook Australia celebrates 12 million total monthly active users on desktop and mobile. Available: http://www.campaignbrief.com/2013/12/facebook-australia-celebrates.html. Accessed 4 June 2014. Australia.
- 11. Bartholomew MK, Schoppe-Sullivan SJ, Glassman M, Kamp Dush CM, Sullivan JM (2012) New Parents’ Facebook Use at the Transition to Parenthood. Family Relations 61: 455–469.
- 12.
News.com.au (2012) Facebook reliance starts with mums- so log off. Available: http://www.news.com.au/technology/facebook-reliance-starts-with-mum-so-log-off/story-e6frfro0-1226352510497?from=public_rss. Accessed 24 May 2012..
- 13.
Maher CA, Lewis L, Ferrar K, Marshall S, De Bourdeaudhuij I, et al.. (2013) Are health behaviour change interventions that use online social networks effective? A systematic review. Journal of Medical Internet Research: In press.
- 14. Cavallo DN, Tate DF, Ries AV, Brown JD, Devellis RF, et al. (2012) A social media-based physical activity intervention: a randomized controlled trial. American Journal of Preventive Medicine 43: 527–532.
- 15.
Napolitano M, Hayes S, Bennett G, Ives A, Foster G (2012) Using Facebook and text messaging to delivery a weight loss program to college students. Obesity: Epub 24 April.
- 16. Valle CG, Tate DF, Mayer DK, Allicock M, Cai J (2013) A randomized trial of a Facebook-based physical activity intervention for young adult cancer survivors. Journal of Cancer Survivorship 7: 355–368.
- 17.
Foster D, Linehan C, Kirman B, Lawson S, James G (2010) Motivating physical activity at work: Using persuasive social media for competitive step counting.. 14th International Academic MindTrek Conference: Envisioning Future Media Environments,. Tampere. 111–116.
- 18. Leahey TM, Crane MM, Pinto AM, Weinberg B, Kumar R, et al. (2010) Effect of teammates on changes in physical activity in a statewide campaign. Preventive Medicine 51: 45–49.
- 19. Leahey TM, Kumar R, Weinberg BM, Wing RR (2012) Teammates and social influence affect weight loss outcomes in a team-based weight loss competition. Obesity 20: 1413–1418.
- 20.
World Health Organisation (2010) Global Recommendations on Physical Activity for Health. Available: http://whqlibdoc.who.int/publications/2010/9789241599979_eng.pdf. Accessed 24 April 2012.
- 21.
World Health Organisation (2012) 10 Facts on Physical Activity. Available: http://www.who.int/features/factfiles/physical_activity/facts/en/index.html. Accessed May 29 2012.
- 22. Haskell WL, Lee I, Pate RR, Powell KE, Blair SN, et al. (2007) Physical activity and public health: updated recommendation for adults from the American College of Sports Medicine and the American Heart Association. Medicine and Science in Sports and Exercise 39: 1423.
- 23.
Azjen I (1985) From Intentions to Actions: A Theory of Planned Behavior. In: Kuhl J, Beckmann, J, editor. Action-control: From cognition to behaviour. 1st ed. Heidelberg: Springer. 11–39.
- 24.
Azjen I (1987) Attitudes, traits and actions: Dispositional prediction of behaviour in personality and social psychology. In: Berkowitz L, editor. Advances in Experimental Social Psychology. 1st ed. New York: Academic Press. 1–63.
- 25.
Volkswagon (2009) The Fun Theory. Available: http://www.thefuntheory.com/. Accessed March 20 2012.
- 26.
W.K. Kellogg Foundation (2004) Logic Model Development Guide. Michigan, USA.
- 27.
Australian Institute of Health and Welfare (AIHW) (2003) The Active Australia Survey: a guide and manual for implementation, analysis and reporting. Canberra: AIHW.
- 28. Brown WJ, Burton NW, Marshall AL, Miller YD (2008) Reliability and validity of a modified self-administered version of the Active Australia physical activity survey in a sample of mid-age women. Australian and New Zealand Journal of Public Health 32: 535–541.
- 29. Turner-McGrievy G, Tate D (2011) Tweets, Apps, and Pods: Results of the 6-month Mobile Pounds Off Digitally (Mobile POD) randomized weight-loss intervention among adults. Journal of medical Internet research 13: e120.
- 30. Vandelanotte C, Spathonis KM, Eakin EG, Owen N (2007) Website-Delivered Physical Activity Interventions: A Review of the Literature. American Journal of Preventive Medicine 33: 54–64.
- 31. van Poppel MN, Chinapaw MJ, Mokkink LB, Van Mechelen W, Terwee CB (2010) Physical Activity Questionnaires for Adults. Sports Medicine 40: 565–600.