In the last few weeks much has been made of the physical activity levels of children and adolescents in England. In late November, as part of the Active Healthy Kids Global Alliance Global Matrix 3.0, the Active Healthy Kids England 2018 Report Card was published. The Report Card, which is led by Professor Martyn Standage (@Martyn_Standage) and colleagues builds on the 2014 and 2016 versions to provide an update on performances on, the provision for, and support for physical activity opportunities for children and young people. Using nationally representative self-report survey (questionnaire) data, it demonstrated that 23% of boys and 20% of girls achieved the UK Chief Medical Officers’ guidelines of at least 60 minutes moderate-to-vigorous physical activity (MVPA) per day. When combined with accelerometer (activity monitor) data from a large study of primary school children in the south-west that reported higher proportions of children meeting the guidelines, the Report Card gave a grade C- for English children’s ‘overall physical activity’.
Hot on the heels of the Report Card in early December, was the Sport England Active Lives Survey for children and young people (ALS). Sport England reported how an impressive 130,000 children completed the online survey in school. The headline results were that 17.5% of children are active (i.e., they engage in MVPA for at least 60 minutes every day), and that 25.7% of children average at least 60 minutes MVPA per day over the week (which is how active/inactive is commonly reported in research studies). Other insights from the ALS include breakdowns of in-school versus out-of-school activity anchored to children achieving an average of 30 minutes MVPA per day. It was also useful to see the physical activity data presented in relation to ethnicity, family socioeconomic profile, and disability, as well as the insights into types of activities that children engage in, volunteering, and perceptions of wellbeing, Other aspects and correlates of children’s activity behaviours (e.g., attitudes) were also reported.
It’s a very positive thing that work of the scale of the ALS is underway, leading to children’s physical activity being discussed in non-academic circles, which helps to raise awareness of the need to improve activity levels, and hopefully lever policy and funding support for this agenda. However, for users of such research and insight evidence, it would be very useful to know how the self-report responses were treated to generate the estimates of MVPA, sedentary time, etc. The ALS technical notes explain how the survey questions were formulated and tested, but little detail is provided about how the accuracy of the resultant physical activity estimates were assessed. Reference is made to an ‘objective study’ but no other information is provided other than this rather vague footnote:
“In the objective measurement study pupils used a wrist worn accelerometer and completed the questionnaire so that their objective data could be compared to the data collected in the questionnaire.”
As a matter of course, physical activity researchers have to report the validity evidence for their chosen methods. This is so research users know that physical activity estimates are generated as a result of rigorous and robustly tested processes. The absence of this level of detail in the ALS is a somewhat surprising omission. There are sources of error or bias in any physical activity measurement method, and it’s well established that self-report questionnaires are prone to specific types of measurement error (e.g., related to recall, specification error, processing error, etc – see Nusser et al., 2012 for a brief summary). Indeed, a recent systematic review of 89 published youth physical activity questionnaires concluded that none had conclusive evidence of acceptable validity and reliability. So, it’s actually the norm for self-report data to lack the same level of validity as that from device-based measures, such as accelerometers. Although the reduced cost and increased usability of accelerometers has improved the possibility of their deployment in large scale studies, there is still a place for self-report tools, particularly when they are administered online.
Good examples of self-report tools that have applied robust statistical methods using accelerometers to produce accurate estimates of activity do exist. At Iowa State University, Saint-Maurice, Welk, and colleagues developed and tested a ‘calibration’ approach to develop the Youth Activity Profile (YAP). This 15-item online survey employs algorithms generated from the calibration of YAP scores with temporally-matched estimates of activity from accelerometers. The published results were impressive, with MVPA from the YAP estimated to within ~3 minutes/day compared to the accelerometer. As is the case with the ALS, the YAP has great reach and has been employed in the NFL Play 60 programme, capturing data from approximately 250,000 children in 1120 schools. In collaboration with colleagues at Iowa we adapted the YAP and conducted our own calibration study with around 400 children and adolescents from northwest England. The results showed that the preliminary UK-specific YAP algorithms generated estimates of school-day and out-of-school MVPA that were within 3.5 min/day and 11.8 min/day respectively, of the equivalent accelerometer-based estimates. We are currently refining the UK YAP algorithms with the aim of increasing their accuracy and hope to publish our findings in 2019. This calibration approach has definitely demonstrated promise, with YAP algorithms also being developed in Spain and the Czech Republic. Moreover, if this calibration approach is coupled with advanced statistical techniques and appropriate study designs to model the measurement error, the accuracy of the resultant estimates of physical activity could be even further improved.
So, it’s unfortunate that more isn’t known about the accelerometer study referred to in the ALS technical notes. In the absence of any information about how the ALS physical activity estimates were validated, it’s difficult for them to be interpreted with great confidence. However, for many, the data in the ALS will be seen as the definitive source of current evidence to describe children’s activity levels. This is a worry, and also great shame, as the ALS has the reach and schools infra-structure to generate valuable physical activity surveillance data, that could be linked to other national-level intelligence such as the National Child Measurement Programme. Hopefully, the next iteration of ALS will include evidence of its validity.
