Methodological domains | Methodological variables | Protocol considerations | Reporting considerations |
1.Target population | 1.1. Population | Cross section of participants across the spectrum of ages*, Three groups: 1. Children (<12 years) 2. Adolescents and healthy adults (13–64 years) 3. Older adults (>65 years) * The basic validation protocol can be performed in healthy participants without any specific disease or condition that can impact a typical gait. The advanced protocol should be completed in special populations with atypical gait characteristics, such as morbid obesity, falls-risk, lower-limb amputees and neurological conditions. In the advanced protocol, participants may use any walking aid if required. | Provide detailed demographics for each group (ie, age, height, weight, BMI, health condition). Report means and ranges. |
1.2. Sex | Equal Sample of males and females in each group. | Report the sex distribution in each group. | |
1.3. Sample size | If the focus of the study is to conduct hypothesis testing about a predefined minimal level of accuracy, a sample size calculation should be completed based on the previously published or pilot study mean and SE of the differences between the devices, using the methodologies outline by Lu et al.44 If sufficient data are not available or this is not the focus of the evaluation, we advise a minimum of 15 participants for each specific validation group (i.e. ≥45 participants). | Detail sample recruited and sample analysed for all groups and in all levels of the study. If a sample size calculation is used, provide details of the assumptions. | |
2.Criterion measure | 2.1. Reference test | Video camera with multiple observer (≥2) or equivalent*. Test the agreement between observers. * Any device or method which has been demonstrated to possess less than 5% measurement error using the laboratory, semifree-living, and free-living validation protocols detailed below. Additionally, this should be specific to the population of interest. | Report camera setup and the level of agreement between observers. If an equivalent method is used, its validity must be reported. |
2.2. Placement | Laboratory and Semifree-living: Ensure the whole scene is in the field of view. Free-living: Camera should be affixed to the body in a manner that does not affect gait pattern, activities of daily living, and allows the evaluator to see the step field of view. For example, attached to a belt around the waist. If an ‘equivalent’ criterion is used, the device should be used as per the methods described within the validation study. | Describe in detail the setup including special situations (eg, going to the toilet). | |
3.Index measure | 3.1 Placement | Should be placed in an ecological body location which the consumer device was designed for. Smartphones: clothing pockets, hand-held, handbags or purses, belt phone holder. Phone should not be mounted to the body in an unnatural way (eg, strapped to chest etc). Wearable Activity Monitor: should be worn as per the manufacturer’s instructions. | Report the exact model and version for the index measure including hardware and software and report in detail the placement protocol. |
4.Testing conditions | 4.1. Laboratory assessment protocol | Walking: Self-selected walking speed.
Running/fast-walking: Self-selected running speed.
Optional: Evaluation on alternate floor surfaces (grass, gravel, linoleum, tarmacadam, tile, incline/decline etc). Estimated time:~25 min | Report cadence/ gait velocity for each trial (eg, 3 min walk test) and for each group (eg, children) Report the overground surface and footwear condition being used. If possible, report the air temperature |
4.2 Semifree-living assessment protocol | Participants are encouraged to complete the tasks as they would do during activities of daily living, at a self-selected pace. Each activity should be completed for ≥3 min unless otherwise stated. Sedentary activities:
Household activities:
Exercise related activity: Exercise difficulty should be selected by the evaluator based on the participants ability.
Estimated time:~45 min | Describe in detail the setup for each trial (eg, computer use) to ensure transparency. Report the overground surface being used. If possible, report the air temperature | |
4.3 Free-living assessment protocol | ≥24 hours free-living period whereby the participant completes activities of daily living in an unconstrained environment (home, work, travelling, etc). A weeklong recording period is optimal to capture interday and weekday-weekend day variation. | Describe in detail the setup to ensure transparency. | |
5.Processing | 5.1 Criterion measure processing * | The recorded video should be reviewed by >2 independent reviewers in real time and a counting device should be used to record steps. * Periods of uncertainty, where it is not clear if a step took place, should be reviewed in slow-motion. | Camera: Report camera frame rate and agreement between reviewers. Equivalent: report all processing methods in detail. |
5.2 Index measure processing | Ensure alignment of epochs from criterion measure and index device. | Detail the index device processing as much as possible. | |
5.3 Epochs for analysis | Laboratory and Semifree-living: Step counts, and video times should be noted after each trial and analysed for each component of the protocol. Steps which occur between trials should be discarded. Cadence (steps/min) and gait velocity (m/s) should be calculated from the video data. Free-living: Step counts and video times should be noted by the evaluator at the beginning and end of the assessment period (≥24 hours). Step counts should be analysed for the full period. Where available, step counts should be divided into shorter hour by hour epochs to facilitate a more granular analysis (eg, hour-by-hour accuracy or activity-by-activity accuracy). This is dependent on the capabilities of the index device. | Laboratory and semifree-living: Report error for the overall protocol and individually for each trial. Free-living: Report the valid wear time for comparison and the error for the entire period or a more granular analysis if available. | |
5.4 Index and criterion synchronisation | Laboratory and semifree-living: Participants should stand still at the beginning and end of each activity trial while the step count and video time are noted. Free-living: Stand still at the start of the period and film the number of steps recorded on wearable/smartphone. Complete the wear period. Stand still at the start of the period and film number of steps recorded on wearable/smartphone. | Describe in detail the synchronisation process allow replication. | |
5.5 Step definition | The act of raising one foot and putting it down in another spot, resulting in the displacement of the centre of mass. | Clearly report the adopted definition. | |
6.Statistical analysis | 6.1 Statistical tests | Device Accuracy
Device precision
| Unless a formal sample size analysis has been conducted, binary conclusions about the validity should not be made. Rather, the level of measurement error and bias should be openly reported, taking into consideration all the proposed validation conditions (lab; semi; free-living), to facilitate contextual interpretation. |
If the evaluator decides not to perform the whole validation protocol, there is a minimal protocol to be followed. At a minimum, the validity should be investigated in those with typical gait (basic protocol), across laboratory (step 1) and semifree-living (step 2). Observers should determine which participants are deemed suitable for the different components of the assessment protocols.
*Any device or method which has been demonstrated to possess less than 5% measurement error using the laboratory, semifree-living, and free-living validation protocols detailed below. Additionally, this should be specific to the population of interest. For the case of free-living validation, the INTERLIVE network is aware that video recording and ‘manual’ step counting by two independent evaluators over ≥24 hours recording period (a 1-week recording period (minimum 3–4 complete days) is optimal to capture interday and weekday-weekend variation) is extremely costly in time and resources and therefore likely not feasible for many. Therefore, we feel that this field needs to move forward, developing and validating alternative methods such as insole sensors that after validation and cross-validation could be used as a new and more feasible gold-standard method for free-living validation protocols.
BMI, body mass index; INTERLIVE®, Intelligent Health and Well-Being Network of Physical Activity Assessment; LoA, limits of agreement; MAPE, mean absolute percentage error.