Table 1

The proposed best-practice protocols for the validation of wearable and smartphone step counting devices

Methodological domains	Methodological variables	Protocol considerations	Reporting considerations
1.Target population	1.1. Population	Cross section of participants across the spectrum of ages, Three groups:* 1. Children (<12 years) 2. Adolescents and healthy adults (13–64 years) 3. Older adults (>65 years) * The basic validation protocol can be performed in healthy participants without any specific disease or condition that can impact a typical gait. The advanced protocol should be completed in special populations with atypical gait characteristics, such as morbid obesity, falls-risk, lower-limb amputees and neurological conditions. In the advanced protocol, participants may use any walking aid if required.	Provide detailed demographics for each group (ie, age, height, weight, BMI, health condition). Report means and ranges.
	1.2. Sex	Equal Sample of males and females in each group.	Report the sex distribution in each group.
	1.3. Sample size	If the focus of the study is to conduct hypothesis testing about a predefined minimal level of accuracy, a sample size calculation should be completed based on the previously published or pilot study mean and SE of the differences between the devices, using the methodologies outline by Lu et al.44 If sufficient data are not available or this is not the focus of the evaluation, we advise a minimum of 15 participants for each specific validation group (i.e. ≥45 participants).	Detail sample recruited and sample analysed for all groups and in all levels of the study. If a sample size calculation is used, provide details of the assumptions.
2.Criterion measure	2.1. Reference test	Video camera with multiple observer (≥2) or equivalent. Test the agreement between observers. Any device or method which has been demonstrated to possess less than 5% measurement error using the laboratory, semifree-living, and free-living validation protocols detailed below. Additionally, this should be specific to the population of interest.	Report camera setup and the level of agreement between observers. If an equivalent method is used, its validity must be reported.
2.Criterion measure	2.2. Placement	Laboratory and Semifree-living: Ensure the whole scene is in the field of view. Free-living: Camera should be affixed to the body in a manner that does not affect gait pattern, activities of daily living, and allows the evaluator to see the step field of view. For example, attached to a belt around the waist. If an ‘equivalent’ criterion is used, the device should be used as per the methods described within the validation study.	Describe in detail the setup including special situations (eg, going to the toilet).
3.Index measure	3.1 Placement	Should be placed in an ecological body location which the consumer device was designed for. Smartphones: clothing pockets, hand-held, handbags or purses, belt phone holder. Phone should not be mounted to the body in an unnatural way (eg, strapped to chest etc). Wearable Activity Monitor: should be worn as per the manufacturer’s instructions.	Report the exact model and version for the index measure including hardware and software and report in detail the placement protocol.
4.Testing conditions	4.1. Laboratory assessment protocol	Walking: Self-selected walking speed. 3 min walk test (30 m walkway) 3 min zig-zag walk test (30 m walkway - three cones) 3 x stair test ascent/descent (minimum 12 step staircase) 3 min stationary cycling 2×1 min steady state treadmill test (consistent, self-selected walking speed). Running/fast-walking: Self-selected running speed. 3 min run test (30 m walkway) 3 min zigzag run test (30 m walkway - three cones) Optional: Evaluation on alternate floor surfaces (grass, gravel, linoleum, tarmacadam, tile, incline/decline etc). Estimated time:~25 min	Report cadence/ gait velocity for each trial (eg, 3 min walk test) and for each group (eg, children) Report the overground surface and footwear condition being used. If possible, report the air temperature
	4.2 Semifree-living assessment protocol	Participants are encouraged to complete the tasks as they would do during activities of daily living, at a self-selected pace. Each activity should be completed for ≥3 min unless otherwise stated. Sedentary activities: simulated sleeping (Get into bed, after 1 min roll over, after 2 min check time on smartphone/wearable, after 3 min get up) writing by hand (sitting) eating/ drinking (sitting) computer use (both typing and mouse use) smartphone use (sitting) simulated video-game (game console controller) Household activities: standing and talking sweeping (5 m squared) vacuuming (5 m squared) folding laundry (no steps while standing) simulated washing/drying dishes (washing for 1.5 min and drying for 1.5 min) simulated shopping (pick up two shopping bags, walk 10 m, empty bag and put items in cupboard—perform task once) Exercise related activity: Exercise difficulty should be selected by the evaluator based on the participants ability. Squat/sit-to-stand (≥20 s) Lunge/split-squat (≥20 s) Low rowing exercise (≥20 s) Estimated time:~45 min	Describe in detail the setup for each trial (eg, computer use) to ensure transparency. Report the overground surface being used. If possible, report the air temperature
	4.3 Free-living assessment protocol	≥24 hours free-living period whereby the participant completes activities of daily living in an unconstrained environment (home, work, travelling, etc). A weeklong recording period is optimal to capture interday and weekday-weekend day variation.	Describe in detail the setup to ensure transparency.
5.Processing	5.1 Criterion measure processing *	The recorded video should be reviewed by >2 independent reviewers in real time and a counting device should be used to record steps. * Periods of uncertainty, where it is not clear if a step took place, should be reviewed in slow-motion.	Camera: Report camera frame rate and agreement between reviewers. Equivalent: report all processing methods in detail.
	5.2 Index measure processing	Ensure alignment of epochs from criterion measure and index device.	Detail the index device processing as much as possible.
	5.3 Epochs for analysis	Laboratory and Semifree-living: Step counts, and video times should be noted after each trial and analysed for each component of the protocol. Steps which occur between trials should be discarded. Cadence (steps/min) and gait velocity (m/s) should be calculated from the video data. Free-living: Step counts and video times should be noted by the evaluator at the beginning and end of the assessment period (≥24 hours). Step counts should be analysed for the full period. Where available, step counts should be divided into shorter hour by hour epochs to facilitate a more granular analysis (eg, hour-by-hour accuracy or activity-by-activity accuracy). This is dependent on the capabilities of the index device.	Laboratory and semifree-living: Report error for the overall protocol and individually for each trial. Free-living: Report the valid wear time for comparison and the error for the entire period or a more granular analysis if available.
	5.4 Index and criterion synchronisation	Laboratory and semifree-living: Participants should stand still at the beginning and end of each activity trial while the step count and video time are noted. Free-living: Stand still at the start of the period and film the number of steps recorded on wearable/smartphone. Complete the wear period. Stand still at the start of the period and film number of steps recorded on wearable/smartphone.	Describe in detail the synchronisation process allow replication.
	5.5 Step definition	The act of raising one foot and putting it down in another spot, resulting in the displacement of the centre of mass.	Clearly report the adopted definition.
6.Statistical analysis	6.1 Statistical tests	Device Accuracy Bland-Altman with LoA Least products regression of the differences against the means MAPE Device precision Intraclass correlation with 95% CIs (calculated for the 2×1 min treadmill test only)	Unless a formal sample size analysis has been conducted, binary conclusions about the validity should not be made. Rather, the level of measurement error and bias should be openly reported, taking into consideration all the proposed validation conditions (lab; semi; free-living), to facilitate contextual interpretation.

If the evaluator decides not to perform the whole validation protocol, there is a minimal protocol to be followed. At a minimum, the validity should be investigated in those with typical gait (basic protocol), across laboratory (step 1) and semifree-living (step 2). Observers should determine which participants are deemed suitable for the different components of the assessment protocols.
*Any device or method which has been demonstrated to possess less than 5% measurement error using the laboratory, semifree-living, and free-living validation protocols detailed below. Additionally, this should be specific to the population of interest. For the case of free-living validation, the INTERLIVE network is aware that video recording and ‘manual’ step counting by two independent evaluators over ≥24 hours recording period (a 1-week recording period (minimum 3–4 complete days) is optimal to capture interday and weekday-weekend variation) is extremely costly in time and resources and therefore likely not feasible for many. Therefore, we feel that this field needs to move forward, developing and validating alternative methods such as insole sensors that after validation and cross-validation could be used as a new and more feasible gold-standard method for free-living validation protocols.
BMI, body mass index; INTERLIVE^®, Intelligent Health and Well-Being Network of Physical Activity Assessment; LoA, limits of agreement; MAPE, mean absolute percentage error.