Table 2

Check-list of items to be considered during the validation protocol development for consumer wearable and smartphone step counters

Target population assessment
Age
 (Children (<12 years)
 Adolescents and healthy adults (13–64 years)
 Older adults (>65 years)
Sex (equal sample of males and females)
  1. Sample size calculation via pilot study

    OR

  2. Sample of convenience (n≥45)

Criterion measure assessment
Video camera with multiple observers (≥2) or equivalent*
Placement to ensure steps are within the field of view†
Index device assessment
Placement according to manufacturer’s instructions
Laboratory testing condition assessment
Walking:
 3 min walk test
 3 min zigzag walk test
 3 x stair test ascent/descent
 3 min stationary cycling
 2×1 min steady state treadmill test (reliability)
Running/fast-walking:
 3 min run test
 3 min zig-zag run test
Optional
 1 x incline/ decline walking test
Semifree-living testing condition assessment
Sedentary activities:
 Simulated sleeping
 Writing by hand
 Eating/ drinking
Household activities:
 Standing and talking
 Sweeping
 Vacuuming
 Folding laundry
 Simulated washing/drying dishes
 Simulated shopping
Exercise-related activities:
 Squat/sit-to-stand
 Lunge/split-squat
 Low rowing exercise
Free-living testing condition assessment
Subject’s wear index and criterion device for a minimum of 24 hours, and if a more feasible gold-standard or pseudo-gold-standard method is developed/validated a week assessment would be ideal.
Processing
Criterion measure processing
 3. Video: Recorded video should be reviewed by >2 independent observers in real time and a counting device should be used to record steps
 OR
Equivalent: automated method which has previously demonstrated sufficient validity (* see footnote)
Index measure processing
 No post processing of the end-user data is allowed
Epochs for analysis
 Note step count at start and end of each trial
 Discard steps which occur between trials
 Record cadence and gait velocity for laboratory and semifree-living
Index and criterion synchronisation
 Participants stand still at start and end of each activity trial while step count and video time (or equivalent criterion count) are noted
 Step definition
 The act of raising one foot and putting it down in another spot, resulting in the displacement of the centre of mass
Statistical analysis
Mean difference or mean relative difference Bland-Altman LoA
Least products regression of the differences against the means
Mean absolute percentage error
Intraclass correlation coefficient (calculated for the 2×1 min treadmill test only)
  • *Any device or method which has been demonstrated to possess less than 5% measurement error using the laboratory, semifree-living, and free-living validation protocols detailed below. Additionally, this should be specific to the population of interest. For the case of free-living validation, the INTERLIVE network is aware that video recording and ‘manual’ step counting by two independent evaluators over ≥24 hours recording period (a 1-week recording period (minimum 3–4 complete days) is optimal to capture interday and weekday-weekend variation) is extremely costly in time and resources and therefore likely not feasible for many. Therefore, we feel that this field needs to move forward, developing and validating alternative methods such as insole sensors that after validation and cross-validation could be used as a new and more feasible gold-standard method for free-living validation protocols.

  • †If an ‘equivalent’ criterion is used, the device should be used as per the methods described within the validation study.

  • INTERLIVE®, Intelligent Health and Well-Being Network of Physical Activity Assessment; LoA, limits of agreement.