Skip to main content

Advertisement

SpringerLink
Log in

A Two-Phased Approach to Quantifying Head Impact Sensor Accuracy: In-Laboratory and On-Field Assessments

  • Concussion Biomechanics in Football
  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

Measuring head impacts in sports can further our understanding of brain injury biomechanics and, hopefully, advance concussion diagnostics and prevention. Although there are many head impact sensors available, skepticism on their utility exists over concerns related to measurement error. Previous studies report mixed reliability in head impact sensor measurements, but there is no uniform approach to assessing accuracy, making comparisons between sensors and studies difficult. The objective of this paper is to introduce a two-phased approach to evaluating head impact sensor accuracy. The first phase consists of in-lab impact testing on a dummy headform at varying impact severities under loading conditions representative of each sensor’s intended use. We quantify in-lab accuracy by calculating the concordance correlation coefficient (CCC) between a sensor’s kinematic measurements and headform reference measurements. For sensors that performed reasonably well in the lab (CCC ≥ 0.80), we completed a second phase of evaluation on-field. Through video validation of impacts measured by sensors on athletes, we classified each sensor measurement as either true-positive and false-positive impact events and computed positive predictive value (PPV) to summarize real-world accuracy. Eight sensors were tested in phase one, but only four sensors were assessed in phase two. Sensor accuracy varied greatly. CCC from phase one ranged from 0.13 to 0.97, with an average value of 0.72. Overall, the four devices that were implemented on-field had PPV that ranged from 16.3 to 91.2%, with an average value of 60.8%. Performance in-lab was not always indicative of the device’s performance on-field. The methods proposed in this paper aim to establish a comprehensive approach to the evaluation of sensors so that users can better interpret data collected from athletes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

References

  1. Bartsch, A., S. Samorezov, E. Benzel, C. Clinic, V. Miele and D. Brett. Validation of an “intelligent mouthguard” single event head impact dosimeter. 2014.

  2. Beckwith, J. G., R. M. Greenwald, and J. J. Chu. Measuring head kinematics in football: Correlation between the head impact telemetry system and hybrid iii headform. Ann. Biomed. Eng. 40(1):237–248, 2012.

    Article  PubMed  Google Scholar 

  3. Camarillo, D. B., P. B. Shull, J. Mattson, R. Shultz, and D. Garza. An instrumented mouthguard for measuring linear and angular head impact kinematics in American football. Ann. Biomed. Eng. 41(9):1939–1949, 2013.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Campolettano, E. T., R. A. Gellner, and S. Rowson. High-magnitude head impact exposure in youth football. J. Neurosurg. Pediatr. 20(6):604–612, 2017.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Cecchi, N. J., D. C. Monroe, J. J. Phreaner, S. L. Small, and J. W. Hicks. Patterns of head impact exposure in men’s and women’s collegiate club water polo. J. Sci. Med. Sport. 23(10):927–931, 2020.

    Article  PubMed  Google Scholar 

  6. Cobb, B. R., A. M. Zadnik and S. Rowson. Comparative analysis of helmeted impact response of hybrid iii and national operating committee on standards for athletic equipment headforms. Proc. Inst. Mech. Eng. Part P 230(1):50-60, 2016.

  7. Cobb, B. R., A. MacAlister, T. J. Young, A. R. Kemper, S. Rowson and S. M. Duma. Quantitative comparison of hybrid iii and national operating committee on standards for athletic equipment headform shape characteristics and implications on football helmet fit. Proc. Inst. Mech. Eng. Part. 229(1):39-46, 2015.

  8. Cortes, N., A. E. Lincoln, G. D. Myer, L. Hepburn, M. Higgins, M. Putukian, and S. V. Caswell. Video analysis verification of head impact events measured by wearable sensors. Am. J. Sports Med. 45(10):2379–2387, 2017.

    Article  PubMed  Google Scholar 

  9. Crisco, J. J., and R. M. Greenwald. Let’s get the head further out of the game: A proposal for reducing brain injuries in helmeted contact sports. Curr Sports Med Rep. 10(1):7–9, 2011.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Cummiskey, B., D. Schiffmiller, T. M. Talavage, L. Leverenz, J. J. Meyer, D. Adams and E. A. Nauman. Reliability and accuracy of helmet-mounted and head-mounted devices used to measure head accelerations. Proc. Inst. Mech. Eng. Part P 231(2):144-53, 2016.

  11. Daniel, R. W., S. Rowson, and S. M. Duma. Head impact exposure in youth football. Ann. Biomed. Eng. 40(4):976–981, 2012.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Duma, S. M., S. J. Manoogian, W. R. Bussone, P. G. Brolinson, M. W. Goforth, J. J. Donnenwerth, R. M. Greenwald, J. J. Chu, and J. J. Crisco. Analysis of real-time head accelerations in collegiate football players. Clin. J. Sport Med. 15(1):3–8, 2005.

    Article  PubMed  Google Scholar 

  13. Eckner, J. T., K. L. O’Connor, S. P. Broglio, and J. A. Ashton-Miller. Comparison of head impact exposure between male and female high school ice hockey athletes. Am. J. Sports Med. 46(9):2253–2262, 2018.

    Article  PubMed  Google Scholar 

  14. Funk, J. R., S. Rowson, R. W. Daniel, and S. M. Duma. Validation of concussion risk curves for collegiate football players derived from hits data. Ann Biomed Eng. 40(1):79–89, 2012.

    Article  PubMed  Google Scholar 

  15. Greenwald, R. M., J. J. Chu, J. J. Crisco and J. A. Finkelstein. Head impact telemetry system (hits) for measurement of head acceleration in the field. Proceedings of the American Society of Biomechanics Annual Meeting. 2003.

  16. Hecimovich, M., D. King, A. Dempsey, and M. Murphy. Head impact exposure in junior and adult australian football players. J. Sports Med. 2018. https://doi.org/10.1155/2018/8376030.

    Article  Google Scholar 

  17. Jadischke, R., D. C. Viano, N. Dau, A. I. King, and J. McCarthy. On the accuracy of the head impact telemetry (hit) system used in football helmets. J. Biomech. 46(13):2310–2315, 2013.

    Article  PubMed  Google Scholar 

  18. Kieffer, E. E., C. Vaillancourt, P. G. Brolinson and S. Rowson. Using in-mouth sensors to measure head kinematics in rugby. IRCOBI. 2020.

  19. Kuo, C., L. C. Wu, B. T. Hammoor, J. F. Luck, H. C. Cutcliffe, R. C. Lynall, J. R. Kait, K. R. Campbell, J. P. Mihalik, C. R. Bass, and D. B. Camarillo. Effect of the mandible on mouthguard measurements of head kinematics. J. Biomech. 49(9):1845–1853, 2016.

    Article  PubMed  Google Scholar 

  20. Lamond, L. C., J. B. Caccese, T. A. Buckley, J. Glutting, and T. W. Kaminski. Linear acceleration in direct head contact across impact type, player position, and playing scenario in collegiate women’s soccer players. J. Athl. Train. 53(2):115–121, 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Lawrence, I. and K. Lin. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 255-68, 1989.

  22. Miller, L. E., E. K. Pinkerton, K. C. Fabian, L. C. Wu, M. A. Espeland, L. C. Lamond, C. M. Miles, D. B. Camarillo, J. D. Stitzel, and J. E. Urban. Characterizing head impact exposure in youth female soccer with a custom-instrumented mouthpiece. Res. Sports Med. 28(1):55–571, 2020.

    Article  PubMed  Google Scholar 

  23. Nevins, D., K. Hildenbrand, J. Kensrud, A. Vasavada and L. Smith. Laboratory and field evaluation of a small form factor head impact sensor in un-helmeted play. Proc. Inst. Mech. Eng. Part P 232(3):242–254, 2018.

  24. Nevins, D., L. Smith, and J. Kensrud. Laboratory evaluation of wireless head impact sensor. Procedia Eng. 112(2015):175–179, 2015.

    Article  Google Scholar 

  25. Ng, T. P., W. R. Bussone, and S. M. Duma. The effect of gender and body size on linear accelerations of the head observed during daily activities. Biomed. Sci. Instrum. 42:25–30, 2006.

    PubMed  Google Scholar 

  26. O’Connor, K. L., S. Rowson, S. M. Duma, and S. P. Broglio. Head-impact–measurement devices: a systematic review. J. Athl. Train. 52(3):206–227, 2017.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Patton, D. A. A review of instrumented equipment to investigate head impacts in sport. Appl. Bionics Biomech. https://doi.org/10.1155/2016/7049743

  28. Patton, D. A., C. M. Huber, C. C. McDonald, S. S. Margulies, C. L. Master, and K. B. Arbogast. Video confirmation of head impact sensor data from high school soccer players. Am. J. Sports Med. 48(5):1246–1253, 2020.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Press, J. N., and S. Rowson. Quantifying head impact exposure in collegiate women’s soccer. Clin. J. Sport Med. 27(2):104–110, 2017.

    Article  PubMed  Google Scholar 

  30. Rich, A. M., T. M. Filben, L. E. Miller, B. T. Tomblin, A. R. Van Gorkom, M. A. Hurst, R. T. Barnard, D. S. Kohn, J. E. Urban, and J. D. Stitzel. Development, validation and pilot field deployment of a custom mouthpiece for head impact measurement. Ann. Biomed. Eng. 47(10):2109–21021, 2019.

    Article  PubMed  Google Scholar 

  31. Rowson, S., E. T. Campolettano, S. M. Duma, B. Stemper, A. Shah, J. Harezlak, L. Riggen, J. P. Mihalik, K. M. Guskiewicz, C. Giza, A. Brooks, K. Cameron, T. McAllister, S. P. Broglio, and M. McCrea. Accounting for variance in concussion tolerance between individuals: Comparing head accelerations between concussed and physically matched control subjects. Ann. Biomed. Eng. 47(10):2048–2056, 2019.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Rowson, S., and S. M. Duma. Development of the star evaluation system for football helmets: Integrating player head impact exposure and risk of concussion. Ann Biomed Eng. 39(8):2130–2140, 2011.

    Article  PubMed  Google Scholar 

  33. Rowson, S., M. W. Goforth, D. Dietter, P. G. Brolinson, and S. M. Duma. Correlating cumulative sub-concussive head impacts in football with player performance—biomed. Biomed Sci Instrum. 45:113–118, 2009.

    PubMed  Google Scholar 

  34. Savino, A. K., L. Huang, J. Yang, N. M. Pizzimenti, M. T. McCarthy, and S. C. Rose. Head impact burden differs between seasons in youth and high school us football players. Ann. Biomed. Eng. 2020. https://doi.org/10.1007/s10439-020-02548-3.

    Article  PubMed  Google Scholar 

  35. Siegmund, G. P., K. M. Guskiewicz, S. W. Marshall, A. L. DeMarco, and S. J. Bonin. Laboratory validation of two wearable sensor systems for measuring head impact severity in football players. Ann. Biomed. Eng. 44(4):1257–1274, 2016.

    Article  PubMed  Google Scholar 

  36. Stemper, B. D., A. S. Shah, J. Harezlak, S. Rowson, J. P. Mihalik, S. M. Duma, L. D. Riggen, A. Brooks, K. L. Cameron, D. Campbell, J. P. DiFiori, C. C. Giza, K. M. Guskiewicz, J. Jackson, G. T. McGinty, S. J. Svoboda, T. W. McAllister, S. P. Broglio, M. McCrea and C. C. I. the. Comparison of head impact exposure between concussed football athletes and matched controls: Evidence for a possible second mechanism of sport-related concussion. Ann. Biomed. Eng. 47(10):2057-2072, 2019.

  37. Tiernan, S., G. Byrne, and D. O’Sullivan. Evaluation of skin-mounted sensor for head impact measurement. Proc. Inst. Mech. Eng. Part H. 233:095441191985096, 2019.

    Article  Google Scholar 

  38. Tierney, G. J., C. Kuo, L. Wu, D. Weaving, and D. Camarillo. Analysis of head acceleration events in collegiate-level american football: A combination of qualitative video analysis and in-vivo head kinematic measurement. J. Biomech. 110:109969, 2020.

    Article  PubMed  Google Scholar 

  39. Tyson, A. M., S. M. Duma, and S. Rowson. Laboratory evaluation of low-cost wearable sensors for measuring head impacts in sports. 34(4):320, 2018.

    Google Scholar 

  40. Urban, J. E., E. M. Davenport, A. J. Golman, J. A. Maldjian, C. T. Whitlow, A. K. Powers and J. D. Stitzel. Head impact exposure in youth football: High school ages 14 to 18 years and cumulative impact analysis. Ann Biomed Eng. 2013.

  41. Viano, D. C., and D. Halstead. Change in size and impact performance of football helmets from the 1970s to 2010. Ann Biomed Eng. 40(1):175–184, 2012.

    Article  PubMed  Google Scholar 

  42. Withnall, C., N. Shewchenko, R. Gittens, and J. Dvorak. Biomechanical investigation of head impacts in football. Br. J. Sports Med. 39(suppl 1):i49–i57, 2005.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Wu, L. C., V. Nangia, K. Bui, B. Hammoor, M. Kurt, F. Hernandez, C. Kuo, and D. B. Camarillo. In vivo evaluation of wearable head impact sensors. Ann. Biomed. Eng. 44(4):1234–1245, 2016.

    Article  PubMed  Google Scholar 

  44. Zhou, Y. J., G. Lu, and J. L. Yang. Finite element study of energy absorption foams for headgear in football (soccer) games. Mater. Des. 88:162–169, 2015.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

The authors thank the Lewis Family Foundation for funding this work. The authors have no conflicts of interest to disclose.

Author information

Authors and Affiliations

  1. Biomedical Engineering and Mechanics, Virginia Tech, Blacksburg, VA, USA

    Emily E. Kieffer, Mark T. Begonia, Abigail M. Tyson & Steve Rowson

Authors
  1. Emily E. Kieffer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark T. Begonia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abigail M. Tyson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Steve Rowson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Emily E. Kieffer.

Additional information

Associate Editor Stefan M. Duma oversaw the review of this article.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kieffer, E.E., Begonia, M.T., Tyson, A.M. et al. A Two-Phased Approach to Quantifying Head Impact Sensor Accuracy: In-Laboratory and On-Field Assessments. Ann Biomed Eng 48, 2613–2625 (2020). https://doi.org/10.1007/s10439-020-02647-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-020-02647-1

Keywords

Search

Navigation