LettersSystematic mixed studies reviews: Updating results on the reliability and efficiency of the mixed methods appraisal tool
Section snippets
Methods
To test the efficiency and reliability of the MMAT-v2011, we used it to appraise studies included in two systematic mixed studies reviews. The first dealt with the key processes associated with outcomes of organizational participatory research. The second explored the transitional care of patients with chronic conditions from the hospital to the home. Three graduate students received MMAT training and then used the tool to independently appraise (i) qualitative and MMS included in the first
Results
A total of 261 studies were appraised. The first review included 140 qualitative studies and 27 MMS. The second review included 72 RCTs and 22 NRS. No quantitative descriptive study was appraised. Overall, the average appraisal time per study was 11.3 minutes, with average number of minutes for mixed methods, qualitative research, NRS, and RCT being 18.7, 12.5, 8.4, and 7.4, respectively. The reliability of the MMAT varied by criterion, from fair to perfect (Souto et al., 2014). In particular,
Discussion and conclusion
These results confirm that the MMAT is an efficient tool, but suggest its reliability needs further improvement, particularly for two items including the sentence ‘appropriate consideration’. We noted that the independent reviewers understood this sentence in a different manner. A reviewer considered that ‘appropriate consideration’ was given when there were at least few details, whereas the other reviewer looked for a detailed description of specific strategies. In some articles reporting
Acknowledgments
Quan Nha Hong holds a PhD bursary from the Canadian Institutes of Health Research (CIHR). Pierre Pluye and Isabelle Vedel hold Investigator Awards from the ‘Fonds de recherche du Québec santé’ (FRQS). The present work is supported by CIHR-funded systematic mixed studies reviews, and CIET-PRAM (Participatory Research at McGill) (http://pram.mcgill.ca/index.php).
Contributions: Rafaella Queiroga Souto and Pierre Pluye proposed an initial version of the paper. All authors contributed to the
References (7)
- et al.
A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed
J. Clin. Epidemiol.
(2011) - et al.
Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review
Int. J. Nurs. Stud.
(2012) - et al.
A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews
Int. J. Nurs. Stud.
(2009)
Cited by (230)
Using telehealth to deliver Qi Gong and Tai Chi programs: A mixed-methods systematic review on feasibility, acceptability and participant engagement factors
2024, Archives of Gerontology and GeriatricsAssociation between greenspace exposure and suicide-related outcomes across the lifespan: A systematic review
2024, Science of the Total EnvironmentDisability, equity, and measurements of livability: A scoping review
2024, Disability and Health JournalNon-clinical intuitions and adaptive heuristics in emergency care: A scoping review
2023, International Emergency NursingPrimary health care roles of community pharmacists in low- and middle-income countries: A protocol for a mixed methods systematic review
2023, Journal of the American Pharmacists Association