Skip to main content

Advertisement

SpringerLink
Log in

Reducing research wastage by starting off on the right foot: optimally framing the research question

  • Special Section: Reducing Research Waste in (Health-Related) Quality of Life Research
  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Strongly framed research questions are clear as to the population (P), the exposures or interventions (E/I), comparison groups (C), outcomes (O), time when relevant (T), and what the investigator wants to know. A solid framework sets up the measurement model, analysis, and anticipated results. The purpose of this study was to estimate the extent to which research questions in journals that focused on patient-reported outcome measures (PROM) and quality of life (QOL) are clear.

Methods

All 440 research articles published in four PROM journals in 2020. excluding reviews, psychometric, and qualitative papers, were reviewed. Research questions were classified as: (i) adequately framed (ii) poorly framed; or (iii) unframed based on clarity criteria. Examples from each journal were presented and reframed to match results in the article.

Results

Of 440 articles, 195 (44.3%) were classified as adequately framed; 230 (52.2%) as poorly framed; and 15 (3.4%) as unframed. There was heterogeneity across journals (Chi-square: 20.8; 6 df; p = 0.002). Only 29% were framed according to what the investigators wanted to know; 72% were framed like a “to do” list; and 6% were framed as a research agenda.

Conclusion

Almost half of the questions were poorly framed or unframed a practice that could contribute to research wastage. Even “adequately framed” questions rarely stated what they wanted to know a priori, increasing the risk of biased reporting. Researchers, reviewers, and editors should encourage the use established frameworks for research questions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Mayo, N. E., Asano, M., & Barbic, S. P. (2013). When is a research question not a research question? Journal of Rehabilitation Medicine, 45(6), 513–518.

    Article  PubMed  Google Scholar 

  2. Oxman, A. D., & Guyatt, G. H. (1988). Guidelines for reading literature reviews. CMAJ, 138(8), 697–703.

    CAS  PubMed  PubMed Central  Google Scholar 

  3. Birch, D. W., Eady, A., Robertson, D., De Pauw, S., Tandan, V., Evidence-Based Surgery Working G. (2003). Users’ guide to the surgical literature: How to perform a literature search. Canadian Journal of Surgery, 46(2), 136–141.

    PubMed Central  Google Scholar 

  4. Thabane, L., Thomas, T., Ye, C., & Paul, J. (2008). Posing the research question: Not so simple. Canadian Journal of Anesthesia/Journal canadien d’anesthésie, 56(1), 71.

    Article  PubMed  Google Scholar 

  5. Richardson, W. S., Wilson, M. C., Nishikawa, J., & Hayward, R. S. (1995). The well-built clinical question: A key to evidence-based decisions. ACP Journal Club, 123(3), A12–A13.

    Article  CAS  PubMed  Google Scholar 

  6. Davies, K. S. (2011). Formulating the evidence based practice question: A review of the frameworks. Evidence Based Library and Information Practice., 6(2), 6.

    Article  Google Scholar 

  7. Sackett, D. L., & Wennberg, J. E. (1997). Choosing the best research design for each question. It’s time to stop squabbling over the “best” methods. BMJ, 315(7123), 1636.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Antonoff, M. B., Nguyen, S., Nguyen, T. C., & Odell, D. D. (2019). Conducting high-quality research in cardiothoracic surgical education: Recommendations from the Thoracic Education Cooperative Group. Journal of Thoracic and Cardiovascular Surgery, 157(2), 820–7.e1.

    Article  PubMed  Google Scholar 

  9. Mattick, K., Johnston, J., & de la Croix, A. (2018). How to…write a good research question. The Clinical Teacher, 15(2), 104–108.

    Article  PubMed  Google Scholar 

  10. Bordage, G. (2001). Reasons reviewers reject and accept manuscripts: The strengths and weaknesses in medical education reports. Academic Medicine, 76(9), 889–896.

    Article  CAS  PubMed  Google Scholar 

  11. Bailar, J. C., III., & Mosteller, F. (1992). Medical uses of statistics Massachusetts Medical Society (p. 146). Wiley.

    Google Scholar 

  12. Nuzzo, R. (2014). Scientific method: Statistical errors. Nature, 506(7487), 150–152.

    Article  CAS  PubMed  Google Scholar 

  13. Chalmers, I., & Glasziou, P. (2009). Avoidable waste in the production and reporting of research evidence. Lancet, 374(9683), 86–89.

    Article  PubMed  Google Scholar 

  14. Munn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018). What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Medical Research Methodology., 18(1), 5.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Barile, J. P., Binder, S. B., & Baker, C. K. (2020). Recovering after a natural disaster: Differences in quality of life across three communities after hurricane sandy. Applied Research in Quality of Life, 15(4), 1151–1159.

    Article  Google Scholar 

  16. Wang, D., & Li, D. (2020). Social capital, policy fairness, and subjective life satisfaction of earthquake survivors in Wenchuan, China: A longitudinal study based on post-earthquake survey data. Health and Quality of Life Outcomes, 18(1), 350.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Cohee, A., Storey, S., Winger, J. G., Cella, D., Stump, T., Monahan, P. O., et al. (2020). A cohort study of quality of life in partners of young breast cancer survivors compared to partners of healthy controls. Journal of Patient-Reported Outcomes, 4(1), 19.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Vyas, A., Kang, F., & Barbour, M. (2020). Association between polypharmacy and health-related quality of life among US adults with cardiometabolic risk factors. Quality of Life Research, 29(4), 977–986.

    Article  PubMed  Google Scholar 

  19. Kivijärvi, A., Aaltonen, S., Forma, L., Partanen, J., Myllylä, M., & Rissanen, P. (2020). Quality of life among young Finnish adults not in employment or education. Applied Research in Quality of Life, 15(3), 757–774.

    Article  Google Scholar 

  20. Andersen, J., Thomsen, J., Enes, Å. R., Sandberg, S., & Aarsand, A. K. (2020). Health-related quality of life in porphyria cutanea tarda: A cross-sectional registry based study. Health and Quality of Life Outcomes, 18(1), 84.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Oreel, T. H., Delespaul, P., Hartog, I. D., Henriques, J. P. S., Netjes, J. E., Vonk, A. B. A., et al. (2020). Ecological momentary assessment versus retrospective assessment for measuring change in health-related quality of life following cardiac intervention. Journal of Patient-Reported Outcomes, 4(1), 98.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Nutakor, J. A., Dai, B., Gavu, A. K., & Antwi, O. A. (2020). Relationship between chronic diseases and sleep duration among older adults in Ghana. Quality of Life Research, 29(8), 2101–2110.

    Article  PubMed  Google Scholar 

  23. Ludwigs, K., Haese, P., Sivy, K., Weber, S., & Schrömgens, R. (2020). Trivago flowlab—a case study on how to improve employees’ well-being in a corporate environment. Applied Research in Quality of Life, 15(5), 1353–1374.

    Article  Google Scholar 

  24. Sorbye, H., Meyer, L. S., Mordal, K. E., Myhre, S., & Thiis-Evensen, E. (2020). Patient reported symptoms, coping and quality of life during somatostatin analogue treatment for metastatic small- intestinal neuroendocrine tumours. Health and Quality of Life Outcomes, 18(1), 188.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Jonsson, Å., Orwelius, L., Dahlstrom, U., & Kristenson, M. (2020). Evaluation of the usefulness of EQ-5D as a patient-reported outcome measure using the Paretian classification of health change among patients with chronic heart failure. Journal of Patient-Reported Outcomes, 4(1), 50.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Wu, Y., Al-Janabi, H., Mallett, A., Quinlan, C., Scheffer, I. E., Howell, K. B., et al. (2020). Parental health spillover effects of paediatric rare genetic conditions. Quality of Life Research, 29(9), 2445–2454.

    Article  PubMed  Google Scholar 

  27. Garrido, D., Carballo, G., & Garcia-Retamero, R. (2020). Siblings of children with autism spectrum disorders: Social support and family quality of life. Quality of Life Research, 29(5), 1193–1202.

    Article  PubMed  Google Scholar 

  28. Shan, X., Chen, Y., Liu, K., Zhang, S., Yu, J., Yin, J., et al. (2020). Health-related quality of life (HRQoL) associated with echinococcosis patients in Tibetan communities in Shiqu County, China: A case-control study. Quality of Life Research, 29(6), 1559–1565.

    Article  PubMed  Google Scholar 

  29. Sánchez-Ferrer, M. L., Adoamnei, E., Prieto-Sánchez, M. T., Mendiola, J., Corbalán-Biyang, S., Moñino-García, M., et al. (2020). Health-related quality of life in women with polycystic ovary syndrome attending to a tertiary hospital in Southeastern Spain: A case-control study. Health and Quality of Life Outcomes, 18(1), 232.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Khalifa, N., Rahman, B., Gaintantzopoulou, M. D., Al-Amad, S., & Awad, M. M. (2020). Oral health status and oral health-related quality of life among patients with type 2 diabetes mellitus in the United Arab Emirates: A matched case-control study. Health and Quality of Life Outcomes, 18(1), 182.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Chapman, S. J., Aldaffaa, M., Downey, C. L., & Jayne, D. G. (2019). Research waste in surgical randomized controlled trials. British Journal of Surgery, 106(11), 1464–1471.

    Article  CAS  PubMed  Google Scholar 

  32. Blencowe, N. S., Boddy, A. P., Harris, A., Hanna, T., Whiting, P., Cook, J. A., et al. (2015). Systematic review of intervention design and delivery in pragmatic and explanatory surgical randomized clinical trials. British Journal of Surgery, 102(9), 1037–1047.

    Article  CAS  PubMed  Google Scholar 

  33. Meneses-Echavez, J. F., Rodriguez-Prieto, I., Elkins, M., Martínez-Torres, J., Nguyen, L., & Bidonde, J. (2019). Analysis of reporting completeness in exercise cancer trials: A systematic review. BMC Medical Research Methodology, 19(1), 220.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Johnson, A. L., Fladie, I., Anderson, J. M., Lewis, D. M., Mons, B. R., & Vassar, M. (2020). Rates of discontinuation and nonpublication of head and neck cancer randomized clinical trials. JAMA Otolaryngology. Head & Neck Surgery, 146(2), 176–182.

    Article  Google Scholar 

  35. Scott, J., Cooper, C. M., Checketts, J. X., Cutler, J., Boose, M., Morris, J., et al. (2018). An observational analysis of discontinuation and non-publication of osteoarthritis trials. Osteoarthritis Cartilage, 26(9), 1162–1169.

    Article  CAS  PubMed  Google Scholar 

  36. Ioannidis, J. P., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., et al. (2014). Increasing value and reducing waste in research design, conduct, and analysis. Lancet, 383(9912), 166–175.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Hennedige, O. (2014). Reducing waste in dental research. Indian Journal of Dental Research, 25(3), 277.

    Article  PubMed  Google Scholar 

  38. Harron, K., & Gilbert, R. (2014). Research: Increasing value, reducing waste. Lancet, 383(9923), 1124.

    Article  PubMed  Google Scholar 

  39. Okumura, Y. (2016). Reducing research waste through good reporting practices. Journal of Epidemiology, 26(8), 397–398.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Brindle, P. (2014). Research: Increasing value, reducing waste. Lancet, 383(9923), 1125.

    Article  PubMed  Google Scholar 

  41. Chan, A. W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gotzsche, P. C., et al. (2014). Increasing value and reducing waste: Addressing inaccessible research. Lancet, 383(9913), 257–266.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Breznau, N., Rinke, E., Wuttke, A., Nguyen, H. H. V., Adem, M., Adriaans, J., et al. (2021). How many replicators does it take to achieve reliability? Investigating researcher variability in a crowdsourced replication. SocArXiv.

  43. Brady, D., & Finnigan, R. (2014). Does immigration undermine public support for social policy? American Sociological Review, 79(1), 17–42.

    Article  Google Scholar 

  44. Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., et al. (2014). Reducing waste from incomplete or unusable reports of biomedical research. Lancet, 383(9913), 267–276.

    Article  PubMed  Google Scholar 

  45. National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and replicability in science (p. 256). National Academies Press.

    Google Scholar 

  46. Mayo, N. E., & Goldberg, M. S. (2009). When is a case-control study not a case-control study? Journal of Rehabilitation Medicine, 41(4), 209–216.

    Article  PubMed  Google Scholar 

  47. Kicielinski, K. P., Dupepe, E. B., Gordon, A. S., Mayo, N. E., & Walters, B. C. (2019). What isn’t a case-control study? Neurosurgery, 84(5), 993–999.

    Article  PubMed  Google Scholar 

  48. Esene, I. N., Mbuagbaw, L., Dechambenoit, G., Reda, W., & Kalangu, K. K. (2018). Misclassification of case-control studies in neurosurgery and proposed solutions. World Neurosurgery, 112, 233–242.

    Article  PubMed  Google Scholar 

  49. Grimes, D. A. (2009). “Case-control” confusion: Mislabeled reports in obstetrics and gynecology journals. Obstetrics and Gynecology, 114(6), 1284–1286.

    Article  PubMed  Google Scholar 

  50. Schlosser, R. W., Koul, R., & Costello, J. (2007). Asking well-built questions for evidence-based practice in augmentative and alternative communication. Journal of Communication Disorders, 40(3), 225–238.

    Article  PubMed  Google Scholar 

  51. Hays, M., Andrews, M., Wilson, R., Callender, D., Malley, P. G., & Douglas, K. (2016). Reporting quality of randomised controlled trial abstracts among high-impact general medical journals: A review and analysis. British Medical Journal Open, 6(7), e011082.

    Google Scholar 

  52. Hopewell, S., Ravaud, P., Baron, G., & Boutron, I. (2012). Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: Interrupted time series analysis. BMJ: British Medical Journal, 344, e4178.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Mbuagbaw, L., Thabane, M., Vanniyasingam, T., Debono, V. B., Kosa, S., Zhang, S., et al. (2014). Improvement in the quality of abstracts in major clinical journals since CONSORT extension for abstracts: A systematic review. Contemporary Clinical Trials, 38(2), 245–250.

    Article  PubMed  Google Scholar 

  54. Hopewell, S., Clarke, M., Moher, D., Wager, E., Middleton, P., Altman, D. G., et al. (2008). CONSORT for reporting randomised trials in journal and conference abstracts. The Lancet, 371(9609), 281–283.

    Article  Google Scholar 

  55. Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., et al. (2014). Reducing waste from incomplete or unusable reports of biomedical research. The Lancet, 383(9913), 267–276.

    Article  Google Scholar 

Download references

Funding

No specific funding was received for this study.

Author information

Authors and Affiliations

  1. School of Physical and Occupational Therapy, Divisions of Clinical Epidemiology, Geriatrics, Experimental Medicine, Department of Medicine, McGill University, Center for Outcomes Research and Evaluation (CORE) McGill University Health Centre (MUHC)-Research Institute, Montreal, Canada

    Nancy E. Mayo

  2. Occupational Science and Occupational Therapy, Faculty of Medicine, University of British Columbia, Foundry Central Office, Vancouver, Canada

    Nikki Ow

  3. Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore

    Miho Asano

  4. School of Occupational Therapy, Dalhousie University, Halifax, Canada

    Sorayya Askari

  5. Department of Physical Therapy, College of Rehabilitation Sciences, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, Canada

    Ruth Barclay

  6. Department of Clinical Research and Leadership, School of Medicine and Health Sciences, George Washington University, Washington, DC, USA

    Sabrina Figueiredo

  7. Centre for Global Health and Equity, Faculty of Health, Arts and Design, Swinburne University of Technology, Melbourne, Australia

    Melanie Hawkins

  8. Brain Health Outcomes Platform, Montreal Neurological Institute and Hospital, Montreal, Canada

    Stanley Hum

  9. School of Physical and Occupational Therapy, Faculty of Medicine and Health Sciences, McGill University, Center for Outcomes Research and Evaluation (CORE) McGill University Health Centre (MUHC)-Research Institute, Montreal, Canada

    Mehmet Inceer & Ana Maria Moga

  10. Department of Physical Therapy, Rehabilitation Sciences Institute, University of Toronto, Toronto, Canada

    Navaldeep Kaur

  11. School of Rehabilitation Science, McMaster University, Hamilton, ON, Canada

    Ayse Kuspinar

  12. Centre for Neurological Restoration, Cleveland Clinic, Cleveland, USA

    Kedar K. V. Mate

  13. Division of Experimental Medicine, Faculty of Medicine and Health Sciences, McGill University, Center for Outcomes Research and Evaluation (CORE) McGill University Health Centre (MUHC)-Research Institute, Montreal, Canada

    Maryam Mozafarinia

  14. School of Physical and Occupational Therapy, Divisions of Clinical Epidemiology, Geriatrics, Experimental Medicine, Faculty of Medicine and Health Sciences, McGill University, Center for Outcomes Research and Evaluation (CORE) McGill University Health Centre-Research Institute (RI-MUHC), Montreal, QC, Canada

    Nancy E. Mayo

Authors
  1. Nancy E. Mayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nikki Ow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miho Asano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sorayya Askari
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ruth Barclay
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sabrina Figueiredo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Melanie Hawkins
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Stanley Hum
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mehmet Inceer
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Navaldeep Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Ayse Kuspinar
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kedar K. V. Mate
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Ana Maria Moga
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Maryam Mozafarinia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nancy E. Mayo.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Consent to participate

N/A.

Consent for publication

N/A.

Ethical approval

N/A.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mayo, N.E., Ow, N., Asano, M. et al. Reducing research wastage by starting off on the right foot: optimally framing the research question. Qual Life Res 31, 2889–2899 (2022). https://doi.org/10.1007/s11136-022-03117-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-022-03117-y

Keywords

Search

Navigation