Tackling methodological challenges in public health reviews

The Cochrane Methods team has defined a programme of work to improve the quality and impact of Cochrane reviews in public health with the support of the Methods ExecutiveMethods Groups and Public Health and Health Systems Network. The programme has been designed to foster collaboration between Methods Groups and Cochrane Review Group (CRG) Networks and to support the ongoing response to COVID-19. It focuses on producing high-priority public health reviews and delivering user-friendly resources to improve the planning, conduct and reporting of methods within them.

Why focus on public health?
Public health reviews, especially those relating to COVID-19, are often highly scrutinised and have wide-ranging impacts on policy and global health outcomes. However, public health reviews typically address complex questions and use a range of complex methods that are considered non-standard in Cochrane, and existing guidance is not always being applied accurately or consistently. In the Cochrane context, public health reviews are defined as those that include diverse sources of evidence (e.g., not solely RCTs), complex interventions that are delivered at the population level (e.g., policies rather than drugs or treatments), exposures that cannot be delivered in randomized controlled trials (e.g. studying the effects of a chemical or viral exposure), and outcomes that measure something other than efficacy (e.g., harms, process outcomes, implementation or costs).

How will the programme support review production?
The programme of work is based on the premise that high-quality, timely review production requires expert methodological support and pragmatic resources that help authors and editors translate Handbook guidance into practice. The topics and approach have been designed to bring CRGs and Methods Groups together to ensure the projects and their outputs are designed with the needs of Cochrane editors and authors in mind.

Though the focus is on public health reviews, the challenges faced within them are often encountered in other topic areas, such as choosing when and how to include non-randomised studies of interventions, assessing bias in non-randomized studies of interventions, planning a useful synthesis in light of complexity and heterogeneity, incorporating qualitative evidence, and reporting results when a meta-analysis has not been possible. By tackling the challenges in a COVID-priority area, the aim will be to develop, adapt and apply the tools more widely across Cochrane. The Methods Support Unit plays an important role in bridging the gap between CRGs and Methods Groups and will be embedded in the projects to support capacity-building.

What has happened so far?
The Cochrane Methods team mapped the most frequently cited challenges from two recent methods surveys of CRGs by methodological area and listed possible projects to address them. The projects were then shortlisted considering extent of author and CRG need, potential for impact, and the anticipated time and resources required to deliver a useful output. The projects have been refined with Cochrane’s Methods Executive and project teams are now being brought together. Each has been designed to deliver a practical tool or decision aid for editors or authors by the end of 2021 that harnesses existing theory so that it can be more easily applied.

  • Project 1: Preferred and accepted risk of bias tools for assessing bias in non-randomised studies of interventions
  • Project 2: Practical guidance to frame public health intervention review questions, define outcomes and choose appropriate study designs
  • Project 3: Standard terminology to help authors describe different study designs appropriately and consistently
  • Project 4: Practical conduct and reporting guidance for assessing risk of bias in non-randomised studies of interventions and/or ROBINS-I
  • Project 5: Practical tools to implement SWiM reporting guidance to improve pre-specification and presentation of findings

In addition to the projects, plans are underway to deliver a suite of learning resources around qualitative evidence synthesis, and to house a bank of exemplar reviews showcasing best practice. A Methods Symposium will also take place in October 2021 covering a range of methodological challenges relating to public health reviews and complex interventions. The symposium will be followed by a methods implementation surgery led by the Methods Support Unit and aimed at CRGs in November 2021 to discuss the outputs that were developed and how they can support editors and authors.

How can I get involved?
Each project will be encouraged to use Task Exchange to call for volunteers across the community to provide feedback on the first draft of the tools and resources being developed. Those interested in getting involved are encouraged to create a profile or contact the Methods Team directly to enquire about other ways to contribute.

Additional resources:

Tuesday, July 27, 2021