How can you assess the strength and validity of a clinical guideline?

We will be discussing Diagnosis and management of endometriosis: a systematic review of international and national guidelines. Hirsch et al. BJOG 10.1111/1471-0528.14838 on Wednesday 29 November, 8pm BST.

A new guideline on the diagnosis and management of endometriosis is released by a prestigious medical society or association. You typically agree with the recommendations and adhere to the guidelines set forth by this group, but tend to be frustrated by recommendations that are based on expert opinion alone.

How can an informed reader of the obstetrics & gynecology literature assess the strength and validity of a clinical guideline and its recommendations?

Description of Research:
Study Design: Systematic review of clinical guidelines for endometriosis
Methods: Four authors assess quality using the Appraisal of Guidelines for Research & Evaluation (AGREE-II) tool.
Comparison: Total guideline score: low quality (0-33%), moderate quality (34-66%), and high quality (67%-100%).
Outcomes: Two guidelines were high quality, four were moderate quality, and one was low quality. No guidelines followed the standardized AGREE-II approach.
Authors’ Conclusions: There is substantial variation in recommendations and methodological quality of endometriosis guidelines. Future guidelines should be developed using high quality methodology.

Discussion Points:
• How are systematic reviews and guidelines similar? How do they differ?
• For endometriosis, what kind of stakeholder groups should be included in the making of guidelines, in your opinion?
• How can writers of guidelines for endometriosis or other topics improve the methods and reporting of recommendations based on expert opinion when data are lacking?
• What benefits may authors expect by including patients or patient advocates in the systematic review process?

1. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Safe Health Care. 2003;12:18-23.
2. Khan K. The CROWN initiative: journal editors invite research to develop core outcomes in women’s health. BJOG 2014;121:1181-2.
3. Duffy JMN et al. Core outcome sets in women’s and newborn health: a systematic review. BJOG 2017;124:1481-1489.

Getting termination right: strategies for cervical preparation

We will be discussing Preoperative effects of mifepristone for dilation and evacuation after 19 weeks’ gestation: a randomised controlled trial.  Shaw et al. BJOG 2017 on Wednesday 25 October, 8pm BST.


You are performing a surgical termination of pregnancy in the second trimester.
How do you prepare the cervix?

 Description of research


Participants 75 participants undergoing termination of pregnancy between
19+0 and 23+6 weeks of pregnancy in two tertiary care teaching
hospitals in California, USA.
Interventions Pre-procedure misoprostol, along with cervical preparation of one
(1) mifepristone alone (n=27),
(2) osmotic dilators with mifepristone (n=27),
(3) osmotic dilators with placebo (n=21)
Comparison Between arms of intervention.
Outcomes Operative time, preoperative cervical dilation and complications,
and participant painm
Study Design Site-stratified, double-blind randomized controlled trial


Discussion Points:

  • What were the interventions used in this trial? Do you use these, or others in your clinical practice?
  • How were participants randomised in this trial? What are the advantages and disadvantages of this method?
  • Were participants and clinicians sufficiently blinded in this study?
  • Second trimester termination of pregnancy is a specialist service that requires travel from those women undertaking the procedure. How does this affect the valuation of admission time and repeated visits?
  • Do you agree with the authors’ weighting of pain and procedure length in the conclusions of this study?
  • Are these study’s conclusions strong enough to support practice change?

Suggested reading

Critical Appraisal Skills Programme (2017). CASP Randomised Controlled Trial Checklist. [online] Available at:

What cervical cancer screening method would you recommend in a low resource setting?

Source: National Cancer Institute \ Winship Cancer Institute of Emory University
Creator: Scott Wilkinson, Adam Marcus

We will be discussing Accuracy of combinations of visual inspection using acetic acid or Iugol Iodine to detect cervical precancer: A meta-analysis.  Catarino et al. BJOG 2017 on Wednesday 27 September, 8pm BST.


You have been asked to develop a screening system for cervical cancer in a low income country.    What method do you recommend?

 Description of research


Participants 101,273 women participating in 23 studies in India (11 studies), Africa (9 studies), Oceania (one study) and Latin America (two studies)
Interventions Four strategies of cervical screening:

(1) visual inspection of the cervix with acetic acid (VIA);

(2) visual inspection with Iugol’s iodine (VILI);

(3) cotesting;

(4) VILI as a triage test of a positive VIA result.

Comparison Histology if a cervical biopsy was performed; colposcopy when no sample was taken
Outcomes Sensitivity and specificity of testing strategy
Study Design Diagnostic test accuracy meta-analysis


Discussion Points:

  • What are the described screening strategies? Do they differ from what you do in clinical practice?
  • What are the properties of an ideal screening test?
  • What is the relationship between sensitivity and specificity, and positive predictive value?
  • Most of the studies included were cross-sectional and prone to bias. What kinds of bias are discussed in this meta-analysis? Are there others you can think of? (1)
  • What other information would you like in order to answer the question in the scenario?
  • Imagine that a funder, hearing of the need for cancer screening in this country and the paucity of evidence, offers research funding for a trial to evaluate different methods. How would you design such a trial?

Suggested reading

Leeflang MMG, Deeks JJ, Gatsonis C, Bossuyt PMM, Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med. NIH Public Access; 2008 Dec 16;149(12):889–97.

How much physical activity is safe and healthy for pregnant women?

We will be discussing: Physical activity and the risk of preterm birth: a systematic review and meta-analysis of epidemiological studies by Aune and colleagues.

Wednesday 30th August 2017, 8pm BST

The first patient on the afternoon clinic schedule is seeing you for her first
obstetric visit. She is a healthy nulliparous 37-year-old business
administrator. She asks whether her morning workout routine will increase the
risk of preterm delivery.

What recommendations do you make about time and intensity of exercise during

Description of research

Participants: Pregnant women in prospective studies of physical activity in
Outcomes: Preterm birth <37 weeks
Study Design: Systematic review and meta-analysis of 41 studies from Europe, North America, South America, Asia, Australia and New Zealand
Authors’ conclusion: Higher leisure-time physical activity in pregnancy is associated
with a 14% relative risk reduction in preterm birth <37 weeks of gestation compared to lower physical activity. The data does not definitively answer the question of “how much” exercise is enough, or too much, but the lowest observed risk for preterm
birth was at 2-4 hours of activity per week.

Discussion points

  • How do you counsel women on physical activity in pregnancy?
  • Do you give the same advice about exercise to women who are underweight? How about those who are overweight or obese?
  • What is the most important conclusion of this study, from a patient-counselling standpoint?
  • Some of the forest plots (e.g Figure 4e) suggest that RCTs and cohort studies give different answers to a clinical question. Is this common in women’s health research? Can you name prominent examples where observational data gave an “opposite” or “wrong” answer to an important clinical question?
  • Describe the difficulties in studying this topic. How could you design the “perfect study” to answer the question of “how much” physical activity is safe and healthy for pregnant women?

Suggested reading

1. Perales et al. Exercise during pregnancy. JAMA 2017;317(11):1113-4.
2. Magro-Malosso et al. Exercise during pregnancy and risk of preterm birth in overweight and obese women: a systematic review and meta-analysis of randomized controlled trials. Acta Obstet Gynecol Scand 2017; 96:263-273.


A pill or a patch for a hot flush?

We will be discussing: Vasomotor symptoms due to natural menopause; systematic review and network meta-analysis (NMA) of treatment effects from the NICE Menopause Guideline by Sarri and colleagues.

Wednesday 26th July 2017, 8pm BST

A 52 year old woman presents to your clinic with hot flushes and night sweats coinciding with the menopause. She does not wish to have an operation. What treatments do you recommend?

Description of research

Participants: 8326 women participating in 47 randomised controlled trials of medications and therapies to control
1) Transdermal oestradiol and progesterone
2) Oral oestradiol and progesterone
3) Isofavones
4) Black cohosh
5) Chinese herbal medicine
6) Multibotanicals
7) Acupuncture
8) SSRIs
9) SNRIs
Comparison: Placebo and sham acupuncture
Outcomes: Odds ratios and means ratios of vasomotor symptoms (hot flushes and night sweats)
Study Design: Bayesian network meta-analysis (NMA)

Discussion points

  • What advice do you currently give women in the situation above?
  • In your experience, do women find pills or patches (a) more acceptable and (b) easier to use?
  • What is a network meta-analysis?
  • What does it mean for such an analysis to be Bayesian? How would you explain the methods used to another clinician?
  • There are many interventions in this analysis and many possible interactions. What problems does this present?
  • Would you change your recommendations to the women you treat as a result of this analysis?
  • Papers such as this use innovative statistical methods to answer real-life clinical questions. Consider your answers to questions 4 and 6. Is your answer to question 6 dependent on your answer to question 4? If so, how and why?

Suggested reading

Ades AE, Caldwell DM, Reken S, Welton NJ, Sutton AJ, Dias S. Evidence Synthesis for Decision Making 7. Med Decis Making. 3rd ed. SAGE PublicationsSage CA: Los Angeles, CA; 2013 Jun 26;33(5):679–91.