This Article  • Full text  • PDF  • Send to a friend  • Save in My Folder  • Save to citation manager  • Permissions  Citing Articles  • Citation map  • Citing articles on HighWire  • Citing articles on Web of Science (9)  • Contact me when this article is cited  Related Content  • Similar articles in JAMA  Social Bookmarking   What's this?

Comparing What Was Done to What Was Planned

Chris A. Silagy, AO,PhD,MB,BS; Philippa Middleton, BSc(Hons),MPH; Sally Hopewell, MSc

JAMA. 2002;287:2831-2834.

Context  Publication of research protocols minimizes bias by explicitly stating a priori hypotheses and methods without prior knowledge of results.

Methods  We conducted a retrospective comparative study to assess the extent to which the content of published Cochrane reviews had changed compared with their previously published protocols and to assess any potential impact these changes may have had in introducing bias to the study. We identified previously published protocols for new Cochrane reviews appearing in The Cochrane Library; 2000, issue 3. The texts of published protocols and completed reviews were compared. Two raters independently identified changes to the different sections of the protocol and classified the changes as none, minor, or major.

Results  Of the 66 new Cochrane reviews, we identified a previously published protocol for 47 reviews. Of these, 43 reviews had at least 1 section that had undergone a major change compared with the most recently published protocol. The greatest variation between protocols and reviews was in the methods section, in which 68% of reviews (n = 32) had undergone a major change. Changes made in other sections that may have resulted in the introduction of bias included narrowing of objectives, addition of comparisons or new outcome measures, broadening of criteria for the types of study design included, and narrowing of types of participants included.

Conclusions  Research protocols, even if published, are likely to remain, at least to some extent, iterative documents. We found that a large number of changes were made to Cochrane reviews, some of which could be prone to influence by prior knowledge of results. Even if many of the changes between protocol and review improve the overall study, the reasons for making these should be clearly identified and documented within the final review.

Author Affiliations: Monash Institute of Health Services Research, Monash Medical Centre, Clayton, Victoria, Australia (Dr Silagy); and UK Cochrane Centre, Oxford, England (Mss Middleton and Hopewell). Dr Silagy is deceased.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter     What's this?

THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES

Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu et al.
JAMA 2009;302:977-984.
ABSTRACT | FULL TEXT  

Reprint--Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher et al.
ptjournal 2009;89:873-880.
ABSTRACT | FULL TEXT  

Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher et al.
ANN INTERN MED 2009;151:264-269.
FULL TEXT  

The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration
Liberati et al.
ANN INTERN MED 2009;151:W-65-W-94.
ABSTRACT | FULL TEXT  

Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement
Moher et al.
BMJ 2009;339:b2535-b2535.
FULL TEXT  

The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration
Liberati et al.
BMJ 2009;339:b2700-b2700.
ABSTRACT | FULL TEXT  

Publishing study protocols
Johnston
Inj. Prev. 2008;14:73-73.
FULL TEXT  

Patient-Care Questions that Physicians Are Unable to Answer
Ely et al.
J. Am. Med. Inform. Assoc. 2007;14:407-414.
ABSTRACT | FULL TEXT  

Methodological Approach: American College of Chest Physicians Guidelines for the Prevention and Management of Postoperative Atrial Fibrillation After Cardiac Surgery
Fleisher et al.
Chest 2005;128:17S-23S.
ABSTRACT | FULL TEXT  

Empirical Evidence for Selective Reporting of Outcomes in Randomized Trials: Comparison of Protocols to Published Articles
Chan et al.
JAMA 2004;291:2457-2465.
ABSTRACT | FULL TEXT  

Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
Soares et al.
BMJ 2004;328:22-24.
ABSTRACT | FULL TEXT