 |
|
Techniques to Improve Physicians' Use of Diagnostic Tests
A New Conceptual Framework
Daniel H. Solomon, MD, MPH;
Hideki Hashimoto, MD, MPH;
Lawren Daltroy, DrPH;
Matthew H. Liang, MD, MPH
JAMA. 1998;280:2020-2027.
ABSTRACT
| |
Objectives.— To review the published literature on interventions aimed at improving physicians' testing practices and propose methodologic standards for these studies and to review selected studies using the PRECEDE framework, a behavioral model that helps categorize interventions based on which behavioral factors are being affected.
Data Sources.— MEDLINE, EMBASE, and HEALTHStar databases were searched for the years 1966 to January 1, 1998, for English-language articles pertaining to diagnostic testing behavior; bibliographies were scanned to identify articles of potential interest; and researchers in health services, health behavior, and behavior modification were contacted for proprietary and other unpublished articles.
Study Selection.— A total of 102 articles were identified that described the results of interventions aimed at changing physicians' testing practices. We included the 49 studies that compared diagnostic testing practices in intervention and control groups.
Data Extraction.— Two investigators independently reviewed each article in a blinded fashion using a standard data collection form to obtain a methodologic score and to abstract the key elements of each intervention.
Data Synthesis.— On a 38-point methodologic criteria scale, the mean±SD score was 13 ± 4.4. The desired behavior change was reported in the intervention group in 37 (76%) of 49 studies. Twenty-four (86%) of 28 interventions targeted at many behavioral factors were successful, while 13 (62%) of 21 studies aimed at a single behavioral factor were successful (P=.12).
Conclusions.— A majority of interventions to improve physicians' testing practices reported in the literature claimed success, with interventions based on multiple behavioral factors trending toward being more successful. While methodologic flaws hamper drawing strong conclusions from this literature, application of a behavioral framework appears to be useful in explaining interventions that are successful and can facilitate interpretation of intervention results.
INTRODUCTION
DIAGNOSTIC TESTING and imaging are essential tools for disease screening, diagnosis, and monitoring. These aspects of medical care represent an enormous expenditure1 and studies suggest wide variation in test ordering behavior for seemingly similar indications.2-4 Several factors that may explain this variation include physicians' inability to estimate test performance characteristics, inaccurate interpretation of diagnostic test results,5-12 and rapid advances in diagnostic technology that make it difficult for clinicians to stay abreast of the most effective testing strategies.
Improving the appropriateness of testing behavior is a major focus of quality improvement, and many interventions have targeted laboratory and radiographic test ordering. Intervention strategies have included educational programs, guideline development and implementation, utilization audits, and economic incentives. Studies of interventions to modify physicians' test ordering behavior suggest that behavior change rarely can be accomplished solely through education, feedback of laboratory utilization data has inconsistent effects, rewarding physicians for improved ordering practices may result in short-term behavior change only, and testing guidelines have a variable effect depending on the implementation of the guidelines.13-17 Nearly every strategy has had both success and failure, providing limited guidance for designing new or more effective strategies.
Past reviews of this subject14-18 have not discriminated among tests used for screening, diagnosis, or monitoring—clinical situations that profoundly impact physicians' motivation and willingness to tolerate uncertainty and their test ordering behavior. Interventions aimed at screening generally have used accepted algorithms and attempt to enhance test usage, quite a different challenge than tests used for diagnosis, which are generally overused. No standardized evaluation of the methodology used in these interventions has been applied. Finally, and perhaps most importantly, the literature provides no evidence that the interventions utilized known behavioral theory.
The PRECEDE model of behavior change18 is a standardized framework that has been used to design successful, large-scale health interventions.19-20 PRECEDE incorporates 3 types of factors that precipitate or inhibit behavior change—predisposing, enabling, and reinforcing—and can be described through a clinical example. A 45-year-old man complains of chronic lower back pain and sciatica. His physician believes that this patient is at risk for metastatic cancer, leading him to order an imaging study. This physician's perception of his patient's risk for cancer exemplifies a predisposing factor, which includes other cognitive attributes such as attitudes or knowledge, underlying testing behavior. Enabling factors are skills, resources, or structural barriers that facilitate or prevent behavior. Extending the above example, if the physician had access to a specialist in his practice whom he could consult, he might learn that sciatica rarely represents metastatic cancer in patients younger than 50 years.21 Hence, proximity to an expert opinion might enable the physician to take a "wait and see" approach instead of ordering imaging. Reinforcing factors reward a specific behavior through feedback. In this example, on follow-up the patient's back and leg pain remits, reinforcing the importance of examining for sciatica and that a wait and see approach is often appropriate in lower back pain.
Predisposing, enabling, and reinforcing factors can facilitate or hinder behaviors. They are not mutually exclusive nor are their effects the same in all settings. For instance, a diagnostic guideline outlining indications for imaging studies printed on a radiology ordering form may guide physicians who do not know current recommendations, thus changing attitudes or predisposing factors, but reinforce the ordering practices of others who are more knowledgeable. These guidelines may also act as a memory aid for other clinicians who have heard about the recommendations, but do not remember them. Thus, the guideline-based order form can act to enable appropriate ordering.
Analysis of behavior change in terms of predisposing, reinforcing, and enabling factors has proven robust in behavior change planning for a variety of behaviors at many levels, including individual, institutional, and community.18 These groupings of variables encapsulate much of what has been learned empirically in the fields of psychology, sociology, and planned change. Because most behaviors are determined by multiple factors, it follows that interventions that address several key factors are more likely to be successful than unidimensional interventions. This observation was made by Oxman et al,22 who have developed a typology of interventions to change physician behavior. We elected to use the PRECEDE framework to categorize studies in our review since we believe it has greater explanatory power.
We reviewed the literature on interventions to modify physician testing behavior for methodologic rigor and whether known and effective behavioral principles were utilized. We hypothesized that multidimensional interventions that target more than 1 behavioral factor would be most effective.
METHODS
Identification and Selection of Literature
We searched the MEDLINE, EMBASE, and HEALTHStar databases for articles published between 1966 and January 1, 1998. Paired MeSH (medical subject heading) terms used for key and text word searching included physician behavior, physician practice patterns, laboratory use, test ordering, decision making, practice guidelines, appropriateness, and education. In addition, we contacted researchers in health services, health behavior, and behavior modification for proprietary and other nonpublished references. In addition, references of all articles obtained were scanned to identify others of potential interest.
Few randomized controlled trials of interventions to modify diagnostic test behavior exist; thus, we evaluated randomized and nonrandomized studies. Studies reviewed met the following criteria: (1) the behaviors of an intervention and a comparison group were examined; (2) the test(s) was used for diagnostic purposes and not solely for screening; and (3) the study was published in English.
Quality of Evidence
We built on prior methodologic standards for studies of therapeutics and supplemented them with behavioral criteria and developed an ad hoc methodologic scoring schema (Table 1).23-24 In our schema, more points corresponded to higher quality, with a maximum score of 38. The features we examined in these criteria included the patient and physician populations, the health care system, the intervention, and analysis including measurement of the outcomes.
|
|
|
|
Table 1.—Methodologic Standards for Evaluating Interventions to Change Physicians' Diagnostic Practices
|
|
|
Patient characteristics that may impact on the decision to test, such as demographic and clinical characteristics, must be balanced or controlled for in the analysis. We examined what methods were used for allocating physicians to control or intervention groups. Quasi-experimental study designs were categorized into those with prospectively allocated intervention and control groups vs those with only a historical control group. To assess generalizability of results, we looked for a description of the demographic characteristics, specialties, and levels of experience of the ordering physicians. These physician attributes had to be balanced between groups and/or controlled for in the analysis.
We examined whether authors described important environmental factors that might influence the use of diagnostic tests, such as patients' health insurance status. This criterion was not applied to interventions from countries with nationalized health insurance systems.
If physicians do not believe that the diagnostic testing intervention will improve care based on clinical evidence, behavior change may not occur. Hence, we examined the strength of evidence supporting a particular intervention. Interventions based on published evidence in peer-reviewed publications were rated highest and those based on consensus were rated lowest. Other aspects of the intervention that are important from a methodologic standpoint include whether it can be replicated based on the description and whether it is generalizable to other settings, for example, from hospitalized patients to ambulatory settings.
We suggest that interventions attempting to modify physicians' testing practices must be based on sound behavioral science; thus, we examined whether a behavioral framework was mentioned explicitly and whether attitudes that may mediate the intervention's effect were measured. Interventions were categorized according to which behavioral factor(s) was primarily targeted in the intervention: educational interventions were considered to target predisposing factors; utilization or charge audits primarily act to provide feedback and thus were included under reinforcing factors; and changes in the administrative structure of test ordering (such as order form revisions) were analyzed as enabling interventions.
Outcomes, when not resource utilization, should be measured with validated instruments in a blinded fashion. We examined whether appropriate end points were measured and whether the effects were desirable. Educational and behavioral interventions are often short-lived; thus, the durability of the results needs to be studied beyond the intervention period. Unintended changes in patient management should be examined. For example, an intervention to reduce imaging of the back might actually increase patient anxiety and prompt an increase in physician visits. Finally, we looked for appropriate power calculations to see if the results were statistically robust.
Two of us (D.H.S. and H.H.), blinded to the authors and institution studied, independently applied these criteria and the readers' agreement was examined by calculating an intraclass correlation.25 A mean methodologic quality score was calculated for each study and these were aggregated appropriately. Studies that reported reduced test volume or charges during the intervention period were considered successful. The Student t test was used for analysis of comparative differences between study types.
RESULTS
Methodologic Quality
One hundred two studies were identified through our search methods and 49 (48%) met all inclusion criteria and were reviewed. Of the other 53 excluded papers, 41 (40%) had no clear description of an intervention group and 12 (12%) examined interventions to change the use of tests primarily for screening. Agreement between the 2 independent blinded reviewers in the methodologic quality scores was high, with an intraclass correlation of 0.92. The methodologic quality scores were generally low (Table 2). No study was given more than 26 points (of a possible 38), with a mean±SD score of 13±4.4. Demographic characteristics of patients were described in only 17 studies (35%). Physicians' age and sex were outlined explicitly in only 1 study (2%). Eight studies (16%) used randomization, 19 (39%) included intervention and control groups, and the remaining 22 (45%) had only a historical control group. Two studies (4%) mentioned a behavioral framework as the basis of their intervention and 10 investigators (20%) examined physicians' intermediate attitudes. Eight (21%) of the studies performed in countries without nationalized health insurance discussed the insurance status of patients. Finally, 33 studies (67%) targeted physicians in training, 18 (37%) observed test ordering for 12 months or longer, and 22 (45%) examined test ordering after the intervention ended.
|
|
|
|
Table 2.—Methodologic Quality Scores by Targeted Behavioral Factors*
|
|
|
Intervention Efficacy
Even though the methodologic quality was low, 76% of the interventions reported reduced volume and/or charges for the test(s) being targeted (Table 3 and Table 4). Not all the studies, however, calculated tests of statistical significance. Sixty-two percent (13/21) of interventions aimed at 1 behavioral factor were successful in changing behavior (Table 3), while 86% (24/28) of interventions targeting more than 1 behavioral factor reported success (Table 4) (P=.12). No statistically significant differences between success rates of interventions targeting predisposing, reinforcing, or enabling factors were evident, but there was inadequate power to find small differences.
|
|
|
|
Table 3.—Unidimensional Interventions to Change Physicians' Diagnostic Practices
|
|
|
|
|
|
|
Table 4.—Multidimensional Interventions to Change Physicians' Diagnostic Practices*
|
|
|
Three (60%) of the 5 interventions involving exclusively physicians' knowledge or attitudes (predisposing factors) were successful. In this category, successful interventions targeted selected tests and the duration of effect was unclear.26, 28, 30 Several researchers coupled an educational intervention (predisposing factors) with reinforcements, such as utilization audits, and successfully changed behavior.47-50,52-53 However, when utilization audits (reinforcing factors) were used without accompanying educational material (predisposing factors), rates of behavior change were lower, with 4 (36%) of 11 interventions producing no change.31, 33, 35, 41
Changing the system or environment to enable behavior change was effective in selected situations. Three (60%) of 5 interventions targeting enabling factors produced change.43-45 Utilization audits used in conjunction with interventions that remove barriers to behavior change were successful, with 9 (75%) of 12 reporting positive results.54-59,62-64 When paired with educational strategies, enabling factor interventions produced change in every study reviewed.66, 68-74
We also examined whether more recent interventions were more successful at producing behavior change. The success rate for studies published before 1985 (59%) was slightly lower than that of studies published since 1985 (81%); however, this difference did not reach statistical significance (P=.12).
COMMENT
We reviewed the English-language literature concerning changing physicians' test ordering behavior, applied methodologic criteria, and overlaid a behavioral framework. The overall quality of this literature was not optimal, but results were generally positive, perhaps a manifestation of publication bias. Interventions that targeted multiple behavioral factors were more successful at producing change, a finding noted previously by Oxman et al.22 Several intervention types were common and warrant separate discussion.
Exercises to develop consensus among providers or other educational meetings were common but relatively weak interventions. Such interventions can be viewed as targeting attitudes or knowledge (predisposing factors). Often a consensus conference led to development of guidelines for appropriate testing, which were then disseminated as the intervention. Mozes et al71 found distribution of guidelines ineffective, but when combined with a change in the laboratory ordering form requiring justification of the test to be performed, ordering volume was reduced. Other selected studies also documented that combining a consensus conference with an audit was effective.48, 62, 69-70 Traditional education, aimed at predisposing attitudes or factors, was a weak intervention, but necessary and effective when coupled with strategies to reinforce attitudes or enable the desired behavior.
Utilization audits or presentation of laboratory charges to physicians were variably successful at producing behavior change. Audits can reinforce a desired testing practice but were often implemented without the proper baseline education. Interventions that first educated physicians as to the optimal diagnostic practice, a predisposing factor, and then gave physicians detailed comparative utilization data produced behavior change.47-50,52-53 We believe that utilization audits can act to reinforce desired diagnostic practices, but only if delivered to physicians who understand, or are predisposed to, the optimal testing strategy.
Several interventions explicitly used continuous quality improvement programs. This process entails assessment, consensus building, targeted behavior change, and reassessment; it has been recognized as an effective means for quality improvement and behavior change in the health care setting.75 For instance, Dowling et al50 and Nardella et al58 assessed baseline utilization of selected tests, developed and disseminated guidelines for test ordering, and then reassessed utilization. Both interventions reduced ordering and increased appropriateness. In addition, Dowling et al found that frequency of test use returned to baseline after the intervention, underscoring the need for continuous assessment.
Environmental or administrative interventions can be incredibly effective at little expense; however, they must be chosen carefully. For instance, deletion of the coagulation profile from standard admitting orders45 and restricting first-year residents to only 8 laboratory tests per patient43 were both successful interventions. On the other hand, Sussman et al46 attempted to eliminate standing orders and did not reduce test orders.
Several limitations should be noted. The proposed criteria for evaluating published studies on improving physicians' use of diagnostic tests build on previous work in this area23, 76 and, though somewhat arbitrary, have face validity. Specific criteria regarding the interventions were added to standardize our considerations of this genre of research. How to weight each criterion to maximize predictive validity is a question for further research.
Since the primary data were generally of low quality, we cannot draw strong conclusions. Many of the studies reviewed were not designed as research projects but as quality improvement programs. As the methodology underpinning this literature improves, we expect more interventions will be designed as research projects, allowing for more rigorous evaluation. Additionally, 2 of us who developed the methodologic quality criteria (Table 1) also applied them. Hence, the intraclass correlation was probably higher than it would be for other researchers using the criteria. We look forward to other applications of this scoring system to assess its reliability.
Pressure to use diagnostic tests in a more cost-effective manner continues. Current laboratory and diagnostic imaging practices need to be improved. Many attempts have been made to change physicians' diagnostic practices, with varying success. We reviewed these interventions applying standardized methodologic criteria. Our results suggest that past studies have been of poor quality and that few investigators have applied behavioral science principles.
Future research attempting to change physicians' behavior should incorporate relevant behavior change models, such as PRECEDE, and apply rigorous evaluations over longer periods of observation. Interventions should target multiple behavioral factors since multidimensional interventions appeared more successful than those aimed at 1 level. When utilization data are being supplied to physicians as part of an intervention, it should be coupled with educational material and documentation of clinical evidence to predispose physicians to the intended behavior. Finally, enabling factors that facilitate the preferred diagnostic behavior through blocking improper test orders or defaulting to the intended practice are most potent.
AUTHOR INFORMATION
This work was supported in part by National Institutes of Health grants AR36038, AI31599, and AR07530-10. Additionally, Dr Solomon is a recipient of an Arthritis Foundation/American College of Rheumatology Physician Scientist Training Award.
Robert Lew, PhD, and Elizabeth Wright, PhD, helped with calculation of the intraclass correlation. The anonymous reviewers made important comments that significantly shaped the manuscript.
Reprints: Daniel H. Solomon, MD, MPH, Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital, 221 Longwood Ave, Suite 341, Boston, MA 02115 (e-mail: dhsolomon{at}bics.bwh.harvard.edu).
From the Department of Medicine, Division of Rheumatology, Immunology, and Allergy, Robert B. Brigham Multipurpose Arthritis and Musculoskeletal Diseases Center, Brigham and Women's Hospital, Harvard School of Medicine (Drs Solomon, Hashimoto, Daltroy, and Liang), and the Department of Health and Social Behavior, Harvard School of Public Health (Drs Hashimoto and Daltroy), Boston, Mass.
REFERENCES
| |
1. Health Care Financing Administration. Health Care Financing Review, Medicare and Medicaid Statistical Supplement. 1997:150-151.
2. Wennberg JE. Dealing with medical practice variations: a proposal for action. Health Aff (Millwood). 1984;3:6-32.
PUBMED
3. Bass EB, Steinberg EP, Luthra R, et al. Variation in ophthalmic testing prior to cataract surgery: results of a national survey of optometrists. Arch Ophthalmol. 1995;113:27-31.
FREE FULL TEXT
4. Daniels M, Schroeder SA. Variation among physicians in use of laboratory tests, II: relation to clinical productivity and outcomes of care. Med Care. 1977;15:482-487.
ISI
| PUBMED
5. Abbott JA, Tedeschi MA, Cheitlin MD. Graded treadmill stress testing: patterns of physician use and abuse. West J Med. 1977;126:173-178.
ISI
| PUBMED
6. Casscells W, Schoenberger A, Graboys TB. Interpretation by physicians of clinical laboratory results. N Engl J Med. 1978;299:999-1001.
ISI
| PUBMED
7. McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996;124:56-62.
FREE FULL TEXT
8. Kanouse DE, Jacoby I. When does information change practitioners' behavior? Int J Tech Assess Health Care. 1988;4:27-33.
9. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700-705.
FREE FULL TEXT
10. Sibley JC, Sackett DL, Neufeld V, Gerrard B, Rudnick KV, Fraser W. A randomized trial of continuing medical education. N Engl J Med. 1982;306:511-515.
ABSTRACT
11. Freemantle N. Are decisions taken by health care professionals rational? a non systematic review of experimental and quasi experimental literature. Health Policy. 1996;38:71-81.
FULL TEXT
|
ISI
| PUBMED
12. McKinlay JB, Potter DA, Feldman HA. Non-medical influences on medical decision-making. Soc Sci Med. 1996;42:769-776.
FULL TEXT
|
ISI
| PUBMED
13. Axt-Adam P, van der Wouden JC, van der Does E. Influencing behavior of physicians ordering laboratory tests: a literature study. Med Care. 1993;31:784-794.
ISI
| PUBMED
14. Eisenberg JM. Physician utilization: the state of research about physicians' practice patterns. Med Care. 1985;23:461-483.
FULL TEXT
|
ISI
| PUBMED
15. Grossman RM. A review of physician cost-containment strategies for laboratory testing. Med Care. 1983;21:783-802.
FULL TEXT
|
ISI
| PUBMED
16. Moskowitz MA. Evaluating doctors' office testing patterns. Clin Lab Med. 1986;6:387-394.
ISI
| PUBMED
17. Young DW. Improving laboratory usage: a review. Postgrad Med J. 1988;6:283-289.
18. Green LW, Kreuter MW. Health Promotion Planning: An Educational and Environmental Approach. 2nd ed. Mountain View, Calif: Mayfield Publishing Co; 1991.
19. Gans KM, Jack B, Lasater TM, Lefebvre RC, McQuade W, Carleton RA. Changing physicians' attitudes, knowledge, and self-efficacy regarding cholesterol screening and management. Am J Prev Med. 1993;9:101-106.
ISI
| PUBMED
20. Murray DM, Kurth CL, Finnegan JR, et al. Direct mail as a prompt for follow-up care for persons at high risk for hypertension. Am J Prev Med. 1988;4:331-335.
ISI
| PUBMED
21. Deyo RA, Diehl AK. Cancer as a cause of back pain: frequency, clinical presentation, and diagnostic strategies. J Gen Intern Med. 1988;3:230-238.
ISI
| PUBMED
22. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423-1431.
ABSTRACT
23. Chalmers TC, Smith H, Blackburn B, et al. A method for assessing the quality of a randomized control trial. Control Clin Trials. 1981;2:31-49.
FULL TEXT
|
ISI
| PUBMED
24. Balas EA, Austin SM, Ewigran BG, Brown GD, Mitchell JA. Methods of randomized controlled clinical trials in health services research. Med Care. 1995;33:687-699.
ISI
| PUBMED
25. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New York, NY: John Wiley & Sons Inc; 1981.
26. Davidoff F, Goodspeed R, Clive J. Changing test behavior: a randomized controlled trial comparing probabilities reasoning with cost-containment education. Med Care. 1989;27:45-58.
FULL TEXT
|
ISI
| PUBMED
27. Eisenberg JM. An educational program to modify laboratory use by house staff. J Med Educ. 1977;52:578-581.
ISI
| PUBMED
28. Ratnaike S, Hunt D, Eilermann LJM, Deam D. The investigation of chest pain: audit and intervention. Med J Aust. 1993;159:666-671.
ISI
| PUBMED
29. Rhyne RL, Gehlbach SH. Effects of an educational feedback strategy on physician utilization of thyroid function panels. J Fam Pract. 1979;8:1003-1007.
ISI
| PUBMED
30. Wong ET, McCarron MM, Shaw ST. Ordering of laboratory tests in a teaching hospital: can it be improved? JAMA. 1983;249:3076-3080.
FREE FULL TEXT
31. Bates DW, Kuperman GJ, Jha A, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med. 1997;157:2501-2508.
FREE FULL TEXT
32. Cohen DI, Jones P, Littenberg B, Neuhauser D. Does cost information availability reduce physician test usage? a randomized clinical trial with unexpected findings. Med Care. 1982;20:286-292.
FULL TEXT
|
ISI
| PUBMED
33. Forrest JB, Ritchie WP, Hudson M, Harlan JF. Cost containment through cost awareness: a strategy that failed. Surgery. 1981;90:154-158.
ISI
| PUBMED
34. Gama R, Nightingale PG, Broughton PMG, et al. Modifying the request behavior of clinicians. J Clin Pathol. 1992;45:248-249.
FREE FULL TEXT
35. Grivell AR, Forgie HJ, Fraser CG, Berry MN. League tables of biochemical costs: an attempt to modify requesting patterns. Med J Aust. 1982;2:326-328.
ISI
| PUBMED
36. Lyle CB, Bianchi RF, Harris JH, Wood ZL. Teaching cost containment to house officers at Charlotte Memorial Hospital. J Med Educ. 1979;54:856-862.
ISI
| PUBMED
37. Pop P, Winkens RA. A diagnostic centre for general practitioners: results of individual feedback on diagnostic actions. J R Coll Gen Pract. 1989;39:507-508.
ISI
| PUBMED
38. Pugh JA, Frazier LM, DeLong E, et al. Effect of daily charge feedback on inpatient charges and physician knowledge and behavior. Arch Intern Med. 1989;149:426-429.
FREE FULL TEXT
39. Schroeder SA, Kenders K, Cooper JK, Piemme TE. Use of laboratory tests and pharmaceuticals: variation among physicians and effect of cost audit on subsequent use. JAMA. 1973;225:969-973.
FREE FULL TEXT
40. Winkens RAG, Pop P, Grol RPTM, Kester ADM, Knottnerus JA. Effect of feedback on test ordering behavior of general practitioners. BMJ. 1992;304:1093-1096.
FREE FULL TEXT
41. Wones RG. Failure of low-cost audits with feedback to reduce laboratory test utilization. Med Care. 1987;25:78-82.
FULL TEXT
|
ISI
| PUBMED
42. Bree RL, Kazerooni EA, Katz SJ. Effect of mandatory radiology consultation on inpatient imaging use: a randomized controlled trial. JAMA. 1996;276:1595-1598.
FREE FULL TEXT
43. Dixon RH, Laszlo J. Utilization of clinical chemistry services by medical house staff: an analysis. Arch Intern Med. 1974;134:1064-1067.
FREE FULL TEXT
44. Golden WE, Pappas AA, Lavender RC. Financial unbundling reduces outpatient laboratory use. Arch Intern Med. 1987;147:1045-1048.
FREE FULL TEXT
45. Groopman DS, Powers RD. Effect of "standard order" deletion on emergency department coagulation profile use. Ann Emerg Med. 1992;21:524-527.
FULL TEXT
|
ISI
| PUBMED
46. Sussman E, Goodwin P, Rosen H. Administrative change and diagnostic test use: the effect of eliminating standing orders. Med Care. 1984;22:569-572.
FULL TEXT
|
ISI
| PUBMED
47. Applegate WB, Bennett MD, Chilton L, Skipper BJ, White RE. Impact of a cost-containment educational program on housestaff ambulatory clinic charges. Med Care. 1983;21:486-496.
ISI
| PUBMED
48. Bareford D, Hayling A. Inappropriate use of laboratory services: long term combined approach to modify request patterns. BMJ. 1990;301:1305-1307.
FREE FULL TEXT
49. Billi JE, Hejna GF, Wolf FM, Shapiro LR, Stross JK. The effects of a cost-education program on hospital charges. J Gen Intern Med. 1987;2:306-311.
ISI
| PUBMED
50. Dowling PT, Alfonsi G, Brown MI, Culpepper L. An education program to reduce unnecessary laboratory tests by residents. Acad Med. 1989;64:410-412.
ISI
| PUBMED
51. Eisenberg JM, Williams SV, Garner L, Viale R, Smits H. Computer-based audit to detect and correct overutilization of laboratory tests. Med Care. 1977;15:915-921.
ISI
| PUBMED
52. Fowkes FGR, Hall R, Jones JH, et al. Trial of a strategy for reducing the use of laboratory tests. BMJ. 1986;292:883-885.
FREE FULL TEXT
53. McDonald CJ. Use of a computer to detect and respond to clinical events: its effect on clinical behavior. Ann Intern Med. 1976;84:162-167.
ISI
| PUBMED
54. Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost education, cost audits, and faculty chart review on the use of laboratory services. Arch Intern Med. 1983;143:942-944.
FREE FULL TEXT
55. Fowkes FGR, Evans KT, Hartley G, et al. Multicentre trial of four strategies to reduce use of a radiological test. Lancet. 1986;1:367-369.
FULL TEXT
|
ISI
| PUBMED
56. Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med. 1980;303:1330-1336.
ABSTRACT
57. Marton KI, Tul V, Sox HC. Modifying test-ordering behavior in the outpatient medical clinic: a controlled trial of two educational interventions. Arch Intern Med. 1985;145:816-821.
FREE FULL TEXT
58. Nardella A, Farrell M, Pechet L, Snyder LM. Continuous improvement, quality control, and cost containment in clinical laboratory testing: enhancement of physicians' laboratory-ordering practices. Arch Pathol Lab Med. 1994;118:965-968.
ISI
| PUBMED
59. Novich M, Gillis L, Tauber AI. The laboratory test justified: an effective means to reduce routine laboratory testing. Am J Clin Pathol. 1985;84:756-759.
ISI
| PUBMED
60. Pozen MW, Gloger H. The impact on house officers of educational and administrative interventions in an outpatient department. Soc Sci Med. 1976;10:491-495.
PUBMED
61. Schroeder SA, Myers LP, McPhee SJ, et al. The failure of physician education as a cost containment strategy: report of a prospective controlled trial at a university hospital. JAMA. 1984;252:225-230.
FREE FULL TEXT
62. Spiegel JS, Shapiro MF, Berman B, Greenfield S. Changing physician test ordering in a university hospital: an intervention of physician participation, explicit criteria, and feedback. Arch Intern Med. 1989;149:549-553.
FREE FULL TEXT
63. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computer predictions of abnormal test results: effects on outpatient testing. JAMA. 1988;259:1194-1198.
FREE FULL TEXT
64. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med. 1990;322:1499-1504.
ABSTRACT
65. Williams SV, Eisenberg JM. A controlled trial to decrease the unnecessary use of diagnostic tests. J Gen Intern Med. 1986;1:8-13.
ISI
| PUBMED
66. Wachtel TJ, O'Sullivan PO. Practice guidelines to reduce testing in the hospital. J Gen Intern Med. 1990;5:335-341.
ISI
| PUBMED
67. Auleley GR, Ravaud P, Giraudeau B, et al. Implementation of the Ottawa ankle rules in France: a multicenter randomized controlled trial. JAMA. 1997;277:1935-1939.
FREE FULL TEXT
68. Berwick DM, Coltin KL. Feedback reduces test use in a health maintenance organization. JAMA. 1986;255:1450-1454.
FREE FULL TEXT
69. Gortmaker SL, Bickford AF, Mathewson HO, Dumbaugh K, Tirrell PC. A successful experiment to reduce unnecessary laboratory use in a community hospital. Med Care. 1988;26:631-642.
ISI
| PUBMED
70. Kroenke K, Hanley JF, Copley JB, et al. Improving house staff ordering of three common laboratory tests: reduction in test ordering need not result in underutilization. Med Care. 1987;25:928-935.
FULL TEXT
|
ISI
| PUBMED
71. Mozes B, Lubin D, Modan B, et al. Evaluation of an intervention aimed at reducing inappropriate use of preoperative blood coagulation tests. Arch Intern Med. 1989;149:1836-1838.
FREE FULL TEXT
72. Schectman JM, Elinsky EG, Pawlson LG. Effect of education and feedback on thyroid function testing strategies of primary care clinicians. Arch Intern Med. 1991;151:2163-2166.
FREE FULL TEXT
73. Stiell I, Wells G, Laupacis A, et al. Multicentre trial to introduce the Ottawa ankle rules for use of radiography in acute ankle injuries. BMJ. 1995;311:594-597.
FREE FULL TEXT
74. Zaat JOM, van Eijk JTM, Bonte HA. Laboratory test form design influences test ordering by general practitioners in the Netherlands. Med Care. 1992;30:189-198.
FULL TEXT
|
ISI
| PUBMED
75. Blumenthal D, Epstein AM. Quality of health care, 6: the role of physicians in the future of quality management. N Engl J Med. 1996;335:1328-1331.
FREE FULL TEXT
76. Detsky AS, Naylor CD, O'Rourke KO, McGeer AJ, L'Abbe KA. Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol. 1992;45:255-265.
FULL TEXT
|
ISI
| PUBMED
 CiteULike  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter
What's this?
RELATED ARTICLES
Effect of Population-Based Interventions on Laboratory Utilization: A Time-Series Analysis
Carl van Walraven, Vivek Goel, and Ben Chan
JAMA. 1998;280(23):2028-2033.
ABSTRACT
| FULL TEXT
Changing Physician Behavior in Ordering Diagnostic Tests
George D. Lundberg
JAMA. 1998;280(23):2036.
EXTRACT
| FULL TEXT
December 16, 1998
JAMA. 1998;280(23):2051-2052.
EXTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Specialty Differences in Primary Care Physician Reports of Papanicolaou Test Screening Practices: A National Survey, 2006 to 2007
Yabroff et al.
ANN INTERN MED 2009;151:602-611.
ABSTRACT
| FULL TEXT
Clinical Pathology Differences in Laboratory Utilization in Hospice and Nonhospice Units
Barbosa et al.
AM J HOSP PALLIAT CARE 2009;26:79-83.
ABSTRACT
Influence of Watchful Waiting on Satisfaction and Anxiety Among Patients Seeking Care for Unexplained Complaints
van Bokhoven et al.
Ann Fam Med 2009;7:112-120.
ABSTRACT
| FULL TEXT
Appropriate laboratory utilization in diagnosing pulmonary embolism
Eva et al.
Ann Clin Biochem 2009;46:18-23.
ABSTRACT
| FULL TEXT
The unbearable lightness of diagnostic testing: time to contain inappropriate test ordering
Schattner
Postgrad. Med. J. 2008;84:618-621.
ABSTRACT
| FULL TEXT
Canadian Quality Circle pilot project in osteoporosis: Rationale, methods, and feasibility
Ioannidis et al.
cfp 2007;53:1694-1700.
ABSTRACT
| FULL TEXT
The effect of automated test rejection on repeat requesting
Sharma and Salzmann
J. Clin. Pathol. 2007;60:954-955.
FULL TEXT
Pitfalls of testing and summary of guidance on safety monitoring with amiodarone and digoxin
Smellie and Coleman
BMJ 2007;334:312-315.
FULL TEXT
Cost Consequences of Implementing an Electronic Decision Support System for Ordering Laboratory Tests in Primary Care: Evidence from a Controlled Prospective Study in The Netherlands
Poley et al.
Clin. Chem. 2007;53:213-219.
ABSTRACT
| FULL TEXT
Factors contributing to inappropriate ordering of tests in an academic medical department and the effect of an educational feedback strategy
Miyakis et al.
Postgrad. Med. J. 2006;82:823-829.
ABSTRACT
| FULL TEXT
What Interventions Should Pharmacists Employ to Impact Health Practitioners' Prescribing Practices?
Grindrod et al.
The Annals of Pharmacotherapy 2006;40:1546-1557.
ABSTRACT
| FULL TEXT
Pediatricians' Clinical Decision Making: Results of 2 Randomized Controlled Trials of Test Performance Characteristics
Sox et al.
Arch Pediatr Adolesc Med 2006;160:487-492.
ABSTRACT
| FULL TEXT
Epidemiology of Uncontrolled Hypertension in the United States
Wang and Vasan
Circulation 2005;112:1651-1662.
FULL TEXT
Unnecessary repeat requesting of tests: an audit in a government hospital immunology laboratory
Kwok and Jones
J. Clin. Pathol. 2005;58:457-462.
ABSTRACT
| FULL TEXT
Comparing cost effects of two quality strategies to improve test ordering in primary care: a randomized trial
Verstappen et al.
Int J Qual Health Care 2004;16:391-398.
ABSTRACT
| FULL TEXT
The Impact of Peer Management on Test-Ordering Behavior
Neilson et al.
ANN INTERN MED 2004;141:196-204.
ABSTRACT
| FULL TEXT
GPs' and physicians' interpretation of risks, benefits and diagnostic test results
Heller et al.
Fam Pract 2004;21:155-159.
ABSTRACT
| FULL TEXT
Effect of a Controlled Feedback Intervention on Laboratory Test Ordering by Community Physicians
Bunting and van Walraven
Clin. Chem. 2004;50:321-326.
ABSTRACT
| FULL TEXT
Effect of a Practice-Based Strategy on Test Ordering Performance of Primary Care Physicians: A Randomized Trial
Verstappen et al.
JAMA 2003;289:2407-2412.
ABSTRACT
| FULL TEXT
From Physician to Consumer: The Effectiveness of Strategies to Manage Health Care Utilization
Flynn et al.
Med Care Res Rev 2002;59:455-481.
ABSTRACT
A Utilization Management Intervention to Reduce Unnecessary Testing in the Coronary Care Unit
Wang et al.
Arch Intern Med 2002;162:1885-1890.
ABSTRACT
| FULL TEXT
Why Do Physicians Not Follow Evidence-Based Guidelines for Preventing Ventilator-Associated Pneumonia?* : A Survey Based on the Opinions of an International Panel of Intensivists
Rello et al.
Chest 2002;122:656-661.
ABSTRACT
| FULL TEXT
Impact of end user involvement in implementing guidelines on routine pre-operative tests
BARAZZONI et al.
Int J Qual Health Care 2002;14:321-327.
ABSTRACT
| FULL TEXT
Rational, cost effective use of investigations
Bulusu et al.
BMJ 2002;325:222-222.
FULL TEXT
Infection Control in the ICU
Eggimann and Pittet
Chest 2001;120:2059-2093.
ABSTRACT
| FULL TEXT
Quality improvement report: A laboratory based intervention to improve appropriateness of lipid tests and audit cholesterol lowering in primary care
Smellie et al.
BMJ 2001;323:1224-1227.
ABSTRACT
| FULL TEXT
Assessment of Decision Support for Blood Test Ordering in Primary Care: A Randomized Trial
van Wijk et al.
ANN INTERN MED 2001;134:274-281.
ABSTRACT
| FULL TEXT
A Report Card on Quality Improvement for Children's Health Care
Ferris et al.
Pediatrics 2001;107:143-155.
ABSTRACT
| FULL TEXT
Benchmarking general practice use of pathology services: a model for monitoring change
Smellie et al.
J. Clin. Pathol. 2000;53:476-480.
ABSTRACT
| FULL TEXT
Despite Financial Penalties, French Physicians' Knowledge of Regulatory Practice Guidelines Is Poor
Durieux et al.
Arch Fam Med 2000;9:414-418.
ABSTRACT
| FULL TEXT
Common Diagnostic Test Panels for Clinical Evaluation of New Primary Care Outpatients in Japan: A Cost-Effectiveness Evaluation
Takemura et al.
Clin. Chem. 1999;45:1752-1761.
ABSTRACT
| FULL TEXT
Analysis of the Practice Guidelines of the Dutch College of General Practitioners with Respect to the Use of Blood Tests
VAN Wijk et al.
J. Am. Med. Inform. Assoc. 1999;6:322-331.
ABSTRACT
| FULL TEXT
Changing Physician Behavior in Ordering Diagnostic Tests
Lundberg
JAMA 1998;280:2036-2036.
FULL TEXT
|
