Login to MyKarger

New to MyKarger? Click here to sign up.

Login with Facebook

Forgot Password? Reset your password

Authors, Editors, Reviewers

For Manuscript Submission, Check or Review Login please go to Submission Websites List.

Submission Websites List

Institutional Login (Shibboleth)

For the academic login, please select your country in the dropdown list. You will be redirected to verify your credentials.

Table of Contents
Vol. 20, No. 4, 2003
Issue release date: 2003
Section title: Review
Free Access
Dig Surg 2003;20:263–269
(DOI:10.1159/000071183)

Perspectives of Evidence-Based Surgery

Wente M.N. · Seiler C.M. · Uhl W. · Büchler M.W.
Center for Clinical Studies in Surgery, Department of General, Visceral and Trauma Surgery, University of Heidelberg, Heidelberg, Germany
email Corresponding Author

Waldemar Uhl, MD

Department of General, Visceral and Trauma Surgery, University of Heidelberg

Im Neuenheimer Feld 110, D–69120 Heidelberg (Germany)

Tel. +49 6221 56 6111, Fax +49 6221 56 5450

E-Mail waldemar_uhl@med.uni-heidelberg.de


Abstract

The assessment of the optimal treatment option based on best current knowledge is called evidence-based medicine (EBM). Considering the cost explosion in public health systems, EBM should also incorporate proper utilization of the restricted economical resources and should enforce quality assurance in medicine. It is imperative that surgeons realize that randomized controlled trials are applicable to the operative specialties in a large scale, and are necessary to provide evidence-based surgery. So far, only 3.4% of all publications in the leading surgical journals are randomized controlled trials. Furthermore, only 44.1% of the published surgical randomized controlled studies compared different surgical procedures, whereas 55.9% of the articles compared medical therapies in surgical patients. Evidence-based surgical therapy is essential for further development of a high- quality surgical standard, which will also provide quality assurance in future surgical care. This article presents the definition of EBM and discusses specific problems involved in the introduction of its principles into the surgical discipline.

© 2003 S. Karger AG, Basel


Related Articles for ""

Close Related Articles


Introduction

Over the last few decades, medicine has seen an information explosion with nearly uncontrollable quantities of information derived from scientific research. At the same time, medical practitioners faced the challenge of integrating this rapidly enlarging knowledge into a health care system with very limited financial resources. This process has led to an increasing recognition of the need for reliable evidence for treatment decisions, forcing the development of the Cochrane Collaboration. This international organization provides systematic reviews of trials of health care interventions. Evidence-based medicine (EBM) supplies different techniques to solve clinical questions and therefore transfers results out of clinical studies to numbers that can be used in surgical practice. Until now, only limited studies can be used for EBM purposes because of a lack in study design, analysis and presentation [1].

Definition of EBM

Sackett et al. [2]have defined EBM as the integration of best research evidence with clinical expertise and patient values. Best research evidence is defined as clinically relevant data from basic sciences and especially quantitative clinical research. Together with the skills, past experience and knowledge of the surgeon and the unique preferences, concerns and expectations of each patient, an integrated clinical decision can be drawn for a specific problem [2].

Randomized controlled trials and their subsequent systematic reviews including meta-analyses represent the top of the hierarchy for finding evidence for medical routine and are therefore often defined as the ‘reference standard’ for clinical studies (table 1). Randomized controlled trials are the best method to compare the effectiveness of different therapeutic interventions and can have a direct impact on patient care. In order to provide all necessary information about external and internal validity and to reach the highest possible standard for reporting results of randomized controlled trials, the Consolidated Standards of Reporting Trials (CONSORT) criteria were established in 1996 and revised in 2001 [3, 4]. These CONSORT were created to improve the quality of research reports of randomized controlled trials and represent the most important standards of high-quality clinical research (table 2). The CONSORT statement found a rapid distribution into the scientific world and is now accepted as a general guideline for reporting of randomized controlled trials [5, 6, 7]. The wide support of the CONSORT checklist, encouraged by leading medical journals, has led to an improvement of reports of randomized controlled trials and thus to a facilitation of effective clinical decision making [8].

Table 1

Levels of evidence (from the web page of the NHS Research and Development – Center for Evidence-Based Medicine)

http://www.karger.com/WebMaterial/ShowPic/310165

Table 2

Checklist for reporting a randomized trial – CONSORT (from Moher et al. [4])

http://www.karger.com/WebMaterial/ShowPic/310164

Introduction of EBM into Surgical Therapy

Randomized controlled trials are not appropriate for all surgical interventions, but should be seriously considered whenever a newly introduced intervention is being studied [9]. The first successful surgical randomized controlled studies were performed and presented in the 1960s, when Goligher et al. [10]randomized patients to different treatments for peptic ulcer disease (table 3).

Table 3

Introduction of the randomized controlled trial (RCT) into surgery (from Solomon and McLeod [15])

http://www.karger.com/WebMaterial/ShowPic/310163

Randomized controlled studies can provide valid results comparing treatment alternatives, and this available evidence should be considered for all therapeutic clinical decisions. Assessments of the integration of the principles of EBM into clinical practice were first performed in the field of general medicine. Ellis et al. [11]studied the treatment in a university-affiliated general medicine team and grouped the treatments in three categories: (1) evidence from randomized controlled trials; (2) convincing nonexperimental evidence, based on nonrandomized prospective trials and large retrospective studies, and (3) interventions without substantial evidence, based on case reports or small clinical studies.

Eighty-two percent of all medical treatments in this study were evidence based (53% group I, 29% group II). In a similar study about the efficacy of surgical practice, it was shown that general surgical practice is evidence based (95% group I and II), but only 25% of the patients received treatment based on randomized controlled trials and thereby presumed to be of the best available quality [12]. In a study about evidence-based general thoracic surgical practice, comparing 50 different types of thoracic surgical treatments, 14% were supported by group I, and 64% by group II level of evidence [13]. Therefore, these studies demonstrate that the majority of commonly performed surgical procedures and treatments are evidence based, but the fraction of evidence from randomized controlled trials is still smaller than in general medicine.

A recent analysis of the MEDLINE database (01/1966–12/2000) confirmed these numbers, as previously reported [1]. Of the 134,689 published randomized controlled trials, only 20,376 (15.1%) dealt with surgical contents (table 4). In a detailed analysis of all articles (n = 45,342) out of 5 leading surgical journals (Annals of Surgery, Archives of Surgery, British Journal of Surgery, World Journal of Surgery, Surgery), only 3.4% (n = 1,530) of all papers were randomized controlled trials (table 5). In further analysis, all randomized controlled trials out of these 5 surgical journals from the last decade (01/1991–12/2000) were categorized according to their main subject. This analysis revealed that 10.8% (n = 78) of the articles defined as randomized controlled trials (n = 728) did not fulfill the criteria or were mislabeled. To improve the quality of publishing randomized controlled trials, all journals should accept the conditions of the CONSORT guidelines [14].

Table 4

Literature analysis (MEDLINE)1966–20001

http://www.karger.com/WebMaterial/ShowPic/310162

Table 5

Randomized controlled trials in leading scientific surgical journals (MEDLINE 1966–2000)

http://www.karger.com/WebMaterial/ShowPic/310161

In addition, 55.9% (n = 363) of all randomized controlled studies (n = 650) in the analyzed surgical journals dealt with medical therapies in surgical patients, such as analgesic, antibiotic or neo-/adjuvant cancer treatment concepts. Only the remaining 44.1% (n = 287) of the articles compared different surgical procedures or operative and conservative treatments (table 6).

Table 6

Detailed analysis of surgical randomized controlled trials (MEDLINE 1991–2000)

http://www.karger.com/WebMaterial/ShowPic/310160

Remarkably, over 80% of all published surgical studies are still case reports or single institutional small series [15]. Despite the fact that they are cheap and easy to perform, they generate only a low level of evidence about the efficacy of different treatment options.

Specific Problems of Randomized Controlled Studies in Surgery

The small number of randomized controlled trials in surgery has been criticized for years. At the same time, the implementation of EBM into the field of surgery faces some specific problems. Compared with randomized controlled studies in internal medicine, which were originally developed to compare two different drug protocols, there are some unique pitfalls and sources of bias in surgical randomized trials.

First, randomized, double-blinded and placebo-controlled trials are the only way to not only control the biased investigator but also the placebo effects. Sham operations have been performed, i.e. internal mammary artery ligation was reported as providing relief for pain in angina pectoris, but the trials were discontinued when, in comparison, a sham operation showed a similar success rate [16, 17]. In recent years, the use of placebo operations and the substantial ethical concerns about sham surgery in particular have been discussed anew. In this presently ongoing discussion, two recently published studies using placebo surgery should be used as prime examples. Freeman et al. [18]used placebo surgery for a cellular-based therapy of Parkinson’s disease and provided a basis for a new discussion about ethical aspects of sham operations [18, 19, 20]. Moseley et al. [21]could prove that there is no difference in pain relief after arthroscopic or placebo surgery for osteoarthritis of the knee. This led to a first version of ethical guidelines on the use of surgical placebo controls in the design of surgical trials [22]. If no sham operation is performed, investigators should be aware of bias through the placebo effect of all surgical procedures [23].

Furthermore, it remains difficult to standardize the tested surgical procedures because the procedures will evolve continuously and because the complications decrease as surgeons overcome the learning curve and gain increasing experience with a new procedure. The results vary with the individual surgeon because the participating operators vary in their surgical skill and experience (table 7). All participating surgeons should undergo appropriate training before the start of a randomized controlled trial to reach a certain minimum of standardization [24, 25].

Table 7

Differences between randomized trials for drugs and surgical procedures (from Bonchek [24])

http://www.karger.com/WebMaterial/ShowPic/310159

Yet, another source of bias in surgical trials is the blinding of patients and surgeons. Unfortunately, it is not always possible to blind all participants as effectively shown in the exemplary cited trial of Majeed et al. [26], where the same wound dressing was used for patients who underwent laparoscopic and small incision cholecystectomy. Especially if the primary outcome criteria are not recurrence of disease or even death, but variable symptoms or quality of life measurements, a lack of blinding procedures may bias the results of these trials. This bias can be minimized by assessment of the outcome of the procedure by independent investigators.

Furthermore, through overly restrictive exclusion criteria, the group of patients eligible for participation in a randomized controlled trial may comprise a misrepresentation of the typical patient population [27]. This can lead to a lack of generalizability, i.e. less external validity of the results. Although nonrandomized studies have a less selective recruitment process, their results are influenced through bias in the selection of treatment options by the preferences of the participating physicians. In addition, there is still a large inexpertness in surgeons about clinical epidemiology and designing and performing a randomized controlled trial [28, 29].

Finally, the financial support for surgical clinical research is limited and there seems to be a lack of interest in funding of surgical trials [15]. Commercial funding for comparison of different surgical techniques is often not available, whereas quite all medical trials are supported by pharmaceutical companies [30, 31]. Therefore, the access to institutional financial resources should be simplified for clinical surgical research, and the lack of interest in funding surgical randomized trials should be battled in the near future.

Implementation of Results from Surgical Randomized Controlled Trials into Day-to-Day Surgical Decision Making

After improving the quality and quantity of randomized controlled trials in surgery, the next logical step must be a feasible transfer and the best possible implementation of the results into the day-to-day surgical decision making. In times of innumerable published papers, even in special fields of medicine every year, it is important for a physician to stay informed about the most recent developments in clinical medicine. The best way to assimilate all findings of evidence-based trials about a specific intervention is a systematic review [32]. The leading institution performing systematic reviews of published randomized controlled trials is the Cochrane Collaboration [33]. This international organization has the goal to improve the quality of health care by ‘preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of health care interventions’ (www.cochrane.org). By presenting these systematic reviews in the ‘Cochrane Database of Systematic Reviews’ in the Internet and quarterly in electronic format, there is a great accessibility of critical summaries of results of randomized controlled trials. Hence, the Cochrane Collaboration became the platform to find guidance for decision making in the day-to-day patient care based on the best evidence.

Conclusion

There is still no increase in well-designed clinical surgical studies that could form a rigorous evidence based on newly introduced procedures, and there are many established procedures supported only by nonexperimental evidence. To accurately assess the evidence for the clinical utility of these procedures, critical analysis is required [34]. Randomized controlled surgical trials are the essential cornerstone of this analysis.

However, randomized controlled trials may provide only low external validity secondly to several factors, including unrepresentative participating surgeons with particular interest in the topic, atypical settings and patients and the difficulty in applying trial results across surgical patient populations. Nevertheless, the randomized controlled trial to compare surgical procedures remains the best and only method creating solid evidence and is the most effective source to identify effects of therapeutic interventions on treatment outcomes. Despite the obvious difficulties in the realization of randomized controlled trials in surgery, there should be an increase in randomized controlled surgical studies to develop and preserve high-quality surgical care in the future.

The surgical world has to work against accusations such as the following from the editor of Lancet [35]: ‘Only when the quality of publications in the surgical literature has improved will surgeons reasonably be able to rebut the charge that as much as half of the research they undertake is misconceived.’

Hopefully, in the future, the following quote from Sackett [36], one of the leading forces of the EBM movement, will apply to the clinical and research aspects of surgical care: ‘As physicians, whether serving individual patients or populations, we always have sought to base our decisions and actions on the best possible evidence.’ The highest available level of evidence, namely the randomized controlled trial, should support in particular surgical treatment decisions because they are often definite and irreversible. We have created a new center at our institution that is dedicated to quantitative clinical research and will continue to conduct randomized controlled trials in surgery.


References

  1. Uhl W, Wente MN, Büchler MW: Chirurgisch-klinische Studien in der praktischen Durchführung. Chirurg 2000;71:615–625.
  2. Sackett DL, Straus SC, Richardson WS, Rosenberg WM, Haynes RB: Evidence-Based Medicine: How to Practice and Teach EBM, ed 2. London, Churchill Livingston, 2000.
  3. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF, Simel D, Stroup DF: Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA 1996;276:637–639.
  4. Moher D, Schulz KF, Altman DG: The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987–1991.
  5. Altman DG: Better reporting of randomised controlled trials: The CONSORT statement. BMJ 1996;313:570–571.
  6. Farthing MJ, Newcombe RG: Reporting of clinical trials in Gut: The CONSORT statement. Consolidated standards of reporting trials. Gut 1997;40:563–564.
  7. Liem MSL, van der Graaf Y, van Vroonhoven TJMV: CONSORT, randomized trials and the surgical scientific community. Br J Surg 1997;84:769–770.
  8. Moher D, Jones A, Lepage L: Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 2001;285:1992–1995.
  9. McLeod RS, Wright JG, Solomon MJ, Hu X, Walters BC, Lossing A: Randomized controlled trials in surgery: Issues and problems. Surgery 1996;119:483–486.
  10. Goligher JC, Pulvertaft CN, Watkinson G: Controlled trial of vagotomy and gastroenterostomy, vagotomy and antrectomy and subtotal gastrectomy in elective treatment of duodenal ulcer. Br Med J 1964;1:455–460.
  11. Ellis J, Mulligan I, Rowe J, Sackett DL: Inpatient general medicine is evidence based. Lancet 1995;346:407–410.
  12. Howes N, Chagla L, Thorpe M, McCullogh P: Surgical practice is evidence based. Br J Surg 1997;84:1220–1223.
  13. Lee JS, Urschel DM, Urschel JD: Is general thoracic surgical practice evidence based? Ann Thorac Surg 2000;70:429–431.
  14. Moher D: CONSORT: An evolving tool to help improve the quality of reports of randomized controlled trials. Consolidated standards of reporting trials. JAMA 1998;279:1489–1491.
  15. Solomon MJ, McLeod RS: Surgery and the randomised controlled trial: Past, present and future. Med J Aust 1998;169:380–383.
  16. Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA: An evaluation of internal-mammary-artery ligation by a double-blind technique. N Engl J Med 1959;260:1115–1118.
  17. Dimond EG, Kittle CF, Crockett JE: Evaluation of internal mammary ligation and sham operation for angina pectoris. Am J Cardiol 1960;5:483–486.
  18. Freeman TB, Vawter DE, Leaverton PE, Godbold JH, Hauser RA, Goetz CG, Olanow CW: Use of placebo surgery in controlled trials of a cellular-based therapy for Parkinson’s disease. N Engl J Med 1999;341:988–992.
  19. Horng S, Miller FG: Is placebo surgery unethical? N Engl J Med 2002;347:137–139.
  20. Macklin R: The ethical problems with sham surgery in clinical research. N Engl J Med 1999;341:992–996.
  21. Moseley JB, O’Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, Hollingsworth JC, Ashton CM, Wray NP: A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002;347:81–88.
  22. Tenery R, Rakatansky H, Riddick FA Jr, Goldrich MS, Morse LJ, O’Bannon JM 3rd, Ray P, Smalley S, Weiss M, Kao A, Morin K, Maixner A, Seiden S: Surgical ‘placebo’ controls. Ann Surg 2002;235:303–307.
  23. Johnson AG: Surgery as a placebo. Lancet North Am Ed 1994;344:1140–1142.
  24. Bonchek LI: Randomised trials of new procedures: Problems and pitfalls. Heart 1997;78:535–536.
  25. McLeod RS: Issues in surgical randomized controlled trials. World J Surg 1999;23:1210–1214.
  26. Majeed AW, Troy G, Nicholl JP, Smythe A, Reed MW, Stoddard CJ, Peacock J, Johnson AG: Randomised, prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. Lancet 1996;347:989–994.
  27. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C: Methods in health services research. Interpreting the evidence: Choosing between randomised and non-randomised studies. BMJ 1999;319:312–315.
  28. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D: Randomised trials in surgery: Problems and possible solutions. BMJ 2002;324:1448–1451.
  29. Solomon MJ, McLeod RS: Surgery and the randomised controlled trial: Past, present and future. Med J Aust 1998;169:380–383.
  30. Johnson AG, Dixon JM: Removing bias in surgical trials. BMJ 1997;314:916–917.
  31. Sauerland S, Lefering R, Neugebauer EA: The pros and cons of evidence-based surgery. Langenbecks Arch Surg 1999;384:423–431.
  32. Clarke M: What is so important about systematic reviews. Acta Psychiatr Scand 2000;102:239–240.
  33. Levin A: The Cochrane Collaboration. Ann Intern Med 2001;135:309–312.
  34. Maddern GJ: Surgery and evidence-based medicine. Med J Aust 1998;169:348–349.
  35. Horton R: Surgical research or comic opera: Questions, but few answers. Lancet 1996;347:984–985.
  36. Sackett DL, Rosenberg WM: The need for evidence-based medicine. J R Soc Med 1995;88:620–624.

Author Contacts

Waldemar Uhl, MD

Department of General, Visceral and Trauma Surgery, University of Heidelberg

Im Neuenheimer Feld 110, D–69120 Heidelberg (Germany)

Tel. +49 6221 56 6111, Fax +49 6221 56 5450

E-Mail waldemar_uhl@med.uni-heidelberg.de


Article / Publication Details

First-Page Preview
Abstract of Review

Published online: July 08, 2003
Issue release date: 2003

Number of Print Pages: 7
Number of Figures: 0
Number of Tables: 7

ISSN: 0253-4886 (Print)
eISSN: 1421-9883 (Online)

For additional information: http://www.karger.com/DSU


Copyright / Drug Dosage / Disclaimer

Copyright: All rights reserved. No part of this publication may be translated into other languages, reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, microcopying, or by any information storage and retrieval system, without permission in writing from the publisher or, in the case of photocopying, direct payment of a specified fee to the Copyright Clearance Center.
Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug.
Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.

References

  1. Uhl W, Wente MN, Büchler MW: Chirurgisch-klinische Studien in der praktischen Durchführung. Chirurg 2000;71:615–625.
  2. Sackett DL, Straus SC, Richardson WS, Rosenberg WM, Haynes RB: Evidence-Based Medicine: How to Practice and Teach EBM, ed 2. London, Churchill Livingston, 2000.
  3. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF, Simel D, Stroup DF: Improving the quality of reporting of randomized controlled trials: The CONSORT statement. JAMA 1996;276:637–639.
  4. Moher D, Schulz KF, Altman DG: The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;285:1987–1991.
  5. Altman DG: Better reporting of randomised controlled trials: The CONSORT statement. BMJ 1996;313:570–571.
  6. Farthing MJ, Newcombe RG: Reporting of clinical trials in Gut: The CONSORT statement. Consolidated standards of reporting trials. Gut 1997;40:563–564.
  7. Liem MSL, van der Graaf Y, van Vroonhoven TJMV: CONSORT, randomized trials and the surgical scientific community. Br J Surg 1997;84:769–770.
  8. Moher D, Jones A, Lepage L: Use of the CONSORT statement and quality of reports of randomized trials: A comparative before-and-after evaluation. JAMA 2001;285:1992–1995.
  9. McLeod RS, Wright JG, Solomon MJ, Hu X, Walters BC, Lossing A: Randomized controlled trials in surgery: Issues and problems. Surgery 1996;119:483–486.
  10. Goligher JC, Pulvertaft CN, Watkinson G: Controlled trial of vagotomy and gastroenterostomy, vagotomy and antrectomy and subtotal gastrectomy in elective treatment of duodenal ulcer. Br Med J 1964;1:455–460.
  11. Ellis J, Mulligan I, Rowe J, Sackett DL: Inpatient general medicine is evidence based. Lancet 1995;346:407–410.
  12. Howes N, Chagla L, Thorpe M, McCullogh P: Surgical practice is evidence based. Br J Surg 1997;84:1220–1223.
  13. Lee JS, Urschel DM, Urschel JD: Is general thoracic surgical practice evidence based? Ann Thorac Surg 2000;70:429–431.
  14. Moher D: CONSORT: An evolving tool to help improve the quality of reports of randomized controlled trials. Consolidated standards of reporting trials. JAMA 1998;279:1489–1491.
  15. Solomon MJ, McLeod RS: Surgery and the randomised controlled trial: Past, present and future. Med J Aust 1998;169:380–383.
  16. Cobb LA, Thomas GI, Dillard DH, Merendino KA, Bruce RA: An evaluation of internal-mammary-artery ligation by a double-blind technique. N Engl J Med 1959;260:1115–1118.
  17. Dimond EG, Kittle CF, Crockett JE: Evaluation of internal mammary ligation and sham operation for angina pectoris. Am J Cardiol 1960;5:483–486.
  18. Freeman TB, Vawter DE, Leaverton PE, Godbold JH, Hauser RA, Goetz CG, Olanow CW: Use of placebo surgery in controlled trials of a cellular-based therapy for Parkinson’s disease. N Engl J Med 1999;341:988–992.
  19. Horng S, Miller FG: Is placebo surgery unethical? N Engl J Med 2002;347:137–139.
  20. Macklin R: The ethical problems with sham surgery in clinical research. N Engl J Med 1999;341:992–996.
  21. Moseley JB, O’Malley K, Petersen NJ, Menke TJ, Brody BA, Kuykendall DH, Hollingsworth JC, Ashton CM, Wray NP: A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002;347:81–88.
  22. Tenery R, Rakatansky H, Riddick FA Jr, Goldrich MS, Morse LJ, O’Bannon JM 3rd, Ray P, Smalley S, Weiss M, Kao A, Morin K, Maixner A, Seiden S: Surgical ‘placebo’ controls. Ann Surg 2002;235:303–307.
  23. Johnson AG: Surgery as a placebo. Lancet North Am Ed 1994;344:1140–1142.
  24. Bonchek LI: Randomised trials of new procedures: Problems and pitfalls. Heart 1997;78:535–536.
  25. McLeod RS: Issues in surgical randomized controlled trials. World J Surg 1999;23:1210–1214.
  26. Majeed AW, Troy G, Nicholl JP, Smythe A, Reed MW, Stoddard CJ, Peacock J, Johnson AG: Randomised, prospective, single-blind comparison of laparoscopic versus small-incision cholecystectomy. Lancet 1996;347:989–994.
  27. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C: Methods in health services research. Interpreting the evidence: Choosing between randomised and non-randomised studies. BMJ 1999;319:312–315.
  28. McCulloch P, Taylor I, Sasako M, Lovett B, Griffin D: Randomised trials in surgery: Problems and possible solutions. BMJ 2002;324:1448–1451.
  29. Solomon MJ, McLeod RS: Surgery and the randomised controlled trial: Past, present and future. Med J Aust 1998;169:380–383.
  30. Johnson AG, Dixon JM: Removing bias in surgical trials. BMJ 1997;314:916–917.
  31. Sauerland S, Lefering R, Neugebauer EA: The pros and cons of evidence-based surgery. Langenbecks Arch Surg 1999;384:423–431.
  32. Clarke M: What is so important about systematic reviews. Acta Psychiatr Scand 2000;102:239–240.
  33. Levin A: The Cochrane Collaboration. Ann Intern Med 2001;135:309–312.
  34. Maddern GJ: Surgery and evidence-based medicine. Med J Aust 1998;169:348–349.
  35. Horton R: Surgical research or comic opera: Questions, but few answers. Lancet 1996;347:984–985.
  36. Sackett DL, Rosenberg WM: The need for evidence-based medicine. J R Soc Med 1995;88:620–624.