The Scientist magazine of October 30, 2000 (Vol. 14 [21]) published a comments article by Edzard Ernst (M.D., Ph.D., FRCP (Edin.), a chair of the department of complementary medicine, School of Postgraduate Medicine and Health Sciences, University of Exeter, UK) entitled: 'Are reviewers biased against unconventional therapies?'. The article discusses a study about bias occuring in medical science.

Here we present citations from the article and abstract of the original paper by Resch et al.:

"The peer-review process can be seen as a method of quality assessment, so my colleagues and I tested the hypothesis that this method is prone to bias. Specifically, our hypothesis was that journal reviewers would be more critical toward a manuscript relating to a study of an unconventional therapy as compared to a study of conventional treatment."

____________________________
To learn more about the study read paper abstract: K.I. Resch et al., Journal of the Royal Society of Medicine, 93:164-7, April 2000.
____________________________

"These results confirmed our initial hypothesis that a reviewer bias against unconventional treatments exists. The implications of this finding, if it proves to be generally applicable, are twofold.

First, this result implies that a prejudice exists against "alternative" interventions. It is conceivable, even likely, that such prejudice extends into other areas, such as research funding. If this is true, it would indicate that the playing field is not level.

Second, one might assume that the phenomenon observed in this study related to a much wider bias against ideas, concepts, or hypotheses that do not follow the trodden path of the mainstream thinking of the present day."

Authors believe that if being true the bias would amount to a serious limitation of today's practice of science, and that ways need to be found to overcome this limitation.

Paper abstract from PubMed database:

Resch K.I., Ernst E., Garrow J.
A randomized controlled study of reviewer bias against an unconventional therapy.
Journal of the Royal Society of Medicine, 93:164-7, April 2000.
Forschungsinstitut fur Balneologie und Kurortwissenschaft, Bad Elster, Germany.

Abstract. A study was designed to test the hypothesis that experts who review papers for publication are prejudiced against an unconventional form of therapy. Two versions were produced (A and B) of a 'short report' that related to treatments of obesity, identical except for the nature of the intervention. Version A related to an orthodox treatment, version B to an unconventional treatment. 398 reviewers were randomized to receive one or the other version for peer review. The primary outcomes were the reviewers' rating of 'importance' on a scale of 1-5 and their verdict regarding rejection or acceptance of the paper. Reviewers were unaware that they were taking part in a study. The overall response rate was 41.7%, and 141 assessment forms were suitable for statistical evaluation. After dichotomization of the rating scale, a significant difference in favour of the orthodox version with an odds ratio of 3.01 (95% confidence interval, 1.03 to 8.25), was found. This observation mirrored that of the visual analogue scale for which the respective medians and interquartile ranges were 67% (51% to 78.5%) for version A and 57% (29.7% to 72.6%) for version B. Reviewers showed a wide range of responses to both versions of the paper, with a significant bias in favour of the orthodox version. Authors of technically good unconventional papers may therefore be at a disadvantage in the peer review process. Yet the effect is probably too small to preclude publication of their work in peer-reviewed orthodox journals.