The Right to be Wrong
Kirjoittanut
Howy JacobsSetting aside the growing threat to the scientific literature from outright fraud, wilful misinterpretation, undeclared conflicts of interest, low-quality peer review and malign pseudo- science, it is an inevitable fact that much of what is put into print today will turn out to be erroneous when revisited in the future. At the very least, it is likely to have been superseded by later findings, even if its primary message was a reasonable conclusion to have drawn at the time. Yet there is a great reluctance to recognize that what is out there might be wrong. Students often have a hard time grasping the fact that a paper recommended by a highly respected professor might be based on a mistaken premise or faulty methodology. We tend not to question the veracity of a manuscript that has passed the rigours of a ‘proper’ peer-review. Rather we accord it respect, as an addition to the ever-growing corpus of human knowledge of which we can collectively be proud. And so many scientists cling to the notion that their own papers are unimpeachable, and that anyone who reaches divergent conclusions are the ones who must be wrong.
However, if we randomly select a dozen or so papers from our fields, published in academically respectable journals say 40 years ago, it is quite likely that the majority of them will contain important errors of fact, unjustified assumptions or conclusions, speculations that led nowhere, or results obtained using flawed or inappropriate methods. Occasionally we also find the opposite: a finding that was forgotten about for decades and then (re)discovered later to great fanfare, despite the original report continuing to be totally ignored. When conducting this exercise for myself, I was also struck by the number of interesting observations that have, to my knowledge, never been followed up by applying the much more advanced technologies at our disposal today.
One reason why papers published 40 years ago might actually end up being more reliable than those published today, especially in the ‘top’ journals, is the influence of peer-review. Editors try to accommodate the views of each peer-reviewer of a manuscript, which often pushes the authors to conduct further experiments to confirm the possibly eccentric view of a reviewer, or include the reviewer’s alternate explanation for the findings in the paper even if they disagree with them. As a result, many of today’s papers lack coherence, follow a meandering course that obscures the main finding or message, or emphasize conclusions peripheral to the main point that aren’t robustly supported by the data.
On balance, we need to be less credulous about the literature, including our own work, and train our students, even at undergrad level, to be more critical of what they read. Not necessarily to disbelieve the documented results of experiments, but to think through whether the finding really justifies the conclusions drawn, what alternative interpretations are conceivable, and what is needed to substantiate the results or follow them up – especially if they contradict previous assumptions.
In regard to our own work, most of us need to exercise due humility. We may not strictly have been ‘wrong’, but did we actually do the right experiment in the first place? Are the statistical tests that we applied the only valid ones (regardless of what the peer-reviewers did or didn’t notice)? Did we overlook something that should have been followed up? Did we quote the work of others accurately and fairly? Did the editor or peer-reviewers force us to say something we don’t really believe?
We currently have no academically accepted mechanism for us publicly to comment on our own past work, highlight wrong turnings we may have taken and which could be instructive for others to beware of or, conversely, point out those insights that still need to be followed up, 4 or 40 years later.
Above all, we absolutely need to assert and vaunt the right to be wrong, which is an essential condition for scientific progress.
