Fraud alert: “There is a paper that shows…”
Load your revolver whenever you hear the phrase "there is a paper that shows...". (2 minute read).
“There is a paper that shows…”, “A new paper claims…”, A scientific study proves…”, “There is a recent paper that…”.
These are all phrases I hear again and again, in different versions and contexts. Whenever I do, I get suspicious.
The main problem is that there is almost always a paper that claims what we want it to claim. Every year, over 5 million papers are published in more than 30 000 journals. It is inevitable that a lot of bullshit sneaks into this flood. Even if we are dead wrong, and there are hundreds of papers that contradict our ideas, we can be pretty sure to find one that supports us.
People often lie when they claim the existence of a paper in this way. First to themselves, and then to others. The temptation to win arguments is strong. Scientific papers usually have very boring names, so they are hard to remember, and often they are so difficult that people readily forgive us for not reading them. This creates confusion, and that confusion makes it easy for us to believe what we want to believe. Besides – even if someone tells a barefaced lie and invents a paper that supports them, they are often entirely right by accident.
But it is still a bad sign if there is only one paper that shows something. It either means that the question is almost unexplored, or more likely, that most papers point in another direction.
In the words of Rory Sutherland: “The more data you have, the easier it is to find support for some spurious, self-serving narrative. The profusion of data in the future will not settle arguments. It will make them worse.”[i]
So the next time you hear something like “a recent paper shows that I am right!” – be careful. All too often, someone is abusing the authority of science like a club.
Jakob Sjölander
[i] Rory Sutherland. Alchemy: The Surprising Power of Ideas That Don’t Make Sense. WH Allen. (2019). Chapter 1.18.