On Academic Fraud and Scientific Stunts
I have to admit that I have a weight problem. I’m
not obese, but I could surely benefit from losing 10-20 kg (22-44 pounds). Occasionally
I try to diet, in the sense of eating more vegetables, less meat, as little as
possible fat, avoid sugar (particularly added sugar) etc. And, YES, if you want
to diet, you have to give up wonderful things such as bacon, chocolate, bagels
and deep-fried food.
Wouldn’t it be wonderful if a dieter could hold
on to one of these incredibly tasty ingredients?
Of course!
But, what if one such ingredient would actually
help to lose weight?
What if EATING FRIED BACON would help you lose
weight?
Who wouldn’t want to eat fried bacon and still
lose weight?
Without being a nutritionist (I just mentioned
I’m overweight), but being able to pass biology in middle-school, I know that
eating fat is not going to make anyone slimmer.
Things are similar when it comes to other (guilty)
pleasures of food. Eating chocolate, bagels, donuts, ice-cream and sweets etc.
is very pleasurable and there is a reason for it.
A few million years ago, when our evolutionary
ancestors (pre-homo sapiens species) lived, it was very good for them to eat as
much sugar and fat as they could. This is simply because sugar and fat were
extremely scarce back then. (Remember, we are talking about a few million years
ago).
Whether we like it or not, we have inherited
this natural appeal for sugar and fat, but in today’s (western) world these
ingredients are (way too) abundant. Therefore it is not in our own benefit to
eat as much of them as we can, even if our instincts tell us to.
But what if the facts above are wrong? What if,
eating donuts is good for our health? It would be great! We could indulge in sugar
and fat (read donuts) without guilt and even with a sense of pride since we are
doing something good for our health.
This would be something many of us would love
to be true. We want it to be true… and because of this, often we skip the scepticism
that is so often needed.
A journalist, John Bohannon, managed to fool
lots of people into believing (considering as true) that eating chocolate is
actually good for dieting. He devised a real experimental study in which this
result was statistically significant.
Unlike many dieting
gurus, John Bohannon didn’t do this study in order to sell his books,
workshops etc. and make a fortune by selling desirable illusions. He did it to
prove a very valuable point: we are not critical enough with the results that
are presented, particularly with the methodology of the study.
You can read his full explanation here (open in new tab and read later).
In very, very brief the methodology was flawed
and the over-emphasis on statistically significant results is a two-edge blade.
In a different part of the (scientific) world,
a new scandal on academic fraud (faking data) erupted. This time it concerned
political science and the main reason why this scandal made such a big bang is that the study published
(using fake data) became very popular.
One of the reasons for which it became so
popular was that the study was published in Science. For the people who don’t
know what Science is: for scientists it is similar to the Hollywood walk of
fame for actors and the Rock and Roll hall of fame for musicians.
Another reason for which the paper became
popular was the fact that Donald Green (a super-star in political science
research) was one of the co-authors. Mr. Green claims that he was not involved
in the actual running of the study and that the other co-author (a grad student)
brought the data.
However, the main reason (in my view) for which
the study became so popular is that it told us what we wanted to hear about a
rather sensitive topic: gay marriage.
The study claimed that attitudes towards gay
marriage can change (from negative to positive) after a 30 minutes visit from a
canvasser who disclosed that he or she is homosexual.
You can read a detailed description of the
fraud and how it was discovered here
The fact that occasionally some people fake
data should not surprise anyone. Academia is not immune to cheating and fraud.
What I find very interesting is that this case
has some similarities with the chocolate helps in dieting paper. Obviously both
are methodologically utterly flawed. But there is more than just poor
methodology.
Telling people that attitudes towards a
sensitive topic such as gay marriage can be changed by a 30 minutes interaction
with a gay person is very similar to saying that eating chocolate (or bacon) is
actually helpful in dieting.
Both are things many of us want to hear and
both are a bit too good to be true (once you become critical).
The beauty of behavioural science and other branches
of science is that they prove our intuitions wrong, that they show asymmetries (i.e.
small input, large output) and that they promise actionable means for practical
applications.
There is, however, a difference between
counterintuitive and too good to be true.
Whereas counterintuitive things go against our
initial beliefs, things that are too good to be true confirm what we want to
believe.
Both chocolate helping dieting and changing
attitudes towards a sensitive topic after a 30 minutes interaction are in
accordance with our desired utopias. Many of us would love to eat chocolate and
lose weight; nonetheless this seems utopic. Similarly, many pro-equality supporters would love to
have found a way to change conservative views on gay marriage in just 30
minutes; again this seems utopic.
If it Seems Too Good to Be True and You’d Love
to Believe it, it Probably isn’t True