Brief preface: I had some misgivings about writing this article. It seems to come from a place of indignation, and I would much rather cultivate more positive thoughts and emotions. But the implications regarding the state of modern science, and yes, my indignation, were too much to sway me away from writing. In the end, it’s a story worth sharing, because the implications are worth our consideration, and it’s worth thinking what we might do to improve situations like this. I have done my best to present it in an impartial manner.
Impressively detailed study here, John. This topic is incredibly important for those of us who hold the scientific method in high regard. In the social sciences, the situation may be even worse, see Alvaro de Menard's piece, which I advise as a complimentary essay to your own.
One additional thought. You speak of disincentivizing the behavior -- the "stick". What about the "carrot"? There are reasons why self-described scientists are behaving in this way, deceiving themselves, their community, and the public. Inquiring and addressing these motivations would be important steps to take. This is far bigger than the scientific community, IMO. In the fields in which I have specific knowledge -- the arts -- motivations are so upside-down, that collaboration and teaching have become impossibly difficult.
If we do not address the role and attainment of social status in our culture, the hiring and tenuring mechanisms in academia, the wholesale replacement of the meaning of success across fields of expertise, and indeed, the roles of expertise and excellence in contemporary Western society, I fear the kind of changes we seek will not come to pass anytime soon.
Sep 11, 2023·edited Sep 11, 2023Liked by John P Sullivan
Great article John! On top of everything discussed here another major issue that significantly contributes to the current crisis in science (leaving aside whether the underlying premises/ethics etc are valid or not) is the failure to use the appropriate statistical methods (this would include not just tests but also randomization, blinding etc) as well as to consult a statistician when designing experiments, leading to erroneous results. Unfortunately, part of the issue is that some labs don't have the resources to run adequately controlled experiments with enough samples, replicates etc. Although many journals now do require more in this vein it is likely to take a while for this to sink in.
Sep 5, 2023·edited Sep 6, 2023Liked by John P Sullivan
Call me sad and cynical, but I long ago came to the conclusion that "science" is every bit as corrupt as politics, finance, religion, medicine, the law, education and the rest of academia. Of course, I realize - and celebrate - that there are many people of honor and integrity like you in all fields of endeavor; but, as we all know, it only takes one bad apple to spoil the bushel. For me, a layman with no scientific training, the peer review system is untrustworthy and of little or no value.
On the plus side, there is a naturopathic physician in the UK, Dr Sarah Myhill, who claims to have success in not only preventing but reversing dementia using B vitamins, fish oils and a low-carb diet:
Call me naive, but I trust her because her livelihood does not depend on tenure, toeing a line of narradigm, or on research grants. Her life would be far easier if she did follow the herd; but she doesn't.
A reader sent me a link to this article by a scientist who complains about leaving stuff out of his paper - sticking to "certain preapproved narratives" - so he could get published in Nature.
"To put it bluntly, climate science has become less about understanding the complexities of the world and more about serving as a kind of Cassandra, urgently warning the public about the dangers of climate change. However understandable this instinct may be, it distorts a great deal of climate science research, misinforms the public, and most importantly, makes practical solutions more difficult to achieve."
After I had finalized this essay, but before publishing, this story came out:
https://retractionwatch.com/2023/08/31/stanford-president-retracts-two-science-papers-following-investigation/
Marc Tessier-Lavigne just retracted two papers from Science. We looked at both of these papers in this essay.
Impressively detailed study here, John. This topic is incredibly important for those of us who hold the scientific method in high regard. In the social sciences, the situation may be even worse, see Alvaro de Menard's piece, which I advise as a complimentary essay to your own.
https://fantasticanachronism.com/2020/09/11/whats-wrong-with-social-science-and-how-to-fix-it/
One additional thought. You speak of disincentivizing the behavior -- the "stick". What about the "carrot"? There are reasons why self-described scientists are behaving in this way, deceiving themselves, their community, and the public. Inquiring and addressing these motivations would be important steps to take. This is far bigger than the scientific community, IMO. In the fields in which I have specific knowledge -- the arts -- motivations are so upside-down, that collaboration and teaching have become impossibly difficult.
If we do not address the role and attainment of social status in our culture, the hiring and tenuring mechanisms in academia, the wholesale replacement of the meaning of success across fields of expertise, and indeed, the roles of expertise and excellence in contemporary Western society, I fear the kind of changes we seek will not come to pass anytime soon.
Great article John! On top of everything discussed here another major issue that significantly contributes to the current crisis in science (leaving aside whether the underlying premises/ethics etc are valid or not) is the failure to use the appropriate statistical methods (this would include not just tests but also randomization, blinding etc) as well as to consult a statistician when designing experiments, leading to erroneous results. Unfortunately, part of the issue is that some labs don't have the resources to run adequately controlled experiments with enough samples, replicates etc. Although many journals now do require more in this vein it is likely to take a while for this to sink in.
Call me sad and cynical, but I long ago came to the conclusion that "science" is every bit as corrupt as politics, finance, religion, medicine, the law, education and the rest of academia. Of course, I realize - and celebrate - that there are many people of honor and integrity like you in all fields of endeavor; but, as we all know, it only takes one bad apple to spoil the bushel. For me, a layman with no scientific training, the peer review system is untrustworthy and of little or no value.
On the plus side, there is a naturopathic physician in the UK, Dr Sarah Myhill, who claims to have success in not only preventing but reversing dementia using B vitamins, fish oils and a low-carb diet:
https://drmyhill.co.uk/wiki/Dementia_-_a_symptom_with_many_causes_from_Alzheimers_to_poisonings
Call me naive, but I trust her because her livelihood does not depend on tenure, toeing a line of narradigm, or on research grants. Her life would be far easier if she did follow the herd; but she doesn't.
I'm honored and thrilled that Retraction Watch has included this essay in their weekly roundup:
https://retractionwatch.com/2023/09/09/weekend-reads-chatgpt-in-papers-a-russia-based-paper-mill-getting-scooped-becomes-an-opportunity/
❤🙏
A reader sent me a link to this article by a scientist who complains about leaving stuff out of his paper - sticking to "certain preapproved narratives" - so he could get published in Nature.
https://www.thefp.com/p/i-overhyped-climate-change-to-get-published
"To put it bluntly, climate science has become less about understanding the complexities of the world and more about serving as a kind of Cassandra, urgently warning the public about the dangers of climate change. However understandable this instinct may be, it distorts a great deal of climate science research, misinforms the public, and most importantly, makes practical solutions more difficult to achieve."
Definitely worth a read!
Thanks for the excellent write up and copious amount of sources, it's much appreciated.