Red Team of Science

PCI Webinar series #13 - Simine Vazire - Recognizing and responding to a replication crisis

In case you have missed Simine Vazire's excellent webinar yesterday, here is the link to watch it online: youtu.be/_vb1CNwC3CM Thanks again @simine.com for staying up so late and thanks to the audience for the great questions!

Congratulations to @simine.com for winning the Einstein Foundation Individual Award! 🎉

A well-deserved recognition for her seminal efforts to improve scientific rigor, which includes instituting detailed checks for errors and computational reproducibility at Psychological Science.

Psychologists think there's a crisis in psychology Results from a new poll

New data: Most psychologists believe crisis persists despite significant progress. Yet evidence suggests there's still an advantage to bending the rules. #metasci

www.redteamofscience.com/p/psychologi...

Among the references:

Bik: The dark side of post-publication peer review.

Ioannidis: A new era of hyper-skepticism in science.

A former COPE council member apparently thought these were 1. not suspicious and 2. didn't click the links.

I wrote something about publication bias at statmodeling.stat.columbia.edu/2025/11/14/t...

Cognitive dissonance and a truly impossible assumption The many unopened boxes of psychology

An outsider opened a box and broke the intentionality taboo in metascience. Revision is long overdue.

www.redteamofscience.com/p/cognitive-...

"The project also stands out for its innovative approach, which is pioneering in cosmology. RedH0T introduces the red-teaming method, inspired by cybersecurity."

Physics defends itself There's not as much math as you'd expect

I appreciate you all! I hope I have given credit where it's due for some good points. Follow up is here: www.redteamofscience.com/p/physics-de...

Physics defends itself There's not as much math as you'd expect

Response here: www.redteamofscience.com/p/physics-de...

Physics defends itself There's not as much math as you'd expect

Counting hypotheses changed GWAS. Why not physics? The role of... insults.
#metasci
www.redteamofscience.com/p/physics-de...

New blogpost on @elife.bsky.social's new Replication study type deevybee.blogspot.com/2025/10/prob.... tldr = I think it's doomed.
#Replication #Publishing #Reproducibility

I think we're arguing about the reputation of physics. On that, the small amount of string theory is a good argument.

1 in 5 chemists have deliberately added errors into their papers during peer review, study finds Conclusion is one of many in a report about how chemists handle errors in manuscripts

More than 20% of chemistry researchers have deliberately added information they believe to be incorrect into their manuscripts during the peer review process, in order to get their papers published. cen.acs.org/policy/publi... #chemsky 🧪

Why do you say that?

If you are new to it, base rate neglect is like saying that you have a new hypothesis that you're sure is true for the combination to a padlock, and then another and another. Observers are free to count the combinations and critique.

Base rate neglect is one of the most serious issues in science. It applies to physics. This post is a drop in the ocean on the topic.

I don't mind this at all. I think you're right. I think physics is at least interesting for what it can still get wrong.

Proposing theories is of course costly if there are googols of theories to propose. The other argument would be that the low base rate theories drown out the rest. I would call any literature with a tiny number of likely true results a crisis.

CC #metasci

Physics has all the ingredients for scientific crisis It turns out all you need is money and a googol of hypotheses to write up

Physics is the bedrock of science and may be the site of its worst case of base rate neglect.

www.redteamofscience.com/p/physics-ha...

Kazimier Smith, Yucheng Lu, Qiaochu Fan: From Funding to Findings (FIND): An Open Database of NSF Awards and Research Outputs https://arxiv.org/abs/2510.10336 https://arxiv.org/pdf/2510.10336 https://arxiv.org/html/2510.10336

Is Science Retracting Enough Papers? As paper mills and fraud proliferate, experts warn the retraction rate should reach 2% of published literature—ten times current levels.

Dr. Oransky warned of consequences for self-governance failure: "If you don't self-police, if you don't correct the record, if you don't sanction people who commit fraud, [...] someone else will come in and do that. And it very well could be a government that you don't like."

If science had star ratings like Goodreads, what paper in your field would be a 5/5 - and why?

Protecting the integrity of university rankings Addressing rare cases of attempted manipulation is built into the DNA of THE’s rankings. But we will go further, says Phil Baty

This is how far scientific critique lags behind science. We're just learning subtraction.

www.timeshighereducation.com/world-univer...

I’m deeply honored to have been named as the recipient of the Victor Cohn Prize for Excellence in Medical Science Reporting from the Council for the Advancement of Science Writing. It recognizes my work on scientific integrity for Science Magazine over the last five years. 1/7 tinyurl.com/zz54fzyr

YouTube video by Sabine Hossenfelder I can't believe this really happened.

There more I think about this, the more disturbing it becomes

www.youtube.com/watch?v=ZO5u...

How extreme is this limitless advocacy? Enough that this author has gotten away with saying that the "rabbit hole" of Hitler's big lie or Orwellian dystopia is a solution to the paradox they would not "prefer."

The way to increase trust in science is to stop making rules to benefit scientists.

Making self-serving rules is so universal in science that the author calls transparency a paradox because it doesn't always help your reputation.

The replication crisis started in much the same way, with a relatively normal paper. www.redteamofscience.com/p/tylenol-an...

Tylenol and autism. Why normal papers make history: