Greg Atkinson

The Optimal Split for an A/B Test Is 50:50

Unless...

I recently saw a post explaining why the optimal split between treatment and control groups should be 50:50.

The optimal split is indeed 50:50, but only under one assumption:

1/

#CausalSky #StatSky #ABTest #EpiSky #MLSky #EconSky

This recent RCT of an "AI stethoscope" claims the technology "shows promise" for diagnosing cardiovascular conditions.

It does not.

It is a textbook example of the risks of conducting unprincipled 'per protocol analyses'. Once again, peer review at a major medical journal has failed.

🧵 1/

Ratios are misleading exposure variables that compromise a regression model unless fundamental scaling assumptions are satisfied: a comment on Valente et al - Journal of Clinical Epidemiology www.jclinepi.com/article/S089...

a close up of a person 's hand holding a gun in a dark room . ALT: a close up of a person 's hand holding a gun in a dark room .

Most university strategies seem to boil down to someone shouting "more"

Assessment of adverse effects attributed to statin therapy in product labels: a meta-analysis of double-blind randomised controlled trials Adverse event data from blinded randomised trials do not support causal relationships between statin therapy and most of the conditions (including cognitive impairment, depression, sleep disturbance, ...

This is a very cool study and, as far as I can tell from a quick read, robust.

Statins probably don't cause most of the side-effects that people think they do.

www.thelancet.com/journals/lan...

Die, Dichotomy We have studied 21 435 unique randomized controlled trials (RCTs) from the Cochrane Database of Systematic Reviews (CDSR).Of these trials, 7224 (34%) have a continuous (numerical) outcome and 14 211 (...

Die, Dichotomy.
www.linkedin.com/pulse/die-di...
A brief post encouraging you to read the original paper with @erik-van-zwet.bsky.social and @f2harrell.bsky.social .
onlinelibrary.wiley.com/doi/epdf/10....

We are delighted to recognise our Exceptional Referees for 2025 🎓 🏆

Visit the webpage below to see the list of referees celebrated for the volume, timeliness, and exceptional quality of their reports & exemplary service to the discipline!

🔗⤵️
buff.ly/H14u9Z1

The decline effect (Protzko & Schooler, 2017) is an observed phenomenon where effect sizes in experiments apparently diminish in size from the first paper demonstrating the effect to later replications. This has been taken as a symptom of an unhealthy scientific ecosystem, possibly caused by the "winner's curse" (selection on significance and regression to the mean), publication bias or opportunistic analyses. I show that decline effects can arise as an artifact from a much simpler source: the original article determining the sign of the effect in a meta-analysis. Moreover, such artifactual decline effects will show correlations with some of the same experimental properties that one would expect from biases from poor behavior, such as the sample size of the original study.

New draft: "Decline effects, statistical artifacts, and a meta-analytic paradox". In this manuscript I show how a common practice in meta-analysis (eg the 2015 Open Science Collaboration) creates artifactual signatures of poor scientific behavior. PDF: raw.githubusercontent.com/richarddmore... 1/x

How to Do Bad Biomarker Research – Statistical Thinking This article covers some of the bad statistical practices that have crept into biomarker research, including setting the bar too low for demonstrating that biomarker information is new, believing that...

Judging by the poor quality of biomarker research I see reported in biomedical journals, my article "How to Do Bad Biomarker Research" must have been hugely influential: www.fharrell.com/post/badb/in... #Statistics #StatsSky #EpiSky

S. Senn: Randomisation is not about balance, nor about homogeneity but about randomness (Guest Post) Stephen Senn Consultant Statistician Edinburgh The intellectual illness of clinical drug evaluation that I have discussed here can be cured, and it will be cured when we restore intellectual primac…

100 years ago RA Fisher set out his views on randomisation. This blog of mine errorstatistics.com/2020/04/20/s... from five years ago tries to explain them.

Check out this #EPicks video from James Betts (@uniofbath.bsky.social), as he chats about his article published in @expphysiol.bsky.social which focussed on physiological rhythms and metabolic regulation!

📽️ buff.ly/gpUtCi7

Read the full article here:
📜 buff.ly/rtm1OFd

Are you passionate about supporting the funding of high-quality methods research in health? If so, please consider applying for the role of Deputy Chair of the NIHR/MRC Better Methods Better Research funding Committee - closing date 22 Jan. See: www.nihr.ac.uk/get-involved...

“Assessing the properties of the prediction interval in random-effects meta-analysis” www.cambridge.org/core/journal...

For some of the replicate crossover trials in Lolli et al.(2025) we have repeated measurements in each period, e.g. to derive AUC glucose response, so we can explore this question maybe. We should chat about this sometime! Absolutely agree that replicate cross is best for transient treatment effects

Thanks very much Anne. We are also interested in whether repeated measurements in the follow-up period for a person x time interaction (you've done valuable work on this I think) is synonymous with the repeated treatment cycles in a replicate crossover for person x treatment effects interaction.

Our letter to editor about Caldwell et al (J Appl Physiol 139: 1220–1227, 2025) is published: journals.physiology.org/doi/full/10....

There seemed misconceptions about variance comparison stats. We also commented on the author's “SD of treatment effects”. Suppl stuff: zenodo.org/records/1777...

This #Editorial from Damian Bailey (University of South Wales), David Poole (@kstate.bsky.social) and Ronan Berg (Rigshospitalet) provides an update on the journal from the perspective of our Editor-in-Chief and Deputies for the USA and Europe!

🎓 buff.ly/ogE6Nol

Reflections on the I-squared index for measuring inconsistency in meta-analysis. Julian P. T. Higgins, José A. López-López. Research Synthesis Methods. bit.ly/48Xsjce

Figure 1: The Rothman-Dahly Evidence Pyramid (original version) An equilateral triangle with a small blue section labelled "Thoughtful, well-conducted studies of any design" at the top, with the remaining space colored red and labelled "The other shit"

‪It has a name now 😜

Many thanks to Ken for agreeing to put his good name to my...artwork. The image is in the public domain (CC 0), but citations to the linked documents are warmly welcomed.

✅ zenodo.org/records/1808...

✅ pubmed.ncbi.nlm.nih.gov/24452418/

Professor Dawid's 2000 paper with a similar title is also a very interesting read: www.tandfonline.com/doi/abs/10.1....

The best bit about the show Tipping Point is when the host, Ben Shephard, says "let's find out what would have happened if you'd decided to play". There's always a nagging doubt at the back of my mind: that's not really what *would* have happened, is it?

COVID Inquiry (slight return) The evidence before the court is incontrovertible

Nice piece on the COVID inquiry by Oliver Johnson
open.substack.com/pub/bristoli...

link 📈🤖
Personalised Decision-Making without Counterfactuals (Dawid, Senn) This article is a response to recent proposals by Pearl and others for a new approach to personalised treatment decisions, in contrast to the traditional one based on statistical decision theory. We argue that this approac

Doug Altman was an internationally renowned statistician who served as The BMJ’s chief statistical adviser.

Read about life and work that made this statistician a "citation millionaire"
#BMJChristmas
www.bmj.com/content/391/...

Check out this #EPicks video from James Betts (@uniofbath.bsky.social), as he chats about his article published in @expphysiol.bsky.social, which focussed on physiological rhythms and metabolic regulation!

📽️ buff.ly/gpUtCi7

Read the full article here:
📜 buff.ly/rtm1OFd

Do not try to reproduce p-values from rounded summary statistics. You can (should) check whether they are consistent - you can achieve this by calculating the largest/smallest p-value consistent with the rounded data. Mark Bolland’s app may help: reappraised.shinyapps.io/check_p_vals...

Mathematical Thinking Isn’t What You Think It Is | Quanta Magazine The mathematician David Bessis claims that everyone is capable of, and can benefit greatly from, mathematical thinking.

The mathematician David Bessis believes that mathematical skill is not innate, but learned. “Genius is not an essence. It’s a state. It’s a state that you build by doing a certain job.”

🚨CALL FOR PAPERS CLOSING SOON🚨
Our 'New approaches for old diseases' call for papers will be closing in THREE DAYS!
Follow the link below for more info on this special issue, and how to submit your article.

🔗 buff.ly/ECNxMp2

Using Monty Python’s silly walks to tackle the obesity pandemic. physoc.onlinelibrary.wiley.com/doi/10.1113/...