Rohit P. Ojha, DrPH, FACE

Published: the paper 'On the uses and abuses of Regression Models: a Call for Reform of Statistical Practice and Teaching' by John Carlin and Margarita Moreno-Betancur in the latest issue of Statistics in Medicine onlinelibrary.wiley.com/doi/10.1002/... (1/8)

Should Graphs of Risk or Rate Ratios be Plotted on a Log Scale? Should graphs of risk or rate ratios be plotted on a logarithmic scale? The conventional answer to this question seems to be yes (1), even to the extent th

Nice Simpsons reference, but symmetry is misleading. Consider this analogy: If you invest $100 in the stock market, you cannot lose more than 100% relative to that amount, but you can gain more than 100%. Additional reasons for leaving as ratios in link.

academic.oup.com/aje/article-...

Can’t imagine how random nonpositivity would be illustrated on a DAG. Could deterministic nonpositivity be illustrated similar to selection?

We must hold AI tools to the same standards as any other clinical intervention. Strong evidence builds trust and supports responsible adoption.

3/3

Benefits and harms associated with the use of AI-related algorithmic decision-making systems by healthcare professionals: a systematic review The current evidence on AI-related ADM systems provides limited insights into patient-relevant outcomes. Our findings underscore the essential need for rigorous evaluations of clinical benefits, reinf...

Too many AI interventions lack high-quality evidence. Most studies so far have high risk of bias and few report patient-relevant outcomes or potential harms.

www.thelancet.com/journals/lan...

2/3

Hope and hype are not substitutes for evidence.

Nevertheless, healthcare AI tools are often promoted without the rigorous evaluation expected of other healthcare interventions. Understanding the benefits and harms is essential before deploying these tools at scale.

#Healthcare #AI

1/3

"People Profit from being ambiguous about their research goals"

Julia concludes by highlighting the need for structural change. Rigorous causal research takes time and thought. That's not possible if we're still expecting PhD students to publish 3-5 papers.

You know Pearl's causal ladder, but Julia introduced a different type of ladder : how can we get cutting edge causal inference methods into applications? #EuroCIM2025 so important!

This 'dataset first' approach leads some scientists to conduct weak research because 'this is the best we can do in our data'.

If a dataset is inappropriate for a particular question, the best you can do is NOT use it.

It shouldn't be our job, as scientists, to be showcasing datasets.

NEW PAPER in the @bmj.com "PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or #artificialintelligence methods"

www.bmj.com/content/388/...

#StatsSky #MLSky #AI #MethodologyMatters

The Unbelievable Scale of AI’s Pirated-Books Problem Meta pirated millions of books to train its AI. Search through them here.

NEW: LibGen contains millions of pirated books and research papers, built over nearly two decades. From court documents, we know that Meta torrented a version of it to build its AI. Today, @theatlantic.com presents an analysis of the data set by @alexreisner.bsky.social. Search through it yourself:

Best wishes for a rapid and full recovery.

OpenAI plans to charge $20,000 per month for a PhD-level AI research agent.

To organizations considering this option from OpenAI: Why not hire a human PhD-level researcher instead?

To PhD-level researchers: Here is our fair market value according to OpenAI.

#EpiSky #StatsSky #AcademicSky

The meaning and use of the area under a receiver operating characteristic (ROC) curve. | Radiology A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient char...

A good starting point is the seminal article about this metric in a healthcare context: pubs.rsna.org/doi/10.1148/...

Additional nuances are discussed here: www.ahajournals.org/doi/10.1161/...

Interesting situation. Perhaps the journal or Editorial Board has a policy to help guide?

1/ When using observational data for #causalinference, emulating a target trial helps solve some problems... but not all problems.

In a new paper, we explain why and when the #TargetTrial framework is helpful.

www.acpjournals.org/doi/10.7326/...
Joint work with my colleagues @causalab.bsky.social

Just like the phrase, “…results should be interpreted cautiously.” As if results should ever be interpreted recklessly.

1/
If you were taught to test for proportional hazards, talk to your teacher.

The proportional hazards assumption is implausible in most #randomized and #observational studies because the hazard ratios aren't expected to be constant during the follow-up. So "testing" is futile.

But there is more 👇

NEW PAPER

Really glad to see this one in print: the harm due to class imbalance corrections in prediction models developed using ML/AI

Excellently led by @alcarriero.bsky.social

onlinelibrary.wiley.com/doi/epdf/10....

Electronic Health Record-Based Prediction Models to Inform Decisions about HIV Pre-exposure Prophylaxis: A Systematic Review Background Several clinical prediction models have been developed using electronic health records data to help inform decisions about HIV pre-exposure prophylaxis (PrEP) prescribing, but the character...

Our systematic review of EHR-based prediction models to identify patients who may benefit from HIV PrEP:

• 7 models identified
• 𝗔𝗹𝗹 𝗺𝗼𝗱𝗲𝗹𝘀 𝗵𝗮𝗱 𝗵𝗶𝗴𝗵 𝗿𝗶𝘀𝗸 𝗼𝗳 𝗯𝗶𝗮𝘀
• Most reports were missing critical information

Pre-print available here:
www.medrxiv.org/content/10.1...

#EpiSky

You aren't allowed to say “risk factor” People often say their goal is to identify “risk factors”. But what does that mean? Some people use the term to indicate potential causes of outcomes. Then just say cause. Others use it to identify predictors of outcomes. Then just say predict. And, sadly, too many others use it as shorthand for factors that are “statistically associated” with outcomes. In this case, say nothing at all, since this “goal” has no clinical utility whatsoever (beyond what it might suggest about causation or prediction). So once you have framed your research question as description, prediction, causation or measurement, there is no longer a need to talk about risk factors. It's basically just a catch-all phrase to cover up muddy thinking.

Please stop telling me about risk factors. 🙏😖

(ICYMI)

statsepi.substack.com/p/sorry-what...

I’m with you and advocate for further inquiry. I also agree that inference requires multiple sources, but the question is whether some studies are even useful for informing policy? Savitz wrote a nice article about the need for policy-relevant research.

academic.oup.com/aje/article/...

Sometimes the available studies are so flawed or do not address the question of interest well enough that meta-analysis is unwarranted and no amount of sensitivity analysis or post hoc remedies can redeem. In such cases, greater value to provide guidance about how to improve the quality of studies.

Standardization and other approaches to meta-analyze differences in means - PubMed Meta-analysts often use standardized mean differences (SMD) to combine mean effects from studies in which the dependent variable has been measured with different instruments or scales. In this tutoria...

A meta-analysis is only as good as the included studies. Most studies in this meta-analysis were riddled with selection, exposure and outcome misclassification, and confounding biases. In addition, the standardized mean difference is problematic for meta-analysis.

pubmed.ncbi.nlm.nih.gov/38761102/

So nice to see ideas statisticians established years ago about prediction models making a resurgence in other contexts.

#StatsSky

www.jclinepi.com/article/S089...

Eliminating Ambiguous Treatment Effects Using Estimands Abstract. Most reported treatment effects in medical research studies are ambiguously defined, which can lead to misinterpretation of study results. This i

What effect is being estimated? Clarifying this question can help decision-makers better use the evidence. Recent developments in defining estimands (i.e., the effect of interest) can help.

Here’s a nice summary to help think about estimands:

academic.oup.com/aje/article/...

#EpiSky #CausalSky

Agree. Such analyses are conditional on knowing when the outcome (mortality) occurred and have little practical value for informing practice change. We cannot intervene after the outcome already occurred.

LLMs may be good for certain tasks, but encoding causal knowledge is not one of them.

Many measures used in evaluating AI prediction models do not measure the intended parameter. AUC, calibration plot, and decision curve analysis are recommended for evaluation.

No problem. Another consideration is that two things that seemingly occur simultaneously may be related to a known or unknown common cause rather than a bidirectional effect. You’re thus searching for a common cause or some time-varying effect. DAGs can encode either and help clarify assumptions.