Winston Lin

Gentle reminder that a correlation coefficient isn’t a particularly great way to quantify the effect of a dichotomous treatment. See also

www.the100.ci/2025/07/28/w...

Interpreting p values and interval estimates based on practical relevance: guidance for the sports medicine clinician Statistical methods are employed in medical research to estimate effects of treatments or health conditions across populations.1 2 This paper presents a framework to avoid common misinterpretations th...

Excellent new editorial and guideline on interpreting p values and interval estimates
bjsm.bmj.com/content/earl...

from Amrhein, Greenland, & McShane ("Retire statistical significance," Nature, 2019) "For example, the authors above could have written: ‘Like a previous study, our results suggest a 20% increase in risk of new-onset atrial fibrillation in patients given the anti-inflammatory drugs. Nonetheless, a risk difference ranging from a 3% decrease, a small negative association, to a 48% increase, a substantial positive association, is also reasonably compatible with our data, given our assumptions.’ "

from Amrhein, Greenland, & McShane ("Retire statistical significance," Nature, 2019) "Whatever the statistics show, it is fine to suggest reasons for your results, but discuss a range of potential explanations, not just favoured ones. Inferences should be scientific, and that goes far beyond the merely statistical. Factors such as background evidence, study design, data quality and understanding of underlying mechanisms are often more important than statistical measures such as P values or intervals."

I like this from @vamrhein.bsky.social et al. I assigned it to my class last semester and tried to explain that p-values measure how compatible (vs. surprising) the data are with the null, given our assumptions. But yeah, tests & CIs are hard to understand!

www.blakemcshane.com/Papers/natur...

Orley Ashenfelter's papers often have good introductions. Here's Ashenfelter & Plant

www.journals.uchicago.edu/doi/abs/10.1...

Koenker & Geling is one of my favorites

www.jstor.org/stable/2670284

The Only Three Reasons to Do a Ph.D. in the Social Sciences If none are true, don’t do it.

Are you or one of your students considering doing a Ph.D. in a social science? I've spent a lot of time talking about this w/ students & finally wrote something up.

IMO, there are only 3 good reasons to do it. One of them needs to be true--otherwise, don't.

medium.com/the-quantast...

How to Write an Effective Referee Report and Improve the Scientific Review Process (Winter 2017) - The review process for academic journals in economics has grown vastly more extensive over time. Journals demand more revisions, and papers have become bloated with numerous robustness...

This is written for economists, but I think it’s very helpful more generally

www.aeaweb.org/articles?id=...

How hazard ratios can mislead and why it matters in practice - European Journal of Epidemiology Hazard ratios are routinely reported as effect measures in clinical trials and observational studies. However, many methodological works have raised concerns about the interpretation of hazard ratios ...

A nice recent article on why you should abandon hazard ratios.

#statssky #episky

See our No-Spin report on a widely-covered NBER study of Medicaid expansion. In brief: Despite the abstract's claims that expansion reduced adult mortality 2.5%, the study found much smaller effects that fell short of statistical significance in its main preregistered analysis.🧵

Starting to look like I might not be able to work at Harvard anymore due to recent funding cuts. If you know of any open statistical consulting positions that support remote work or are NYC-based, please reach out! 😅

In case this is of interest, even ANCOVA I is consistent and asymptotically normal in completely randomized experiments (though II is asymptotically more efficient in imbalanced or multiarm designs)

Issues with interpreting p-values haunts even AI, which is prone to same biases as human researchers. ChatGPT, Gemini & Claude all fall prey to "dichotomania" - treating p=0.049 & p=0.051 as categorically different, and paying too much attention to significance. www.cambridge.org/core/journal...

CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials Critical appraisal of the quality of randomised trials is possible only if their design, conduct, analysis, and results are completely and accurately reported. Without transparent reporting of the met...

NEW: CONSORT 2025 now published!

Some notable changes:
-items on analysis populations, missing data methods, and sensitivity analyses
-reporting of non-adherence and concomitant care
-reporting of changes to any study methods, not just outcomes
-and lots of other things

www.bmj.com/content/389/...

How To Write a Response to Reviewers

How to write a response to reviewers. www.sciencedirect.com/science/arti...

Why Econometrics is Confusing Part 1: The Error Term | econometrics.blog “Suppose that \(Y = \alpha + \beta X + U\).” A sentence like this is bound to come up dozens of times in an introductory econometrics course, but if I had my way it would be stamped out completely.

Very nice explainer by @economictricks.bsky.social

www.econometrics.blog/post/why-eco...

today we will all read imbens 2021 on statistical significance and p values, which is a strong contender for having the best opening paragraph of any stats paper

pubs.aeaweb.org/doi/pdf/10.1...

Email to Stata explaining that Welch (1949) "welched" on Welch (1947), but refraining from asking if their co-founder Finis Welch was related to B. L. Welch. :) Continued in next screenshot

Second half of email, suggesting that the "unequal" option be described as using "the formula of Satterthwaite (1946) and Welch (1949)" (because what many people call the Welch test is what Stata's "unequal" option does, not what their "welch" option does)

Btw here's an email I sent Stata in 2012, suggesting a clarification to their descriptions of the "unequal" and "welch" ttest options. Got a polite reply but I don't think they changed it :)

Imbens & Kolesar give a nice overview and simulations

www.jstor.org/stable/24917...

Here's some older, related stuff (from me) aimed at political scientists.

Related paper #1

"Arguing for a Negligible Effect"

Journal: onlinelibrary.wiley....

PDF: www.carlislerainey.c...

"The Need for Equivalence Testing in Economics"

from Jack Fitzgerald (@jackfitzgerald.bsky.social)

Preprint: osf.io/preprints/met...

We know that "not significant" does not imply evidence for "no effect," but I still see papers make this leap.

Good to see more work making this point forcefully!

www.researchgate.net/publication/...

Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends (September 2022) - This paper discusses two important limitations of the common practice of testing for preexisting differences in trends ("pre-trends") when using difference-in-differences and relate...

Isn’t the Hausman approach likely to lead to undercoverage, similar to what @jondr44.bsky.social wrote about in a different context?

www.aeaweb.org/articles?id=...

Robustness of ordinary least squares in randomized clinical trials There has been a series of occasional papers in this journal about semiparametric methods for robust covariate control in the analysis of clinical trials. These methods are fairly easy to apply on cu....

For RCTs, another reference with simulation evidence on the robustness of OLS is Judkins & Porter (2016). But average marginal effects from logit are also robust

doi.org/10.1002/sim....

In the '90s when I worked at Abt and MDRC, I wrote an email that initially had the subject header "OLS without apology". I shared a later version with Freedman, who cited it as "Lim (1999)" in his "Randomization does not justify logistic regression"

Clarification: my paper doesn’t advocate a specific estimator. That’s one of the meanings of “agnostic” in the title :)

A warning and a plea - As fields start to use more advanced quantitative / "cause" methods, there is a desire to help consumers of the research (four journal reviewers, editor) to easily assess the study quality and validity (e.g. JAMA causality language) ... - We need to push against this - need ways to help people understand and assess the (inherently untestable) assumptions in many studies.

An important plea from @lizstuart.bsky.social in today's SCI-OCIS Special Webinar Series:

I don't wanna put words in Rosenbaum's mouth ("spectrum" is just a word that came to my mind for a quick Bluesky reply) and I'd really encourage anyone interested to read his papers and the Stat Sci discussion in full, and then critique them. :) But here's a screenshot from his reply to Manski

competing theories. He has an interesting debate with Manski on external validity in the comments on the Stat Sci paper (I'll send you some excellent responses that my undergrad students at Yale wrote).

can lead to badly misleading literatures; (3) we can sometimes learn from collections of studies with different designs & weaknesses (I think he's partly influenced by the literature on smoking & lung cancer, which he cites elsewhere); (4) we should try to falsify or corroborate predictions of 2/

Thanks, Alex! I think that's a small part of his message. It's hard for me to do justice to these papers in a short thread, but I think he's also saying (1) credibility is on a spectrum and we should try to learn from all sorts of designs; (2) repeating the same design with the same weaknesses 1/