Christopher Boyer

#statsky If I’m writing a protocol for 1) developing a prediction model and 2) an RCT to test implementation of said model within a health system would you publish 1 and 2 together or separately or neither?

Yes just got it!

Thank you!

Prospective benchmarking of an observational analysis in the SWEDEHEART registry against the REDUCE-AMI randomized trial Prospective benchmarking of an observational analysis against a randomized trial increases confidence in the benchmarking process as it relies exclusively on aligning the protocol of the trial and the...

Also frontier seems to be concurrent prospective trial plus observational study (although obviously this has been around forever eg WHI) pmc.ncbi.nlm.nih.gov/articles/PMC...

Not to say that both use cases don’t still have their issues of course!

Increasingly think that some of the best uses of TTE are complimentary, e.g. 1) postmarket in a space where you have premarket RCT that has helped set bounds of reasonable effects and maybe identified negative controls, or 2) in an emerging area to help argue for potential equipoise for an RCT

Depends on whether you’re interpreting it as SATE or PATE. If former then your sample has no power users and repeated randomizations using same procedure over fixed sample produces no bias. Now if this error raises suspicion of other randomization errors then maybe not?

Time Sensitive: Public comment on Schedule F (removing civil service protections for many federal employees INCLUDING THOSE WITH AUTHORITY OVER GRANTS) ends tomorrow. Notice is here: www.govinfo.gov/content/pkg/... . Comment here (green button) www.federalregister.gov/documents/20... .

I think most of those “sames” in the emulation section of the protocol table in TTE paper could be replaced with “look it’s not actually the same but we at least tried to think about this aspect of the trial we are emulating instead of ignoring it. Be glad we didn’t just do Cox PH and call it a day”

Thanks! Will read. I’m interested in estimands at group and individual levels (possibly under peer effects) and the resulting analytic choices.

Yeah it comes up quite frequently across a range of public health programs that are delivered in groups (when they aren’t pre-existing). To be clear, what I described where only intervention are grouped, is partially-nested?

Thanks I hadn’t heard of nesting! I had been using RCT with random group formation (rolls off the tongue) based on previous discussion.

Follow up: what is the estimand being targeted here?

You have a trial with following characteristics: participants are recruited and randomly assigned 1:1 to an intervention delivered in groups or control. Those assigned to intervention are further randomized to the group they are to receive intervention in.

QUESTION: is this a “cluster” RCT? #stats

Identification and estimation of vaccine effectiveness in the test-negative design under equi-confounding The test-negative design (TND) is frequently used to evaluate vaccine effectiveness in real-world settings. In a TND study, individuals with similar symptoms who seek care are tested for the disease o...

New preprint with Kendrick Li, Xu Shi, and Eric Tchetgen Tchetgen! We revisit the test-negative design (TND) and propose alternative identifiability assumptions and estimators of vaccine effectiveness (VE) under “equip-confounding".
arxiv.org/abs/2504.20360

Identification and estimation of vaccine effectiveness in the test-negative design under equi-confounding The test-negative design (TND) is frequently used to evaluate vaccine effectiveness in real-world settings. In a TND study, individuals with similar symptoms who seek care are tested for the disease o...

New preprint with Kendrick Li, Xu Shi, and Eric Tchetgen Tchetgen! We revisit the test-negative design (TND) and propose alternative identifiability assumptions and estimators of vaccine effectiveness (VE) under “equip-confounding".
arxiv.org/abs/2504.20360

Estimating and evaluating counterfactual prediction models Counterfactual prediction methods are required when a model will be deployed in a setting where treatment policies differ from the setting where the model was developed, or when a model provides predi...

Also plug for our paper arxiv.org/abs/2308.13026

Ie just because it’s really hard shouldn’t mean we get to do the “easier” thing and then throw our hands up when people start to use it to address the harder question.

Agree but very slight pushback would be that people were often using clinical prediction when their research question was better articulated as a form of counterfactual prediction.

In silico trials and digital twins are emerging as transformative medical technologies as they offer a unique way to design medical innovations, optimize their application, and evaluate their utility. Their utility spans from individual care – appropriating the technology for personalized decision, to population care – presenting an alternative to design, supplement, or replace clinical trials. They effectually offer a new way to efficiently qualify, quantify, and personalize healthcare innovations in advance or in conjunction with their clinical application. While much progress is underway to advance these technologies across diverse developments, realizing their full potential requires a cohesive goal to unify separate activities towards a common objective. Such a cohesive goal – a moonshot – can be defined as forming and fostering a digital twin of every single human person, owned by the individual, progressively updated with new data, and used to deliver optimized care, technology assessment, and real-world evidence. This vision builds upon a growing body of work in computational modeling, regulatory science, and digital healthcare, underscoring its feasibility. Bringing this vision to reality requires ownership and active engagement of all stakeholders to contribute diverse expertise and resources for transforming medicine and medical appropriation towards a more accurate, efficient, and quantitative future.

These people are dangerous, and I mean that.

academic.oup.com/pnasnexus/ad...

Thesis: most published findings are false/waste and we need a movement to
fix this.

Antithesis: science reform is broken, falls prey to same tendency to overclaim, and is politically weaponized.

Synthesis: ????

I have confirmation from my (HI) State DOH that PRAMS is paused as of last Thurs. All new data collected after Jan 31 are rejected.

Which epis in other states, esp those without large MCH research presence, are interested in standing up alternatives?

Also an opportunity to educate on data! 📩 me!

I agree that there's a lot of bad science out there and am, er, somewhat sceptical of the quality control mechanisms we have to put the point mildly. But, like, point estimates of the amount of research waste are, to me, just on their face very implausible. I don't think I could possibly trust that?

Yeah it's a mess out there (here?)

But there is a reason I am working in the science reform movement (such as it is):

There are actual potential paths forward that build on what the science reform movement has made over the last decade. But no easy solutions.

Thesis: most published findings are false/waste and we need a movement to
fix this.

Antithesis: science reform is broken, falls prey to same tendency to overclaim, and is politically weaponized.

Synthesis: ????

Yes and therefore messaging about the overall economic impact mostly just reproduces existing partisan cleavages. But many rural areas are connected to a blue or purple metropole through (specialty) healthcare and to the extent that cuts reduce quality or increase cost maybe that moves the needle. 🤷‍♂️

brimr.org/brimr-rankin...

This table misses several of the large hospital systems that bring in big NIH dollars (like mass general who are usually in top 5-10)

I feel like there hasn’t been enough reporting on the fact that there’s effectively a 21st century patronage system now in which far more people are bound financially to the “success” of the dear leader than any 19th century political boss could have dreamed.

Very concerning news about the future of USAID from Sen Murphy below: