Rohit Satija

Target prioritization is to bio foundation models as next token prediction is to language models

Learn More –

Blog: rohitium.github.io/2025/08/13/L...

Code: github.com/rohitium/LLM...

In my latest blog post, I propose an order parameter, inspired by the statistical physicist’s favorite toy model, the Ising model, to analyze and interpret thermodynamics of a GPT trained on a corpus of Shakespeare’s writing.

For several reasons, I believe we stand to gain a deep understanding of LLMs if we view them as physical, rather than biological, systems. In particular, a branch of physics called statistical mechanics happens to have a considerable amount of overlap with the equations underlying these models.

Anthropic had recently published an illuminating study called “On the Biology of a Large Language Model” that used circuit tracing, inspired by concepts from neuroscience, to investigate the flow of signals in LLMs during inference.

As AI becomes embedded in our everyday work, it has become increasingly relevant to understand how LLMs learn natural language – to ensure their outputs are safe/aligned, to develop more efficient architectures, and to probe the origins of human intelligence.

Let’s train a simple Generative Pre-trained Transformer (GPT) model and analyze its training thermodynamics using tools from statistical physics 🧵

Photo of Newton Cho

🎉 Congratulations to Newton Cho, the 2025 winner of the Science & PINS Prize for Neuromodulation for his work identifying a novel therapeutic target to improve walking after spinal cord injury.

Learn more: scim.ag/44WlGF0

Learn More –

Blog: rohitium.github.io/2025/07/28/S...
Code: github.com/rohitium/sar...
App: rohitium.github.io/sarepta-fda-...

In my latest blog post, I argue for an evidence-first approach to drug development and biotech capital investments. When patients’ lives are at stake, expert opinions, beliefs, or emotions matter much less than the treatment’s safety and efficacy data.

Since there is no cure for DMD, Elevidys was approved despite lack of sufficient evidence of clinical benefit – via an Accelerated Approval pathway that accepts efficacy on a surrogate endpoint that *may* predict clinical outcomes.

Elevidys is the first gene therapy approved for Duchenne Muscular Dystrophy (DMD), a rare genetic disease that affects young boys and is often a death sentence.

Earlier this month, FDA requested Sarepta to suspend distribution of its flagship gene therapy, Elevidys, owing to multiple patient deaths from acute liver failure. In a rare move, Sarepta initially refused (!) but later relented to FDA’s request.

Let’s dive into Sarepta’s recent feud with FDA by building a RAG application on publicly accessible FDA documents, SEC filings, Press Reports, Publications, and Abstracts 🧵

Learn more --

Blog: rohitium.github.io/2025/07/14/D...
Code: github.com/rohitium/tar...
Demo: rohitium.github.io/target-dili-...

One way to augment preclinical tox models is to prioritize targets with lower risk, using historical data from in-human clinical trials, e.g. the Drug Induced Liver Injury Rank (DILIrank) Dataset published by FDA

This guidance comes on the heels of increasing recognition that failure of pre-clinical models translating into the clinic is one of the biggest reasons for high costs of drug development, especially in case of first-in-class targets

FDA’s recent guidance on de-prioritizing animal testing places heavy emphasis on developing better computational models for estimating drug toxicity, immunogenicity, and pharmacodynamics in humans

Let’s build a data-driven, biologically relevant Liver Injury risk score using Open Targets, Pathway Commons, and FDA DILI rank dataset (100% open source) 🧵