@joelye.bsky.social
NeuroAI PhD student @ Carnegie Mellon. Being a brain-computer interface.
NDT3 is brought to you by a huge team of data contributors from @corticalbionics.bsky.social
@raeedcho.bsky.social
@jenpitt.bsky.social
. ? (Apologies for the shortlist, not wired into the bsky community yet)
Link: doi.org/10.1101/2025...
Code: github.com/joel99/ndt3
. Weights are posted on HuggingFace, includes a notebook for fine-tuning NDT3 on a new dataset.
Lowlights:
- NDT3 scales weakly, likely due to cross-subject transfer issues (but there's some rzn to believe cross-attention encoders, like in POYO, wouldn't fare better)
- NDT3 will still overfit some types of stereotypy in neural data, e.g. reach angle. Post-training likely needed to resolve
Highlights:
- Pretrained on 2000 hours. By data/params, NDT3 brings motor BCI to the BERT era!
- Evaluated on 8 datasets, (including 3 from FALCON, go beat NDT3 snel.ai/falcon
!)
- Gains ~generalize to distribution shifts in neural data -- e.g. in time, posture
What it is:
- NDT3 is "just" an autoregressive Transformer, for decoding behavior from neural spiking activity
- We tokenize behavior timeseries by splitting and neural data by patching (a la NDT2/ViT). Thus, the model can ingest most motor brain-computer interface datasets
Neural Data Transformer 3 is preprinted! We think it can be an off-the-shelf model for all your intracortical motor decoding needs. Decoding gains over your multisession model expected if you have <1.5 hours of data.
07.02.2025 14:21 — 👍 18 🔁 5 💬 2 📌 1
