Aidan Sirbu

🧵 Everyone is chasing new diffusion models—but what about the representations they model from?
We introduce Discrete Latent Codes (DLCs):
- Discrete representation for diffusion models
- Uncond. gen. SOTA FID (1.59 on ImageNet)
- Compositional generation
- Integrates with LLM
🧱

New preprint! 🧠🤖

How do we build neural decoders that are:
⚡️ fast enough for real-time use
🎯 accurate across diverse tasks
🌍 generalizable to new sessions, subjects, and even species?

We present POSSM, a hybrid SSM architecture that optimizes for all three of these axes!

🧵1/7

Preprint Alert 🚀

Can we simultaneously learn transformation-invariant and transformation-equivariant representations with self-supervised learning?

TL;DR Yes! This is possible via simple predictive learning & architectural inductive biases – without extra loss terms and predictors!

🧵 (1/10)

This can be a game changer for embodied #NeuroAI.

Or it *could* be, if it were open source.

Just imagine the resources it takes to develop an open version of this model. Now think about how much innovation could come from building on this, rather than just trying to recreate it (at best).

See my inner physicist hates the whole "doesn't matter as long as it works" sentiment in the ML community 😂. I want to UNDERSTAND not just accept... jokes aside though I see your point for the purposes of this discussion. I think we've identified a lot of potential in this stream of inquiry 🧐

That's somewhat along the lines of what I was thinking as well :)

Also good point about o1. I'd be very interested to see how it performs on the ToM tests!

Give the results and discussion a read as well it's super interesting! There's reason to believe perfect performance of Llama on the faux pas test was illusory (expanded upon in the discussion). That bias you mention is also elaborated upon in the discussion (and I briefly summarize above).

This all now begs the question of whether this makes LLMs more or less competent as practitioners of therapy. I think good arguments could be made for both perspectives. 🧵/fin

This fact is of course unsurprising (as the authors admit) since humanity's embodiment has placed evolutionary pressure on resolving these uncertainties (i.e. to fight or to flee). This dis-embodiment of LLMs could prevent their commitment to the most likely explanation. 🧵/2

I stand corrected. However, LLM's failure at the faux pas test underscores the need for further discussion. The failure: "not comput[ing] [mentalistic-like] inferences spontaneously to reduce uncertainty". LLMs are good at emulating human-responses, but the underlying cognition is different. 🧵/1

Uniform manifold approximation and projection Nature Reviews Methods Primers - Uniform manifold approximation and projection is a dimensionality reduction technique used to visualize and understand high-dimensional data. In this Primer, Healy...

I recently wrote a primer on UMAP for Nature Reviews Primers. If you are looking for an overview of the method, a getting started primer, or best practices it is a good place to start.

rdcu.be/d0YZT

I'd argue that until LLMs can implement theory of mind, they'd be much better at diagnostic-oriented therapy. Being able to truly understand a human, form hypotheses, and guide a patient towards resolution is very different from recommending treatment based off a checklist made using the DSM.

1/ I work in #NeuroAI, a growing field of research, which many people have only the haziest conception of...

As way of introduction to this research approach, I'll provide here a very short thread outlining the definition of the field I gave recently at our BRAIN NeuroAI workshop at the NIH.

🧠📈

I'm making an unofficial starter pack with some of my colleagues at Mila. WIP for now but here's the link!

go.bsky.app/BHKxoss

Mind if I wiggle my way into this 🐛

From double descent to grokking, deep learning sometimes works in unpredictable ways.. or does it?

For NeurIPS(my final PhD paper!), @alanjeffares.bsky.social & I explored if&how smart linearisation can help us better understand&predict numerous odd deep learning phenomena — and learned a lot..🧵1/n