Emanuele Marconato

Joint work with Sebastian Weichwald, Sebastien Lachapelle, and Luigi Gresele 🙏

For more info, check the full paper 👇

arxiv.org/abs/2410.235...

🧵Summary

A mathematical proof that, under suitable conditions, linear properties hold for either all or none of the equivalent models with same next-token distribution 😎

Exciting open questions on empirical findings remain🤔 - check Section 6 (Discussion) in the paper!

8/9

3⃣ We demonstrate what linear properties are shared by all or none LLMs.

🔥 Under mild assumptions, relational linear properties are shared!

⚠️ Parallel vectors may not be shared (they are under diversity)!

7/9

We also describe other linear properties: linear subspaces, probing, steering, based on relational strings (Paccanaro and Hinton, 2001).

💡They arise when the LLM can predict next-tokens for textual queries like: "What is the written language?" for many context strings!

6/9

2⃣ We reformulate linear properties of LLMs based on textual strings, depending on how LLMs predict next tokens

💡Parallel vectors arise from same log-ratios of next-token probs

E.g. same ratio for "easy"/"easiest" and "strong"/"strongest" in all contexts => parallel vecs

5/9

💡The extended linear equivalence underlies that two models' representations are linearly related, but in a subspace

‼️Outside that subspace, representations can differ a lot!

4/9

1⃣We extend the results by Khemakem et al. (2020), Roeder et al. (2021), removing a diversity assumption.

For the first time, we relate models with different repr. dimensions & find that repr.s of LLMs with same distribution are related by an “extended linear equivalence”!

3/9

Contributions:

1⃣ An identifiability result for LLMs
2⃣A 𝙧𝙚𝙡𝙖𝙩𝙞𝙤𝙣𝙖𝙡 reformulation of linear properties
3⃣ A proof of what properties are 𝙘𝙤𝙫𝙖𝙧𝙞𝙖𝙣𝙩 (~to Physics, cf. Villar et al. (2023)): hold for all or none of the LLMs with same next-token distribution

2/9

🧵Why are linear properties so ubiquitous in LLM representations?

We explore this question through the lens of 𝗶𝗱𝗲𝗻𝘁𝗶𝗳𝗶𝗮𝗯𝗶𝗹𝗶𝘁𝘆:

“All or None: Identifiable Linear Properties of Next-token Predictors in Language Modeling”

Published at #AISTATS2025🌴

1/9

All or None: Identifiable Linear Properties of Next-Token... We analyze identifiability as a possible explanation for the ubiquity of linear properties across language models, such as the vector difference between the representations of “easy” and “easiest”...

@yanai.bsky.social this is very interesting!! FYI, we studied the ubiquity, rather than emergence, of linear relational properties here:

openreview.net/forum?id=XCm...

All or None: Identifiable Linear Properties of Next-Token... We analyze identifiability as a possible explanation for the ubiquity of linear properties across language models, such as the vector difference between the representations of “easy” and “easiest”...

Now in Thailand to present our paper at #AISTATS2025 🇹🇭🌴

📍Today at 3:00-6:00 pm, poster number 118!

More details here:

openreview.net/forum?id=XCm...

Only yesterday I discovered that my PhD thesis has been made public to everyone 😅

I worked three years on "Learning concepts" and I tried to spot the connection between #concepts, #symbols, and #representations, and how they're used in ML today 👾🪄

etd.adm.unipi.it/t/etd-012620...

Hey hey! We have an accepted paper at #AISTATS2025!!
Time to prepare for Thailand 🪷🏖️🌴🐒

Huge thanks to my coauthors
Luigi Gresele, Sebastian Weichwald, and @seblachap.bsky.social for all the joint effort!

More details soon 👇

arxiv.org/abs/2410.235...

Don't miss the chance to learn more about our new #benchmark suite @ #NeurIPS2024

📊 New benchmarks to test concept quality learned by all kinds of models: Neural, NeSy, Concept-based, and Foundation models.

🤔 All models learn to solve the task but, beware, do they learn concepts??

Spoiler: 😱

📣 Does your model learn high-quality #concepts, or does it learn a #shortcut?

Test it with our #NeurIPS2024 dataset & benchmark track paper!

rsbench: A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts

What's the deal with rsbench? 🧵

The #NeurIPS experience is about to start! ✈

Drop me a line if you want to chat about #neurosymbolic reasoning #shortcuts, human-interpretable machine #concepts, logically-consistent #LLMs, or human-in-the-➰ #XAI!

See you in Vancouver!

For those attending #NeurIPS2024 go to UNIREPS @unireps.bsky.social workshop to know more about representations similarity. Nice work led by @beatrixmgn.bsky.social 🌟

🚨 Interpretable AI often means sacrificing accuracy—but what if we could have both? Most interpretable AI models, like Concept Bottleneck Models, force us to trade accuracy for interpretability.

But not anymore, due to Concept-Based Memory Reasoner (CMR)! #NeurIPS2024 (1/7)

Ask you to please add me :)

We are the premier conference on #uncertainty in #AI and #ML since 1985 🧓

Hello, 🦋!

Follow us to reduce uncertainty!

Then I agree 😄

What is the precise definition of feature?

I would like to ask for some back stabs to reviewer 2 🤬

I know @looselycorrect.bsky.social well enough eheh

A symbol is a "physical token" (Harnad 1990, arxiv.org/html/cs/9906...)

Do you have answers? 😁