@bminixhofer.bsky.social
Instruction tuning unlocks incredible skills in LLMs, but at a cost: they become dangerously overconfident.
You face a choice: a well-calibrated base model or a capable but unreliable instruct model.
What if you didn't have to choose? What if you could navigate the trade-off?
(1/8)
Models from our paper, including Gemma-2B and Llama-3B instruction-tunes transferred to byte-level, are up on Hugging Face ๐ค
huggingface.co/collections/...
Check out the paper for lots of details.
We are also releasing our code as part of `tokenkit`, a new library implementing advanced tokenization transfer methods. More to follow on that๐
Paper: arxiv.org/abs/2503.20083
Code: github.com/bminixhofer/...
w/ Ivan Vuliฤ and @edoardo-ponti.bsky.social
2๏ธโฃWe also use ALM to directly transfer knowledge from a large teacher (with one tokenizer) to a smaller student (with another tokenizer).
We test this by distilling a large maths-specialized Llama into a small Gemma model.๐ข
1๏ธโฃcontinued: we can also transfer different base models to the same tokenizer, then ensemble them by combining their logits.
This would not be possible if they had different tokenizers.
We try ensembling Gemma, Llama and Qwen. They perform better together than separately!๐ค
We investigate two use cases of ALM in detail (but there's definitely more!)
1๏ธโฃTokenizer transfer: the teacher is the model with its original tokenizer; the student is the same model with a new tokenizer.
Here, ALM even lets us distill subword models to a byte-level tokenizer๐ฎ
Chunks of tokens with different tokenization biases are not fairly comparable!โ ๏ธโ ๏ธ
We thus develop a method to find chunks with low tokenization bias differences (making them *approximately comparable*), then learn to match the likelihoods of thoseโ
Our greatest adversary in this endeavour is *tokenization bias*.
Due to tokenization bias, a sequence of subword tokens can leak information about the future contents of the text they encode.
Most distillation methods so far needed the teacher and the student to have the same tokenizer.
We lift this restriction by first identifying comparable chunks of tokens in a sequence (surprisingly, this is not so easy!), then minimizing the difference between their likelihoods.
Image illustrating that ALM can enable Ensembling, Transfer to Bytes, and general Cross-Tokenizer Distillation.
We created Approximate Likelihood Matching, a principled (and very effective) method for *cross-tokenizer distillation*!
With ALM, you can create ensembles of models from different families, convert existing subword-level models to byte-level and a bunch more๐งต
Two amazing papers from my students at #NeurIPS today:
โ๏ธ๐ฅ Switch the vocabulary and embeddings of your LLM tokenizer zero-shot on the fly (@bminixhofer.bsky.social)
neurips.cc/virtual/2024...
๐ Align your LLM gradient-free with spectral editing of activations (Yifu Qiu)
neurips.cc/virtual/2024...
