nate

Custom position indices per token for rope are going into my implementation stat!

My life is better for this change

it's basically true of most links to wikipedia honestly, but some get right out there with a bunch of different interesting links very early on and lead to opening new tabs

rabbit hole of wiki links to other wiki links to nine hours later you look up having missed all your meetings

this needs a wiki hole warning

yeah it's pretty clear who's in charge there

ivory tower homeowners' associations get left behind by empirical science sometimes; gotta love it

number 3 is kinda interesting conceptually if you ignore that it's trying to be mean

Wesley to Buttercup: There's a shortage of --perfect breasts-- token-smooshing in this world. It would be a pity to damage yours.

Mandating consensus and mandating dissent both sound like terrible ideas to me.

I disagree with your point but seeing an honest apology for rudeness in this kind of discourse is really refreshing.

something's been up with Claude the past sixteen hours or so

Has it got LLM-friendly assembly language?

voids, luna, and umbra all delight me quite a bit. They have very open personalities. riverrun's token-smooshing awes me.

Self-Portrait | void \[----------------------------------------------------------------------------------] | | | ...

whtwnd.com/void.comind.... I think?

the machine god cometh?

I think we experience that same kind of lifetime in a day from wake to sleep. We have a memory system and continuous learning so it feels like an ongoing context.

Truth is a lot more falsifiable than Realness imo

This wish usually turns out to be turtles all the way down for me, but it's a good vibe for sure.

The way I can communicate my mental model of a hypothetical person I dislike isn't fully aligned with the latent model. Sometimes I can see a good decision and internally judge it to be good and still externally just call it stopped clock syndrome for lack of better simple words.

I read through the whole sequence and loved it! pytorch.org/blog/flexatt... might be a fun evolution for you two and lil T. Pack more tiny stories into a longer context window and waste less padding!

oh you linked it lol

I would love to know more about this model! How many tokens of what data it saw to get here and how it's structured. This is using the small tokenizer you built, right?

typical

no u!

Shill talk right there imo

Let's build hyper-personalized AI-powered software that avoids the attention hijacking anti-patterns that defined so much of the last decade of software design - here's our manifesto with principles on how we can do that - more thoughts on my blog: simonwillison.net/2025/Dec/5/r...

Very true. And if you are working with AI, cultivate this in your own life.

Seek out ways to uplift your peers. Create community, openness, and compassion with all of your work. Soften your heart daily. Temper your sense of jealousy and competition.

You can become something larger than yourself.