aria 🍊😶‍🌫️👁️

it hasn't made it into any public models yet. i think it's one of my favorite things by DeepSeek, because it's a perfect microcosm of their whole philosophy

it is simultaneously a cool sparse attention scheme and also a highly-efficient attention kernel with proper memory/compute balancing

in theory you could just do NSA on a sliding window or something but DeepSeek correctly point out that this would murder inference throughput, so a very clever trick would be needed to make this happen

i think the most likely "solution" to tokenizers being shit is going to be something like DeepSeek's Native Sparse Attention (which allows for compression of adjacent embeddings within a fixed block pattern) but more dynamic

such that the model can just cope with byte-level separation natively

new OpenAI chart crime just dropped

personally I am not confident that GPT-5 actually fixes this

it works by routing requests automatically to one of three models, which will apparently also be selectable individually. unless the router is ~perfect users will see essentially random variation in latency and intelligence. sucks

what do you mean

you can use 4o or if you want extended thinking you can use o3 or o4-mini (o4-mini is worse than o3, naturally). there's also GPT-4.1, which is better than GPT-4.5

you can also activate various "tools" which actually swap to various unnamed internal models or multi-model systems

bafflingly, when logged out of ChatGPT or using it on mobile, the UX does not expose the existence of other models to you or tell you which one you're using

this has convinced millions of people that AI has stagnated for the past year (because it's been the same model for the last year)

this sort of problem is apparently a very high priority for the UX surrounding GPT-5

an OpenAI guy on the Other Site has referred to their new unified model router as the "stop doctors from using 4o-mini" update

at minimum I am going to be really annoyed if OpenAI calls their open model "GPT-OSS" and it's licensed restrictively

as we all know, doing a decent job predicting the contents of literally any possible text document is a task that requires no intelligence or understanding of the world

the inhumanity of a bureaucracy is most visible when it is fucking you over, but in other ways it is helpful to have a system that cares very little about people's sensibilities

most people's sensibilities kind of suck

they also sometimes remember ethereum exists, which they're pretty sure is the one that eats gaming GPUs

overall I find my productivity more augmented by the autocomplete than the big LLM, which is a big change from last year when I found the autocomplete grating and nearly always wrong or overzealous

stuff like Cursor's custom autocomplete model has been getting more advanced on this front but the big LLMs have generally not

at best they get access to linter and compiler errors after writing the buggy code

one day I really want to make a barebones instruct model purely with base model prompting and RL over rubrics

explicit "no sycophancy" and "you are not ChatGPT" rules to suppress slop

Somehow despite coming up with a robust rubric evaluation RL system for their safety tuning, Anthropic is still using human gig workers and RLHF for personality. Makes no sense to me, personally.

a substantial number of people have convinced themselves that the primary driver of price is that every company is making a high-double-digit profit margin on everything, and therefore material or labor inputs represent a basically negligible component compared to "greed"

it's like butterflies taking detours in their migration to avoid a long-eroded mountain range, pure muscle memory

we have invented a way to print panels that generate electricity with no fuel and require only dusting and the occasional replacement

and everyone is still acting like the path of least resistance is rationing. this sucks

if the fired project leaders had their way, the game would've come out next month with two biomes (one of which is just the edge of the map), 7 POIs, 12 fish, and "8-10 hours of story". their original early access target was 2023 with way more content

in the most "we did it reddit" moment in a while, the subnautica community noticed that SN2 was delayed and the leaders fired and immediately began a campaign to boycott it

only to find out 3 days later that they were fired for cutting most of the content to rush it out after 2 years of delays

i have come to the opinion that there are a number of people who, consciously or not, believe that the purpose of a protest is to be gunned down and spark civil unrest

nah, this is "Grok 4", which they seemingly RL'd to act like this

I never understand why they all have legs

they probably spend as much money on the legs as they do the arms, when the arms are clearly the value-producing part of the machine

if the answer is "burn into camera module" this will:
1. not work, people will figure it out anyway
2. limit adoption so heavily that it will be like 10 years before "I shot this on an old phone without a verified camera module" is not a valid excuse

i have never understood what is meant to stop an extracted key from defeating the whole system

the people who react with "nose-wrinkling contempt" to someone saying they don't use LLMs are dwarfed in scale by the people who invent a new synonym for "degenerate" whenever someone says they used ChatGPT to write an email

The modal person running a blocklist is more unpleasant than the modal person on one

i wonder how much of Lifestyle Creep is attributable to "kids have basically no preferences about their parents' lifestyle so everyone just assumes they are maximalists"

one imagines it will be completely offset by the predictable economic devastation of this bill anyway