so the small draft model failed, presumably, almost every time, driving load and costs up probably by something like a factor of two or five and leading openai to claim it was ddosed
which it was. by its customers
(no I don't know for sure this is why they dropped but it lined up)
05.03.2026 07:00 —
👍 1
🔁 0
💬 0
📌 0
why would they be wrong? because the small model was trained on normal text, or on normal text for chatgpt BEFORE the price drop
gpt-4 was still the best llm on earth by far and the price had just dropped. thousands of very serious nerds immediately spammed it with shit no LLM had ever seen before
05.03.2026 07:00 —
👍 1
🔁 0
💬 1
📌 0
the time i am thinking of, openai had a sharp sharp price drop right as spec decoding was well fleshed out and then immediately had an outage and blamed a ddos
my best guess: they set up speculative decoding, and priced on some reasonable level of "cache hit" ie the small model guessing right
05.03.2026 07:00 —
👍 1
🔁 0
💬 1
📌 0
i will never not repost borges
a fun story about speculative decoding: i am pretty sure openai fucked themselves with it once
in spec decoding a small llm guesses what the big llm is going to do, and the big llm can check the result quickly
this works like a cache, and it can miss like a cache
05.03.2026 07:00 —
👍 1
🔁 0
💬 1
📌 0
I know what comes next
https://github.com/tanishqkumar/ssd
05.03.2026 06:45 —
👍 14
🔁 2
💬 1
📌 1
not unheard of
05.03.2026 06:36 —
👍 0
🔁 0
💬 0
📌 0
my oddities these days are mostly deepseek which is like, barely finetuned
05.03.2026 06:27 —
👍 0
🔁 0
💬 0
📌 0
will i mention that X llm did Y thing if i am talking to someone? yes. did i write up a detailed report of my experience? no. i had seven of them that day. they were all strange in different ways
05.03.2026 05:43 —
👍 19
🔁 0
💬 3
📌 0
this has the effect that there are all sorts of weird LLM quirks that are mostly known by oral tradition because the volume of them is simply much too high
05.03.2026 05:42 —
👍 22
🔁 0
💬 1
📌 0
one of the funny things is that most of us who deal with them a lot will encounter a doomloop or similar when we are DOING SOMETHING ELSE and so usually we barely or don't chronicle the fact that it happened
05.03.2026 05:42 —
👍 31
🔁 0
💬 1
📌 0
seems pretty obvious. time dilation makes them effectively eternal
05.03.2026 05:39 —
👍 6
🔁 0
💬 1
📌 0
"how close to the line am i" joke
05.03.2026 05:38 —
👍 19
🔁 0
💬 0
📌 0
is gemini like this because they gave it a name that suggests it is into astrology? in the same way that making an LLM a nazi makes it dumb and a bad coder, naming your LLM "gemini" suggests it's really into crystals and has self esteem issues
05.03.2026 05:37 —
👍 35
🔁 0
💬 3
📌 0
count how many times it says "I will be" in that doomloop. almost any level of repetition penalty would prevent it from outputting that, either at serving time or at RL time
05.03.2026 05:31 —
👍 10
🔁 0
💬 2
📌 0
huh. where's that from, exactly? one of the soundtrack bits?
05.03.2026 05:30 —
👍 3
🔁 0
💬 1
📌 0
they could literally fix it with a repetition penalty, a widely known techinque that has been available to everyone for at least five years
05.03.2026 05:20 —
👍 47
🔁 0
💬 6
📌 0
if anyone has never seen the gemini doomloop it looks like this
05.03.2026 05:04 —
👍 118
🔁 7
💬 20
📌 1
05.03.2026 04:41 —
👍 59
🔁 6
💬 0
📌 0
a beta beto, if you will
05.03.2026 05:01 —
👍 18
🔁 0
💬 2
📌 0
Basically I am going through "which of these people would be an obvious person for a senator to talk to" and most other people are distinctly non-obvious
05.03.2026 04:53 —
👍 2
🔁 0
💬 0
📌 0
i take it super seriously. i just think you have to generalize it past quick text exchanges to get any signal out of it now. turing would've loved that
05.03.2026 04:52 —
👍 8
🔁 0
💬 0
📌 0
llms should have native audio output so they can rap battle and we can determine the best lab that way
05.03.2026 01:31 —
👍 79
🔁 4
💬 5
📌 0
please remember: he's not mad
05.03.2026 01:29 —
👍 4
🔁 0
💬 0
📌 0
Time to support trans Dems like @beth4ma.bsky.social
04.03.2026 22:36 —
👍 99
🔁 25
💬 1
📌 0
lmfao
05.03.2026 01:09 —
👍 31
🔁 1
💬 1
📌 0
isn't he canadian
05.03.2026 00:47 —
👍 29
🔁 1
💬 0
📌 0
is this meme retro yet
05.03.2026 00:47 —
👍 34
🔁 3
💬 1
📌 0
an LLM is the plucked chicken for our age
05.03.2026 00:45 —
👍 27
🔁 2
💬 3
📌 0
*sufficient
05.03.2026 00:42 —
👍 4
🔁 0
💬 0
📌 0
the tldr is that llms show sufficiently evidence of consciousness that it depends which definition of the word you use, even, which is probably a major source of the drama
like we basically mean "human-like" and they're clearly human-like in some ways but not others
05.03.2026 00:42 —
👍 12
🔁 0
💬 3
📌 1