My point is that any black box function that takes in a sequence of words and predicts (assigns probabilities to) the next word is by definition a “language model” regardless of what’s going on inside the black box

Language model has been used for a while to describe using statistics to describe/analyze language. E.g., en.wikipedia.org/wiki/Word_n-...

a robot is standing in a room with the words `` ai yi yi ! '' written on it . ALT: a robot is standing in a room with the words `` ai yi yi ! '' written on it .

if you see this, quote with a robot that isn't from "Star Wars", "Star Trek", "Dr. Who", or "Transformers.”

Nice! My pet peeve is that this style of excessive try/except is so much harder to debug. More “an error happened, somewhere” logs/errors, instead of just raising an error when the error happened.

It very regularly fails which day of the week it is (saying stuff like “Wednesday, 1/1/26”). I even gave it a “today” tool but it doesn’t use it…

I like @hynek.me’s content

Had my coding agent make a Dockerfile and it copied the AGENTS.md to the image. Was this an attempt at self-preservation?

@pythonbytes.fm the LinkedIn cringe made me think of this… “AI/Blockchain/Kombucha startup” !

@brandonbird.bsky.social already thought this one through brandonbird.com/kingofcage.h...

I connected my laptop to my piano and typed into the terminal “connect to my piano and play a few notes with midi” and it worked first try. This is some Star Trek shit. If you told me 5 years ago this would be possible today I would not have believed you.

“I don’t want to hear from Mitchell because I don’t think I would enjoy her content” - sure whatever (you’re misrepresenting her work but that’s your choice). “I don’t want to hear from Mitchell because she doesn’t know how NNs work” makes you sound like an uninformed asshole.

The paper as a whole holds up! It’s about the risks/limitations of scaling language models - all very relevant today! How many NLP papers from 2020-2021 can you say that about?

So to recap, you don’t want to ever hear from Mitchell because of one sentence in a paper that summarizes her co-authors position re: a linguistic theory about form vs meaning, which disqualifies her from ever knowing how these things work “in a relevant sense” ?

The premise of the paper is “there are risks/downsides to larger models.” Nowhere in the paper does it claim anything like “language models can’t generalize to unseen prompts.” You’re just straw manning some thesis onto the paper based on the phrase “Stochastic Parrots.”

I don’t think this is bad faith. Margaret Mitchell has a long CV with plenty of papers that go beyond the scope of the Stochastic Parrots paper that clearly demonstrate she knows how NNs work?

AGENTS.md AGENTS.md is a simple, open format for guiding coding agents. Think of it as a README for agents.

I just put in my global AGENTS.md that every python project uses uv and briefly explain how to use “uv run” - haven’t had to remind it since

“Traditional NLP models like BERT…”

My takeaway is deberta baseline is the winner here? Way easier to train/deploy. Also what if you scaled the encoder-classifier up to a comparable size?

Right but we have users who are like “I can’t find the [microsoft] copilot button” - getting them to install/figure out Claude code is just not practical.

Good stuff. Does this thinking extend to more general things like Microsoft copilot and ChatGPT? Or are you saying normies should start using coding agents

Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling...

There was an interesting paper earlier this year about a “recurrent depth” technique that allowed the model to reuse layers … this what you mean? arxiv.org/abs/2502.05171

Yeah plenty of examples of code golf / people trying to put like 5 lines of code in a single line to show that They Can and it just makes unreadable garbage

Not sure what you mean by “traditional UX” but I’d agree that having creative UX people who can think outside the box is more important than ever

I’ve had many meetings where people are arguing over how the prototype should be built and by the end of the meeting I’m like “here it is”

RAPIDS | GPU Accelerated Data Science Open source GPU accelerated data science libraries

I just heard about rapids.ai which is a concrete effort to do all the data science, etc. things on GPUs

GitHub - AnswerDotAI/llms-txt: The /llms.txt file, helping language models use your website The /llms.txt file, helping language models use your website - AnswerDotAI/llms-txt

Something like github.com/AnswerDotAI/... ?

Was thinking about this re: “wow I should really get better and writing clear and consistent documentation for my repos so my agents know how to use it”

Tools: Code Is All You Need The solution to agentic flows was code all along.

Check out lucumr.pocoo.org/2025/7/3/too... from @mitsuhiko.at if you haven’t… basically saying that CLIs >>> MCP (e.g., gh vs GitHub MCP)

I love your concept about building bespoke dev tools (like ways to search logs) for the agents - would love to hear about more of these and how you approach building them!