Nasim Rahaman

You’ve probably heard this before, but chances are that you know a decent amount of math, just in a different language. E.g. if you can code well, clearly you’re good at symbol manipulation (most of coding is that, in a sense).

Math is like a different (and at times esoteric) programming language.

… [2/2] and (b) how easy is it to extract, or “render” those answers.

The true craft of scientific writing is figuring out these functions. Everything else (LLMs or not) is irrelevant.

I actually think this is the wrong way to think about (scientific) writing.

The purpose of a scientific document is to help answer questions about some topic.

The quality of the document is a function of (a) how large is the space of questions you cover, … [1/2]

I defended my PhD today, and would like to take this moment to thank the folks who made it worth the journey -- ya'll are awesome.

IMO the practical value of impossibility theorems often lie in formulating those assumptions — they might reveal what conditions are probably violated in practice (if the statement of the proof is intuitively off). Eg. the IIA assumption in Arrow’s Impossibility Theorem.

I do wonder how much of that fat tail is lost in simulation, and what the downstream effects are. Model autophagy disorder is not something you’d want for models making life or death decisions.

The purpose of a document is to help an LLM answer questions about said document.

Imagine where we would be today if an overpowered Nazi Germany did not have US and USSR to counterbalance. Or just watch Man in the High Castle.

Dario Amodei — On DeepSeek and Export Controls On DeepSeek and Export Controls

Re: Dario’s post (darioamodei.com/on-deepseek-...) — the current state of the US has left me scared shitless about a unipolar world. A single, powerful pole with compromised institutions is the worst possible outcome for human civilization.

I’m low key disappointed r1 doesn’t swear in its CoT.

Tired: “Wait…”

Wired: “hol the fuck up”

And in that world, the quality of the weapon is how cheap it runs for how good it is.

I won’t be surprised if nation states eventually train and host their own models. Heck, some LLM shops seem to be betting on that, e.g. Mistral and AlephAlpha with “European LLMs”.

We’re approaching an era of memetic warfare where LLMs are the weapons. We’re not there yet — the values espoused by Chinese LLMs aren’t all that different from American ones — but that’s for now.

But once LLMs become our primary interface with the outside world, it’s bound to happen.

Towards Benchmarking LLM Diversity & Creativity Discussion of possible tasks to measure LLM capabilities in soft ‘creative’ tasks like brainstorming or editing, to quantify failures in creative writing domains.

An underappreciated takeaway from Gwern’s recent post (linked) is that LLMs are like vessels of human culture. They’re distribution channels for values.

gwern.net/creative-ben...

Looks nice! Some FastAPI endpoints + a docker image should help adoption. :)

The @jina.ai team is low key cracked. No yapping just shipping. 🫡

Finished the first season of Severance. Brb implementing memory for my LLM agents.

This is fascinating work, congratulations!

Question: the point that architectural constraints (locality + equivariance) are sufficient is well demonstrated.

But do you think it is necessary? I.e. would you expect a diffusion transformer to learn these constraints?

In other words, code 1 is more “multi-agent” than others.

What do I mean when I say “agent”? A system that we’d like to abstract away like a black box (Rovelli’s definition). Of that, I count three in code 1, and 1 in both codes 2 and 3.

Because code 1 is most explicit about the structure of the computational graph. :)

bsky.app/profile/nasi...

If I were to force an answer, I’d say code 1 (prompt chaining) has more agent energy than the others.

Claude is something special.

Paper here:

Have the token-level LLM predict “concept tokens”. The hidden states for these tokens go in to an adapter, and out come concept representations. Concept tokens attend to previous concept tokens, and perhaps also the span between itself and the previous concept token.

Very cool work from Meta AI: Large Concept Models. Idea: autoregress in the space of sentence level representations.

I think an interesting next step would be to layer this on conventional LLMs / token prediction models.

Here’s how that could work: ⤵️

GitHub - microsoft/markitdown: Python tool for converting files and office documents to Markdown. Python tool for converting files and office documents to Markdown. - microsoft/markitdown

Microsoft just released a tool that lets you convert Office files to Markdown. Never thought I'd see the day.

Google also added Markdown export to Google Docs a few months ago.

github.com/microsoft/markitdown

Alien Recombination: Exploring Concept Blends Beyond Human Cognitive Availability in Visual Art While AI models have demonstrated remarkable capabilities in constrained domains like game strategy, their potential for genuine creativity in open-ended domains like art remains debated. We explore t...

See paper for more.

+ Alejandro is at NeurIPS and figuring out where to do his PhD. Wink wink nudge nudge.

www.linkedin.com/in/alejandro...

Results? Good stuff.

🧵⤵️

The idea is to model two things:
(a) if concepts fit together to make good art, and
(b) if people have already thought about that combination of concepts (“cognitive availability”).

Seek out the combos for which (a) is true but (b) isn’t, and ask a text-to-image model to render that.

🧵⤵️

My MSc advisee (& gang) cooked.

tl:dr — a cute technique to get machines to be more creative.

🧵⤵️

If you aren’t already doing this: share that pdf with Claude.