Alexander Doria

It's going to be much faster. Everything is going to be much faster (and that's a major issue).

I’d need 1-2 hours of careful reading but one the most significant factor is the introduction of a dedicated model system for geometry/spatial reasoning. Auto-regressive LLMs are notoriously deficient in this front.

The one paper to read this week-end. ByteDance seems to have achieved a major breakthrough in AI math: new LLM systems claim to solve half PutnamBench — with previous SORA from DeepSeek and Goedel at 10-15%. arxiv.org/abs/2507.23726

Correct and the reason you don’t see a lot of AI regulations in the foreseeable future:

I'm increasingly suspicious a lot of gender/sex issue stems straight from the lack of sex positive discourse for men.

Kind of a black pill for quick AGI: no lab seemingly attempted to compete at the International Physics Olympiad.

Also code models frequently include a "fill in the middle" feature which does include future tokens at prediction time, but not predicting backwards.

Not directly but one of the interesting features of diffusion models is kind of close: since they generate the whole answer in one batch and denoise they do reassess past tokens in light of (likely) future tokens.

Someone got access to the weights and attempted to reverse engineering inference without success so far. Still plenty of architecture details. gist.github.com/main-horse/a...

None yet but I really Sam Altman committed to an actual open one.

Someone reposted what seems to be the config on discord. Maybe a mixture of experts after all.

New OpenAI open source model briefly appeared on HuggingFace and it’s not only phone-sized: 120 billion parameters, apparently dense. There’s also a 20b. Unclear if either of them is Horizon-Aloha currently on preview on OpenRouter.

Since they focused on triplet combinations, almost surprising they did not work straight from Wikidata or other RDF sources.

Very intriguing paper on agentic search from Qwen team. Introduce advanced synth methods for query and retrieval with heavy symbolic approach (almost look like a math prover - they do cite deepseek prover). arxiv.org/pdf/2507.15061

Relatedly I was surprised people in symbolic AI are so negatively polarized against generative AI given the many opportunities for crossover. But now I'm going deep in expert systems it does make some sense: really really hard to get a LM working reliably.

2027: you need 2FA digital id to read wikipedia article on contraception all the while your phone generates 5 hours of endless hentai.

local models improving a lot in a context of widespread internet censorship will certainly be funny contrast

Hum je pense vraiment que c'est plutôt un choix d'architecture — probablement mix auto-regressif + diffusion avec un drift naturel vers les couleurs chaudes. La donnée synthétique est utilisée depuis un moment pour les modèles d'image (enfin on parlait plus de data augmentation à l'époque)

Been creating fake synthetic wikipedia articles as I plan to use real ones as seeds for math problem and some are accidentally funny.

I dare say the concept of synthetic playground has completely changed the way I approach pre/mid/post-training (is it training now?). Next Pleias releases will be very different.

We now get a major preview of the next chapter of Physics of language model. And it's not toy models anymore: systematic synthetic data/design experiments to create SOTA 7B, especially for memorization/recall. github.com/facebookrese...

anyway, thanks mastercard and visa for reviving the big tent anti-censorship movements from the early 2010s. i can confirm it isn't any more popular on the other website

Just started it and goes really straight to the theme. Really well written and interesting, surprised it’s not more well-known.

Except we might now argue computers can somewhat "think", not much has changed

Randomly bought one of the few (?) novels about digital humanities. From 1968 but completely fitting the definition: narrator is going to do text statistics on Italian cultural heritage with a giant IBM computer.

This is especially welcome as models deal much more with structured data. For our semantic data experiments I ended up dropping the classic rdf triplets as they were too token consuming: no longer the case here.

Great breakdown of h-net, the new rising alternative to vanilla tokenization. With a demonstration of its strength: encoding low information/repetitive patterns (like sequences of the same letter) as a single token. main-horse.github.io/posts/hnet-i...

Yes! As far as I know they seem to have used some parts of common corpus (and associated tooling as well)

En réalité je pense qu’il n’y a plus rien d’inaccessible en terme de courtes productions écrites/visuelles. Suffit d’avoir les données et une manière de modéliser l’objectif à atteindre (pas forcément vérifiable : là l’essentiel de la recherche en reinforcement learning est sur les soft rewards)

Le problème de ce type d’argument c’est qu’on court derrière les évolutions de la technique. Les labs commencent à investir le créneau esthétique comme les évaluations de math sont saturées : OpenAI préparerait un modèle "littéraire" et k2 (Moonshot) se fait en partie remarquer sur ce plan