Alexander Doria

yeah, the correct implementation would be something like agentic gimp (+ capability to generate artifacts on demand on top)

Right now the most limiting factor for AI capabilities is synthetic environment design. It's scalable but costly, as you need simultaneously high-level AI training, data design and in-domain skills. Not worth it for small markets, so art will likely stay for a while with semi-bad diffusion

+1. Existing agentic models could be considerably better at art/illustration design (not just creating an image in one batch, but iteratively enhancing it, adding details, etc.) if there was the incentives for it. But programming/law/management are just much more lucrative.

Full list of conferences/interventions, some with usual suspects (obviously Wikimedia)

If you're ever in Delhi next week: Pleias has been invited to the AI Impact Summit and my co-founder will give several conferences on SLMs and building AI infrastructure as a commons.

In my experience: they are good provided they have some ability to backtrack/self-correct. Might be a trade-off with the overwhelming agent design.

Could it be delegation to Haiku? Opus self-corrected course by itself, but just yesterday it had offloaded some annotation tasks to Haiku sub agents, and a few of them settled on bad rule-based design that generated pure junks.

Censure ? Colle avec en tout cas pour la séquence 1805-1814 qui voit vraiment la naissance du feuilleton.

I still have the older DH pilled version (in French). vintagedata.org/pclanglais And maybe next blogpost will be about seeing synth generation through DH lenses.

As a result, did the long delayed/much needed update of personal web page: vintagedata.org

this time absolutely not better of this side of the Atlantic. France/Italy have nearly everything blocked.

I’m not sure if it’s limited to the EU but sex censorship of the web is getting absolutely ridiculous. 1950s level.

i guess i’m really an ai researcher now

Tiens me rappelle un peu en.wikipedia.org/wiki/Celesti... (si jamais tu l'as pas déjà lu)

Wouldn’t have bet +2 years ago on Bender & co getting stuck in a weird cultish mood while EA people counteract very sensibly, but here we are. benthams.substack.com/p/the-ai-con...

Part of the issue: it’s difficult to say what is actually "hard" for a model and frequently counter-intuitive. It could work for more compositional problems of even agentic trajectories but we just don’t know for now.

Yes it's curriculum training. Used to be more discussed 1-2 years ago, still roughly what mid-training is doing in a way, but in a very broad sense. Current consensus is rather in the direction of pooling everything from the start and let the model sort it out.

Was literally one of my SYNTH slide.

(unfortunately practical here as people are too confused otherwise with mid/post-training, but I trust the plan)

Meanwhile, further independent evaluation confirms SYNTH performs better than Nanochat/Fineweb across nearly all benchmarks.

(Humaneval to be expected: we did not out any code inside yet)

Hopefully more like months :)

(Very fun practical industrial use case too, millions of daily users)

*a tentative typology of different shapes of synthetic compilations from memorization, to hardwired logic,
*speculative discussions drawn from model interpretability research (quanta & reasoning primitives),
*and why in the end, maybe we should actually plan for greatness.

Enjoy.

Featuring:
*first attempt at systematic definition informed from practical realities on the field (scaling + scheduling),
*historical context going back from Phi 1.5, why it failed and why you need to treat data as a research axis to make it happen,

It took me weeks, but finally it's there: an overlong blogpost on synthetic pretraining. vintagedata.org/blog/posts/s...

maybe it's awfully european of me, but i'm not convinced dumping a mass of non-contextualized private documents is ever a good thing

seems we’re dangerously close to a point where someone will tweet you can slurp three gas town juices in a moltbot - and kill a few ai bubbles.

Common Corpus paper is going to @iclr-conf.bsky.social !

My point is that it’s not at all a contrarian idea anymore. In the EU fast becoming default funding option for grant as I’m afraid people internalize LLM/verbal reasoning research is beyond grasp, so actually damaging.

If I see another headline about Yann le Cun "contrarian" bet, I’m killing a non-verbal model.

Now starting.

From what I see generally compared to Joyce and Kafka but for now the obvious comparison point is Robert Musil (did Musil read it?)

If you ever wondered if SYNTH could be usable in mid-training for larger models: Step-DeepResearch (from StepFun) is now out as private beta. stepfun.ai/deep-researc...