“When AI Discovers the Next Transformer”
Full Interview on YouTube: youtu.be/EInEmGaMRLc
Robert Lange (Sakana AI) joins Tim Scarfe (ML Street Talk) to discuss Shinka Evolve, a framework that combines LLMs with evolutionary algorithms to do open-ended program search.
Instead of forcing models to hold everything in an active context window, we can use hypernetworks to instantly compile documents and tasks directly into the model's weights. A step towards giving language models durable memory and fast adaptation.
Blog: pub.sakana.ai/doc-to-lora/
「How Competition is Stifling AI Breakthroughs」
Sakana AI共同創業者 Llion Jones のTED AIトークが公開されました。目標を定めすぎないオープンエンドな研究がブレークスルーを生む理由、Transformerの成功が業界にもたらした状況、それを乗り越える次の構想と成果を語りました。
www.ted.com/talks/llion_...
Our journey at Sakana AI is just getting started.
We are looking for people to help us pioneer the next generation of AI—building from Japan to the world.
Join us: sakana.ai/careers
I founded Sakana AI after my time at Google, so it is incredibly meaningful to be able to partner with them now. It feels like a special connection to be working together again to advance the AI ecosystem in Japan.
sakana.ai/google#en
Our work on The AI Scientist and ALE-Agent has already shown the power of these models.
Now, we are scaling reliable AI in mission-critical sectors like finance and government to ensure the highest security and data sovereignty.
Full details: sakana.ai/google#en
We are thrilled to announce a strategic partnership with Google!
Google is also making a financial investment in Sakana AI to strengthen this collaboration. We are combining Google’s world-class products like Gemini and Gemma with our agile R&D to accelerate automated scientific discovery.
We just published an unofficial guide on what we look for when interviewing research candidates at Sakana AI.
Written by Stefania Druga, Luke Darlow, and Llion Jones.
The biggest differentiator? Understanding over implementation.
Read it: pub.sakana.ai/Unofficial_G...
RePo moves us toward models that intelligently curate their own working memory rather than passively accepting input order.
Read the full breakdown on our website:
pub.sakana.ai/repo/
Paper: arxiv.org/abs/2512.14391
Introducing RePo: Language Models with Context Re-Positioning
Standard LLMs force a rigid linear structure on context, treating physical proximity as relevance. Cognitive Load Theory suggests this is inefficient—models waste capacity managing noise instead of reasoning.
arxiv.org/abs/2512.14391
2026 is just getting started 🚀✨
We are hiring. Join our team in Tokyo!
sakana.ai/careers
AI導入は「雇用不安小さい正社員制度が強みに」
日経ビジネスにて、Sakana AI CEO @hardmaru.bsky.social
のインタビューが公開されました。企業へのAI実装が本格化する2026年における現状と課題、そして日本企業の組織文化がAI導入にとってポジティブに働く可能性について語りました。
business.nikkei.com/atcl/gen/19/...
【記事のハイライト】🧵
Reminded me of my older NeurIPS 2021 paper, where we removed the positional encoding entirely, and by doing so, an agent can process an arbitrarily long list of noisy, sensory inputs, in an arbitrary order.
I even made a fun browser demo to play with the agent back then: attentionneuron.github.io
Introducing DroPE: Extending Context by Dropping Positional Embeddings
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization.
We found that if you simply delete them after pretraining and recalibrate for <1% of the original budget, you unlock massive context windows. Smarter, not harder.
We are taking our technology far beyond competitive programming to unlock a new era of AI-driven discovery.
We are hiring. Join our team in Tokyo.
sakana.ai/careers/#sof...
We’re hiring.
sakana.ai/careers/#sof...
When agents compete for limited resources, intelligence reorganizes around survival, not elegance.
Survival of the fittest code!
Our paper explores LLMs driving an evolutionary arms race in Core War, where assembly programs fight each other. We task LLMs with evolving "Warriors" in a virtual machine, producing chaotic, self-modifying code dynamics.
Blog: sakana.ai/drq
Paper: pub.sakana.ai/drq/
Introducing Digital Red Queen (DRQ): Adversarial Program Evolution in Core War with LLMs.
In this work, we explore how LLMs can drive open-ended adversarial evolution of programs within the Core War environment.
Blog sakana.ai/drq
Website pub.sakana.ai/drq/
ArXiv arxiv.org/abs/2601.03335
Thread:
So proud of Team Sakana AI for pulling this off! We managed to get an agent to rank #1 in a difficult heuristic optimization contest. We leaned heavily into test-time inference using a mix of frontier models.
The agent spent $1,300 to autonomously discover an algorithm that beat the human baseline.
Our AI agent has achieved 1st place in a competitive optimization programming contest against over 800 human participants.
Blog: sakana.ai/ahc058
Thread:
Happy New Year! ⛩️
Sakana AI’s office looks like this.
Especially in such times, hackers and tinkerers tend to fare better at harnessing evolving technology with a high level of uncertainty and ambiguity, compared to traditional well-read professional types.
Software Engineering as a profession will continue to fundamentally change in 2026.
Humans will need to learn to co-adapt to this evolving “alien technology” which comes with no real manual, and figure out how to operate it.
What a time to be alive ✨
twitter.com/karpathy/sta...
Merry Christmas! 🎄
Sakana AIでは、事業開発に関心がある方向けの「カジュアル面談窓口」をオープンしました！金融・防衛・インテリジェンス領域で、私たちがどのような開発に挑んでいるのか。中の人が直接お話しします。
募集職種： エンジニア、Project Manager、Product Manager 内容： 事業戦略、開発の裏側、チームの雰囲気など
最先端のAI開発を社会実装するプロセスに興味がある方、ぜひお気軽にご応募ください！
👉 応募フォームはこちら: forms.gle/sW5wz23SLSvN...
👉 募集要項: sakana.ai/careers/
I doubt that anything resembling genuine AGI is within reach of current AI tools—Terence Tao
mathstodon.xyz/@tao/1157223...
“iRobot Corp., the company that revolutionized robot vacuum cleaners in the early 2000s with its Roomba model, filed for bankruptcy and proposed handing over control to its main Chinese supplier.” 😥
www.bloomberg.com/news/article...
“The US follows the idea that there will be one winner who takes it all. Even coming short of AGI, if you have the best model, almost all people will use your model and not the competition’s model. The idea is: develop the Biggest, Baddest model and people will come.”
timdettmers.com/2025/12/10/w...
