hardmaru

Instead of forcing models to hold everything in an active context window, we can use hypernetworks to instantly compile documents and tasks directly into the model's weights. A step towards giving language models durable memory and fast adaptation.

Blog: pub.sakana.ai/doc-to-lora/

How competition is stifling AI breakthroughs Llion Jones cowrote "Attention Is All You Need," the seminal paper that introduced the transformer — the architecture that launched the generative AI revolution. Now he warns that the industry that gr...

「How Competition is Stifling AI Breakthroughs」

Sakana AI共同創業者 Llion Jones のTED AIトークが公開されました。目標を定めすぎないオープンエンドな研究がブレークスルーを生む理由、Transformerの成功が業界にもたらした状況、それを乗り越える次の構想と成果を語りました。

www.ted.com/talks/llion_...

Our journey at Sakana AI is just getting started.

We are looking for people to help us pioneer the next generation of AI—building from Japan to the world.

Join us: sakana.ai/careers

I founded Sakana AI after my time at Google, so it is incredibly meaningful to be able to partner with them now. It feels like a special connection to be working together again to advance the AI ecosystem in Japan.

sakana.ai/google#en

Sakana AI Sakana AI、Googleとの戦略的パートナーシップ締結を発表

Our work on The AI Scientist and ALE-Agent has already shown the power of these models.

Now, we are scaling reliable AI in mission-critical sectors like finance and government to ensure the highest security and data sovereignty.

Full details: sakana.ai/google#en

We are thrilled to announce a strategic partnership with Google!

Google is also making a financial investment in Sakana AI to strengthen this collaboration. We are combining Google’s world-class products like Gemini and Gemma with our agile R&D to accelerate automated scientific discovery.

An Unofficial Guide to Prepare for a Research Position Application Authors: Stefania Druga, Luke Darlow, and Llion Jones Disclaimer: This guide is written by a few researchers at Sakana AI who have interviewed many candidates, and does not reflect the view of the entire organization. Each team may have their own preferences and styles for interviewing and finding the people that they can work closely with. This document, written by Stefania, Luke, and Llion, provides a glimpse into how some parts of our research org conduct interviews.

We just published an unofficial guide on what we look for when interviewing research candidates at Sakana AI.

Written by Stefania Druga, Luke Darlow, and Llion Jones.

The biggest differentiator? Understanding over implementation.

Read it: pub.sakana.ai/Unofficial_G...

RePo: Language Models with Context Re-Positioning In-context learning is fundamental to modern Large Language Models (LLMs); however, prevailing architectures impose a rigid and fixed contextual structure by assigning linear or constant positional in...

RePo moves us toward models that intelligently curate their own working memory rather than passively accepting input order.

Read the full breakdown on our website:
pub.sakana.ai/repo/

Paper: arxiv.org/abs/2512.14391

Introducing RePo: Language Models with Context Re-Positioning

Standard LLMs force a rigid linear structure on context, treating physical proximity as relevance. Cognitive Load Theory suggests this is inefficient—models waste capacity managing noise instead of reasoning.

arxiv.org/abs/2512.14391

2026 is just getting started 🚀✨

We are hiring. Join our team in Tokyo!

sakana.ai/careers

サカナAIのデビッド・ハCEO、AI導入は「雇用不安小さい正社員制度が強みに」 AIファーストを掲げる企業が増加する中、経営者はAIのリスクを正しく理解し、適切に導入を進める必要がある。国内最大級のユニコーンで、企業向けのAIソリューション開発を行うSakana AI（サカナAI、東京・港）のデビッド・ハ最高経営責任者（CEO）に、日本企業のAI導入における課題を聞いた。

AI導入は「雇用不安小さい正社員制度が強みに」

日経ビジネスにて、Sakana AI CEO @hardmaru.bsky.social
のインタビューが公開されました。企業へのAI実装が本格化する2026年における現状と課題、そして日本企業の組織文化がAI導入にとってポジティブに働く可能性について語りました。

business.nikkei.com/atcl/gen/19/...

【記事のハイライト】🧵

Reminded me of my older NeurIPS 2021 paper, where we removed the positional encoding entirely, and by doing so, an agent can process an arbitrarily long list of noisy, sensory inputs, in an arbitrary order.

I even made a fun browser demo to play with the agent back then: attentionneuron.github.io

Introducing DroPE: Extending Context by Dropping Positional Embeddings

We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.

arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE

One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization.

We found that if you simply delete them after pretraining and recalibrate for <1% of the original budget, you unlock massive context windows. Smarter, not harder.

We are taking our technology far beyond competitive programming to unlock a new era of AI-driven discovery.

We are hiring. Join our team in Tokyo.

sakana.ai/careers/#sof...

We’re hiring.

sakana.ai/careers/#sof...

When agents compete for limited resources, intelligence reorganizes around survival, not elegance.

Survival of the fittest code!

Our paper explores LLMs driving an evolutionary arms race in Core War, where assembly programs fight each other. We task LLMs with evolving "Warriors" in a virtual machine, producing chaotic, self-modifying code dynamics.

Blog: sakana.ai/drq
Paper: pub.sakana.ai/drq/

Introducing Digital Red Queen (DRQ): Adversarial Program Evolution in Core War with LLMs.

In this work, we explore how LLMs can drive open-ended adversarial evolution of programs within the Core War environment.

Blog sakana.ai/drq
Website pub.sakana.ai/drq/
ArXiv arxiv.org/abs/2601.03335

Thread:

So proud of Team Sakana AI for pulling this off! We managed to get an agent to rank #1 in a difficult heuristic optimization contest. We leaned heavily into test-time inference using a mix of frontier models.

The agent spent $1,300 to autonomously discover an algorithm that beat the human baseline.

Our AI agent has achieved 1st place in a competitive optimization programming contest against over 800 human participants.

Blog: sakana.ai/ahc058

Thread:

Happy New Year! ⛩️

Sakana AI’s office looks like this.

Especially in such times, hackers and tinkerers tend to fare better at harnessing evolving technology with a high level of uncertainty and ambiguity, compared to traditional well-read professional types.

Software Engineering as a profession will continue to fundamentally change in 2026.

Humans will need to learn to co-adapt to this evolving “alien technology” which comes with no real manual, and figure out how to operate it.

What a time to be alive ✨

twitter.com/karpathy/sta...

Merry Christmas! 🎄

Sakana AIでは、事業開発に関心がある方向けの「カジュアル面談窓口」をオープンしました！金融・防衛・インテリジェンス領域で、私たちがどのような開発に挑んでいるのか。中の人が直接お話しします。

募集職種： エンジニア、Project Manager、Product Manager 内容： 事業戦略、開発の裏側、チームの雰囲気など

最先端のAI開発を社会実装するプロセスに興味がある方、ぜひお気軽にご応募ください！

👉 応募フォームはこちら: forms.gle/sW5wz23SLSvN...

👉 募集要項: sakana.ai/careers/

“But perhaps this can be resolved by the realization that while cleverness and intelligence are somewhat correlated traits for humans, they are much more decoupled for AI tools (which are often optimized for cleverness), and viewing the current generation of such tools primarily as a stochastic generator of sometimes clever - and often useful - thoughts and outputs may be a more productive perspective when trying to use them to solve difficult problems.”

I doubt that anything resembling genuine AGI is within reach of current AI tools—Terence Tao

mathstodon.xyz/@tao/1157223...

Robot Vacuum Roomba Maker Files for Bankruptcy After 35 Years iRobot Corp., the company that revolutionized robot vacuum cleaners in the early 2000s with its Roomba model, filed for bankruptcy and proposed handing over control to its main Chinese supplier.

“iRobot Corp., the company that revolutionized robot vacuum cleaners in the early 2000s with its Roomba model, filed for bankruptcy and proposed handing over control to its main Chinese supplier.” 😥
www.bloomberg.com/news/article...

“The US follows the idea that there will be one winner who takes it all. Even coming short of AGI, if you have the best model, almost all people will use your model and not the competition’s model. The idea is: develop the Biggest, Baddest model and people will come.”
timdettmers.com/2025/12/10/w...

「人の成長を加速させるパートナー」それこそが生成AIに求めたい価値｜THE NEXT X 変革の扉 - 日経ビジネス電子版Special 生成AI （人工知能）を活用し、業務 効率化の域を超えるような成果を上げた例は決して多くない。どのように活用すれば、新たな価値を生み出すような成果を得られるのか――。

全文はこちら：
special.nikkeibp.co.jp/atclh/ONB/25...