Kunal Jha

Person standing next to poster titled "When Empowerment Disempowers"

Still catching up on my notes after my first #cogsci2025, but I'm so grateful for all the conversations and new friends and connections! I presented my poster "When Empowerment Disempowers" -- if we didn't get the chance to chat or you would like to chat more, please reach out!

Evolving general cooperation with a Bayesian theory of mind | PNAS Theories of the evolution of cooperation through reciprocity explain how unrelated self-interested individuals can accomplish more together than th...

Our new paper is out in PNAS: "Evolving general cooperation with a Bayesian theory of mind"!

Humans are the ultimate cooperators. We coordinate on a scale and scope no other species (nor AI) can match. What makes this possible? 🧵

www.pnas.org/doi/10.1073/...

Really pumped for my Oral presentation on this work today!!! Come check out the RL session from 3:30-4:30pm in West Ballroom B

You can also swing by our poster from 4:30-7pm in West Exhibition Hall B2-B3 # W-713

See you all there!

I'll be at ICML next week! If anyone wants to chat about single/multi-agent RL, continual learning, cognitive science, or something else, shoot me a message!!!

Oral @icmlconf.bsky.social !!! Can't wait to share our work and hear the community's thoughts on it, should be a fun talk!

Can't thank my collaborators enough: @cogscikid.bsky.social y.social @liangyanchenggg @simon-du.bsky.social @maxkw.bsky.social @natashajaques.bsky.social

For the paper and code, see kjha02.github.io/publication/...

Thank you so much to my collaborators @cogscikid.bsky.social @liangyanchenggg @
@simon-du.bsky.social @maxkw.bsky.social @natashajaques.bsky.social for making the first publication of my PhD a fun one!!!

The big takeaway: Environment diversity > Partner diversity

Training across diverse tasks teaches agents how to cooperate, not just whom to cooperate with. This enables zero-shot coordination with novel partners in novel environments, a critical step toward human-compatible AI.

GitHub - wcarvalho/nicewebrl: Python library for easily making web Apps to compare humans and AI Python library for easily making web Apps to compare humans and AI - wcarvalho/nicewebrl

Our work used NiceWebRL, a Python-based package we helped develop for evaluating Human, Human-AI, and Human-Human gameplay on Jax-based RL environments!

This tool makes crowdsourcing data for CS and CogSci studies easier than ever!

Learn more: github.com/wcarvalho/ni...

Why do humans prefer CEC agents? They collide less and adapt better to human behavior.
This increased adaptability reflects general norms for cooperation learned across many environments, not just memorized strategies.

Human studies confirm our findings! CEC agents achieve higher success rates with human partners than population based methods like FCP and are rated qualitatively better to collaborate with than the SOTA approach (E3T) despite never having seen the level during training.

Using empirical game theory analysis, we show CEC agents emerge as the dominant strategy in a population of different agent types during Ad-hoc Teamplay!

When diverse agents must collaborate, the CEC-trained agents are selected for their adaptability and cooperative skills.

The result? CEC agents significantly outperform baselines when collaborating zero-shot with novel partners on novel environments.

Even more impressive: CEC agents outperform methods that were specifically trained on the test environment but struggle to adapt to new partners!

We built a Jax-based procedural generator creating billions of solvable Overcooked challenges.

Unlike prior work studying only 5 layouts, we can now study cooperative skill transfer at unprecedented scale (1.16e17 possible environments)!

Code available at: shorturl.at/KxAjW

We introduce Cross-Environment Cooperation (CEC), where agents learn through self-play across procedurally generated environments.

CEC teaches robust task representations rather than memorized strategies, enabling zero-shot coordination with humans and other AIs!

Current AI cooperation algos form brittle strategies by focusing on partner diversity in fixed tasks.

I.e. they might learn a specific handshake but fail when greeted with a fist bump.

How can AI learn general norms that work across contexts and partners?

Our new paper (first one of my PhD!) on cooperative AI reveals a surprising insight: Environment Diversity > Partner Diversity.

Agents trained in self-play across many environments learn cooperative norms that transfer to humans on novel tasks.

shorturl.at/fqsNN%F0%9F%...

That's true! I think the significance of not assigning the same meaning to symbols may only matter when we're interacting with the agent, but definitely room to explore what we mean by "understand" here and how anyone can learn to grasp the full affordances of objects with underspecified properties!

I guess, but aren't RL agents technically "understanding the world" just through the lens of decision making for a given problem (the MDP they're solving is their world)? Or do you mean Nethack requires having good priors about the real world that brute force alone would take too long to get?

Can it solve nethack?

Really excited to present my work this Sunday @NeurIPS on how we might approach training a generalist agent capable of cooperation at scale: coordinating with many novel partners on many novel tasks has never been easier!

Come by the IMOL workshop to check it out and chat more!

Also on Sunday, Kunal Jha @kjha02.bsky.social l will be presenting his recent work InfiniteKitchen: Cross-environment Cooperation for Zero-shot Multi-agent Coordination at the Intrinsically Motivated Open-ended Learning workshop imol-workshop.github.io

thinking of calling this "The Illusion Illusion"

(more examples below)