Kale-ab Tessera

Reading group today at 2pm BST!

We are starting our NeurIPS series with Sable and Oryx, sequence models for scalable multi-agent coordination from the RL Research Team at InstaDeep. 🚀

Papers:
- Sable: bit.ly/3Lme7jH
- Oryx: bit.ly/47GJb4T

Meeting:
- bit.ly/3JoEbtU

📢 RL reading group Thursday @ 16:00 BST 📢

Speaker: Alex Lewandowski

Title: The World Is Bigger: A Computationally-Embedded Perspective on the Big World Hypothesis 🌍

Details: edinburgh-rl.github.io/reading-group

Refreshing to see posts like this compared to "we have 15 papers accepted at X" 🙌

None of our impactful papers have had an easy path through traditional venues.
Most cited paper? Rejected four times.
Most impactful paper? Poster at a conference.
But none of it matters because arxiv makes everything work

Great first couple of days at DLI @deeplearningindaba.bsky.social in Kigali 🇷🇼, some highlights include amazing talks talks by @verenarieser.bsky.social and Max Welling, great pracs and tuts, and of course the opening party ( before the rain 😢) 🎉 #DLI2025

We’re excited to unveil the first #DLI2025 lineup of tutorials and practicals:

✨ Machine Learning Foundations
✨ Generative Models & LLMs for African languages

All tutorial content will also be available online after the Indaba. Don’t miss out, subscribe here 👉 lnkd.in/eCgXRqsV

🙌🎉

🇨🇦 Heading to @rl-conference.bsky.social next week to present HyperMARL (@cocomarl-workshop.bsky.social) and Remember Markov (Finding The Frame Workshop).

If you are around, hmu, happy to chat about Multi-Agent Systems (MARL, agentic systems), open-endedness, environments, or anything related! 🎉

We are thrilled to announce our next keynote speaker
@wellingmax.bsky.social, Professor at the University of Amsterdam, Visiting Professor at Caltech and CTO & Co-Founder of CuspAI.
Catch his talk “How AI could transform the sciences” on August 18 at 4:30 PM GMT+2.
#DLI2025

RL reading group TODAY @ 15:00 BST 🔥

Speaker: Cam Allen (Postdoc, UC Berkeley)

Title: The Agent Must Choose the Problem Model

Details: edinburgh-rl.github.io/reading-group

Always nice to see when simpler methods + good evaluations > more complicated ones. 👌

Reading group is back for those interested in RL/MARL/agents/open-endedness and alike... First session today at 3pm BST, @mattieml.bsky.social is presenting the Simplifying TD learning/PQN paper. 🎉 Meeting link: bit.ly/4lfdaGR Sign up: bit.ly/40xNQDR

Hello world! This is the RL & Agents Reading Group

We organise regular meetings to discuss recent papers in Reinforcement Learning (RL), Multi-Agent RL and related areas (open-ended learning, LLM agents, robotics, etc).

Meetings take place online and are open to everyone 😊

This has happened to me too many times 🤦‍♂️ Also doesn't help that Jax and PyTorch use different default initialisations for dense layers.

Well done & well deserved!! 🎉🎉 It has been awesome to see this project evolve from the early days.

Edinburgh RL Reading Group Please add your details so that you can remain on the mailing list for the RL Reading Group.

The Edinburgh one will be back and running soon. We are just updating the website and other things. There is this form for people interested - forms.gle/DAbkpN9b4cUt...

Forgot to also add ⚡ quickstart link for people who like to experiment on notebooks: github.com/KaleabTesser...

Thanks for checking it out! 👍 Good point, there might be an interesting link between MoEs and hypernets. We used hypernets since they're simpler (no need to pick or combine experts), and maximally expressive (gen weights directly).

Lol yes, will had a .gitignore, missed it when copying things over.

HyperMARL: Adaptive Hypernetworks for Multi-Agent RL Adaptability to specialised or homogeneous behaviours is critical in cooperative multi-agent reinforcement learning (MARL). Parameter sharing (PS) techniques, common for efficient adaptation, often li...

🎯 TL;DR: HyperMARL is a versatile approach for adaptive MARL -- no changes to the RL objective, preset diversity, or seq. updates needed. See paper & code below!

Work with Arrasy Rahman, Amos Storkey & Stefano Albrecht.

📜: arxiv.org/abs/2412.04233
👩‍💻: github.com/KaleabTessera/HyperMARL

⚠️ Limitations (+opportunity): HyperMARL uses vanilla hypernets, which can inc. param. count esp. MLP hypernets. In RL/MARL this matters less (actor-critic nets are small), and params grow ~const with #agents, so scaling remains strong. Future work could explore chunked hypernets.

🔎 We also do ablations and see the importance of the decoupling and the simple initialisation scheme we follow.

📊 We validate HyperMARL across various diverse envs (18 settings; up to 20 agents) and find that it achieves competitive mean episode returns compared to NoPS, FuPS, and modern diversity-focused methods -- without using diversity losses, preset diversity levels or seq. updates.

💡To address the coupling problem, we propose 𝐇𝐲𝐩𝐞𝐫𝐌𝐀𝐑𝐋: a method that explicitly 𝐝𝐞𝐜𝐨𝐮𝐩𝐥𝐞𝐬 obs- and agent-conditioned gradients with hypernetworks. This means obs grad noise is avg. per agent (Zᵢ) before applying agent-cond. grads (Jᵢ) -- unlike FuPS, which entangles both.

Specialisation matrix game.

Performance and gradient interference plots.

🔬 We isolate FuPS’s failure in matrix games: shared policies struggle when agents need to act differently. Inter-agent gradient interference is at play -- especially when obs and agent IDs are 𝐜𝐨𝐮𝐩𝐥𝐞𝐝. Surprisingly, using only IDs (no obs) performed better and reduced interference.

❓Existing methods add a diversity loss, use sequential updates or require knowing the optimal task diversity level beforehand. These can be hard to tune or inefficient. We ask: can shared policies adapt without any of the above?

⚖️ 𝐖𝐡𝐚𝐭’𝐬 𝐭𝐡𝐞 𝐢𝐬𝐬𝐮𝐞? In MARL, optimal performance requires representing the right behaviours. Separate networks per agent (NoPS) enable agent specialisation but is costly & sample-inefficient; shared networks (FuPS) are efficient but lack agent diversity/specialisation.

📜🤖 Can a shared multi-agent RL policy support both specialised & homogeneous team behaviours -- without changing the learning objective, requiring preset diversity levels or sequential updates? Our preprint “𝘏𝘺𝘱𝘦𝘳𝘔𝘈𝘙𝘓: 𝘈𝘥𝘢𝘱𝘵𝘪𝘷𝘦 𝘏𝘺𝘱𝘦𝘳𝘯𝘦𝘵𝘸𝘰𝘳𝘬𝘴 𝘧𝘰𝘳 𝘔𝘶𝘭𝘵𝘪-𝘈𝘨𝘦𝘯𝘵 𝘙𝘓” explores this!

Ah nice, looks like a fun competition!

For many, the Indaba was their first exposure to ML/AI, and this helps give more people that opportunity. Please donate if you can: gofund.me/61b012e4

🪘The 2024 Impact Report is here!
Our last Indaba’s theme, Xam Xamlé (Wolof for "To Gather Knowledge and Share It"), beautifully reflected our mission: Empowering and educating through African AI. Read our report 🔗https://deeplearningindaba.com/blog/2025/04/xam-xamle-our-latest-indaba-impact-report/