Florent Delgrange

Mind the GAP!

we've had a few works proposing techniques for enabling scaling in deep rl, such as MoEs, tokenization, & sparse training.
ghada sokar and i looked further & found a bit more clarity into *what* enables scaling, leading us to simpler solutions (see GAP in figure)!
1/

PhD Position in Causal Agent-based Modelling of Complex Social Systems Join this exciting interdisciplinary research project at the Centre for Complex Systems Studies and study causal agent-based modelling!

🎓 PhD position available!

Join our interdisciplinary research project on causal agent-based modelling!

🔍 Looking for curious minds with a MSc degree (or near to completing one) in CS/AI/related fields.
📍 Location: Utrecht University, NL
🗓️ Deadline: 16 June 2025
📩 Info: www.uu.nl/en/organisat...

5/ This work is a first step towards improving the reliability of learning agents by unifying RL and reactive synthesis.

I'm very grateful to my co-authors for this great collaboration!
Check my blogpost for more insights!

I'll present the paper in a few weeks at @aamasconf.bsky.social

4/ This approach allows for
- a separation of concerns
- formal guarantees through (PAC) bounds on both the world model quality and policy performance
- reusability and scaling to domains where synthesis was not applicable

3/ Given the map, the learned low-level models/policies, and a formal specification describing what the agent should do or not, we apply reactive synthesis to obtain a high-level planner.

2/ We consider scenarios where a "map" describing the environment's high-level structure can be provided as a graph. Each vertex is a "room," where we apply RL to get low-level policies. In addition, we learn a world model of each room that can be formally verified.

1/ RL enables agents to learn efficient policies in complex domains, but lacks formal guarantees — a challenge in high-stakes scenarios. In contrast, when the environment model is accessible, reactive synthesis offers formal guarantees, but struggles to scale.

Composing Reinforcement Learning Policies, with Formal Guarantees | Florent Delgrange Synthesizing controllers in large domains from verified world models and reinforcement learning policy composition.

Happy to share our new paper (AAMAS 2025)!
We combine reinforcement learning 🤖🧠 & reactive synthesis ⚙️ for learning scalable safe policies in complex tasks with formal guarantees.

📑paper: arxiv.org/abs/2402.13785
✍️blogpost: delgrange.me/post/composi...

A thread🧵⤵️

Still 7 days to submit your work to the ALA workshop at AAMAS! We welcome full papers, work in progress, and 2-page abstracts of recently published journal papers. All the info is available at ala-workshop.github.io.

📌

Thanks!

Hi, I'd be pleased if you could add me too if there's still room :-)

Thanks!

Hey, I’d love to be added!

Empirical Design in Reinforcement Learning Empirical design in reinforcement learning is no small task. Running good experiments requires attention to detail and at times significant computational resources. While compute resources available p...

Another must read for reinforcement learning. Answers many key questions for researchers;
-Do I need multiple training runs?
-How do I report model confidence?
-And a great section on common mistakes to fend off reviewer 2
🧪
#DRL
#reinforcementlearning
#AI
arxiv.org/abs/2304.01315

In addition to the Deep Learning Theory starter pack, I've also put together a starter pack for Reinforcement Learning Theory. Let me know if you'd like to be included or suggest someone to add to the list!

go.bsky.app/LWyGAAu

hey! working on RL and formal verification, do you mind adding me? :-)

If you're an RL researcher or RL adjacent, pipe up to make sure I've added you here!
go.bsky.app/3WPHcHg