Introducing 🥚EGGROLL 🥚(Evolution Guided General Optimization via Low-rank Learning)! 🚀 Scaling backprop-free Evolution Strategies (ES) for billion-parameter models at large population sizes
⚡100x Training Throughput
🎯Fast Convergence
🔢Pure Int8 Pretraining of RNN LLMs
hello 🦋 ! I'll be at NeurIPS next week presenting our work on using learnability to select levels for RL autocurricula. If you're there, I would love to chat about curricula and RL generalisation more broadly. Please DM if you'd like to grab a coffee :)
06.12.2024 10:24 — 👍 5 🔁 1 💬 0 📌 0The good reviewer party™, a party at conferences for reviewers whose ACs said they did a great job. ML operates on FOMO so I think it'd work
24.11.2024 20:09 — 👍 97 🔁 8 💬 5 📌 1Hello BlueSky! Joao Henriques (joao.science) and I are hiring a fully funded PhD student (UK/international) for the FAIR-Oxford program. The student will spend 50% of their time @UniofOxford and 50% @MetaAI (FAIR) in London, while completing a DPhil (Oxford PhD). Deadline: 2nd of Dec AOE!!
23.11.2024 14:35 — 👍 51 🔁 18 💬 1 📌 4