Mansi Sakarvadia

Topology-Aware Knowledge Propagation in Decentralized Learning Topology-Aware Knowledge Propagation in Decentralized Learning

✨ TL;DR: Making decentralized learning aware of network topology boosts performance & resilience—vital for federated learning, edge AI, and IoT. Check out the paper for technical details & results! (6/6) mansisak.com/topology_awa...

19.06.2025 23:19 — 👍 1    🔁 0    💬 0    📌 0

📈 Experiments show significant improvements: faster knowledge propagation compared to traditional, topology-agnostic approaches in various real-world network settings. (5/6)

19.06.2025 23:19 — 👍 1    🔁 0    💬 1    📌 0

🧠 The system prioritizes information from well-connected or "influential" nodes, ensuring that high-quality knowledge quickly reaches more isolated or less-informed devices. No one gets left behind! (4/6)

19.06.2025 23:19 — 👍 1    🔁 0    💬 1    📌 0

🔗 This work introduces topology-aware knowledge propagation, which tailors how models and information are shared based on each device’s place in the network, leading to more effective learning overall. (3/6)

19.06.2025 23:19 — 👍 1    🔁 0    💬 1    📌 0

🌐 Decentralized learning often involves many devices (like edge or mobile) collaborating without a central server. However, the way these devices connect—the network topology—can heavily impact learning quality. (2/6)

19.06.2025 23:19 — 👍 0    🔁 0    💬 1    📌 0

🤝 New advances in decentralized learning! "Topology-Aware Knowledge Propagation in Decentralized Learning" proposes a novel way to improve how information flows in distributed machine learning systems. Let’s break it down! 🧵 (1/6) (arxiv.org/abs/2505.11760)

19.06.2025 23:19 — 👍 2    🔁 0    💬 1    📌 1

Mansi, Aswathy, and Nathaniel are presenting their poster.

Mansi presenting

Mansi, Aswathy, and Nathaniel are standing infront of a nice building

@mansisakarvadia.bsky.social, Aswathy (PhD students), and @nathaniel-hudson.bsky.social (postdoc) presented their work on identifying and ablating memorization in #LLMs at the 2024 MSLD workshop! 🎉 Their research is also accepted to ICLR 2025 — check it out: mansisak.com/memorization/

29.04.2025 03:31 — 👍 3    🔁 2    💬 0    📌 0

7/ 🙏 Special Thanks:
A huge shoutout to my incredible co-authors from multiple institutions for their contributions to this work:
Aswathy Ajith, Arham Khan, @nathaniel-hudson.bsky.social , @calebgeniesse.bsky.social, Yaoqing Yang, @kylechard.bsky.social , @ianfoster42.bsky.social, Michael Mahoney

04.03.2025 18:15 — 👍 2    🔁 0    💬 0    📌 0

6/ 🌍 Scalable Impact:
Our methods aren’t just for small models! We show that they scale effectively to larger LMs, providing robust memorization mitigation without compromising performance across different sizes of models. Exciting progress for real-world applications!

04.03.2025 18:15 — 👍 1    🔁 0    💬 1    📌 0

5/💡Best Approach:
Our proposed unlearning method, BalancedSubnet, outperforms others by effectively removing memorized info while maintaining high accuracy.

04.03.2025 18:15 — 👍 1    🔁 0    💬 1    📌 0

4/🧪 Key Findings:
Unlearning-based methods are faster and more effective than regularization or fine-tuning in mitigating memorization.

04.03.2025 18:15 — 👍 1    🔁 0    💬 1    📌 0

3/⚡Introducing TinyMem:
We created TinyMem, a suite of small, efficient models designed to help test and benchmark memorization mitigation techniques. TinyMem allows for quick experiments with lower computational costs.

04.03.2025 18:15 — 👍 1    🔁 0    💬 1    📌 0

2/ 🚨 Main Methods:
We test 17 methods—regularization, fine-tuning, and unlearning—5 of which we propose. These methods aim to remove memorized info from LMs while preserving performance.

04.03.2025 18:15 — 👍 1    🔁 0    💬 1    📌 0

1/🧵ICLR 2025 Spotlight Research on LM & Memorization!
Language models (LMs) often "memorize" data, leading to privacy risks. This paper explores ways to reduce that!
Paper: arxiv.org/pdf/2410.02159
Code: github.com/msakarvadia/...
Blog: mansisak.com/memorization/

04.03.2025 18:15 — 👍 4    🔁 2    💬 1    📌 1

Mitigating Memorization In Language Models Language models (LMs) can "memorize" information, i.e., encode training data in their weights in such a way that inference-time queries can lead to verbatim regurgitation of that data. This ability to...

Congratulations to @mansisakarvadia.bsky.social and team on their "Mitigating Memorization in Language Models" paper being selected as a Spotlight paper at ICLR '25! Check out the pre-print here: arxiv.org/abs/2410.02159

12.02.2025 17:28 — 👍 3    🔁 3    💬 0    📌 0

How can we stop LLMs from memorizing information? @mansisakarvadia.bsky.social's paper, which develops new methods for unlearning memorized information and introduces a suite of small LMs for testing these methods, was just accepted to ICLR '25!
arxiv.org/pdf/2410.02159

28.01.2025 02:14 — 👍 2    🔁 1    💬 1    📌 0

A Year In Review: C.S. PhD Student Edition | Mansi Sakarvadia A recap of my academic journey in 2024.

Reflecting on my 2024 PhD journey: passed my qualifying exam, spent the summer at Berkeley, mentored undergrad students, and tackled the fast pace of AI/ML research. It’s been a year of milestones and growth! Read more here: mansisak.com/blog/2025/ye... #PhDJourney #AIResearch

05.01.2025 03:13 — 👍 9    🔁 2    💬 0    📌 0