Malikeh Ehghaghi

Next week at NeurIPS 2025 in San Diego, we’re bringing this story to the community. Join us if you are interested. See you soon💜

Despite its simplicity and scalability, model merging is still underappreciated.
But it fundamentally challenges how we think about:
🧠 training
🧩 specialization
🌐 generalization
It's one of the few ideas that redefine what "training a model" can mean.

That sparked a deep dive through merging papers:
🔹 Git Re-Basin
🔹 TIES
🔹 DARE-TIES
🔹 Fisher-weighted averaging
… and later working on MergeKit, the elegant implementation by @chargoddard.bsky.social.

My first encounter with this idea was back in Jan 2024 at
Arcee AI, when my colleague said:
"Have you heard of model merging?
Imagine no GPUs… just weights… and boom — a multitasker."
It truly felt like magic.

But model merging changes the story.
With a simple interpolation in weight space:
✨ no retraining
✨ no shared pipeline
✨ no GPU time
You can fuse their strengths into a single multitask model.
A collaboration that survives time zones.

Ever wondered what happens when experts train models across the world 🌍:
🇯🇵 multilingual in Tokyo
🇨🇦 coding in Toronto
🇩🇪 math in Berlin
…all on their own schedules, no overlap, no coordination?
In classic deep learning, these stay 3 separate models forever.

cc: @vectorinstitute.ai, @utoronto.ca, @hf.co, @edoardo-ponti.bsky.social, @sarahooker.bsky.social, @chargoddard.bsky.social

Excited to be at @neuripsconf.bsky.social next week co-presenting our tutorial: "Model Merging: Theory, Practice, and Applications" 🔥

Proud to do this with my PhD advisor, Colin Raffel, our postdoc research fellow @mcicc.bsky.social, and an incredible panel of speakers 💙

#NeurIPS2025 #ModelMerging

💡 Should LLMs reason in natural language or symbolic language? Why not both?
In our latest WiAIR episode, we sit down with Simeng Han to discuss HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning. (1/8 🧵)
#WiAIR

We’re thrilled to congratulate Dr. Abhilasha Ravichander (@lasha.bsky.social) and her team for receiving the Outstanding Paper Award at #acl2025 for their work titled "HALoGEN: Fantastic LLM Hallucinations and Where to Find Them"! 🏆✨

#ACL #LLMs #Hallucination #WiAIR #WomenInAI

🎙️ New WiAIR Podcast Ep is LIVE!

We dive deep into the future of decentralized AI, scaling beyond scaling laws, and building inclusive, multilingual LLMs with Wanru Zhao.

#WiAIR #DecentralizedAI #FederatedLearning #WomenInAI

Let's talk robotics!💜🤖

What if you could automatically detect and remove mislabeled image-caption pairs?🤯
🎙️ In our new WiAIR episode, we sat down with Aparna Balagopalan @AparnaBee about LEMoN, a method that detects label errors in image-text datasets AUTOMATICALLY. 🧵(1/10)
#WiAIR #WomenInAI

🎓 Is research in industry really that different from academia?

In this short video, Aparna - a current PhD student at MIT - shares the most valuable insights she gained from doing research in industry, and how it affected her career path💡

#PhDLife #AIResearch #WomenInTech

"Achieving Failure" in ML for Healthcare? 🩺 💊 🥼
#WiAIR #WomenInTech #MLforHealthcare

YouTube video by Women in AI Research WiAIR Responsible AI for Health, with Aparna Balagopalan

🚀 Our new episode is LIVE! 🎙️
In Episode 3, we talk with @aparnabee.bsky.social about:

🏥⚠️ Unique challenges of applying AI in medical contexts
📊🧑🏽‍🤝‍🧑🏻 Data quality and bias
👩‍⚕️🩺 Importance of collaboration with clinicians

Watch and subscribe!
youtu.be/DEdJltlFg4I

#MLforHealth #WiAIR #WomenInAI

We are back with the third episode of the
Women In AI Research (WiAIR) podcast, launching Wednesday, April 23rd. This week, we’ll be joined by @aparnabee.bsky.social to discuss the topic of "Responsible AI for Health".

#WiAIR #ResponsibleAI #AIforHealthcare

Another Relevant Paper: arxiv.org/pdf/2403.03874

💡If AI rewrites your voice, is it still your voice?
We had the pleasure of hosting
@CurriedAmanda
in our latest episode, where she walked us through her impactful research on “Impoverished Language Technology: Social Class in NLP.”

#WiAIR #SocialBias #AIFairness

Misleading Empathy:
AI can reflect your emotions — even the dark ones.
It might cheer for pride…
Even when it’s pride in something terrible. 😶‍🌫️

That’s misleading empathy. And it’s dangerous. ⚠️

#ResponsibleAI #WiAIR #WomenInAI #AIEthics

🎙️ In our latest WiAIR episode, we had the pleasure of diving into the thought-provoking work of our brilliant guest Amanda Cercas Curry, “Angry Men, Sad Women: Large Language Models Reflect Gendered Stereotypes in Emotion Attribution”

#WiAIR #GenderBias #WomenInAI #AIEthics

Who defines the "standard" in AI? And what if you don’t fit in?

In our upcoming episode, we join Dr. Amanda Cercas Curry to explore the power dynamics shaping AI development and language - don’t miss it.

#WiAIR #WomenInAI

#WomenInAI #WomenInTech #ResponsibleAI #AIEthics #DataFeminism #WiAIR

🚀 Exciting news! The second episode of the WiAIR Podcast is dropping soon! We’ll be talking about the social and ethical impact of biased AI—why it matters and how researchers are working to build fairer, more inclusive systems. Don’t miss this important conversation! 🎙️✨

Nouha Dziri speaks about the reasoning capabilities of transformers. "Is it true reasoning?"

Watch the full episode here: youtu.be/SL8reoM8pGE

#WomenInAi #WiAIR #AIResearch

The wait is almost over! Our teaser is out, and there's just 1 day left until the big launch! ⏳💜

#womeninai

Following up on my last post - it's time for the big reveal! 🎉

Thrilled to announce that @j-novikova-nlp.bsky.social and I are launching a podcast called Women in AI Research! We're excited to bring you inspiring stories from women in AI.

Follow @wiair.bsky.social for all the updates!

#womeninai

📢 Big announcement coming up later this week!
My friend @j-novikova-nlp.bsky.social and I have been working on something very special. Can't wait to reveal what we have been up to. Stay tuned! ✨

#WomenInAI