Agam Goyal

Excited to be in Albuquerque for #NAACL2025 next week!

I'll be presenting our work on SLMs for governing online communities on 4pm Wednesday, April 30th in Oral Session D (CSS.1) in Room Ruidoso .

Come by if you’re interested in LLM for governance or computational modeling of online communities!

25.04.2025 20:38 — 👍 5    🔁 0    💬 0    📌 0

🔗 Read the full paper here: arxiv.org/pdf/2410.13155

Thanks to my collaborators Xianyang Zhan, Yilun Chen,
@ceshwar.bsky.social, and @kous2v.bsky.social for their help!

#NAACL2025

[7/7]

25.04.2025 20:35 — 👍 0    🔁 0    💬 0    📌 0

⚖ Practical implications:

- SLMs are lighter and cheaper to deploy for real-time moderation
- They can be adapted to community-specific norms
- They’re more effective at triaging potentially harmful content
- The open-source nature provides more control and transparency

[6/7]

25.04.2025 20:35 — 👍 0    🔁 0    💬 1    📌 0

🌐 Cross-community transfer learning also shows promise! Fine-tuned SLMs can effectively moderate content in communities they weren’t explicitly trained on.

This has major implications for new communities and cross-platform moderation techniques.

[5/7]

25.04.2025 20:35 — 👍 0    🔁 0    💬 1    📌 0

📊 Our error analysis reveals:

- SLMs excel at detecting rule violations in short comments
- LLMs tend to be more conservative about flagging content
- LLMs perform better with longer comments where context helps determine appropriateness

[4/7]

25.04.2025 20:35 — 👍 0    🔁 0    💬 1    📌 0

💡 Key insight: SLMs adopt a more aggressive moderation approach, leading to higher recall but slightly lower precision compared to LLMs.

This trade-off is actually beneficial for platforms where catching harmful content is prioritized over occasional false positives.

[3/7]

25.04.2025 20:35 — 👍 0    🔁 0    💬 1    📌 0

🔍 We evaluated fine-tuned SLMs against LLMs across 15 popular Reddit communities. Results show SLMs surpass zero-shot LLMs with:

- 11.5% higher accuracy
- 25.7% higher recall
- Better performance on realistic imbalanced datasets

Even ICL didn’t help LLMs catch up!

[2/7]

25.04.2025 20:35 — 👍 0    🔁 0    💬 1    📌 0

🚨 New #NAACL2025 paper alert: “SLM-Mod: Small Language Models Surpass LLMs at Content Moderation”

We show that fine-tuned small language models outperform LLMs like GPT-4o for content moderation tasks, with higher accuracy and recall!

🔗 arXiv: arxiv.org/pdf/2410.13155

🧵[1/7]

25.04.2025 20:35 — 👍 4    🔁 0    💬 1    📌 3