Dear ChatGPT, Am I the Asshole?
While Reddit users might say yes, your favorite LLM probably wonβt.
We present Social Sycophancy: a new way to understand and measure sycophancy as how LLMs overly preserve users' self-image.
@pranavkhadpe.bsky.social
Human-Computer Interaction researcher | Ph.D. candidate @cmuhcii https://pranavkhadpe.github.io/
Dear ChatGPT, Am I the Asshole?
While Reddit users might say yes, your favorite LLM probably wonβt.
We present Social Sycophancy: a new way to understand and measure sycophancy as how LLMs overly preserve users' self-image.
@narijohnson.bsky.social and Sanika have a FAccT paper about this (and also compiled cases of abandoned algorithms): arxiv.org/abs/2404.13802
18.03.2025 18:00 β π 5 π 1 π¬ 1 π 0Hi Derek, very happy to hear the project resonated with you! You might also enjoy: continueandpersist.org
24.02.2025 00:38 β π 0 π 0 π¬ 0 π 0This work was done with my mentees Olivia Xu (who worked on the initial deployment) and Anna Gu (who's working on the newer extension), and my advisors
Chinmay and Geoff. And it was made possible by a Google Award for Inclusion Research to Chinmay.
What next? We're working on a CMU-wide deployment of the extension. CMU spread the smiley so I think we've always been big on communicating delight :-)
28.11.2024 16:55 β π 0 π 0 π¬ 1 π 0Finally, who has claim to the appreciation when the work is collaborative?
Contributorsβ reactions were influenced by how much they felt they had contributed to the package that was thanked.
Hug Reports were also viewed as a valuable metric of use (ofc value determined by adoption). Today, contributors have little visibility into how their packages are used beyond download numbers. Insights into usage are valuable for raising funding and for project decisions.
28.11.2024 16:55 β π 1 π 0 π¬ 1 π 0But it is hard to be specific when praising.
Contributors ofc preferred notes that were more specific but users often didn't know what to say beyond "this is great". Newer version includes scaffolding for users.
For why praise is less differentiated see
Overall, contributors were surprised and happy to receive the Hug Reports.
In our field study, 18 users interacted with the extension for 3 weeks, resulting in messages of appreciation to 550 contributors, 26 of whom participated in subsequent research.
The extension builds on learnings from our field study of a more rudimentary version. We wanted to get a quick sense of how developers and users feel when sending Hug Reports.
Read about it in our #CSCW2025 paper: arxiv.org/abs/2407.20390
Some highlights below.
Extension: marketplace.visualstudio.com/items?itemNa...
It renders a π next to Python and JS package imports. Right-click it to send thanks to the package or specific modules.
Thanks are sent in batches to authors of 20 most recent commits of the corresponding repo/file, after manual moderation.
Maintaining open-source packages is a thankless job. Users report issues but stay silent when all works well.
We made a VS Code extension for "Hug" Reports. Try it (link inπ§΅) and send thanks today!
How do contributors feel receiving Hug Reports? Read our CSCW paper(link inπ§΅)