Neil Sehgal

Demographic cues (eg, names, dialect) are widely used to study how LLM behavior may change depending on user demographics. Such cues are often assumed interchangeable.

🚨 We show they are not: different cues yield different model behavior for the same group and different conclusions on LLM bias. 🧵👇

@pennldi.bsky.social @pennengineering.bsky.social @upenn.edu @pennchibe.bsky.social @pennmedicine.bsky.social @pminnovation.bsky.social

We hope these findings help health systems design more effective & scalable outreach to close preventive care gaps.

Thoughts welcome!
w/
@manueltonneau.bsky.social, @alison-buttenheim.bsky.social, @sharathg.bsky.social + team

Effect of Static vs. Conversational AI-Generated Messages on Colorectal Cancer Screening Intent: a Randomized Controlled Trial Large language model (LLM) chatbots show increasing promise in persuasive communication. Yet their real-world utility remains uncertain, particularly in clinical settings where sustained conversations...

💡 Bottom line:
🔹 LLMs can generate short, tailored, clinically appropriate messages that move intent particularly for lower-barrier behaviors.
🔹 These messages can fit into portals, texts, or mailed materials.
🔹 They’re low-cost & scalable.

Read more: arxiv.org/abs/2507.08211

📈 Results:
✅ Both AI formats significantly boosted stool-test intent (+13 pts) over expert material.
🩺 For colonoscopy, no AI advantage over expert material.

Surprisingly: single AI message ≈ chatbot – despite participants choosing to spend 3.5 minutes longer with the chatbot!

🧪 In a randomized trial (n=915), we compared:
1️⃣ No intervention
2️⃣ Expert-written patient materials
3️⃣ Single AI message
4️⃣ AI chatbot using motivational interviewing techniques

Outcome: intent to screen (stool test & colonoscopy) over 12 months.

🩺 Why it matters:
Colorectal cancer is the 2nd leading cause of cancer death in the US – but ~1/3 of eligible adults aren’t screened.

We need scalable, persuasive tools to close this gap. Can AI help?

🚨 New study!
We tested whether AI-generated messages – single static messages vs. conversations – can boost intent to screen for colorectal cancer.

Turns out: short, tailored AI messages outperform expert-written materials & match conversations, at a fraction of the time! 🧵👇

tagging some others who may be interested!
@chrisbail.bsky.social @hugoreasoning.bsky.social @tnfalpha.bsky.social @emollick.bsky.social @jennyallen.bsky.social @noelbrewer.bsky.social @julieleask.bsky.social @pminnovation.bsky.social @susanmichie.bsky.social

@pennldi.bsky.social @pennengineering.bsky.social @upenn.edu @pennchibe.bsky.social @pennmedicine.bsky.social

Shout-outs to inspiring work: @gordpennycook.bsky.social @dgrand.bsky.social @tomcostello.bsky.social, @jeffhancock.bsky.social @kobihackenburg.bsky.social on AI persuasion & others pushing this field forward 🙌

Thoughts welcome!
w/ @manueltonneau.bsky.social, @sharathg.bsky.social, @alison-buttenheim.bsky.social
+team

Conversations with AI Chatbots Increase Short-Term Vaccine Intentions But Do Not Outperform Standard Public Health Messaging Large language model (LLM) based chatbots show promise in persuasive communication, but existing studies often rely on weak controls or focus on belief change rather than behavioral intentions or outc...

Take-home: chatbots can nudge short-term intent, but add little over high-quality public-health materials. AI looks best as an add-on, not a replacement, in vaccine communications.
Link here: arxiv.org/abs/2504.20519

In a 15-day follow-up, gains from the reading arm stuck (+7 pts) while chatbot effects faded to ≈0. We also found no spill-over to flu/COVID or general vaccine hesitancy

In an RCT with 930 parents (US/CA/UK, with kids old enough for the HPV vaccine): chatbots raised vaccine intent vs. no intervention—but neither variant beat simply reading official public-health materials, with the conversational chatbot doing significantly worse.

🚨 New preprint on AI persuasion and public health 🚨

A 3-min conversation with GPT-4o nudged HPV-vax-hesitant parents (who obv knew it was AI & consented!)—BUT reading standard public-health material still outperformed chatbots in impact and longevity. Details below 👇

Exploring Socio-Cultural Challenges and Opportunities in Designing Mental Health Chatbots for Adolescents in India Mental health challenges among Indian adolescents are shaped by unique cultural and systemic barriers, including high social stigma and limited professional support. Through a mixed-methods study invo...

🚨 New CHI'25 EA paper! 🚨
How can we design culturally sensitive mental health chatbots for Indian adolescents? 🇮🇳📱
Our mixed-methods study reveals key design insights—from stigma to personalization.
Read it here: arxiv.org/abs/2503.08562
#CHI2025 #HCI #MentalHealth #AIforGood #India

Exploring Socio-Cultural Challenges and Opportunities in Designing Mental Health Chatbots for Adolescents in India Mental health challenges among Indian adolescents are shaped by unique cultural and systemic barriers, including high social stigma and limited professional support. Through a mixed-methods study invo...

Thanks for compiling this! Would you be able to add our paper on designing mental health chatbots for Indian adolescents? arxiv.org/abs/2503.08562

LDI Senior Fellows Neil Sehgal, Anish Agarwal, Raina Merchant, Sharath Chandra Guntuku, and colleagues analyzed Yelp reviews of health care facilities to asses how patient sentiment toward changed before and after COVID-19.