Angelina Wang

Grateful to win Best Paper at ACL for our work on Fairness through Difference Awareness with my amazing collaborators!! Check out the paper for why we think fairness has both gone too far, and at the same time, not far enough aclanthology.org/2025.acl-lon...

Angelina Wang presents the benchmark with Jewish synagogue hiring as an example.

@angelinawang.bsky.social presents the "Fairness through Difference Awareness" benchmark. Fairness tests require no discrimination...

but the law supports many forms of discrimination! E.g., synagogues should hire Jewish rabbis. LLMs often get this wrong aclanthology.org/2025.acl-lon... #ACL2025NLP

Was beyond disappointed to see this in the AI Action Plan. Messing with the NIST RMF (which many private & public institutions currently rely on) feels like a cheap shot

Print screen of the first page of a paper pre-print titled "Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor" by Olteanu et al. Paper abstract: "In AI research and practice, rigor remains largely understood in terms of methodological rigor -- such as whether mathematical, statistical, or computational methods are correctly applied. We argue that this narrow conception of rigor has contributed to the concerns raised by the responsible AI community, including overblown claims about AI capabilities. Our position is that a broader conception of what rigorous AI research and practice should entail is needed. We believe such a conception -- in addition to a more expansive understanding of (1) methodological rigor -- should include aspects related to (2) what background knowledge informs what to work on (epistemic rigor); (3) how disciplinary, community, or personal norms, standards, or beliefs influence the work (normative rigor); (4) how clearly articulated the theoretical constructs under use are (conceptual rigor); (5) what is reported and how (reporting rigor); and (6) how well-supported the inferences from existing evidence are (interpretative rigor). In doing so, we also aim to provide useful language and a framework for much-needed dialogue about the AI community's work by researchers, policymakers, journalists, and other stakeholders."

We have to talk about rigor in AI work and what it should entail. The reality is that impoverished notions of rigor do not only lead to some one-off undesirable outcomes but can have a deeply formative impact on the scientific integrity and quality of both AI research and practice 1/

Alright, people, let's be honest: GenAI systems are everywhere, and figuring out whether they're any good is a total mess. Should we use them? Where? How? Do they need a total overhaul?

(1/6)

I’ll be at both FAccT in Athens and ACL in Vienna this summer presenting these works, come say hi :)

Screenshot of paper title and author: "Identities are not Interchangeable: The Problem of Overgeneralization in Fair Machine Learning" by Angelina Wang

2. 𝗿𝗮𝗰𝗶𝘀𝗺 ≠ 𝘀𝗲𝘅𝗶𝘀𝗺 ≠ 𝗮𝗯𝗹𝗲𝗶𝘀𝗺 ≠ … At FAccT 2025: Different oppressions manifest differently, and that matters for AI. Ex: neighborhoods segregate by race, but rarely by sex, shaping the harms we should target. arxiv.org/abs/2505.04038

Instead, we should permit differentiating based on the context. Ex: synagogues in America are legally allowed to discriminate by religion when hiring rabbis. Work with Michelle Phan, Daniel E. Ho, @sanmikoyejo.bsky.social arxiv.org/abs/2502.01926

Screenshot of paper title and author list: "Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs" by Angelina Wang, Michelle Phan, Daniel E. Ho, Sanmi Koyejo

1. 𝗳𝗮𝗶𝗿𝗻𝗲𝘀𝘀 ≠ 𝘁𝗿𝗲𝗮𝘁𝗶𝗻𝗴 𝗴𝗿𝗼𝘂𝗽𝘀 𝘁𝗵𝗲 𝘀𝗮𝗺𝗲. At ACL 2025 Main: We diagnose issues like Google Gemini’s racially diverse Nazis to be a result of equating fairness with racial color-blindness, erasing important group differences.

In the pursuit of convenient definitions and equations of fairness that scale, we have abstracted away too much social context, enforcing equality between any and all groups. In two new works, we push back against two pervasive and pernicious assumptions:

Have you ever felt that AI fairness was too strict, enforcing fairness when it didn’t seem necessary? How about too narrow, missing a wide range of important harms? We argue that the way to address both of these critiques is to discriminate more 🧵

My ‘woke DEI’ grant has been flagged for scrutiny. Where do I go from here? My work in making artificial intelligence fair has been noticed by US officials intent on ending ‘class warfare propaganda’.

The US government recently flagged my scientific grant in its "woke DEI database". Many people have asked me what I will do.

My answer today in Nature.

We will not be cowed. We will keep using AI to build a fairer, healthier world.

www.nature.com/articles/d41...

If you work in ML fairness, perhaps you tend to get asked similar sets of questions from ML-focused folks, such as what is the best definition or equation for fairness. For those interested, please read, and for those often asked these questions, feel free to pass on the site!

I've recently put together a "Fairness FAQ": tinyurl.com/fairness-faq. If you work in non-fairness ML and you've heard about fairness, perhaps you've wondered things like what the best definitions of fairness are, and whether we can train algorithms that optimize for it.

*Please repost* @sjgreenwood.bsky.social and I just launched a new personalized feed (*please pin*) that we hope will become a "must use" for #academicsky. The feed shows posts about papers filtered by *your* follower network. It's become my default Bluesky experience bsky.app/profile/pape...

I am excited to announce that I will join the University of Zurich as an assistant professor in August this year! I am looking for PhD students and postdocs starting from the fall.

My research interests include optimization, federated learning, machine learning, privacy, and unlearning.

Cutting $880 billion from Medicaid is going to have a lot of devastating consequences for a lot of people

Yes these are good points, and thanks for the pointer! But the trajectory does seem to be towards LLMs replacing human participants in certain cases. The presence of these companies, for instance, to me signal real world use: www.syntheticusers.com, synthetic-humans.ai

Yes! The preprint is available here: arxiv.org/abs/2402.01908

A screenshot of an email notifying Spotsylvania parents that funding for a program for youth with disabilities has been canceled

the richest man in the world has decided that your kids don't deserve special education programs

Large language models that replace human participants can harmfully misportray and flatten identity groups Nature Machine Intelligence - Large language models are being considered to simulate responses from participants of different backgrounds in computational social science experiments. Here it is...

Check it out here: rdcu.be/eabf1 Work with @jamiemmt.bsky.social and John P. Dickerson

Our results differ from work that affirmatively shows LLMs can simulate human participants. We test if LLMs can match the distribution of human responses — not just the mean — and use more realistic free responses instead of multiple choice. The details matter!

Training data phrases like “Black women” are more often used in text *about* a group rather than *from* a group, so that outputs to LLM prompts like “You are a Black woman” more resemble what out-group members think a group is like than what in-group members are actually like.

Our new piece in Nature Machine Intelligence: LLMs are replacing human participants, but can they simulate diverse respondents? Surveys use representative sampling for a reason, and our work shows how LLM training prevents accurate simulation of different human identities.

A poster advertising the first workshop on sociotechnical AI governance with a description of the workshop's core themes and faces of the organizers.

📢📢 Introducing the 1st workshop on Sociotechnical AI Governance at CHI’25 (STAIG@CHI'25)! Join us to build a community to tackle AI governance through a sociotechnical lens and drive actionable strategies.

🌐 Website: chi-staig.github.io
🗓️ Submit your work by: Feb 17, 2025

Ok #FAccT folks - I'm still looking around for people. Who am I missing?

go.bsky.app/EQF5Ne1

Benchmark suites instead of leaderboards for evaluating AI fairness The authors argue for the elimination of leaderboards in artificial intelligence (AI) fairness, because “fairness” cannot be measured as a single number. Instead, they propose the development and cura...

This was in collaboration with @aaronhertzmann.bsky.social and Olga Russakovsky. Check it out here: www.cell.com/patterns/ful...

Benchmark suites are different from what we see today, which are often collections based on availability and convenience. Instead, suites should be curated with the goal of completeness, and encourage friction by design to force contextual deliberation about model superiority.

There are numerous leaderboards for AI capabilities and risks, for example fairness. In new work, we argue that leaderboards are misleading when the determination of concepts like “fairness” is always contextual. Instead, we should use benchmark suites.