Textual Entailment and Token Probability as Bias Evaluation Metrics
Measurement of social bias in language models is typically by token probability (TP) metrics, which are broadly applicable but have been criticized for their distance from real-world langugage model u...
π’ New Preprint! π’
arxiv.org/abs/2510.07662
TL;DR: textual entailment and token probability behave very differently as bias evaluation metrics, even on the exact same bias definitions.
Also, I'm looking for summer 2026 research internships in responsible AI - please reach out!
10.10.2025 22:15 β π 1 π 0 π¬ 0 π 0
blasting Fates Warning in the Temple of Time
Director of Research, @dairinstitute.bsky.social
Roller derby athlete
https://alex-hanna.com
Book: thecon.ai
Pod+newsletter: https://dair-institute.org/maiht3k
πͺπ¬β²β²β²§β²£β²β²β²Μβ²β²β²β²
π³οΈββ§οΈ she/ΩΩ
πΈ @willtoft.bsky.social
Repπ @ianbonaparte.bsky.social
βπ½ β’ RACE AFTER TECHNOLOGY: Abolitionist Tools for the New Jim Code β’ VIRAL JUSTICE: How We Grow the World We Want β’ IMAGINATION: A Manifesto πwww.ruhabenjamin.com
Writer of Queer Computer | PhD candidate researching queer community online @RMIT
https://queercomputer.com
CS PhD candidate @UCBerkeley. Interested in multilingual and low-resourced language NLP + HCI. @SIGHPC CDS Fellow. Interned @MBZUAI. Current intern at DAIR
Website: https://hhnigatu.github.io
Research Interest: Responsible AI in Global South | RTs β endorsement
AI governance research @ Safe AI Forum
Fellow @ Yale Digital Ethics Center
Interests: AI, XR, most other tech, China, climbing, baking.
ex-Oxford, ex-SWE
My tech newsletter: https://www.ethicalreckoner.substack.com
Professor, Santa Fe Institute. Research on AI, cognitive science, and complex systems.
Website: https://melaniemitchell.me
Substack: https://aiguide.substack.com/
Professor of Psychology & Human Values at Princeton | Cognitive scientist curious about technology, narratives, & epistemic (in)justice | They/She π³οΈβπ
www.crockettlab.org
Personal Account
Founder: The Distributed AI Research Institute @dairinstitute.bsky.social.
Author: The View from Somewhere, a memoir & manifesto arguing for a technological future that serves our communities (to be published by One Signal / Atria
Computer Science PhD Student @ Stanford | Geopolitics & Technology Fellow @ Harvard Kennedy School/Belfer | Vice Chair EU AI Code of Practice | Views are my own
Prof (CS @Stanford), Co-Director @StanfordHAI, Cofounder/CEO @theworldlabs, CoFounder @ai4allorg #AI #computervision #robotics #AI-healthcare
Exploring ethical AI, automation & techβs impact on society | Making tech accessible to all | Unsponsored & unaffiliated | All views my own
https://ethicsbiastechblog.blogspot.com
#AI #TechForGood #EthicsInTech
The Center for Democracy & Technology. Shaping technology policy and architecture, with a focus on equity and justice. @cdteu.org⬠for our EU-based team.
https://cdt.org
Professor of Technology and Regulation, Oxford Internet Institute, University of Oxford https://tinyurl.com/3rkmbmsf
Humboldt Professor of Technology & Regulation, Hasso Plattner Institute https://tinyurl.com/47rkrt6c
Governance of Emerging Technologies
Making data & AI work for people & society.
Sign up for our fortnightly newsletter: https://nuffieldfoundation.tfaforms.net/149
Critical AI's new issue is out! https://read.dukeupress.edu/critical-ai/issue
Email us at criticalai@sas.rutgers.edu
Our website and blog: https://criticalai.org/
The AI Now Institute produces diagnosis and actionable policy research on artificial intelligence.
Find us at https://ainowinstitute.org/
Princeton computer science prof. I write about the societal impact of AI, tech ethics, & social media platforms. https://www.cs.princeton.edu/~arvindn/
BOOK: AI Snake Oil. https://www.aisnakeoil.com/
It is said that there may be seeming disorder and yet no real disorder at all
