Why AI evals need to reflect the real world
Opinion: Rumman Chowdhury and Mala Kumar argue that we need better AI evaluations β and the infrastructure and investment to do them
"We must design, build and reward systems that complete work predictably in messy environments, rather than building ones that simply ace static quizzes under lab conditions," write @ruchowdh.bsky.social and Mala Kumar.
12.09.2025 18:23 β π 4 π 3 π¬ 0 π 0
Political neutrality, for an AI model, is simply βnot a thing,β Chowdhury said. βItβs not real.β
For example, she said, if you ask a chatbot for its views on gun control, it could equivocate by echoing both Republican and Democratic talking points, or it might try to find the middle ground between the two. But the average AI user in Texas might see that answer as exhibiting a liberal bias, while a New Yorker might find it overly conservative. And to a user in Malaysia or France, where strict gun control laws are taken for granted, the same answer would seem radical.
One problem with an EO requiring AI models to be politically neutral: There's no such thing, says @ruchowdh.bsky.social. www.washingtonpost.com/technology/2...
24.07.2025 16:59 β π 66 π 16 π¬ 2 π 1
AI companies are spending millions to get the laws they want.
They're not trying to cure cancer, or save America.
These companies want to make $100 billion overnight, and they're willing to sponsor dangerous laws to make it happen.
-----
More Perfect Union is an Emmy-winning, nonprofit newsroom w
What OpenAI Doesnβt Want You to Know
What OpenAI Doesnβt Want You to Know | βͺ@moreperfectunion.bsky.socialβ¬
with commentary from @bcmerchant.bsky.social, @smw.bsky.social, @theodoraskeadas.bsky.social⬠and @ruchowdh.bsky.social
www.youtube.com/watc...
#regulations #discrimination #AI #law
14.07.2025 18:30 β π 4 π 2 π¬ 0 π 0
Expert Views on Advancing AI for Good
Exclusive interviews with Rumman Chowdhury (Humane Intelligence), Eric Loeb (Salesforce) & Anna Koivuniemi (Google DeepMind)
#AIforGood βViews from the Summitβ series expands upon speakersβ on-stage presentations. We look forward to more of these interviews at the 2025 Global Summit in July. Donβt miss 3 standout conversations from 2024.
@humaneintelligence.bsky.social @ruchowdh.bsky.social
medium.com/startingupgo...
25.06.2025 15:57 β π 3 π 3 π¬ 0 π 0
This is a service to humanity
16.04.2025 10:36 β π 3 π 0 π¬ 0 π 0
Can you explain to me how submitting to this is helping? Do you think the Trump administration is going to take with seriousness and equal weight, the input of ethics groups?
07.04.2025 10:51 β π 0 π 0 π¬ 1 π 0
Observing the deepening faultlines in American society in the early 1920s, F Scott Fitzgerald guessed rightβ¦The novelβs prescience lies not in foretelling specific events but in diagnosing a culture where power enjoys impunity and cruelty rubs out its traces β a society run by careless people.
07.04.2025 03:20 β π 48 π 14 π¬ 1 π 2
I hope they feature Angela Saini who wrote the books on this years ago.
06.04.2025 15:39 β π 2 π 0 π¬ 0 π 0
Reposting this to make sure we take a look at who submitted to this farce and importantly, who didnβt.
06.04.2025 15:09 β π 16 π 5 π¬ 1 π 0
Or - hear me out
- look up whoβs speaking. Ed is great and I admire him but Iβve been working on ethics and AI before he cared about the topic.
06.04.2025 13:06 β π 5 π 0 π¬ 0 π 0
Thatβs how we opened it and that was the theme of the entire discussion
06.04.2025 13:04 β π 5 π 0 π¬ 0 π 0
Thats bc they know life on them streets; you donβt say no to a fresh meal and stable roof
05.04.2025 12:21 β π 31 π 0 π¬ 0 π 0
This tells us more about the writer and editors at WSJ than it does about anything else. Sorry their daddies didnβt love them enough.
05.04.2025 12:16 β π 52 π 8 π¬ 1 π 1
My *third* time on @scifri.bsky.social but first time recording in the NYC studio! Hear me and @willdouglasheaven.bsky.social bring AGI conversations to reality and talk about the issues that really matter, like who controls the power behind AI.
05.04.2025 12:12 β π 25 π 5 π¬ 0 π 0
One of the best parts of coming back to social media has been seeing your efforts - thank you!
23.03.2025 17:20 β π 8 π 0 π¬ 0 π 0
TESLA INSIDERS ARE DUMPING SHARES
EMPLOYEE STOCK TRADES IN THE LAST 3 MONTHS:
NUMBER OF SHARES SOLD:
745,228
NUMBER OF SHARES BOUGHT: 0
I see we have entered the finding out phase
22.03.2025 14:12 β π 52 π 10 π¬ 3 π 0
I am a commissioner at the Federal Trade Commission. Earlier today, the president attempted to illegally fire me. This is corruption, plain and simple. I will see the president in court. My full statement:
19.03.2025 01:40 β π 15005 π 4184 π¬ 251 π 180
"Chaos is not where good work happens." @ruchowdh.bsky.social at #SXSW 2025.
Dr. Rumman Chowdhury, CEO and Founder of Humane Intelligence, shared her concerns about Elon Muskβs impact on institutionsβfrom Twitter to the U.S. government.
techcrunch.com/2025/03/13/e...
15.03.2025 20:47 β π 12 π 5 π¬ 0 π 0
This article is great btw thank you
15.03.2025 14:36 β π 1 π 0 π¬ 0 π 0
Correct, it gave some philosophers a way to be paid to give talks and be on panels; it gave others a fig leaf to say big words while doing nothing.
15.03.2025 14:36 β π 1 π 0 π¬ 0 π 0
One would think
15.03.2025 14:34 β π 0 π 0 π¬ 0 π 0
Straight from elons mouth to the AISI.
15.03.2025 13:03 β π 1 π 0 π¬ 0 π 0
Public Comment Invited on Artificial Intelligence Action Plan
WASHINGTON, D.C. β President Trumpβs recent Artificial Intelligence (AI) Executive Order shows that this Administration is dedicated to Americaβs global
Thereβs a new call for contributions for the new AI action plan and I donβt think anyone who cares about ethics should contribute and be complicit. Iβm not looking forward to the slew of linked in clout-seeking strongly worded letters; donβt do it.
www.whitehouse.gov/briefings-st...
15.03.2025 13:02 β π 16 π 4 π¬ 4 π 1
Under Trump, AI Scientists Are Told to Remove βIdeological Biasβ From Powerful Models
A directive from the National Institute of Standards and Technology eliminates mention of βAI safetyβ and βAI fairness.β
βHuman flourishingβ has long been on my list of bullshit terms wrt AI - some memed Aristotle and it became the bullshit term of AI ethics - but now itβs a right wing dog whistle.
Keep an eye on how the AISI will be weaponized as the Trump/Musk AI Act is written. 1/
www.wired.com/story/ai-saf...
15.03.2025 12:56 β π 60 π 17 π¬ 6 π 4
YouTube video by Mala Kumar
Why Private Sector Solutions Won't Replace US Government Funding
I made a video about why the private sector wonβt replace US government funding. Feel free to like and share with those people in your life who still donβt get why privatization in all aspects of life wonβt work.
youtu.be/AUHr0IgPKO8?...
13.03.2025 14:36 β π 9 π 6 π¬ 0 π 0
A publication about the power and politics of transformative AI. Subscribe for free: http://transformernews.ai/subscribe
Philosopher and Pro-Rector at CEU Vienna. Director of Research, FWF Cluster of Excellence, 'Knowledge in Crisis'. Author of The Mechanical Mind, Elements of Mind, The Objects of Thought, Aspects of Psychologism, The Meaning of Belief www.timcrane.com
Sociotechnical network/systems thinker, visionary, inventor, pioneer | Author: @TechPolicyPress, FairPay | Nonresident Senior Fellow, Foundation for American Innovation
WSJ tech columnist. Dog person. Author of Arriving Today, an unfortunately timely book about the global system of trade we're currently flushing down the toilet: https://www.harpercollins.com/products/arriving-today-christopher-mims
Writer/co-writer: The Christophers (2025), The Spot (2026), Full Circle, No Sudden Move, Men in Black, the 3 Bill&Ted movies, Mosaic, Now You See Me 1&2, Charlie's Angels, Itβs Garry Shandlingβs Show, Laverne&Shirley. Fiction: Vanity Fair, The New Yorker
Entertaining & educational conversations about science, tech, + more. Hosted by Ira Flatow and Flora Lichtman. From WNYCStudios.
if you die in the game, you feel sad in real life
AI/ML infrastructure, security, SRE, k8s, robotics, co-ops, roguelikes, Battletech, guitar, cooking, gardening
π΄π³οΈβπ - en/es/δΈζ - Boston
For the little guy. Former FTC commissioner. Current Bad Bunny stan
We are a NONVIOLENT direct action group committed to opposing, disrupting, and defeating any government act that threatens democracy, equality, and our civil liberties.
https://www.riseandresist.org/
Mom, Investor, Tech & Physician Exec β βKick(ing) down doors for others to walk through.β Missy Elliott. Views & photos are my own.
For the past two years, Humane Intelligence has pioneered methods of system-level evaluation by operationalizing, designing, and implementing test methods to understand and mitigate frontier AI risk.
Learn more: www.humane-intelligence.org
PhD @ MIT. Prev: Google Deepmind, Apple, Stanford. π¨π¦ Interests: AI/ML/NLP, Data-centric AI, transparency & societal impact
π Nonfiction Author
π SFF Author (.5 of R.A. Sinn)
π Media Prof at AU in DC
πΏ Bassist/Producer/Composer
(jazz, reggae, worldbeat)
sinnreich.com
tinyurl.com/tslodata
tinyurl.com/ASCFYbook
linktree.com/wassako
Live in Chicago, write for Wired, got a great attitude
Send me tips: kate_knibbs@wired.com / Signal: kateknibbs.09
Former public defender, now U.S. Congresswoman, proudly serving the good people of Texasβ 30th Congressional District. βπΎ
i do comms and other things / still here for the apΓ©ro
I am American πΊπΈ, and I will always stand with Ukraine πΊπ¦.