Abigail Jacobs

Wow! Sounds like a huge decision and a wonderful outcome. Good luck with the move!

Feeling so excited + grateful to be representing this paper at #ICML! Please stop by to talk about how to do more valid measurement for evaling gen AI systems!

Work led by the incomparable @hannawallach.bsky.social and @azjacobs.bsky.social as a part of Microsoft’s AI and Society initiative!!

“If ___ ran a mini nuclear power plant” seems like a strong vibe for the day

Opinion | Look Past Elon Musk’s Chaos. There’s Something More Sinister at Work. Everything is content.

As ever, Tressie McMillan Cottom has the most astute analysis of how to read Musk's behavior. www.nytimes.com/2025/02/12/o...

"The bureaucracy is an “unelected, fourth, unconstitutional branch of government, which has, in a lot of ways, currently, more power than any elected representative,” insisted Mr. Musk, who serves as an unelected appointee with vast reach across the government."

cool

At Oval Office, Musk Makes Broad Claims of Federal Fraud Without Proof (Gift Article) The billionaire, whose federal cost-cutting team has been operating in secrecy, asserted that he had uncovered waste and fraud across the bureaucracy, without providing evidence.

big day to submit an article on how "efficiency" is used to undermine legitimacy of the administrative state

www.nytimes.com/2025/02/11/u... (gift link)

Oh of course good people know each other

Thrilled to have you here regardless !

(Bonus rant for the enshittification of search. Me trying to fix my house, sample AI suggestions from recent tasks: “use vinegar.” “Not flammable since the 80s!”

…more clicks to experts: “OMFG do not fucking use vinegar.” “Yes still flammable so many people died that way”)

This! Not that individualizing the problem is the solution, but so many don’t know how big of a jump it is. Also: bring back lmgtfy (let me google that for you) as a design intervention

"there's a lot of qualitative work that goes into designing quantitative metrics" -- @azjacobs.bsky.social

"how do we translate between benchmark performance and what it will really be like to use a model" -- Su Lin Blodgett

"Overall, the starting list constitutes at best a narrow coverage of the risks the technology is likely to pose, & at worst a (partial) red herring poised to direct significant risk mitigation efforts to building on inappropriate foundations." @yjernite.bsky.social et al on the AI Act Systemic Risks

Evaluating Generative AI Systems is a Social Science Measurement Challenge Across academia, industry, and government, there is an increasing awareness that the measurement tasks involved in evaluating generative AI (GenAI) systems are especially difficult. We argue that thes...

Evaluating Generative AI Systems is a Social Science Measurement Challenge: arxiv.org/abs/2411.10939

TL;DR: The ML community would benefit from learning from and drawing on the social sciences when evaluating GenAI systems.

Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice We articulate fundamental mismatches between technical methods for machine unlearning in Generative AI, and documented aspirations for broader impact that these methods could have for law and policy. ...

New paper on why machine "unlearning" is much harder than it seems is now up on arXiv: arxiv.org/abs/2412.06966 This was a huuuuuge cross-disciplinary effort led by @msftresearch.bsky.social FATE postdoc @grumpy-frog.bsky.social!!!

The AI Interdisciplinary Institute at the University of Maryland (AIM) is hiring

40 new faculty members

in all areas of AI, particularly:
- accessibility,
- sustainability,
- social justice, and
- learning;

building on computational, humanistic, or social scientific approaches to AI.

>

👋🏻 (and see you in Vancouver this weekend?)

Today via listening to HeidiWorld and this : bsky.app/profile/azul...

Ok this is now the second time I have found this post hoping I would find Molly Lambert instead

Answering the question that won me the Ig Nobel prize: Are cats liquid? A liquid is traditionally defined as a material that adapts its shape to fit a container. Under certain conditions, cats seem to fit this definition.

www.pbs.org/newshour/amp...

Cartoon of a man eating milk and cookies. The first panel shows a glass half full of milk and some cookies. He says, “oh no I have some milk left. I need a cookie.” Next we see him say, “oh no half a cookie left. Need some milk” and pouring in more milk to his glass, next to a half eaten cookie. Now we see him looking at an empty plate and an empty glass, with the carton of milk and the platter of cookies just off to the side. Last panel, we see him pouring some milk into his empty glass, saying, “oh nooooo”

oh nooooo

Working on #bias & #discrimination in #NLP? Passionate about integrating insights from different disciplines? And do you want to discuss current limitations of #LLM bias mitigation work? 🤖
👋Join the workshop New Perspectives on Bias and Discrimination in Language Technology 4&5 Nov in #Amsterdam!

Not enough people are worried about this

I'm excited to share that the journal version of our paper, "An archival perspective on pretraining data", is now available (open access) from Patterns!

This project was led by @madesai.bsky.social, along with Irene Pasquetto, @azjacobs.bsky.social, and myself

www.cell.com/patterns/ful...

1/n

Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence ... By the authority vested in me as President by the Constitution and the laws of the United States of America, it is hereby ordered as follows:      Section 1.  Purpose.  Artificial intelligence (A...

New executive order just dropped, with lots of roles for sociotechnical work around AI with relatively fast turnaround… but also a bat flying around the White House logo because the executive order is also spooky 🦇🦇
www.whitehouse.gov/briefing-roo...

One of the many, many concepts we discuss in our upcoming report is Measurement Modeling as it relates to AI governance. Discussed this with @azjacobs.bsky.social earlier this year.

“ruined”

Sam Bankman-Fried, Effective Altruism, and the Question of Complicity Leaders of the social movement had no way to know that FTX would collapse. But they also had every incentive to ignore warnings.

The Sam Bankman-Fried of it all served as both a quieting moment of increasing hype… but I believe also a polarizing or radicalizing moment. https://www.newyorker.com/news/annals-of-inquiry/sam-bankman-fried-effective-altruism-and-the-question-of-complicity

And EA has a lot of utilitarian tech optimism, ranging from GiveWell to extreme altright logics. Roughly, the current “AI Safety” movement focuses on x risk and is getting intl attention, including in congress (Sam Altman), computer scientists who don’t/aren’t trained to think about society/harms

Ahh sorry. EA=effective altruism, and x risk refers to folks worried about the existential risk of AI (flippant description: Skynet bad; current algorithmic harms and structural inequities not so relevant). Emile P Torres has written on the epistemic foundations of x risk…

Spent the last week at #FAccT2023 - filled with some exciting work and exciting scholars on measurement, policy, social impacts emerging from ML. Happy to see this letter come out of it:
https://facctconference.org/2023/harm-policy.html