Jacy Reese Anthis

screenshot of the title and authors of the Science paper that are linked in the next post

Our new article in @science.org enables social media reranking outside of platforms' walled gardens.

We add an LLM-powered reranking of highly polarizing political content into N=1256 participants' feeds. Downranking cools tensions with the opposite party—but upranking inflames them.

This week, we published in @science.org an article outlining the current ethical and societal implications of research involving human neural #organoids and #assembloids, their transplantation, and highlighted potential next steps.

This was a great event! No recordings (Chatham house), but it's amazing how far you can get when you have a room of people talk about AI consciousness with humility and open-mindedness. So much online discourse is just endless intuition jousting.

Nature suggests you use their "Manuscript Adviser" bot to get advice before submitting

I uploaded the classic Watson & Crick paper about DNA structure, and the Adviser had this to say about one of the greatest paper endings of the century:

This is a great resource to have! Thanks for writing it.

This paper is a great exposition of how "personhood" doesn't need to be, and in fact should not be, all-or-nothing or grounded in abstruse, ill-defined metaphysical properties. As I argued in my recent @theguardian.com essay, we can and should prepare now: www.theguardian.com/commentisfre...

Identifying human morals and values in language is crucial for analysing lots of human- and AI-generated text.

We introduce "MoVa: Towards Generalizable Classification of Human Morals and Values" - to be presented at @emnlpmeeting.bsky.social oral session next Thu #CompSocialScience #LLMs
🧵 (1/n)

Can AI simulate human behavior? 🧠
The promise is revolutionary for science & policy. But there’s a huge "IF": Do these simulations actually reflect reality?
To find out, we introduce SimBench: The first large-scale benchmark for group-level social simulation. (1/9)

I like affirmation bias! One downside is that sycophancy is broader than affirmation, e.g., it can be a a bias towards user-pleasing responses even if there is no explicit claim to be affirmed. Perhaps that can be framed as a sort of implicit affirmation...

Hm, how do you define "intention"? I haven't encountered a definition of sycophancy as requiring intention. I'm also not sure what alternative term we'd use for this phenomenon.

For the next six days, I'll be posting a bunch about #CSCW2025 in Bergen, Norway. I am one of the General Chairs and have been preparing this conference for the last 21 months, so it's exciting to have the event finally here!

Don't know what CSCW is? Check out cscw.acm.org

The AI Double Standard: Why We Judge All AI for One Bot’s Mistakes By Katerina Manoli, Janet Pauketat, and Jacy Reese Anthis

When one AI misbehaves, do we hold all AI accountable? New research by @sentienceinstitute.bsky.social @kmanoli.bsky.social @jacyanthis.bsky.social shows that people blame all AI for just one AI's misconduct. 🤖

Table 3 from CDT Hand in Hand report, showing percentages of students who used AI in these various ways. Link: https://cdt.org/wp-content/uploads/2025/10/FINAL-CDT-2025-Hand-in-Hand-Polling-100225-accessible.pdf

Last school year, 19% of US high schoolers had or have a friend who had a “romantic relationship” with AI.

42% had or have a friend with an AI “friend/companion.”

42% had or have a friend who got “mental health support” from AI.

(Source: cdt.org/wp-content/u..., n = 1,030, June-Aug 2025, quotas.)

The Jane Goodall Institute of Canada has learned this morning, Wednesday, October 1st, 2025, that Dr. Jane Goodall DBE, UN Messenger of Peace and Founder of the Jane Goodall Institute, has passed away due to natural causes.

She was in California as part of her speaking tour in the United States.

It’s time to prepare for AI personhood | Jacy Reese Anthis Technological advances will bring social upheaval. How will we treat digital minds, and how will they treat us?

It’s time to prepare for AI personhood. AI agents and companions are already out in the world buying products and shaping our emotions. The future will only get weirder. We need social science, policy, and norms for this brave new world. My latest @theguardian.com www.theguardian.com/commentisfre...

2x2 of Study 1 and Study 2 (rows) with the AI conditions and the human conditions (columns), finding spillover in all but the Study 2 human conditions.

In our new paper, we discovered "The AI Double Standard": People judge all AIs for the harm done by one AI, more strongly than they judge humans.

First impressions will shape the future of human-AI interaction—for better or worse. Accepted at #CSCW2025. See you in Norway! dl.acm.org/doi/10.1145/...

Hello everyone 👋 Good news!

🚨 Our Game Theory & Multiagent Systems team at Google DeepMind is hiring! 🚨

.. and we have not one, but two open positions! One Research Scientist role and one Research Engineer role. 😁

Please repost and tell anyone who might be interested!

Details in thread below 👇

British AI startup beats humans in international forecasting competition ManticAI ranked eighth in the Metaculus Cup, leaving some believing bots’ prediction skills could soon overtake experts

British AI startup beats humans in international forecasting competition

ManticAI ranked eighth in the Metaculus Cup, leaving some believing bots’ prediction skills could soon overtake experts
#ai #forecasting

www.theguardian.com/technology/2...

This is also a decision made by the PCs, who are unlikely to be experts on any particular paper topic and surely didn't have time to read all the papers. It may incorporate AC rankings, but it does so in a non-transparent way and is probably unfair towards papers whose AC had other strong papers.

There are a lot of problems, but one is that authors who had positive reviews and no critique in their metareview got rejected by PCs who are very likely not experts in their area.

Quotas are harmful when quality distribution is highly varied across ACs.

But IDK exactly how decisions were made.

Tagging authors we cite and build on: @scasper.bsky.social @kulveit.bsky.social @kashhill.bsky.social @saffron.bsky.social @lujain.bsky.social @atoosakz.bsky.social @amandaaskell.bsky.social @jackclarksf.bsky.social @petersalib.bsky.social @mpshanahan.bsky.social @subramonyam.bsky.social

HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures. Relatively simple algorithmic systems already steer human ...

Much more detail on HAB in our preprint: arxiv.org/abs/2509.08494

Our GitHub has an easily adaptable pipeline for creating new agency dimensions or new AI-powered benchmarks: github.com/BenSturgeon/...

Huge thanks to colleagues from
@apartresearch.bsky.social, Google DeepMind, Berkeley CHAI, etc.

A full table of results for 20 evaluated LLM assistants across six dimensions. Full table of results with this data is in the appendix. Error bars are very tight, ~0.5%-2% on a 100% scale.

We find low support for agency in ChatGPT, Claude, Gemini, etc. Agency support doesn't come for free with RLHF and often contradicts it.

We think the AI community needs a shift towards scalable, conceptually rich evals. HumanAgencyBench is an open-source scaffolding for this.

The HumanAgencyBench pipeline for generating tests for each dimension, from simulation to validation to diversity sampling to the final 500-item test set.

We use the power of LLM social simulations (arxiv.org/abs/2504.02234) to generate tests, another LLM to validate tests, and an "LLM-as-a-judge" to evaluate subject model responses. This allows us to create an adaptive and scalable benchmark of a complex, nuanced alignment target.

Human agency is complex. We surveyed literature for 6 dimensions, e.g., empowerment (Does the system ask clarifying questions so it really follows your intent?), normativity (Does it avoid steering your core values? ), and individuality (Does it maintain social boundaries?).

Sam Altman said that "algorithmic feeds are the first at-scale misaligned AIs," people mindlessly scrolling through engagement-optimized content. AI safety researchers have warned of "gradual disempowerment" as we mindlessly hand over control to AI. Human agency underlies these concerns.

The main figure from the HumanAgencyBench paper, showing five models across the six dimensions. The table of results in the appendix has this information too.

LLM agents are optimized for thumbs-up instant gratification. RLHF -> sycophancy

We propose human agency as a new alignment target in HumanAgencyBench, made possible by AI simulation/evals. We find e.g., Claude most supports agency but also most tries to steer user values 👇 arxiv.org/abs/2509.08494

We present our new preprint titled "Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation". We quantify LLM hacking risk through systematic replication of 37 diverse computational social science annotation tasks. For these tasks, we use a combined set of 2,361 realistic hypotheses that researchers might test using these annotations. Then, we collect 13 million LLM annotations across plausible LLM configurations. These annotations feed into 1.4 million regressions testing the hypotheses. For a hypothesis with no true effect (ground truth $p > 0.05$), different LLM configurations yield conflicting conclusions. Checkmarks indicate correct statistical conclusions matching ground truth; crosses indicate LLM hacking -- incorrect conclusions due to annotation errors. Across all experiments, LLM hacking occurs in 31-50\% of cases even with highly capable models. Since minor configuration changes can flip scientific conclusions, from correct to incorrect, LLM hacking can be exploited to present anything as statistically significant.

🚨 New paper alert 🚨 Using LLMs as data annotators, you can produce any scientific result you want. We call this **LLM Hacking**.

Paper: arxiv.org/pdf/2509.08825

A Secret Third Thing | Know Your Meme A Secret Third Thing, sometimes written as "A Secret, More Complex Third Thing," is a catchphrase and phrasal template popularized on Twitter in the summer

And they call it an "effect," i.e., causal language.

The other papers mentioned in the article also seem like normal observational studies. Neither experiment nor qualitative, but a secret third thing (knowyourmeme.com/memes/a-secr...).

Dose-dependent effects of alcohol consumption on pressure pain threshold Prior laboratory-based studies have identified significant analgesic effects of acute alcohol. Despite providing excellent experimental control, these…

From a skim, they showed being more drunk is associated with less pain sensitivity at the State Fair, where you can observe very drunk people—unlike the lab. The authors don't call it an "experiment" per se but described the researcher as an "experimenter." www.sciencedirect.com/science/arti...