Joon Sung Park

Thank you so much for the kind words Seth!!!

1000 people is a lot! You continue to impact us with agents-at-scale work, Joon.

Thank you so much DJ!! That means a lot!

Interesting new study! Heh, it makes me want to participate and see how well the agent replicates me. Would I be proud to get a lower score than average?

Thank you so much for the kind words Amy!! haha, yes, I would also like to have you participant at some point (maybe I can ask for advice on things from generative Amy when you are busy!)

Crazy interesting paper in many ways:
1) Voice-enabled GPT-4o conducted 2 hour
interviews of 1,052 people
2) GPT-4o agents were given the transcripts & prompted to simulate the people
3) The agents were given surveys & tasks. They achieved 85% accuracy in simulating interviewees real answers!

So proud of this work with an amazing team!

Thank you to my coauthors, @mbernst.bsky.social, Percy Liang, @robbwiller.bsky.social, Carolyn Zou, @aaronshaw.bsky.social, @mako.cc, Meredith Ringel Morris, and Carrie Cai. And thank you Akaash Kolluri for helping out with the open source release. (14/14)

In closing, doing great interdisciplinary work that respects the tradition and rigor of each field is beyond any one person. This work would not have been possible without an all-star team that embodied its interdisciplinary nature, intersecting AI and social sciences. (13/14)

GitHub - joonspk-research/genagents Contribute to joonspk-research/genagents development by creating an account on GitHub.

For those interested, here is an open-source repository and a Python package for this work:

Github: github.com/joonspk-rese...
(While we are not releasing the participant data, I have included my personal generative agent in the repo. :)) (12/14)

So, to support research while protecting participant privacy, we (Stanford authors) plan to offer a two-pronged access system in the coming months: 1) open access to aggregated responses on fixed tasks, and 2) restricted access to individual responses on open tasks. (11/14)

We spent countless hours discussing ethics with the team, the IRB, and participants. Here’s what we believe: systems hosting generative agents of real people must, at a minimum, support usage audits, provide withdrawal options, and respect individuals' consent and agency. (10/14)

At the same time, this work points to the beginning of an era in which generative agents can represent real people. This ought to bring both excitement and concerns: how can we balance the potential benefits while safeguarding individuals' representation and agency? (9/14)

In sum, this work opens the door to simulating individuals. We believe that accurately modeling the individuals who make up our society ought to be the foundation of simulations. The resulting agent bank of 1,000 generative agents will further facilitate this function. (8/14)

In addition, our interview-based agents reduce accuracy biases across racial and ideological groups compared to agents provided with demographic descriptions. We attribute this to the agents in our study reflecting the myriad idiosyncratic factors of real individuals. (7/14)

Our finding: the agents perform well. They replicate participants' responses on the General Social Survey 85% as accurately as participants replicate their own answers two weeks later, and perform comparably in predicting personality traits and experimental outcomes. (6/14)

To achieve this, we turned to a foundational social science method: interviews. We developed a real-time, voice-to-voice AI interviewer that conducted two-hour, semi-structured interviews to teach us about these individuals’ lives and beliefs. (5/14)

We found our answer in models of individuals—creating generative agents that reflect real individuals and validating them by measuring how well they replicate the individual's responses to the General Social Survey, Big Five Personality tests, economic games, and RCTs. (4/14)

But we felt our story was incomplete: to trust these simulations, they ought to avoid flattening agents to demographic stereotypes, and measurement of their accuracy needs to advance beyond replication success or failure on average treatment effects. (3/14)

When we presented generative agents last year, we pointed to a future where we can simulate life to understand ourselves better in situations where direct engagement or observation is impossible (e.g., health policies, product launches, or external shocks). (2/14)

Simulating human behavior with AI agents promises a testbed for policy and the social sciences. We interviewed 1,000 people for two hours each to create generative agents of them. These agents replicate their source individuals’ attitudes and behaviors. 🧵

arxiv.org/abs/2411.10109