Austin van Loon

Excited to continue learning about the latest #CSS at @ic2s2.bsky.social! I’ll be at the Social Prediction Session, presenting the mixed subjects design on combining human and LLM data in experiments. Paper with Michael Howes and @austin-van-loon.bsky.social. Come join us! doi.org/10.1177/0049...

New in Sociological Methods & Research: Soc PhD candidate @davidbroska.bsky.social, @austin-van-loon.bsky.social, & Michael Howes show how combining human subjects and large language models can yield precise estimates at low cost, with implications for scientific productivity
doi.org/10.1177/0049...

Looking forward to presenting my Class Gap work at the RNIM Online Seminar on Zoom this Friday!

Anyone can register to attend here: sites.google.com/view/rnim-se...

“…the mixed [subjects] design… is the most reasonable approach I’ve seen in the LLMs for social science literature for integrating LLM simulations into confirmatory-style experiments…”

Also provides thoughtful reflections on the limitations of the approach!

My work with David Broska and Michael Howes on augmenting human subjects with LLM simulations was discussed in the Stat Modeling blog!

👀

statmodeling.stat.columbia.edu/2025/06/12/b...

NYU Social Psychology is looking for people passing through NYC next year who want to give a research TALK in our Speaker Series.

We welcome self-nominations and scholars from all career stages (you can apply multiple times).

Apply here: docs.google.com/forms/d/e/1F...

Mixed feelings about silicon subjects (LLM predictions of human behavior) as replacements for human subjects? Consider the mixed subjects design.
🚨Now published at Sociological Methods and Research🚨
doi.org/10.1177/0049...

New from Microsoft Research & Harvard Business School colleagues

"Shifting Work Patterns with Generative AI" by Eleanor Wiske Dillon, Sonia Jaffe, Nicole Immorlica, Christopher T. Stanton

arxiv.org/abs/2504.11436

I’ve never felt so seen 🥹

New paper from the computational culture lab, forthcoming @amjsoc.bsky.social!
Building on (largely untested) sociological intuitions, we show how positions in the organizational network relate to identification with the organization, using a language model:
www.journals.uchicago.edu/doi/abs/10.1...

Want to learn about computational social science *for free* and launch interdisciplinary research projects? We are so excited to announce there will be *26* Summer Institutes in Computational Social Science this year! Apply to one of them here: sicss.io/locations

Our paper is out today in @pnas.org! 🎉 In a large-scale experiment on a YouTube-like platform, we find that giving people politically “slanted” video recs doesn’t shift beliefs or viewing behaviors.

In other words, online filter bubbles may not be as polarizing as we think…

Thank you @davidbroska.bsky.social and Michael for your leadership on this paper. It’s been a joy working with you both. Onwards to more breakthroughs in computational social science!

💡 LLMs aren’t going anywhere. As social scientists, we can either ignore them or work to integrate them rigorously into the research process. We hope this is a step in the right direction—leveraging what LLMs can tell us about human behavior while preserving scientific rigor.

But wait, there’s more! Since many social scientists are new to PPI, we built tools—like a PPI power analysis. With an estimated treatment effect and assumed interchangeability, our tool optimizes budget allocation between expensive human data and possibly biased LLM responses (!!)

3️⃣ The more informative LLM behavior is about human behavior, the more we learn from the silicon data—and the more precise the estimate becomes. As models and prompting improve in the future, the estimate ✨automagically✨ adjusts!

Key properties of this estimator:

1️⃣ It adjusts for silicon bias—if your human-only estimate is unbiased, so is this estimate.

2️⃣ If LLM data adds no info, the mixed estimate is as precise as the human-only one.

How does it work? We collect a small set of LLM responses that “match” gold-standard human data (think silicon twins). Leveraging prediction-powered inference, we use these to estimate and correct for biases in the LLM, then combine with more LLM data. More details in the paper!

What if you didn’t need the interchangeability assumption? My coauthors (David Broska and Michael Howes) and I propose a “mixed‐subjects” approach: collecting human and silicon data together to produce estimates about humans that are unbiased and more precise. 🙂➕🤖

However, silicon sampling today often relies on an untested interchangeability assumption—that LLMs act like humans (at least on average). But, we a TON of evidence now that LLMs misrepresent human behavior/opinions in all kinds of ways. There’s good reason to be skeptical. 🤔

If effective, silicon sampling could make social science more efficient, safe, and equitable. Clearly LLMs contain a lot of information about us—it would be foolish (and irresponsible) to just ignore it.

Since the generative AI boom, researchers have flirted with “silicon sampling”—using LLMs to simulate human subjects. For instance, ask an LLM how it’d answer survey questions after experiencing an experimental stimulus, then treat its responses as data.

🚨 ACCEPTED AT SMR 🚨
Confused by colleagues who seem to want to study LLMs instead of humans? Frustrated by skeptics (e.g., myself 8 months ago) who dismiss LLMs as a potential source of data on human behavior? Check out our paper for a new way forward: osf.io/j3bnt_v3/

I am hiring a predoc at MIT to start this summer!

We will work on various topics in labor economics - unions, minimum wages, imperfect competition, etc.

Please apply and share !

#predoc @mitiwer.bsky.social #EconSky

apply.interfolio.com/163492

The Department of Social Science at the University of New Brunswick in Saint John invites applications for a tenure track position in Sociology. This position is at the rank of Assistant Professor and is housed within an interdisciplinary department that includes Social Work, Sociology, Communication Studies and Education. The Department of Social Science offers major, double-major, minor, specializations, and honours programmes in Sociology. The position will commence as of July 1, 2025 and is subject to budgetary approval. The discipline is seeking a scholar with demonstrated expertise in quantitative methods. The discipline is especially interested in candidates whose research program also engages with one or more of the following topics: immigration and migration, global determinants of health and well-being, mental health and addictions, and social policy. Candidates will be expected to teach at both the introductory and advanced levels and to work with graduate and honours students. Course assignments will include a core course in quantitative methods. Candidates should have a Ph.D in Sociology or cognate discipline or be near completion. Candidates with a Ph.D. nearing completion will be considered for appointment as Lecturer. Applicants are requested to submit electronically (as a single PDF) a curriculum vitae, statement of their research interests, and a teaching dossier (that includes evidence of teaching competence, as indicated by teaching evaluations, testimonials from individuals familiar with the candidate's capabilities) to socsci@unb.ca Applicants should arrange for 3 reference letters to be sent by referees directly to socsci@unb.ca. Questions concerning this position may be directed to the department chair, June M. Madeley at jmadeley@unb.ca

UNB's St. John campus is hiring a TT position in Sociology with an emphasis on quantitative methods. Although there are rules about prioritizing Canadian candidates, I would encourage US folks to apply.

universityaffairs.ca/search-job/?...