Jamie Cummins

Introduction to INSPECT-SR Training Workshop November An introductory 2-hour online workshop will introduce participants to the INSPECT-SR tool for assessing trustworthiness of randomised controlled...

Introductory online INSPECT-SR workshop. November 6th, 12-2pm UK-time. Free, places limited. BOOK: www.trybooking.com/uk/events/la...

thanks for the mention! 😊

RDM Weekly - Issue 016 A weekly roundup of Research Data Management resources.

Issue 16 of RDM Weekly is out! 📬

It includes:
- Data is Not Available Upon Request @ianhussey.mmmdata.io
- AI Generated Participants in Social Science @jamiecummins.bsky.social @science.org
- Why’s it Hard to Teach Data Cleaning? @randyau.com
and more!

rdmweekly.substack.com/p/rdm-weekly...

AI-generated ‘participants’ can lead social science experiments astray, study finds Data produced by “silicon samples” depends on researchers’ exact choice of models, prompts, and settings

Interesting article/paper.

I'm much less anti-AI than a lot of people on my feed. But pretty skeptical it can simulate human behavior effectively for social scientific purposes -- at least in cases where variation among humans, rather than acting like an average human, is what's important.

WCL winners, they’ll never sing that 😉

My article "Data is not available upon request" was published in Meta-Psychology. Very happy to see this out!
open.lnu.se/index.php/me...

I’ve seen @malte.the100.ci recently using one that looked very cool

OMG I can’t wait to listen!

Hacking proteins with AI

New episode of HARD DRUGS!

AlphaFold, ProteinMPNN & other AI tools are transforming biology and drug design.

But how do they work? What can’t they do? And can we use them to make a vaccine against Strep A for the very first time?

In this episode, Jacob and I talk about hacking proteins with AI.

Large Language Models Do Not Simulate Human Psychology Large Language Models (LLMs),such as ChatGPT, are increasingly used in research, ranging from simple writing assistance to complex data annotation tasks. Recently, some research has suggested that LLM...

Our work on simulated participants also makes a small appearance arxiv.org/abs/2508.06950

AI-generated ‘participants’ can lead social science experiments astray, study finds Data produced by “silicon samples” depends on researchers’ exact choice of models, prompts, and settings

@cathleenogrady.bsky.social has just published the story "AI-generated ‘participants’ can lead social science experiments astray, study finds" for Science. It is, once more, a reason to be careful when relying on LLM-generated data in empirical research. www.science.org/content/arti...

Looking forward to reading this, and I’m glad you’ve written it!

Excellent 🧵 about LLM synthetic data (silicon samples etc) and why they don't solve any particular problem in human research.

FWIW, in addition to results and considerations like these, I've argued elsewhere that the entire question is ill-formed: quantuxblog.com/synthetic-su...

There isn't really a fixed term tbh, people use a few different ones depending on field/domain/preference. Silicon samples seems to be the most common but there are a bunch of others, like synthetic samples/synthetic participants/etc.

Couldn't agree more.

Clearly I missed my true career-calling as a diplomat lol

OMG. Did not catch this one during my lit review. Wow.

Starting to feel eerily like Severance....

that should have been my full abstract!

👀 studying real humans better for understanding humans than not

@science.org just dropped a story covering this preprint! Check it out below, and thanks to @cathleenogrady.bsky.social for the great write-up! www.science.org/content/arti...

Yeah this paper was hugely inspirational for me!

Science is grounded in observation. Measurement is a tool for observation. Measurements should be evaluated for validity and reliability/uncertainty. Scientists who use measurements without understanding their properties are not really scientists at all.

AI-generated ‘participants’ can lead social science experiments astray, study finds Data produced by “silicon samples” depends on researchers’ exact choice of models, prompts, and settings

A lot of psych is already conducted with online convenience samples & ppl are probably excited about silicon samples bc it would allow them to crank out more studies for even less 💸

How about we reconsider the idea that sciencey science involves collecting own data.
www.science.org/content/arti...

YouTube video by sammyuri I built ChatGPT with Minecraft redstone!

Forget running DOOM on your calculator; someone created a 5 million parameter language model in Minecraft. www.youtube.com/watch?v=VaeI...

tabs are an objectively better approach for Python and I will not be hearing otherwise

Thanks Mitch! I hope if nothing else it will come in handy for reviewers who can point to it when reviewing this type of work and say “you’re generalising too much, and haven’t tested enough”

First Reaction: "Why would researchers even consider that?"
Response: "It is so much easier than actual humans, and LLM responses look kinda like human responses.

Help! ─ 𝗔𝗽𝗽𝗹𝗶𝗲𝗱 𝗦𝗰𝗶𝗲𝗻𝘁𝗶𝘀𝘁 open to work ─ Remote/Austin → www.rexdouglass.com
With new tools, I've been cooking:
▸ Machine-vision pipelines — RIOS Intelligent Machines
▸ Information-extraction pipelines — Microsoft
▸ Interactive SMS surveys — Pantheon Insights

So a spurious correlation from an observational study is “gold standard science”, but testing vaccines in multiple RCTs with over 30,000 participants each is not?