Nathaniel Haines

wooahh! need to give this a read 🤓

Ah yeah most of the work im familiar with skews toward generative models. e.g. like this classic piece, which captures various interactions between Stroop conditions, learning effects, etc.

psycnet.apa.org/record/1990-...

Probably not what you are referring to, but the Stroop task is our main example here: psycnet.apa.org/fulltext/202...

simply amazing

A Series on Building Formal Models of Classic Psychological Effects: Part 1, the Dunning-Kruger Effect | Computational Psychology Introduction The Dunning-Kruger effect is perhaps one of the most well-known effects in all of social psychology, defined as the phenomenon wherein people with low “objective” skill tend to over-estim...

It has been quite a while since I posted my blog on the modeling the Dunning-Kruger effect. I intended for this to be the first of a series on modeling some classic effects in psychology/cognitive science.

Any recommendations for the second post in the series?

haines-lab.com/post/2021-01...

yeah this is a great set

Wish I could have made it in person!

i have now vented on reddit and bsky so maybe i will feel better about it now

last of us S2 E2, more like last episode amirite

the moment some unintuitive simulation and empirical results finally click and you know what needs to be done 🤌

a feeling vibe coders can only dream of

tariffs amirite

Thanks for the endorsement! Awesome to hear it's been influential to your work 😁

And yes the KL finding is super interesting by itself, happy to hear someone found it buried in the supplement ha

Thank you! Glad you have found it useful 🤓

it was horrible, there were 10+ papers based on the initial preprint/blog that were in print before this one eventually made it there 😅

SIX YEARS after the initial blog post, this paper is finally published.. what a wild ride

- the blog: bit.ly/3GbOqQa
- original tweet thread: x.com/Nate__Haines...
- published (open access) paper: doi.org/10.1037/met0...

Yeah I've found LLM tech good for stuff that is mostly boilerplate (e.g. some general software development stuff), but when it comes to modeling work that is necessarily bespoke, they are quite bad

sounds like a black mirror episode

That's called MDMA

But I have not read into this lit much tbh, so I have a biased sampling of what people think 😁

"Regressions to the mean = statistical artifact = the effect is not real" is my read based on papers like this

My read is quite different, e.g. www.sciencedirect.com/science/arti...

hey we grow good bourbon and that's food

Yes that's exactly what I mean—the claim that, because regression to the mean captures the effect, it is a statistical artifact and not something with a psychological explanation

I think they could potentially be good items for the classroom! It is a fun item set at least, although not sure about calibration as I really only looked at quartiles

Yeah IMO, I think the ideal design would be something within-person such that we could sample people across their individual skill distributions so that we could understand if the effect arises at the person level

"That explanation ... would not be able to explain self estimated performance going up again at very low ability"

I still think it is a better explanation than the "stats artifact" explanation. The models I used are too simple on purpose—I didn't have data for fitting anything more complicated

I feel there is some talking in circles going on here lol character limits be damned.. my stance is that the mechanism generating the noise is not well understood, but that psych-motivated models help provide an explanation. To get more fine grained (empirically) we need better datasets

Ah yeah this form of probability distortion is generally attributed to Goldstein & Einhorn (1987). e.g. see eq ~23

sci-hub.se/https://psyc...

I mean snipping the quote like that is a bit misleading ..

there are of course many different ways that noise could arise. the two models have different implications re mechanisms, but the data I had available were not able to distinguish between them

If you build a model with these assumptions based in, you will get the DK effect. In the same way, if you simulate multivariate normal data with one mean higher than the other and a moderate covariance, you get the DK effect. IMO the former is a plausible explanation, the latter just a description