Anne Scheel

It's harder to understand reality than we intuitively think

New paper, on a worrying trend in meta-science: the practice of anonymising datasets on, e.g., published articles. We argue that this is at odds with norms established in research synthesis, explore arguments for anonymisation, provide counterpoints, and demonstrate implications and epistemic costs.

One approach to the age-period-cohort problem: Just don’t. Just to cause yourself more problems, you seek for something. But there is no need for you to seek anything. You have plenty, and you have just enough problems. Shunryū Suzuki in a 1971 talk A ...

New blog post about the age-period-cohort identification problem!

In which, for the first time ever, I ask "What's the mechanism?" and also suggest that sometimes you may actually *not* be interested in causal inference.

www.the100.ci/2026/02/13/o...

Adding to the confusion, entrepreneur Dr. Jing Liang noted on X that publicly available trial data appear to show Moderna conducted analyses involving higher-dose comparators in older adults. If accurate, it would raise further questions about the basis for issuing a refusal-to-file letter in the first place. No matter which account ultimately proves correct, this episode leaves us with two uncomfortable possibilities. If the FDA changed its regulatory position after years of clinical development, that reinforces a persistent concern about regulatory inconsistency. If, alternatively, Moderna met the agency’s highest level of stated expectations, actually did use a high-dose comparator and still received a refusal-to-file letter, the implications are even more serious, raising questions about transparency and internal coherence at the agency.

Great post @ruxandrabio.bsky.social on FDA’s refusal to review Moderna’s flu vaccine, changing its position on what trial design was acceptable.

Regulatory uncertainty doesn’t just mean missing out on this vaccine, but also reduces future R&D investment:
clinicaltrialsabundance.blog/p/the-modern...

Like the “uneducated” explanation, the “religious” one depends on a confusion between correlation and causation. Even if all women today with large families are religious, not all women who are religious have large families. Religion is correlated with total fertility, but not obviously causal.12 That is, something else mediates the association between religion and fertility. We don’t know what that is.

Some people confuse correlation and causation, some people confuse causation and “deterministic direct 1:1 causation” 🥲

I agree that assumptions underlying tasks in dev psych are often heroic and untested, but I don’t see how that frees you from the responsibility of discussing the assumptions underlying your own task?

I wasn’t expecting a response to my review obviously, but I was expecting a serious engagement with substantive issues affecting the interpretation of the results of this exceptionally big (and likely very impactful) study that you were made aware of before resubmitting this paper.

The comparison is literally the first sentence of your Results section.

Then I misunderstood you. I thought with “such assumptions” you meant the assumptions I had listed above. So those are not mentioned/discussed in the paper, right?

bsky.app/profile/elma...

Re “external validity” (it sounds like you mean criterion validity?): I’m not claiming that your task doesn’t track _something_, but that it tracks a bunch of things beyond what you claim it does. That many of these are correlated with cognitive development is unsurprising but also not v informative

Very possible that I missed something! Can you show me where in the paper you mention these assumptions and discuss their implications?

> inter-cultural differences may be due to changes/differences in any of these assumptions. I personally think that some of the group differences you found may quite plausibly be due to different priors about e.g. eyeball shape/size and distances between houses and hedges in different populations.

As I wrote in my review, this means that you cannot interpret the “accuracy” of participants’ responses (because there is no unambiguous ground truth). It also means that responses will depend on participants’ assumptions about all parameters listed above. Developmental changes as well as >

* Of course it _can_ be inferred, but that requires many more assumptions than you claim it does, namely about the size and shape of the agent’s eyes and the balloon, and about the distances between all relevant features in the video. The “true” values for all these are undetermined (it’s a cartoon)

The problem with your 2D cartoon abstraction is that it has no true solution. You coded the “true” target location as some point on the screen, but that point can’t be inferred from the agent’s pupils in a test trial.*
The paper you linked seems to use the same task so I’m not sure what it adds?

I reviewed an earlier submission of this paper and raised what I think is a very serious issue with your task — in short, that the exact location of the target of the agent’s gaze is unknowable. I’m surprised to see that this doesn’t seem to be acknowledged in the publication at all.

I learned, in the same semester, how methodological rigor is the bedrock of all progress in psychological science, and that a moderated mediation model based on questionnaires from 86 undergrad students is rock-solid proof for a theory in social psychology.

Interesting to know!

The things I have in mind are neither new (ie, not a reaction to observed standards) nor very hard. It’s really basic stuff like “define your target population”.

Yeah, good point and probably true. Though I feel like the disconnect starts in-house already — at least in my experience, empirical practices and thesis projects don’t tend to live up to the intro stats ideal. But IMO that’s because as researchers, we’re used to ignoring a lot of important steps.

Fair!

If this was indeed the core problem, I think there would’ve been way more progress over the past 10 years. What I see now in paper after paper are fantasy constructs studied either in glorified convenience samples or with obviously terrible measures (or both) that correlate for trivial reasons.

Yes, 100% this! Thanks for the tip, I’ll check out your paper.

Thanks, but this strikes me as a very odd and unhelpful definition of “garbage”. If anything, I think one of the things that have hampered progress in psychology is an obsession with “theoretical progress” before establishing any robust phenomena that make mature theories possible in the first place

…which includes an awful amount of active un-learning in psych. I just wonder if that’s normal in, say, biology as well

Of course there’s variation. But what I have in mind are really basic things that are covered in all intro courses I’ve encountered so far. Sampling methods, validity, error inflation, you name it

Is this a thing in other fields too?
The (IMO large) bad portion of the psych literature typically violates basic principles that we teach undergrads in intro methods & stats. Feeling increasingly embarrassed when I think of how awful published papers look compared to what I’m asking of my students

Bad news: additional capacity goes to patients who look remarkably similar to those already receiving care. Hence, outcomes are nonreactive. Thus, expanding the workforce doesn't inevitably reach those with the greatest need. 7/n

Literally

violin plot of results by testosterone measure

violin plot of result by risk pref measure

New meta finds no correlation between testosterone and risk preference, buuuuut testosterone was mainly measured using 2D:4D (nonsense) or saliva (still bad) and we know from Frey et al. 2017 that the risk measures don't tap into a coherent preference.
www.sciencedirect.com/science/arti...