Daniel Heck

Diagram showing four phases of methodological research (Theory, Exploration, Systematic Comparison, Evidence Synthesis) with an arrow indicating that preregistration usefulness increases from early to late phases. Each phase lists its aim, elements, outcome, and an example from factor retention research.

Does it make sense to preregister simulation studies?
This question has sparked a lot of debate.

▶️We* work through the why, when, and how
▶️We discuss different phases of methodological research to clarify where preregistration might (or might not) add value

📝 Preprint: doi.org/10.31234/osf...

Thanks!

Yes, it's a classic paper experimental researchers should know about.

I was surprised that ANOVA of proportions has still been used relatively often in the literature on the truth effect.

Finally published in Memory & Cognition:

Multinomial models of the repetition-based truth effect: Investigating the role of prior knowledge

link.springer.com/article/10.3...

Great, thanks for the detailed and constructive feedback and suggestions! You are right that the current framing might not highlight the core issue of aggregation clearly enough.

The paper is currently submitted, but we will consider PCI: Psych in the future, thanks for the hint.

Thanks for this remark, we will mention in a revision that the field has already been changing. That's good to see!

Agreed!👍

However, it took me a while to understand that all main effects in the probit GLMM are simply effects on the response bias c, whereas all interactions with the factor indicating noise/signal of the stimuli can be interpreted as (main or interaction) effects on the discriminability d'.

Link to preprint: osf.io/preprints/ps...

New preprint by @semihaktepe.bsky.social 🎉

We compare ANOVA/SDT/GLMM for binary judgments in 20 datasets of the truth effect. #lme4

Main conclusion:
"GLMMs are a theoretically sound and practically robust method and thus superior for analyzing binary judgments in social and cognitive psychology.”

Thanks!

Postdoctoral Researcher in Quantitative Psychology / Data Science in Psychology

Interesting postdoc position on modeling psychological resilience at the DKFZ Hector Cancer Institute, Mannheim:

jobs.dkfz.de/en/jobs/1682...

--> dynamic/latent-variable models, Bayesian hierarchical models, causal inference, time-series analysis, cognitive modelling

The latest version of Sigmund, the #AI #research assistant, is able to fully control the #OpenSesame interface. This allows you to build #psychology experiments entirely through conversation. This is no AI slop, but real functionality with real benefits! Tutorial 👉 osdoc.cogsci.nl/4.1/tutorial...

Still think this was one of the best power moves of all time

www.sciencedirect.com/science/arti...

The method worked better than simple aggregation for validation words such as "fifty-fifty chance", "never" or "always" (indicated in the plot by the black intervals compared to the gray interval areas).

New paper by @matzekloft.bsky.social in Psychometrika🎉

We developed a model for aggregating response intervals which are obtained if a participant judges that the word "most" covers the range [86% - 97%].

The model aggregates such intervals while considering people's proficiency & item difficulty.

Halloween in our lab 🎃

Beware of divergent transitions!

There still seems to be a lot of confusion about significance testing in psych. No, p-values *don’t* become useless at large N. This flawed point also used to be framed as "too much power". But power isn't the problem – it's 1) unbalanced error rates and 2) the (lack of a) SESOI. 1/ >

Simulation studies have a conflict of interest problem. The same team:
- develops a new method
- designs a simulation study to evaluate it
However, the new method has to show good performance to get published.

We propose living synthetic benchmarks to address the issue (doi.org/10.48550/arX...).

Welcome to @lauragroot.bsky.social as a new member in the Psychological Methods Lab at @unimarburg.bsky.social! 🎉

Laura just started as a PhD student and will work on Bayesian statistics and multinomial processing tree (MPT) modeling.

We built the openESM database:
▶️60 openly available experience sampling datasets (16K+ participants, 740K+ obs.) in one place
▶️Harmonized (meta-)data, fully open-source software
▶️Filter & search all data, simply download via R/Python

Find out more:
🌐 openesmdata.org
📝 doi.org/10.31234/osf...

Welcome to @timangelike.bsky.social as a new member of our lab at @unimarburg.bsky.social! 🎉

Tim just started as a postdoc in a Momentum project funded by @volkswagenstiftung.de:

"Bridging the Gap Between Verbal Psychological Theories and Formal Statistical Modeling with Large Language Models"

Mit diesen vier Methoden schüchtert die AfD Lehrkräfte ein Wir haben in den vergangenen Monaten unzählige vermeintliche Einzelfälle analysiert. Nun ist klar: Der Angriff der Rechtsextremen auf das Schulsystem folgt einem klaren Muster.

krautreporter.de/kinder-und-b...

Laut CDU-naher Studie: Rechte Parteien durch Kooperation zu zähmen, funktioniert nicht Der Rechtsruck beschäftigt nicht nur Deutschland, in anderen EU-Ländern versuchen sich rechte Parteien in Regierungsverantwortung. Die Konrad-Adenauer-Stiftung hat untersucht, was das für Konservative...

Konservative 🤝Rechtsextreme = politischer Selbstmord?

In vielen EU-Ländern versuchen konservative Parteien durch Kooperationen, rechtsextreme Parteien zu „zähmen“. Die CDU-nahe Konrad-Adenauer-Stiftung hat untersucht, was das für Konsequenzen hat. Spoiler: Es funktioniert nicht. 1/6🧵

That's interesting - thanks for comparing the methods and for letting me know!

Analytical solutions are of course more elegant than numerical integration.

For all who use Bayesian hierarchical models, have a look at our new preprint, out now together with @linushof.bsky.social @nunobusch.bsky.social and @thorstenpachur.bsky.social

osf.io/preprints/ps...

Still, it is worthwile to have a paper elaborating on this in detail!

TreeBUGS: An R package for hierarchical multinomial-processing-tree modeling - Behavior Research Methods Multinomial processing tree (MPT) models are a class of measurement models that account for categorical data by assuming a finite number of underlying cognitive processes. Traditionally, data are aggr...

Just a remark regarding "The same incorrect computation appears in implementations of latent-trait multinomial processing tree models":

The issue has been mentioned in the literature (shorturl.at/Ajx0W), and the R package TreeBUGS has the function probitInverse as a solution (shorturl.at/mRp1S).

Looks and reads great, I especially like the concise figure!

This is an important issue which is often overlooked, so the contribution will be useful for many modelers. 👍

Assessing the truth effect’s reliability and test–retest stability The finding that repeating a statement typically increases its perceived validity is referred to as the truth effect. Research on individual differenc…

New paper showing that both the test–retest stability and the split-half reliability of the repetition-based truth effect is close to zero:
www.sciencedirect.com/science/arti...

At the FORRT Replication Hub, our mission is to support researchers who want to replicate previous findings. We have now published a big new component with which we want to fulfill this mission: An open access handbook for reproduction and replication studies: forrt.org/replication_...