Peder M Isager

Episode 74: Notiones Vague | Nullius in Verba In this episode, we discuss the problems associated with vague concepts in psychological science. We talk about the jingle-jangle fallacy, the trade-off between broad concepts and more precise…

New episode of Nullius In Verba! We discuss the jingle-jangle fallacy, the problem of vague concepts, how the incentive structures promote vagueness, why people who prefer more rigour have to be called the validity 'police', and much more!

nulliusinverba.podbean.com/e/episode-74...

Anyone know of any good papers on the role of postdiction/retrodiction in theory appraisal?

#philsci

Being able to explain which replication efforts are important and why should be helpful both when asking for funding, when convincing journals to take a replication report, and when motivating reserarchers to take on replication in their own work.

Thanks so much for helping to move the discussion!

Wonderful! So happy to hear that workshops are including discussion on this :) I think giving researchers clearer direction on how to formulate replication goals and use them to pick targets is going to help a lot with actually getting more replications done in practice.

Figure 1 from the paper. Econ: from 2008 to 2024, methods aiming for causal inference have increased, theoretical work has decreased. Psych: Mostly experimental or descriptive correlational work.

Just learned about this study looking at methodological trends in psych and econ over time: online.ucpress.edu/collabra/art....

Matches my perception well: Nobody in psych bothers to (explicitly) try causal inference unless they conducted an experiment, not a lot of theoretical work either.

Very cool to hear! What was the topic of the workshop? :)

Very cool. Speaking of front-door, I have also added features to let you statistically control for variables in the DAG to simulate e.g. back-door criterion.

Because of course the Germans have their own Pokémon names 😂 Take notes, Språkrådet!

Introducing Causion: A web app for playing with DAGs | Peder M. Isager Personal website of Dr. Peder M. Isager

New blog post introducing Causion - a web app for causal inference teaching and learning: pedermisager.org/blog/causion....

... generate a "plausible" DAG for your research problem that you can play with and simulate data from. I had some success with this for a class project already. Not guaranteed to work of course (AI is not very good at causal inference yet), but it might be worth a try!

3. If you are working in a field with a large existing literature, you may have some success collaborating with AI to create a plausible SCM for the effects you are studying, including likely confounders, mediators, etc. You can copy-paste such SCMs into the SCM panel of Causion to automatically....

2. You might find the "simulate data" feature in Causion interesting. For example, you can simulate data with a certain level of noise and plug that data into a statistical analysis program to help design an analysis plan for identifying causal effects that you can then preregister.

... to communicate your causal beliefs to them than verbally stating the causal relationships you think are at play. Since Causion is a DAG drawing tool, it can help you with this process.

That is a very interesting use case. I think you might find three features particularly interesting.
1. DAGs help to formalize intuitions. If you have verbal theories about variables involved in your research and how they're related, presenting your readers with a DAG is often a much clearer...

Please do! I am also planning to use it for teaching in the coming months. All feedback and suggestions for fixes and improvements are most welcome! Because I am building with Codex I can normally implement fixes and changes to the UI (like adding buttons etc.) quite fast.

Thanks! It is lightly inspired by the Star Treck Discovery intro :)

Tagging some potentially interested people: @raphaelmerz.bsky.social @dingdingpeng.the100.ci @jamessteeleii.bsky.social @epiellie.bsky.social @elluetravel.bsky.social @annemscheel.bsky.social @lakens.bsky.social @annaveer.bsky.social @pwgtennant.bsky.social @fdabl.bsky.social @nickchk.com

NB: The app is coded with the help of ChatGPT Codex. I have tested it as well as I can, but it may contain bugs and errors. If you spot any, let me know and I will fix them.

GitHub - pederisager/causion: An app for visualizing the expected consequences of changing variable values in a DAG. Gain an intuition for the relationship between correlation and causation. Created w... An app for visualizing the expected consequences of changing variable values in a DAG. Gain an intuition for the relationship between correlation and causation. Created with ChatGPT. - GitHub - pe...

The app will always be free to use, and the app code is open source on Github: github.com/pederisager/....

I hope Causion can be of use to methods teachers and students who want to learn about causal inference and its role in statistics. The app is designed to be easy to use interactively (e.g. as part of a lecture, or to whip up an example from a course book while reading).

YouTube video by Peder Isager Causion App Tutorial: Build, manipulate, and simulate data from DAGs

If you want to try it out, I’ve posted a tutorial video on youtube that explains all the core functionality: www.youtube.com/watch?v=C3fb....

The app is available online at causion.pedermisager.org

For many years I’ve wanted a tool to help me visualize causal inference rules, ”play out” causal mechanisms in DAGs, and quickly study what happens in data when I change something in a DAG. Causion lets me do all of this in a simple and intuitive way.

Introducing Causion: A web app for playing with DAGs | Peder M. Isager Personal website of Dr. Peder M. Isager

New blog post introducing Causion - a web app for causal inference teaching and learning: pedermisager.org/blog/causion....

That's what I was thinking of at least :) The paper, all commentaries and our response to commentaries are now all open access in Meta-Psychology: open.lnu.se/index.php/me...

🚨 SynthNet is out 🚨
Researchers propose new constructs and measures faster than anyone can track. We (@anniria.bsky.social @ruben.the100.ci) built a search engine to check what already exists and help identify redundancies; indexing 74,000 scales from ~31,500 instruments in APA PsycTests. 🧵1/3

Regardless, this issue does pose problems for the usefulness of RV. I discuss this some in my PhD thesis discussion. See figure 6.3.C for a similar example: pure.tue.nl/ws/portalfil...

Test severity may at once improve the ability of a replication study to reduce uncertainty, but it can also leave less uncertainty to be reduced to begin with.

”paradox: test severity seems to imply both increased and reduced uncertainty reduction, depending on how you think about it”
I think this is not a paradox per se. Rather, I think test severity can have two causal effects that work against each other.

I think it is possible to reduce uncertainty about such claims through replication, even without a SESOI (e.g. if you’re willing to be a Bayesian).